because you can't just open up the data. you have to do things like this, where you get the community together or you have people actually talking about it because the demand side, as we were talking about it, will be there because there is going to be someone there. there have to be people working with it who are getting out there. i think this is what this city is going to be really powerful. in terms of other cities doing as well, chicago is doing some really interesting stuff. scary cool stuff. they're taking 3 in 1 data, pothole request and crime report and matching it up with social media. they're getting this really deep and rich picture of what is going on in the city. and you can do that with data when you think about it creatively. philadelphia as i mentioned, they are really active in open data. and new york, again with 3 in 1 is doing smart analytics. i think that's what you'll see happening as well, government starts to become smarter, make better decisions, better policies. this term algorithmic regulation, whic

i talk about some of the data sets we use. most people still don't know they exist despite great efforts from the city and others. and, you know, a lot of entrepreneurs that we're looking for is the way to get leverage, a way to build an experience or build a product before we have 100,000 users using it. open data is the key way to do that. you can start out with this experience, very lively and robust with very few users. and then when the first users come in, there is actually something for them to do and see. and i think that you'll start to see as the data sets get more robust and a little bit better and hopefully as we get to improve them ourselves and put them back into the public domain, the innovation that comes off with them is just going to grow exponentially. >> this seems like a really important moment. the world of civic innovation until now has really been focused on sort of open public data and the private data world has been separate. the fact that the city realizes that it is private data that adds to that and

here we have data from the city. we also have data from private data sources that read -- not going to mess with it. there's one of the variables now, the bottom you see there is median household income. we're pulling in all these different data sources, creating a beautiful visualization to tell a story at the neighborv level of how the city is growing. and some of the things you're seeing on the map here are a pipeline of information about both residential permits over the past decade or so, how has the city allocated permits throughout different neighborhoods in the city. and some of the other things you're seeing on the map is the approved businesses, the businesses are currently doing business in the city. one of the things we said once we started visualizing on the map the slow and more rapid growth of residential -- residential property in soma and then in 2011 you just saw obviously a huge residential boom in the downtown area. so, we've just actually -- we're a company in southern california. we just relocate

because you can't just open up the data. you have to do things like this, where you get the community together or you have people actually talking about it because the demand side, as we

also the opportunity to have better data sets from the work that you've done, scrubbing them and harmonizing them. i think there is also this really great opportunity to generate whole new types of data sets like motion loft is doing. and i think that this is going to present something kind of back to the city, showing where some of our gaps are and hopefully filling in some of those gaps with those data sets. so, i think that there is something about having more city data sets, existing data sets as well as creating new types of data. this is really exciting for us. >> we're going to tell a quick story on how private data can stimulate civic innovation. this summer in singapore, we had about 30 data sets opened p. up. one data set was about a million or so records of taxis all over town. typically taxi data stuff, it's hard to get, private companies, they don't want to open it up competitively. one taxi company at the last minute opened up, motion loft made it efficient to see what would happen. because they contributed to the stuff, we had five separate teams working on could we do collabo

the data is intended to transform the individual data for the services in san francisco. it follows a national movement to give residents access to their own data. it establishes technical glz for the data coordinator. this will make the data more assessable. the date will be released and the documents for rp s and licensing requirement. the data will be available in memorandum and other formats using the most update standards. the following deadlines are going to be effected. winning within a month the mayor is going to appoint a chief officer to oversee the policy within his or her department. and they will have access to the data. within six months the department should publish a catalog that can be available. within one year the chief data officer shall present a plan to help the board of supervisors and the mayor. those deadlines insure with remain educational to the community so the public is made readily aware. one of the priorities of the united states in office is to improve the efficiency in office. i want to thank all my staff and in a particular to hannah who's

traditional or the most common data sources like the census. like are kids safe and what are that credit savings we families and the housing situation and finally the data base. we just launched it in the case management system and we are now working on phase two which is actually developing the second phase where partners can use it collaboratively. this is our data and evaluation framework. we really want to know at the population level, is the promise neighborhood better off as a result of these integrated services that we will be providing to our residents. so we have a common set of measures there. we also want to know are children's families receiving the service they need for success and are these pipelines effective and partners working together to ensure children and families are better off. we want to look at all of these levels of analysis to get at. is this working? if it's not, what can we do to change it? so we are taking this, we aren investing heavily in the data and values component. we have a 5-person evaluation team. we have

we're making sure that data belongs to our city, not the vendor. and second, we're making sure that any software that we buy or build hatx a public api or some equivalent. we don't want to be held hostage by a vendor or by technology. this data belongs to our constituents. we are simply stewards of it. in closing, i want to thank the hatchery, i want to thank our city leadership, mayor, as well as president chiu and partnering with us on this legislation. and i want to thank all of you in our community who have really done amazing things with this data. it's just a celebration of the good work that you're doing that we're here. open data would not exist without our community. so, with that, i'm going to actually hand it off to 100 plus to do a really quick demo and then we're going to do a little q & a and we'll have them come up next. >> 100 plus, we're based here in san francisco. we are interested in small healthy behaviors, ways to be healthy that don't involve going to the gym. we created a system where we recommend hops or help opportunities.

the internet. we want to build solutions -- data centers over the internet. we want people to build solutions for this kind of data centers. >> before the technology became popular, not many people thought about we need to build up all these cloud computing data centers and they probably did not foresee the impact technology would have on people's lives. based on your experience and knowledge about the industry, what would be the next thing people likely to develop in the years ahead based on the experience we have accumulated in the development of cloud computing technologies? >> probably we are going to see thinner and thinner devices because more and more applications are moved to the cloud. everyday lives, we don't keep things around. we put things on the cloud. this issue of security and longevity is true but eventually it will all be worked out. people are worried about putting their credit cards for e- commerce. nowadays people do it all the time. the issue is whether all these services can be sustainable or not? is there a good business model where the

the data centers move to where power gets generated. what is the next hardest thing to move? big data. did it is still really hard to move, even though we have lots of wavelets. that says that the application moves to the data is. if you think about facebook and google and all those folks, they build data center's right were the power is generated. typically near hydro plants and so on. what you find is that more and more applications will move to where the data is. moving those big chunks of data is very difficult. in terms of enterprises in the cloud, there is certainly no reason to suspect that the systems used by cloud vendors like amazon are not capable of geographic replication and redundancy. it is absolutely the case that someone like netflix, for example, could survive an outage of two simultaneous amazon did a centers through geographic redundancy and so on. this stuff exists, and the technology exists within the cloud providers to make sure that once your data gets there, it is not going away. it is just not a cloud provider solution. most enterprises will have thin

a student he proposed his own geological theory it was important to go to antarctica to collect the data he needed. this i didn't find anything new for. here there are no influence. like t.v. or anything like this. you have to sit and think he's simply sitting and thinking. the stations ionosphere is to keep his instruments in a corner of the same room his job is to monitor high altitude conditions all the data he collects has to be sent to an institute in st petersburg and the equipment needs regular adjustment. in fact every polar job has its own specifics such details are handed down to each new generation of polar explorer. these balloons can rise to forty two kilometers. carry sensors that information to a console of the station. is very important we make aeronautical charts for aircraft like how high helicopters can fly for instance similar balloons are released all over the world at midnight g.m.t. russian polar explorers have long invented new ways to make the process more efficient such as how to make them easier to release. it is so short but we can still set up the balloon lik

the n.s.a. pass along raw data and intelligence to israel not just about u.s. citizens but also on canadians new zealanders brits and australians what do you make of this recent information you know over the years the united states has shared intelligence with israel and israel has shared intelligence with the united states. as in so many cases i think with the intelligence community there has been too little congressional oversight. you know these things that are done in the name of the security of the american people. should be reviewed and. overseen by the. representatives of the american people. to see that things are done really in the best in the national interest and in the best way. we know that with surveillance of americans the n.s.a. went way overboard outside the law probably against the constitution in. surveilling americans collecting data. really personal data about americans without probable cause without good court oversight the court has now said it's come to light in recently released papers that they felt they were lied to the court was lied

site is called about the data. 's supposed to give consumers access to the personal information that ax i don't know gathers about you, and sells to marketers. when i went in there, it had my wrong car, it had my income was way off, my merit gang was way off, it didn't know i had children. and it took a guess as to who i voted for and they were wrong. not only is all that information very personal, it's -- it's wrong. now what does that mean for big data and the problems with it. >> well, what you did is you saw the problems of a small data universe. and what axiom has is small little data points with you in which they are trying to make very large inferences. big data, whos differently. let's take your location. gp sing is very accurate, but it is not perfectly accurate. we will know within a meter or two where you are. that's not very food if we want just one data point as a discreet moment of time to find out exactly where you are. but if we take your patterns across every single day, and where you are at every m

and all of that. >> no, it is with the texting and the data and all. yes. >> thank you. >> and i guess that the special shout out there should go to fong of our telecommunication and his connection to verizon that made that happen. >> anything further for the chief? >> thank you. >> chief. >> commissioner loftus? >> my mind was elsewhere. just to follow up, i know that we have, and you have covered this before, i would just ask that if the people are watching miss brown did talk about the unsolved homicide of her son and i know that the question that she left and i want the public to be clear on is what more can bedo and we talk about the need for evidence in that case and i know that we talked about this before. but if you would not remind what is needed in that case if there is anything that folks can help. >> we did speak before and i will ask again tomorrow of homicide if we received any information since we discussed it on the anniversary sadly. on the 7th anniversary. and i am familiar with the names, that we see. these are the folks that people a

for all the chatter about big data in the technology world, the truth is only 10% of the data out there is actually structured in one of these businesses. what about the other 90%? the unstructured data. that's where splunk comes in. much of this unstructured data, this stuff is what's known as machine data, which includes all the data generated by all the systems running in data centers, all the connected devices, applications, servers, security device, machine to machine remote infrastructure that powers any given enterprise. until splunk came along nobody was really doing much with this machine data. it's a catch-all category. there are literally thousands of distinct machine data formats. that makes aggregation very difficult. but splunk's platform takes the machine data and makes it accessible. and most af all, valuable to its clients. thus the heart of the big data theme you can use this information to diagnose security problems, assess the health of your remote equipment, and demonstrate compliance to your regulators, among so many other great applications. splunk has figured out

let's take the weather data as an example like mikey you know you have all the data and the prediction and no one else will work out with. the palm and there will be no information from there anymore and with all the normals the two are used to. the boy. generally almost all of the scientific work in antarctica when winter comes down to monitoring investigations and observing different processes. science doesn't play the main role of what we want to know even though all of us pretend to do it here the main idea. is geopolitics. is the most important thing here. science is just for show that they are where you are goes i'm going. to have to make sure we have a claim here. water for the station is drawn from. at the progress station they can automatically get water from a nearby lake but the polar explorers don't like it they're used to going to the remote lakes. seems to taste better but it's an illusion all water here is the same after distillation. and because of the lack of minerals polar explorers constantly suffer from dental problems. better to fill it. table and forget to take. a

we put 95 different monitors within the collection system and all this data gets into the model to no, it. we're very concerned we quash baseline conditions so we'll be cpa capturing other conditions. we engage portland and portland has been a great resource and new york city and we had a brown bag and we integrate that into our program and feel we have a responsibility planning and monitoring program in place. this was what one of the monitor looks like. we capturing the information and it end up going to southeast where we capture all the flow conditions so that all the data can be compared and the data is reviewed by the california institute and they'll be producing the reports it's a little bit of 0 third party green infrastructure data. but the purpose is to predict how well, the infrastructure is going to be. >> can i - >> so the monitoring - >> say that again so the california - >> so when we get our data downloaded they're looking the infrastructure. >> so then is a report general rated. >> we'll gotten newland and leeland and that might be something when we sit down the 3 tho

the data would support that choice. >> christina what is your reaction to that? there has been so much told to women in the past decades about not drinking while pregnant. >> i think that topic is a tough one and i think it goes back to something emily actually put in her book. you've got to take the information and you've got to analyze it with your own personal value system and make an educated decision from that. for me personally my grandmother actually was a court appointed advocate for a child with fetal alcohol effect. and so i saw firsthand the impact that alcohol had on that young man, and now he's a grown adult. and for me, that choice became fairly simple. and i figured, you know, i really, for one drink or a beer here or there, i didn't want to risk it. but again that's my personal decision. and i think you can't make a global decision for anybody. i think it's got to be unique to any woman as they're carrying their child and frank reply every pregnancy because every pregnancy is different. >> emily the fetal alcohol disorders study group recently like

we can't store the data, the consumer won't like it. somebody else should do it. >> find that interesting in the sense i find it contradictly to what i might want. if a stroke is coming on, god forbid i hope that's several years away. if a stroke is coming on and picked it up, part of me wishes it would go to the cloud, my doctor, an ambulance will be called. >> it will. it will be your data. we have the commodity models to do that. >> you will be able to store your data over time and observe behavior. >> you will choose if you want to sell it or not. we don't want to be a data mart. >> lets take a quick break. we'll go to commercial, come back and start with your question in just a minute. >>> welcome back to "press: here." before the break i cut you off so we could take the break. go ahead. >> you built the device. three times when we were talking you refer to yourself as algorithm people but you're suggesting being part of a much larger kind of business around data, visualization of what people have done over time. perhaps insurance

i didn't feel that way but when you actually realize it's here when the data set and your dream is ready to come true it's tough. what's next i don't know if i like it i'll keep doing it. we'll both flying to antarctica from cape town. the landing strips can only operate for short periods. the weather is too unreliable and the distance too great. for example even in early autumn the temperature at the station falls to sixty degrees below zero skids can get no traction when the snow is that cold. in a month's time the academic field. the food and fuel for the station. will spend the winter the ship is a floating headquarters. of the seasonal expedition. and the head of the winter team which. right now i spend less time at home. of course my family is waiting for me back home but i think they get fed up with me after a while but. they are used to living with me that's just the way it is. gave me one toilet roll he told me it would be enough until i get home i said for a year and he answered when i say home i mean. those who are experienced are already used to it the newcomers have this mix