ballmer was part of the churchill club event, and when he was asked about the data, he said it is a machine-learning problem. today we have very remarkable people on this panel, and they are uniquely qualified to have this discussion about machine learning. a bit startling -- let me get started. i am jeremy howard, the president and chief scientist of kaggl.e >> peter norvig. >> gurjeet singh. >> machine learning has been around for some time.

why is it hot now? tell us about trends. -- machinening learning appeared in the 1950 dos, developing a computer program to learn about checkers, and they learn they did not know enough about checkers. they set out broad parameters about how one would play checkers and get the computer to play against itself. as a result, he found himself with a program that could beat him at checkers. it goes back a long way. i have been involved in machine learning for 20 years. euro then i was doing n marketing. things have not changed a whole heap. back then we were working on large data sets, and we were identifying people come and some

of the same conversations we're having today. was there was maybe three or four people working in the field, and maybe four or five companies with the amount of data and money to invest. today every company, if they are not working with machine learning, is shortly to go out of business because the folks leveraging this technology like google, most particularly, and the really big trend today is that there are now two classes of machine learning technologies which require much less handholding and specific expertise. ensemble, the other is data networks, that can handle nearly every problem, from image recognition to automatic discovery, and those are the two fields which are making a big change right now.

>> i think for the audience you might be familiar with shadow learning. eatwill talk more about b learning. peter, one of the trends i hear a lot about is everywhere, and we know machine learning is everywhere in google. it is almost in the dna of google. the loss about what you see is partly new and what are some of the lessons you're learning, because you have thought -- you are far ahead of the game compared to the rest. >> for us what is new is looking at new sources of information, so we started off saying our domain is the web and the text is on the web and there is a technique for dealing with texts. now there is so many sources of information. we are dealing with images and video now, we are dealing with maps, we are dealing with links

between people, relationships of who is sending e-mails to whom, any messages of various kinds, and what does that tell us about it. we are also trying to do more with what we have. ofstarted out, that type ross is a we would do, is what are similar words? if a person types a couple words into the search process, what are the citizens to that that we should add into that query to happen find something they did not know they were interested in? now we are doing it more with like if you ask a question, do we know a fact, and some of these facts we can find in places like a pdf and will they database of them and retrieve them when we think you are asking about them. others we find on webpages and we piece together, we find them in tables on webpages. part of the webpage, it looks like this is a column editor and

a row, any entry here is a fact. that's store that way. we find it in text here, in the sentence cannot at this company was acquired i this company by a certain amount of money. or types of information trying to integrate them altogether. now we have the problem of what happens when you have conflict, and there has been two approaches to that. one is to upfront say we will only accept stuff that we know guardianso we have a entry into the database, saying that we will only allow it well- curated information in, and what people process will not be easy. a good come out, but answer of the question-answering system that goes with that approach. afterward,re it out but we have to have techniques to say not only can we retrieve, which was what we did in the

house, but can we evaluate? what is true come up what can you trust, and what can you not trust? that is where a lot of the action is now. >> that you and a system is curious because as some of you know watson had built the entire system built on q and a and hypothesis. the problem that we see is the system as you are forced to give a single answer, and usually there are multiple answers or question. it is very gray, and you cannot expect one answer. good data -- better you go with this type of a model. i thought it was a great model, the core search and discovery. approach -- am very impressed with the approach. can you talk about the approach and the changing and 19 new ways of doing things. >> absolutely. background is we started out from stanford many years ago.

there wasallenge was a change in the way things were doing things. large complexw data sets. they figure people who were making a large, the best outlooks data sets were not the people to analyze it. we have this problem that they were investing hundreds of millions of dollars in the creation of data, and next to nothing in understanding of data. their challenge to us, way back startwas how could you with an automatically built model on large complex data, and at stanford that is what we attempted to do. -- and an old batch of we said we are going to be -- we ant to be able to build machine learning model about data and then we want to compare them. it is not an easy problem. we worked on it for quite a while, and that is what we do at our company now.

the fundamental idea is every company in the world needs more data scientists, and there are not enough of them. there are a few ways to go about it. you can hire people somehow come external people, to be data scientists for you, you can train scientists and make them more effective at the jump, or you can get massive numbers of people who know something about data and make them into data's. that is what we are about. we are about empowering people to be able to do more, because it will actually quite a bit of what people manually would have done otherwise. that is our approach to the problem. instead of manually writing code, query some of you use computation. the competition gets cheaper over time and we will be able to do more. >> how do you see the adoption code? we have lived in a world of

databases forever. hadley emerge out of this database -- how do we either of the database world? >> interesting, because it is a very polar world. when we go to our customers were two executives, we get thrown out of offices and have been quite a bit or it is the case that people understand that the onus of coming up with hypotheses from it should not be realized solely on human beings come and that machines should be -- withcome up with-out hypotheses about it. it is interesting for a company like us. we're just getting started. we are only 16 people. there is a small people best small set of people who love us. eventually all the world will us. it is very interesting. we have more than enough demand in the market right now that we are very happy with the growth that we are seeing. setit is a different mind

to say i am going to discover something from my data. >> absolutely. jeremy, your enterprise is interesting. their cycles are long. out to you see the adoption happening with machine learning? >> there are leaders in that space. like ge is the leader of the leaders. they have invested very heavily, building a large center. is --ally the ceo there has recognized that a company like that either leverages technologies or they get outplayed by the players if they have not even come across them yet. kodak, it is too late by the time you realize that a company with 11 people has just eaten your lunch and you're out

of business. >> the question whether codec really knew what business it was in. what does it in the business of making memories or in the business of making -- >> clear is that, but there is a sophisticated understanding of what modern technology can do to change the market place, and in , there is noata business. we worked in every industry and i have never come across an enterprise who when i said to them this is my first question, tell me your top five strategic priorities, and at least we have seen things where a data-ribbon bridge could be transformation. that change is cultural. they will get to the top of enterprises because they are the main experts and have been for decades, and by the get -- by the time they have their they have surrounded with themselves with other experts they can trust. they can say, we are investing in machine learning, you have

approaches for decision-making. that is tough. even tougher is we are going to transform the organizational culture so that we put those data-driven decisions at head of our inter-street x ortiz. there -- -- our expertise. those are the things that was a break countries like ge, or they are saying this is what we're are doing, from companies where the senior executive team is not equally passionate about taking this approach. >> talk about google brain chemicals i think it is fascinating that google can come up with -- mimic the mental space, basically, and here is machineing beap learning. tell us about the background and how you see it going, where you see going. long beforeround is

google -- and it is called google brain -- i think of it more as mathematics vendor all at the about but there is two metaphors that drive it. to do machine learning in a way that builds up intermediate representations. if machine learning is mapping .rom inputs to outputs in high school, you drew these x-ywife pots and -- plots and do a mathematical equation, and that is what machine learning is. you have the x input, the y output, what is the extradition? what machine learning is we are not going to do a shallow position, but we will make up some stuff in between. we're going to try to learn this lack box to test this black box

to take with them through steps and we are not going to do that with direct mapping, which will be hard to but we will invent the intermediate pieces. this has been a goal for decades in machine learning. it started to come in to play in 1996 when a group that urkel he that can -- at berkeley that combine computer scientist and neurologists who were studying the visual cortex that we were are going to do image recognition to go from some axel to say that is a dog or cat or whatever, and we are going to do that by building up the intermediate steps, but saying what are the components of the picture rather than going from pixel to dog, we will go from pixel to some little line segments to the line segments coming together to some bigger thing. , we of having one function are coming up with multiple functions, but the hard part is we do not know what the in puts rfid intermediate level

functions. one way i like to think of it is if there is a problem of image recognition of but we can think of it as image duplication, to imagen we present this that i give it in a more compact way am a with a representation to its function. is say,n analogy i give you were a jigsaw puzzle manufacturer and have a bunch of jigsaw puzzles in boxes. and now one of your engineer says let's allow some of these to upload a picture and we will make that excel puzzle for them. but your guy in inventory says, it is too much trouble to create custom jigsaw puzzle pieces for every public -- should they get uploaded. let's have a set number of pieces out and when somebody put in an order, we will act them out of inventory and put them into a box. the question is, what are the right pieces that you need? to have someood

eyes, because a lot of pictures have eyes in them, and maybe some noses of skin colors. then a lot of blue for the sky in the background, the green for the grass. they do not have to get them exactly right, but approximately right. what machine learning says is th most the thousand important puzzle pieces so that when anybody upload a picture andng i can pick this up, understanding a human, you come up with ideas like eyes and noses and so on. up until 1996, nobody had been able to do that, nobody got a computer to advance those exit pieces. then they got it to invent lies. they said pictures are made out of lies. people kept working on it. in 2006, a group finally came up with eyes and noses where they did this and they look at the intermediate level and said, aha, it invented the notion of an eye and the nose. did angroup in 2009

advance on that, and now we've got these multiple levels of here is a line, here are the eyes and noses, here are the faces, and here they are all combined together. we're able to do that by saying, writing a mathematical formula that says tell me the best pieces that if you assemble them will give me this picture. so that has worked out. we were able to assemble most of the players that laid key parts there -- that played key parts there at google, saying we got all the data in the computers, and we have some of our top not computerntists, but architecture scientists, saying how can we put this together to fulfill the computer that will be more efficient than in the past? we were able to build models that were about a hundred times bigger than what had been done in the past and have shown great results. we were able -- the standard

databases of image recognition tasks, and we were able to cut the error rate in half on that. and speech recognition, speech recognition is another field that has a 30-year history. you are getting to the point where if you make a change and if you eke out a 1% improvement, it is a high fives all around. we put in deep learning, and decades of research them all in one technology. field, and oneng of the important areas now. >> you mentioned you are interested in exponential technology. just say why this is important, why human might be thinking that they found eyes and noses, it is ok, put people before that people do that by hand, the algorithms. the thing about what is just

described is exponential and note your, where we have come to the point now where the progress i've seen in deep learning in the last year eclipses the previous 30 years, and the next year will eclipse that time. we are at the point where we are finding stuff itself -- >> so you have gained momentum snively. >> and the computer creates momentum. now we are dealing with the big issue,hem a architecture and a perfect example of google, they have graphics processing units that can do these things millions at a time. we have hit the exponential point in deep learning, and that is why everybody will see it in the next two or three years. >> there's a lot of room for improvement there any architecture. we have had a race internally between the basic google architecture that was existing in our eta centers -- in our data centers and the specialized gdu's. dataoogle eta centers --

centers won. for us it is a way to see in the future, in a way that xerox in $250,000 work a station on every researcher's desk because in a few decades they knew that would be a thousand dollar workstation. we are killing this problem with a data center, where we know at this -- at some point it will be the gpu hardware that will be much more efficient and lower cost. it isinteresting point is such an exciting field, but it is inaccessible to most people .utside of places like google actually, maybe there are more places in silicon valley in which the excess to these technologies -- in which you have access to these technologies. you need a lot of competitions. it requires a bunch of computers

at google to make an advance. it requires the best minds in the world, all working together, and in working at google, there is jeffrey now. these are resources that are not ordinarily accessible to most of the companies of the world. that is something that we see, even for other machine learning areas, not only for deep learning, but there are technologies which can actually help many companies in the world, but they are simply not accessible. those companies in the world are not able to hire the expertise and do not have the expertise in the infrastructure itself. that is something we are going to bridge somehow. we want to be able to make these tings accessible to the wider -- these things accessible more widely available. >> the work that you did in -- you're making it accessible to the world by patching up the software. accessibility of the world lost this brand in data scientists,

we're making it available to everybody who comes to our website and talks to them, and say it is creating a service that anybody can use. deep technology -- to build them, but they are becoming increasingly accessible to enterprises, individuals come and so forth. >> i agree. i think our two companies take a complementary approach to the problem. our mission is the same, right, to make it accessible to a larger number of people. security,alk about because i want to say national security and machine learning. this is an interesting area. especially because machine learning has grown so much. what do you guys see is playing there, what are the gaps, what are the implications? >> ok. look, security is an area in

which is in an adversarial situation and the adversary is changing on you very, critical view. systemsast, most of the available in industry for dealing with these robins were -- these problems were rule based, coming up with a bunch of roles, and when something happens, you say, most of my rules that do not do this job, and maybe we will start this and we are going to do. it works for well enough. it is the case that with technologies getting accessible to your adversaries, now it is a gpu's aretpu's are -- available, and they are available for cracking passwords, and so it is not good enough anymore to just say these and i am going

to do a rethink by these rules and we will see what happens. if you need a feedback loop, you constantly keep rebuilding work, updating your selves, and you have to be on your toes. aboutise -- and i think peter, you were saying, you have seen this problem at google, and is a complementary situation in that sense. turning an adversary is a problem you're seeing in the different areas. >> the question is, can we take machine learning and making more dynamic? [indiscernible] the world is changing fast. >> you can make rules by yourself -- you cannot make rules by yourself anymore. that is not visible. >> and revelation came out this morning that some of you saw and "the guardian," and it turns intersection, that

the nsa has machine learning applications that they can apply mesthe feed that copme straight off the cables. they are using an anomaly detection which is identified in a stream of data something that looks unusual, but it has the same virtue of describing, in that now the nsa does not have to type in a specific query, but tell me about something that is unusual. >> absolutely, it is not feasible anymore. >> let's talk about the gaps. where do you see that gaps in machine learning and the indications of that? >> one of the things we're talking about now is moving these traditional outdoor rhythms from that to stream- oriented. the traditional algorithms say you give me a set of data, input output, and i learned a model, and i you make a query for that, make a new input, and i were but it the best output. they were written assuming that

data was a finite resource that you were given once. that is not really true anymore. in at is data is coming in continual stream and you have to update the model over time, and you got to decide how much to trust what i learned yesterday, versus how much do i trust what is happening today, and is it different, and should i then forget what i knew before, should i remember it because next year at this time may be the same as last year at this time, and so all these models in ahave now been rebuilt stream-oriented fashion. >> is there a concept of learning, certain types of learning stay the same, and then other types of learning in motion? aboutht now it is largely count, the gaps. >> who touched the pond this a

little bit. the performance of the 100,000 uses of a certain modeling competition to make sure who is best at making accurate reductions. i know that guys at the top of that community. the top 10 do stuff that the top hundred cannot dream of. stuff 100ousand do times faster than the top 10,000 can. ofs is a massive curve flexibility. there is very little at the moment around how do we train the next generation of those top tens, how do we identify and then. i was telling peter beforehand about competitions, nearly all of the winners from our past year learned about machine learning by watching youtube

lectures, by literally the best place to be trained right now because they are online courses. it is not clear to me how this talent cap is going to be filled and how we are going to build and identify the billion -- the brilliant data scientists. possible tot is augment our telik gap with the software to a very large degree, and if it is the case that the top end of people are constantly winning in the competitions, then it must be the case that they do not reinvent the wheel for every competition. they must have built parts of acrosst that they reuse different composition. possible toat is build software, they can do heavy lifting for you, that you do not have to be inventive. i think the other point is one of the biggest gaps in industry

and i as i was saying early, is culture. either you are going to trust data and make decisions or you're going to be obsolete. you have to pick one. >> i would agree with that. there's this notion of the hippo, the highest paid person's opinion, where everybody resents their point of opinion, and the guy who gets paid the most gets to make a final call, and you want to decide is that the type of organization you are, or are you driven by one of the data streams? along with that is the notion of openness, that a lot of companies, everyone has been trained to say it is my data, and i get to keep it, so that my division will keep performing better than anybody else's division. that you have to change that to this notion of this data has some value to you, but it also has value to everybody else in the company. >> especially now like the

government's example is that they conduct a hundred 50,000 data sets. you have to be sharing stuff internally. >> the final piece is embracing this notion of uncertainty, to say that there is no one right answer. it is not that your programs are going to write something absolutely correct. it is that you can make predictions, and some of them are going to be wrong. we have seen the spreadsheets of quarterly growth and so on and they all have precise numbers and they always go up and to direct them and they are always wrong. why do we keep doing that? why can't we do something where we say here is the production, but here's a range, and your addictions,her here's the low one, the high one, there's the black swan, the one in a thousand event, and if it happens it is really, really important. we have to consider that as well as just entering our single point. >> is funny that has not taken

back more. 20 years we were using an excel and a flat crystal ball and everybody was expected to do an analysis of the model. if anything, it has disappeared rather than taken off. >> on -- quantum computers are here. leave the audience with one key take away as a big opportunity. big technology opportunity is taking advantage [indiscernible] in a particular way, which is [indiscernible] well let you through as much data as you like. degree ofres a high creativity, soothing of all the things you can throw at it, and you got to get out of the mindset of filtering, which is that will not work, that it will not work, and that is the top approach that art and the scientists use. quantumkeptical of

computing and he will be great if works out, but is not here today, and they're so much that you can do today with the data and computing power that you and that to focus on that, as new technologies come in, use them and they are available. >> from my point of view i think -- i want to reinforce we should law that computers are going to get cheaper and faster. we should let them do more. we should not allow them to do all the work. moreters can do a fair bit . [indiscernible] >> excellent. under the audience has a lot of questions. we will open it up for questions.

>> there was a lot of talk about ap.ling the talent gua what do you think -- this is directed at jeremy howard -- everybody has mentioned some of scientists, solid engineering, machi learning. what you thing about 18-building approach in which you can leverage those specialties into individual people that are specialists in that area and how would you find a way to allow them to communicate in such a way in which they are all on the same page, almost as if they were close to one individual? >> for those who have not seen a

talk, what he is referring to is we analyzed the top talent to find their common characteristics, and we found there was for -- tenacity, creativity, and good encoding. you cannot take a person with each of those four things and expect them to be that person. what you can do is bring expert and amain great data scientists and expect them to do great things together. to main expert's role is come through the conversation with a deep understanding of two things. one of the specific actions that this organizations can change question mark we can change where we put our next door. we can't change the color of our next wall. what are the strategically important things for this company, so we know what drives profit. scientists whota

says whether it be with using software like i asked you to identify interesting patents or using a supervised approach like deep learning to identify drivers of outcomes, the conversation is, ok, i am trying to achieve this and these are to changes that i get it, and the scientists says, the approach looks like this, do you have that data? it combined the software and the main action work and the data scientists to gather in that conversation. >> there is a couple of questions. >> [indiscernible] of the competition for machine learning. next three to five years, do you think you will have this service available in the cloud, or will it all be machine learning as a service?

>> we already have a cloud-based service. friendly, the other half does not use it because it can change. they would love to do it accepted data [indiscernible] or it [indiscernible] that there areng enterprises who are much more t. and othersh i. who are not. from my perspective, what really matters is the and business users. it is their problem that they need to get solved. if it is the case that you are in an organization where the ideas are supported and you have legal constraints, it should not you are notn, and as advanced or you can change on your data, then you go to the cloud-based model. from our perspective, we are happy to work in either

situation, but clearly operating in the cloud and fire and meant to seek -- environment is you a system in which the economies will help you. >> the problem is as the data comes in, does machine learning as a service create a model of the data. what if the model changes? depends on you can re- create the model every so often. you do not have to rely on the same model, and you should not anyway. whichre are algorithms are specifically designed to understand that component of data over time. the other thing and it is surprising how little that happens. bottles describe kind of a connection between inputs and outputs in terms of causality. that is what they do behind the scenes. my experience is once you build a model, the input predators change a lot about but the

underlying way they combined to create outcomes, other than in an adversarial situation, does not change that often. inyou have an application which you have new data and your model is not fit for that anymore. >> and if your model has a temporal, spatial hundred, you add that to the data, then your data will change. ok. next question. >> to follow up on the question, one of the problems of taking this approach in machine learning is you're looking at correlations and you do not get a lot of insight into causality, and you do not know that under the correlations stopping true and your model breaks and you cannot predict when it is going to break. in fact we solve that by developing theories and abstractions read what can machine learning do to help us get toward the theory generation do you think that is impossible or du think we are making that kind of progressz >> there are a few ways to

address the problem. the first way to think about it causality is built into the data. a good example of this is in certain situations, when you're talking about drug discovery, you have data about dna, about gene expression and so on. there aremetimes that changes in your genetics and genetic expression, and there is causality built into that situation. you can explore that. there are situations in which causality is not put into the data, and then if you are going to make decision based on data, you only have correlations to rely on, you cannot do much else. >> unless you build some sort of reasoning -- >> reasoning is a funny concept. it works for some situation if you have a bunch of text, if you can read some books, and if

there is text about a problem, you can come up with if you do not have any prior knowledge about it, then, yeah -- >> you can discover causality unless you're allowed to do experiments. you do not know, but if you can do an intervention, then you can discover that. >> this is why one of the big things to me, the problems of the last few years, is data exhaust. it means you only use the data you already have, which means you're not doing experience -- experiments to find causality. >> the active learning is what is the next data i need, and our formalisms are designed to represent causality. the work from the last three years has been concentrating on causality and how drugs at it. >> we can measure the effectiveness on those algorithms.

and thousands of variables, where we know the causality, have been inserted and people have to reverse engineer them, and we will have exact measurements of identifying causality. this is not something we do not know about. >> next question. you should not trust machine learning. we know that in prior times, have been building models and testing them out. thatn make it argument that probably is not a good thing. what are the areas where you would not trust machine learning? have been manyir ideas in machine learning. it is the case that people are not greedy and did some wrong things. we basically packaged up things

some security that would solve -- they should not have done that and they knew they were not allowed there. machine learning, i think it has been successful in finance. they would not have ordinarily been able to do things that they have been able to do. mechanism question of design rather than machine guarding. there is this idea and giving the mechanisms we had, people try to optimize them, whether they had to use regime learning or intuition, that was immaterial. what was important was the mechanisms we had were such that they could get us into this place i was unstable. if you are clever about setting the rules and having governors, you could say traits can only be

made at a certain velocity, and you have other ways of damping down these wild swings, you would be in a better position. >> it is difficult for a lot of alks to kind of select machine learning vendor or technology, because frankly there are a lot of snake oil salesmen in the industry right now, and i would be terrified of setting my business on a piece of machine learning technology which i either did not understand or have a vigorous weight to test the effectiveness of because, if you think of it as this black box which is going to do next, then you have got this ability to destroy your own business using black magic that you do not understand. that is where i would be careful with machine learning. >> there was a conversation about you have to establish baseline, you have to understand muchsion and law and how is a variance of all these things before you're going to go into it. around frequency and size and

make sure those things are in place. >> questions? >> a lot of the work in corporations [indiscernible] a lot of the work in corporations is focused on human religion ships, which are stored of by definition ever changing and never well find. influenced by many factors that are not in the data. so the use of machine learning from marketing has a relatively roche -- low risk, that to find those core patterns that describe all of human relationships, is that some work that is being done? >> absolutely. in fact, we were talking about this beforehand, but the exact areas of the more sophisticated out gordon's work this, is worthy under rules that govern the things that we do not have a

strong intuition for. understanding about how human favor changes is complex and is the kind of thing that if you provide the inputs and the outputs, modern regime learning lessons are not good at defining the incredible complexity of underlying causation systems going on inside. that is what actually machine learning for marketing is perhaps the hardest area, like investment, from everything from models doertising to so forth is an area where machine learning has been very successful. of you is an example give people a choice of which article of clothing do like the best, and you show them three, and they say one of the most predictive variables is the one on the right. nobody says i chose the one on the right because it was the one on the right. they said i like the color or the texture or so on. >> and outgrow them will find that. >> the algorithm will find that

very fast. >> ok. ?uestion question >> what kind of problems do think machine learning is beyond the realm of machine learning? i can tell you one. natural language. just fluent, natural language. if you look at reviews and blogs and things like that, that people speak in grammar that is not very structured them a prayer difficult form a machine to learn that, because you do not understand the words. for example, birds fly. the penguin flies. the penguin is a bird. we know these things inherently, but machines do not. they have to learn context. you have to learn grammar, lots of things behind singh, and then you have to bring in machine learning and nsp together. >> it is amazing what we can do. we ran a project master where we took 3000 handwritten school

essays which had been graded by two teachers and try to find out if anybody could come up with an outgrowth of which would great them the same way the teachers did. the best algorithm was more consistent with the teachers than the teachers were with each other. we already have our core rhythms that can read student essays with all that québec city and -- that complexity and right how good and as a they are other than the teachers can. >> another example would be the machine translation. it used to be you build a system by hiring teams of linguist and having them write all these roles. that we do it by gathering examples. here's a document in english. here's one in french. use that as an example. a line the peoples of -- the pieces of the english to french, and you have a model. that works great, and in many places it works as well as the human translator. it is not quite there yet. the places where it falls down is where you do not have examples very few have not seen

this text before, and mostly where it interacts with the real world. i i give you a sentence like dropped the brick on the glass itle, it broke, we know that refers to glass table, and i now have to translate it to an language where table is macgillis can and brick is feminine, i can make the right choice, but that is a fact about physics, not a fact about language. of getting all those types facts about how the world works, integrated into the language, that is where we are going to have trouble. eventually i think we are going to get those figured out. the contradiction in data, we will be there. >> you can find these places that machine learning is not good enough when you look at places where they are using humans. problemg i like is solving. we do not have algorithm's that do a good job with problem

solving. game, andcomputer that has resulted in papers of new proteins that have been devised because better art that -- because humans are better than computers. was one successful story about that, and that is great, but isn't it true that the number of human beings attempted to solve the problem was -- >> it is taking advantage of something that humans are good at. we have some connections in our really great at seeing things. on the other hand, the result of that is now they said that to the algorithm researchers and our rhythms are approving. software, our brains are fine- tuned, in recognizing spaces and images. we're still better than computers are now. >> we will get there.

>> our brains are good at two- dimensional and three- dimensional data. questions? >> this is mostly for peter. one way to think of google is as a machine learning company, learning each of us to be able to deliver things to us more easily and effectively. can you share with us your view of your predictions about where the conceptual model for modeling us is going, how integrated it will be, how sophisticated it will become a and how do you guys think about that internally or the next sort of handful of years? >> the first thing to say is there's two parts of it. each of us and all of us. i think more of it is in all of i do not really want a model of you, i want a model of everybody, and something that everybody knows. it varies among companies.

model ofally wants a you. they want to know what did you buy last time, so they can sell you something like that the next time, because you're going to be consistent in your buying habits. google wants more of a model of everybody rather than a model view because most of the time when you come to google, it is not because of something you already know, but it is because something you do not know, so you pushing your boundary to where we do not know anything about you, so it is better to go on what we know about everybody than you. we are tying to build both of what, a better model of each individual is, but also combining everybody's knowledge together. and that goes for the words you take, youaths you typed something to do, it is not quite right, we say, did you mean, we give a suggestion, and we're able to do that not

because we know about a few, but because we know about other people, and because other -- in the past, other people typed the same things. now we will take a directory to the path that they eventually ended up at. question.e >> [indiscernible] trust anding about risk. in the case of google, and correct me if i'm wrong, you're supplying information and people trust that they will make their own decision about what to do with it. in the case of machine learning algorithms that invest your retirement funds or chooses a something that is very high risk. the decision-making side is still machine-based. how do you develop trust in those kinds of things, where the experiment is not possible? >> i guess i had that situation

at my last set up which was doing in georgia pricing, and insurance pricing is the most report interest buddies can make , and we wanted to get to the point where our burdens could make the decision, but we had to trust. this is where you have to basically draw pictures. you need to basically show what the model is saying in a way that humans can understand it, and then you visit three phases. the first phase is sure the humans the pictures and let the humans use their expert judgment to decide what to do, supported by that predict the model and visualization. the second set is to get the computer some limited ability to make their own changes in situations in which they are very confident, it is a well understood area, and it would be something that is time sensitive, and in the third step would be on the trust that you build up through the stages would lead to slowly decrease

the number of -- that peter is describing, we give it more ability to change things, and you are moving -- you are removing the constraints. >> in most situations eventually there is a human being in the loop. if an outgrowth of -- if and how your call -- it is at the end of the day. similarly, if you're making an investment decision, there are suggestions that can be made, but is a case you have to make your own decision, and to the extent you can make the user possible like easier, then that is the way to go. >> [indiscernible] >> you cannot protect yourself from a malicious data scientist when there are only a few thousand of them around. >> there are a few of them

around. [applause] that is it for the questions. we will be around. you can ask us individually. this [applause] the supreme court announced it will hear two cases regarding the affordable care act and whether businesses can use religious objections to risk the evercore to cover birth control for employees. the assistant -- "the nuclear for profit curve operations could not rely on opt outs objections to of compliance with the law. the white house in a statement after the court + announcement said the administration has already acted to ensure no church or similar religious institution will be forced to revive contraception coverage and has made a commonsense accommodation for nonprofit

religious organizations that object to contraception on religious grounds. the speaker of the house in a statement, he said the administration must and eight is an attack on ridges written, and i'm hopeful it will be reversed by the court. the court will hear this case is, an argument in march. later on c-span, or from that unh decision -- more from our q and a series. coming up at 7:15 eastern, and we will have the president coming up in a few minutes at 3:15. the director with charles bolden, talking about nasa. here's a look at that program this evening. >> many years from now, we will still be operating to my hope, on the international space station as our toehold to the universe. i would've to say 10 years from now humans will land on mars am a but that is not the course on which we are embarked. the present challenge us to put

humans on mars or in the martian environment in the 2030's. that is outside the 10-year window. we should be there. we should have been there now. but there may be humans on the moon inside that 10-year window if massachusetts s oh in fostering the development of commercial space and entrepreneurial space to the extent we are doing up. there are some private enterprises who believe they can put humans on the moon. have formal agreements with some of them to provide engineering expertise and other assistance in a nonreimbursable basis. it is conceivable. my belief is that it probably is a little bit outside the 10-year time frame. different.0's were [laughter]

of things a lot happening with race, the breakdown, of the structure of society. i was only out of the seminary and in new england. but there were no rules. ms. -- things were falling apart. and without structure, it is very difficult to navigate. i was extremely fortunate to be at holy cross. i was extremely fortunate to still have had a residual mob of the way i was raised i -- and the structure that the nuns had given me, the structure that the seminary had given me. i was also extremely fortunate because i had already and in predominantly white schools. i was the only black in my high school in savanna. the transition to a school with a veryw blacks in difficult set of circumstances, academically and otherwise, i had sort of a jump start. i was ahead of the game. i had something.

to continue do well even though it was very, very difficult. >> thanksgiving, air from -- hear from clarence thomas followed by linda kagan -- elena kagan. also this weekend, four days a book tv on c-span2, including evora sulman on the life and art of norman rockwell, -- including deborah solomon on the life and art of norman rockwell. >> president obama coming up in about 15 minutes in california talking about jobs and economy. until then it, a conversation from this morning's "washington journal" on poverty in the u.s..

host: welcome back. one in six americans are in poverty. what does that mean and who is the hardest hit? guest: the average listener is thinking significant material deprivations. the average newscaster will flash to a homeless family, but the reality is roughly 46 million americans who are labeled by the government as poor, only one percent of them are homeless at any given point in time. close to 50% of all poor families actually own their own homes, typically a three-bedroom house with 1.5 bath. the typical poor family,

according to census numbers, lives in a perfectly large house or apartment that is in good repair. about 80% have air conditioning. two thirds have cable or satellite tv. half of them have imputed. half of them have internet access. about a third of them have widescreen hdtv. if you ask them during the course of the year, were your children hundred very -- hungry for a single day during the course of the year. four out of five people will say no. four out of five adults will say they were not hungry for a single moment during the day. it does not mean these families are not struggling. they have to work very hard to make these ends meet. the normal picture of material the privation, living in a home that is cold because you cannot eat it or has a hole in the roof

or a family -- homeless family in the back of a van, that has nothing to do with poverty as the government explains it. host: how does the government defined poverty? guest: it is to say anyone below a certain income level that was arrived at by looking at the cost of the basic food budget, and then adding a little more for shelter, clothes, and other necessity. for a four person family, the poverty line is around $22,000 per year. a poor person family -- four person family is expected to have that much money to meet the basic needs. more than that, they are not poor. less than that, they are considered poor. host: one to bring you into the conversation. for democrats the number is (202) 737-0001 for republicans (202) 737-0002 for independents (202) 628-0205 when we think about poverty, we

think about inner cities. where is poverty concentrated in the united states? guest: it is more concentrated in the inner-city. while we have a picture about poverty and that this condition of extreme deprivation, and i do not want to suggest that those families and extreme it deprivation do not exist at all, that would be ridiculous, but to say they are one in six families in the united states is just wrong. they are a much smaller portion. one thing when the government goes to define poverty is is as a society we spent close to and $20 million on a program for low income americans. roughly one in three americans receive benefits from the anti- poverty system. the average cost is around 9000

dollars per recipient for 100 million ethiopians. -- recipients. when the government goes to count how much income household has, virtually none of that massive amounts of assistance, larger than the entire economy, none of that is counted as income. it is a very misleading depiction i believe. host: if i understand correctly, the obama administration rolled out a new way of measuring poverty. how is that different? guest: i think it is an improvement, and what it does is if you include the non-cash benefits for the poor gets, you get a slightly different picture than you do if you just count cash income.

we have moved very far away from providing people with what we used to call welfare or cash income towards providing services and support for making any assistance they get conditional on work. most of the assistance we get now, except for the disabled and a few other groups is conditioned out on work. i think that it's a really important for people to understand. so i would say poverty is in the eye of the beholder. i would say robert is suggesting there are not that many truly poor people in the united states. we are an affluent country after all. when you think about, could you live on $22,000 per year and have to support a family of four?

you have to define -- decide for yourself if it is a reasonable standard or not. host: $22,000 spent a lot differently here in washington than other places. guest: there is some attempt to build in geographic variation in the new experimental measure that the admin -- administration is using now. it has not been traditionally the case we have done that. guest: traditionally is black across the board. i would argue the new measure is a better measure, because it is a race with a moving goal line. as soon as you get close to crossing the finish line, they move it back so you can never cross it. you can define poverty income level relative to the average standard of society. a pretty good way to ensure the poor is always with you.

if we were to get a magic wand and ways that at everyone's income and it doubled this evening thomas that would be a good thing, right cap go there is no reduction in poverty at all because the poverty income levels would also double. i think it is extraordinarily misleading and does not really get us closer to the truth. we want to know how many families are hungry. how many families are not nourished. how many live in housing that is really unhealthy. this does not answer that. we also need an honest count to understand how much people are really receiving government aid. if week spent close to $1 trillion on the year on

assisting poor all, cash, food, and housing and convert that to cash, close to six times the amount needed to raise every families income above the poverty level, even if you just take the cash, food, and housing aid, more than twice what is needed to wipe out poverty in the u.s. we spend a lot more than people realize, and effectively the government does not give the government credit for that. -- taxpayer credit for all of that. host: we want to bring some of our viewers into the conversation now. first up, passed on the line for democrats. -- pat on the line for democrats. caller: i was listening to your conversation about 23,000 dollars per year being poverty guidelines. i wonder where you feel where individuals say their social security check or small retirement check is $600 per

month as opposed to the amount you are thinking. where does that leave them? people struggle every day, and yet there are many people that do not have housing, do not have medical and the other things that you just mentioned. your conversation sounded very rosy in cheerful, but that is not the way it really is. guest: well, i think there are some people that do not have medical but we do spend close to half $1 trillion for year getting medical care to low income americans, and that does not include medicare so the elderly generally have medicare, and i am not saying there is no one out there who is not struggling.

in fact, i think most people are struggling day to day to make ends meet, but i am saying if you ask poor old were you hungry of any point during the year, four out of the five of them will say hungry -- they know, we were not hungry. >> if you asked them did you have a medical need you could not get attended to, almost all of them would say they do. most people have satellite, cable television. so the picture is much different than you ordinarily see on the network news. not to say there are no families that really do face deprivation, but a much smaller number than the 46 million we usually hear as being in poverty. guest: i think we have to recognize one of the reasons people are poor is because they have lost a job or cannot find one. it might have -- they might have television or an air- conditioning unit or what have

you, because you do not move every month to fit what income is coming into the household. so a lot of what this is about this huge insecurity, not knowing where the next job will come from, how you are going to get one and if you're going to be able to keep it. particularly during the time when unemployment rates are very high, i think we really have to focus on the fact that a lot of people are jobless, who would like to work, and they cannot find a full-time job, or a job that would enable them to support their family. guest: i would agree. the lack of jobs or insecurity is a bigger issue. it does not do you much good if you have internet access, a computer, a television, and air- conditioning when you are

worried about if you can keep those things. host: eight, ohio. j, on our line for democrats. -- julia on our line for democrats. caller: good morning. i am listening with disdain. for robert to say one in six americans are going without, he is completely out of touch. we just went through one of the greatest recessions since the great depression. we have one of the highest unemployment numbers we have seen in a long time, and right now, food bank shelves are empty because people are not getting enough money through snap to get their children said, their families said --fed, their families fed. i work with these people. robert needs to get in touch

with a few of them. i am talking to people that have lost their jobs and have to pick between food and heating the house, and inhaler for their daughter, or putting food on the table, or paying the bills to make sure their lights stay on. it is a real problem in the united states. making additional cuts and allowing the snap program to expire was one of the worst things you could do with -- for these people, especially so close to the holiday season. look at walmart. they are having food drives for their own employees that can not afford to put food on the table. walmart sells food. their employees do not have enough money to buy the food. every is a problem with the working poor. part of the solution is having a

minimum wage inflation and does not stagnate for multiple years. we have to help americans and not make them feel criminalized because they are not making enough money. guest: well, it is simple for advocacy individuals such as that to say i have seen this, i have seen that. i have seen things, too, can information i am getting comes directly from the agriculture department, which runs these food programs, and they tell us when they are asking for people if they had enough food to eat, were you ever hungry, and 96% of them say our children were never hungry even for a single day, and 80% of adults were not hungry for a single day. i feel sorry for the 20% that were hungry, but to shape a policy that meets real needs, we have to be honest about what is out there. i am not making these numbers

up. they come from the agency that runs these food programs and spend a lot of money feeding poor people. we are spending close to $9,000 for each person in the bottom third of the population. if we could spend close to $1 trillion a year on these individuals and still have massive numbers of people that do not have enough food to eat on a daily basis, that is a huge indictment of the welfare state beyond anything i could say. it is simply not true. we spent a lot of money, the money goes to assist the poor people, and we do a reasonable job with that, not a perfect job. i disagree that in the long these programs promote work. there are over 80 different programs for poor people.

only two of them have work requirements. we have to work toward a system that helps to give aid and requires more of the poor to help themselves in the long term. host: an article in "the new york times" writes about how the reduction in snap benefits has impacted more than 47 million people, the largest wholesale cut in the program since the act was first passed and it touches about one in seven americans. isabel sawhill, can you respond? guest: robert is calling on data that does exist, but he is only giving you part of the picture because a lot of the data shows that people are very food insecure. they might not be going to bed hungry, but they might be cutting back on what the adults eat themselves to keep food in the mouths of the children. it is a much more mixed picture that i think you are getting from him. on food stamps, is one of our largest anti-poverty programs. it is the basic program that we

used to put out the minimum floor under people's standard of living, and it was cut back on november 1. it had been bumped up as a result of the anti-recession program that was enacted to fight unemployment. it has now been brought back to where it was before, and i am not so concerned about these recent cuts that occurred in november, as i am about what is on the congressional agenda right now, which is a farm bill that would further reduce the safety net in america, and particularly the food stamp program. there is a huge difference between the senate and the house.

the house bill is very restrictive. they will be a debate and i think there should be a debate about work requirements. in other words, do we require people to work in return for getting food stamps, especially if they are able-bodied and do not have children. the issue there is are there jobs available, training programs available for those that cannot find jobs? we need to have that debate. until we have jobs and training programs, we should be very careful about imposing work requirements on families that do not have any other income. host: roy in kentucky on our line for republicans. caller: good morning, ma'am, good morning, ma'am, good morning, sir. i have been to about four food banks. i go to different states, looking for jobs. there ain't no jobs in this country. obama care is so expensive i cannot hit obamacare, -- get

obamacare, and i want to know, am i going to pay a fine, or wind up in jail? what can you all say about obama care? i think we should vote to impeach the president of the united states, impeach him, get some new people in the white house and some new people in the congress. this is terrible. i have to hit four food banks because you all want to cut food stamps. guest: while i sympathize and recognize the huge problem of the media time about a lack of jobs, the cut in the food stamps constitutes a cut of about one half of 1% of the cuts, and i would be willing to exchange those for changes in the way the food stamp program and the

welfare program works. 96% of americans believe that those that receive aid from the government should be willing to work or look for work. 80 programs assist poor people. only two of them require any type of work. i know it sounds odd to be talking about requiring the work in a climate where there is a lack of jobs, but to put a work requirement in place takes many years to implement. if we start today, we could be talking 4, 5 years from now, for example, in the food stamp program, taking the able-bodied adults, with no kids to support, requiring them to come to the

office one day a week. when we did that in the 1990's, what we saw was a dramatic reduction in the number of people that needed aid, a huge surge in employment, and a reduction in poverty that was quite substantial. we need to follow that same progress with all of the welfare programs over the long term, not necessarily over the next 12 months. it takes a long time to change the welfare state. it is like a giant oil tanker. it does not turn on a dime. it is time to turn the system around so that it is not a handout, but gives aid to everyone who needs it, but ..expects everything -- something back in the -- in return. guest: i think it is important to make it clear to your viewers here we are talking about.

we are talking about programs for the disabled. are we going to expect them to work? we are talking about programs that serve the elderly, children, that help with aid to education, health care. you do not expect a child to go work because their school is getting extra help because they are a poor child. i am in favor of work requirements for adults that are getting direct assistance that supports their income from the government. that does not mean we can have a work requirement for all of these programs. this is a set of programs that is not welfare. these programs support training, education, health care, social services, problems when children are neglected and abused -- we have to do those things in our society. guest: clearly, we're not talking about making disabled

people were, and when i talk about $1 trillion of aid, about half of that does go to disabled or elderly people who no reasonable person could expect to work, but the other half goes to families that contain someone that is capable of working, and when you are giving cash to those individuals when they are not working, we should be doing something to help them become more self-sufficient, and for the most part we do not do that in the welfare system. host: st. james city, florida. martin. independent. caller: i hear people touching on the real problem, but dancing around it. walmart is the most egregious employer in the country, and we have richard trumka, the most gutless leader in the history of unions, who was never held a

real job in his life. mr. obama was -- was told by his own committee to use small collectives to facilitate national healthcare. that means unions. if mr. richard trumka would approach everyone punching a cash register -- host: do you have a question about poverty? caller: yes, why not addressing that unions are being left by the website -- wayside? the real important issue is the deportation of illegal immigrants that have reduced our workplace economy by two thirds. as a 14-year-old child i beg to do the work that americans do not want to do, and i got paid three times the minimum wage. now, these liberals are asking to raise the minimum wage. you do not need to raise the minimal wage. you have to deport and get rid of the actual, literal slaves

that are being allowed into this country. guest: i agree with the caller that you illegal immigrants do push low skilled american workers out of the labor market and they tend to push wages down. i do not think we should have 7 million illegal immigrants working in the u.s. economy when we have such a huge level of unemployment, legitimate immigrants and us-born citizens. i think jobs belong to u.s. citizens first, not to those that have come here and violated our borders. guest: as an economist that has looked at all of the literature on whether or not a legal immigrants reduce wages for americans, i do not see much evidence that that has happened at all, and virtually all economists that have looked at that have the consensus that it is a tiny effect, if any effect at all, on wage levels. i think that is a myth that we need to be more careful about.

host: a couple of questions from twitter and e-mail. ralph in kalamazoo, michigan is asking should the snap food program be cut, and by how much? isabel sawhill? guest: there are some savings that can be made in the snap program. there are bills in both houses of congress. the bill the senate has enacted or past has some reasonable cuts in it. on the house side, the cuts that are being proposed are very draconian and should not go forward. but, there is no right answer to what is the correct amount to spend on food stamps. it comes back to the debate we're having about whether we think that people with very low incomes and a lot of economic insecurity and difficulty finding work should get some help from the rest of us, and how much help should they get. we will have different views about that, and i think it is good that you are sponsoring

this discussion, but i just want to make sure that all sides of the discussion are brought into play here. guest: i think there are two things that need to be done in the food stamp program. the first is that through a number of bureaucratic tricks, the asset rules in food stamps have been removed, so you literally could be somebody with $1 million in the bank and no job and you can get food stamps. now, the program was not supposed to work that way, and we need to put the asset rules back, because that would save a significant amount of money over time. the second thing is looking to the future, food stamps should not be a one-way handout. food stamps, in the future, should allow able-bodied adults to at least look for work as a result of assistance.

there are 4 million able-bodied adults on the program that do not have kids they need to support, and we give them aid and expect nothing back in return. i would like to see that those individuals at least ought to come down and look for work under supervision at least one day, or half a day a week, and what we expect from previous experience is the of -- the unemployment rate will go up, and the number of people receiving aid will go down. it helps you separate people who truly need assistance from those that need a nudge to get back and start to support themselves. i think i would be a good policy and i think the overwhelming majority of americans would agree with that. guest: if i could just make sure that everyone understands the current law a right now -- you can have three -- three months of food stamps for every 36 months to tide you over if you

lose your job or get sick. that seems reasonable. some states have been able to waive the requirement during the depths of the recession, but it is not as if someone that is able to work and support themselves gets continuous help. guest: 44 states have waived the program. guest: i understand that, but that was a special -- guest: they waved did before. the key is to make the requirement, that i had to draft, and make it real, and make it a good policy. a policy that exists only on paper does not really do anything. host: minnesota. bob on our line for democrats. caller: thank you for taking my call. i think it is the wrong thing to do in a downturn economy to cut back as much as they are talking about cutting back on the food stamp program.

i do believe that people should make an attempt to go to work before they get a handout of any kind, but it is affecting a lot of other people, too, that the working poor and the people that cannot find enough work -- a lot of these jobs are part-time and they will not give them full time. if there are any members of the clergy out there listening, you take food out of the mouths of kids, and you are going down the wrong road. if you want to support people that do that, and call yourself clergy, well, i do not see that cutting back at a time like this on a food stamp program is the right thing to do at all. also, the farm bill -- you know, unless they fix some of the

things like that -- i do not know exactly what is going to happen, but i have heard the price of a gallon of milk could double or triple, and this is the wrong time for stuff like this to be going on. guest: i think that the reality about the welfare system and welfare spending is that anytime a program is ostensibly cut, we have a huge amount of controversy about it, but when those programs are enlarged, nobody actually knows about it, and as a result of that, we have expanded, after adjusting for inflation, assistance for the poor, 20 fold since the beginning of the war on poverty. when we started the war on poverty, if you took all the anti-poverty programs and added them together, $.50 was spent for every dollar that would be

needed to eliminate poverty, and today we are spending close to $5.50 for every dollar that is needed to eliminate poverty. the system is very large. when we talk about the cuts in food stands, this is half of 1% of a system that is growing by close to 33% over the last four or five years, and, again, even those cuts, i think it might be reasonable to postpone those in exchange for fundamental reforms in the food stamp program requiring that able-bodied adults work or at least look for work in the future as a condition for getting aid. i have been promoting that change for 25 years and i always run into a left-wing roadblock in congress. maybe we can finally get it through today. guest: i very much agreed with the last caller that this is a very bad time to be cutting back on what is already a fairly meager safety net in this

country. robert gives you a lot of statistics that makes it sound impressive and makes it sound like we are a lot more generous than we are, and we cannot have a debate here on television about whose numbers are right, and i do not want to be disrespectful of his numbers. he is a good analyst, but he is a different point of view. i believe and work requirements. i believe we can do a better job of delivering help to the poor. i think it is very important to understand that a lot of the growth he is understanding -- talking about is because of the growth in health-care costs that has affected all of us. that is one of the biggest anti- poverty programs we have, health care for the poor, especially poor children. we have two programs that loom large in the numbers here.

one is dedicated, which goes to low income elderly, in nursing homes, and mothers and their children that are poor. another program for children that is for low income children. i do not think anyone in the country wants to deny health care to the poor. you have to be careful when you talk about this data about exactly what you are talking about and what is going on here. i think that we have a much stingier safety net in the united states than any other advanced country that you could look at, and i think we are an affluent society, and although we could do a better job and you do like self-sufficiency, and i am in favor of moving people toward self-sufficiency, there are a lot of people who because of the job market, because they did not get a good education, or because they have various

disabilities, need some help. guest: i would disagree strongly that we have a smaller, stingier safety net than european countries, with all due respect, and i think the point about medical care is right, but if you take all of the medicare -- medical care out, the fact of the matter is we are spending twice as much as would be needed to raise every single person's income above the poverty level. it is therefore a misleading impression about actual living conditions. guest: as you know well, robert, one of the reasons there is money going to people that do not appear to be poor is because if we cut off all assistance to moment somebody reaches the poverty line, there would be no incentive for someone to get out of poverty, take a job, and see their earnings go up.

there are various constraints on how we could do this that i think you would support as much as i would. it is important to design programs in a way that they do not just cut you off the moment you achieve the poverty line. there is no way around that problem. guest: i agree 100%, at the fact of the matter is you are still raising enough money to raise everyone to essentially twice the poverty line, and we are still pretending we have 46 million people that are homeless and do not have food to eat. if that was true, it would be the worst indictment of this system, we spent close to $1 trillion a year but we have tens of millions of people that do not have enough food to eat. there is no way the system can be that any efficient. i find myself in the ironic position of defending the liberal welfare state, but i am saying even the government could

not be that ineffective. we have a large system that rivals anywhere in the world. guest: we will have to disagree about that. guest: the problem is not -- is the money is spent in a way that does not encourage people to help themselves. host: we are talking with robert rector and isabel sawhill. a couple of your tweets -- 83% of food stamp recipients work full-time from don ritchie. from jan, with snap, medicaid, section eight, school lunch, heating assistance, how much would one need to learn for the same benefit? a response for either of those. guest: the average family is getting probably about $20,000 to 25,000 -- we have thousand dollars a year. 83% of people working full-time

is not accurate, but if you are working full-time, the working requirement does not apply to you. we think in the long-term individuals would be benefited by a nudge it would move them more into the labor force and that would be best for the taxpayers as well. guest: we have not talked very much about wages. you could work full-time and your making them wage, you are making $15,000 a year. i would ask people to think about trying to support a family on 15 -- on $15,000 a year. these programs, including food stamps, help to bump that up, and in an important way, and the debt is just that if we did not have these programs at all, we would have 37 -- 30% of the population that are poor instead

of about 15%. the first safety net that i described, it is doing a lot of good. i think we should not be tampering with it too much unless we are willing to see a little more suffering than we have now. guest: i think the important point here is i am not advocating doing away with food stamps or the earned income tax credit. i am saying we need to be honest about how much we are spending, honest about whether or not there are 46 or 50 million people in stark deprivation,

which there are not. guest: i agree with that, by the way. people in this country are not destitute. that is an exaggeration. guest: then, importantly, we need to talk about the future of these programs. i believe to start out, we ought to say that any program that provides assistance to able- bodied adults, those individual should be required in the long- term -- not tomorrow -- in the long-term, they should be required to work, or at least look for a job to get that aid. another huge problem here that both ms sawhill and i agree on is the collapse of marriage. when poverty -- the war on poverty began, 6% of americans were born outside of marriage. now the number is 42%. one of the problems with the welfare state is most of these programs penalize couples when they get married. that is probably not a good idea. we ought to look for a way to put the family back together

again rather than assuming it just goes on forever. host: kelly in rome, georgia, on our line for republicans. caller: yes. no disrespect to ms. sawhill, but i believe a lot of the problems might not have to do exactly with poverty, but maybe with decisions with people who receive all of the assistance that they receive. my husband happens to work. he is a major contractor for one of the largest companies in northwest georgia that builds and maintains section eight housing, and i can tell you that it is very hard for him to accept when he shows up at 8:00 a.m. yet a 20-year-old truck -- in a 20-year-old truck, and you

ride around section eight housing with people who have brand-new nissan's, and they are living in a free housing, free food, free water, heating, air, and you walk into their homes and they have brand-new xboxs, flatscreen tvs, and there is mcdonald's, people and stakes, and they claim they are in poverty. -- t-bone stakes, and they claim they are in poverty, and they are sitting on the couch. they do not get up. there are screaming kids, and they are receiving everything free.

i a lot of them get refunds. host: did you have a question for the guest? guest: -- caller: the point is a lot of it has to do with the decisions they make with their money. when the housing crisis was hard, and we were trying to keep our home, we lowered our bills. we only in out once every six months. -- eat out once every six months. we do not eat mcdonald's. guest: i will agree that there is a lot of bad decision-making, no question about that, so i can understand her frustration over her husband's frustration, but it is difficult because we do not believe in the united states that we should be interfering in people's private lives by saying we will give you a certain

amount of support, but for heaven stakes, -- heaven sakes, do not buy a t-bone steak with it. there will be mistakes made. people will make terrible decisions. there will be people who are actually doing counterproductive things in terms of moving toward self-sufficiency. i do recognize that. guest: i also think there are some more fundamental decisions here. one of the basic rules that a -- about welfare and poverty that both dr. sawhill and i agree on, is if an individual does three things in life -- graduate from high school, get any kind of job and stick with it when they can, and they are married before they had children, that individual will not be poor or will rarely be poor in the course of their

lifetime. the way you get consistent poverty in the u.s. is individuals who break those rules and make bad decisions. i am not saying you made a bad decision, so we will not give you assistance, but we need to modify the welfare system so that the decisions are less frequent. another issue that the caller is bringing up is there is a lot of cheating. people get aid, and they steal other sources of income from the government, or they have other people in the household with income that is hidden from the government. a work requirement goes a long way to solving that cheating. if you have a hidden job while you are getting benefits, if the government requires you to show up at the site to train, you cannot do your hidden job and show up at the same time. that is why when you put those

requirements on, a lot of welfare recipients simply jump off of the roles. host: another question from twitter. boring file clerk is asking what is the current definition of the middle class? guest: there is no such definition. it does not exist. i think a lot of people would say informally that it is people with incomes of around $50,000, $60,000 a year, because that is about the average in this country for a family. maybe it is all of those that are in those middle ranks between $40,000 and $70,000, but definitions are all over the map. guest: i would agree with that. it is an imprecise term. >> we are leaving this now to go live to glendale, california.

president obama has been touring dreamworks animation. the chairman of dreamworks animation, melody hobson, is speaking now. >> people always say one thing when they say i am from chicago, they say barack obama is from chicago. the next question that they always ask, without a doubt, is do you know him. i actually had the good fortune of being able to say yes, i do. seat watchingrow him move from community organizer to senator, state senator, skyrocketing into the white house. i saw that firsthand up close. i knew that i was coming to see you all today, that i was meeting with storytellers, so i thought i should tell a story about the experience of knowing the president, a story that was

extraordinary and poignant for us at my firm, arial investments. close to the end of the first campaign and his team thought there was a possibility that they could win, they called and said that -- if he wins we need a place for him to work for a few days because he does not have an office yet. he will have a transition office but he will be in limbo for a few days. can he come to work at arial investments? we were thrilled. we thought this would be the neatest thing ever if it happened. we all remember the night in chicago where he became the president, where he stayed up late into the night, i am sure speaking to the press around the world, this is what struck us. the next morning, we wake up early in the investment business, he walked in. he walked in and what i was struck by was his demeanor. there was no big grin, no hive

five, no victory lap. he looked at me and said -- melody, we have a lot of work to do. with that he little leak did -- literally disappeared into a conference room where he was more or less in and out for the next three days with his team. i thought about that a lot, about the fact that during that time he was so deliberate, so calm. remember what was going on in the financial world. the financial world was falling apart at that point in 2008. ultimately i think it meant a lot to the country. that is one of the words about the president that is underrated. beliefs, sticking to them. one of the things about it is that we do not really appreciate

-- when he think about really smart, talented, great leaders, they were able to have a strong point of view and also at the same time admit when they make mistakes and admit when they learn from something. very, very few people actually do that well, where they can hold true and stay true to their values. and ire we are today think about this president and what he has stayed focused on since he was elected. growing this economy, which went through a very bad spell, and creating jobs in this country. where are we? in the last 44 months, 7.8 million jobs have been created. circle in my story, full circle. there are no high fives, no victory laps, and i think that what he will say to us that i look forward to hearing is that

there is still a lot of work to be done. with that it is my great honor and privilege to introduce the president of the united states. chief] ♪ the >> hello, everybody. [cheers and applause] inis good to be nla -- to be l.a. at theolder in d.c. moment. colder in chicago. 70 degree weather is something to be thankful for. it is great to be a dreamworks animation. i would like to work here. jeffrey.ked the only concern that i had was the lights were kind of dim in the offices.

i am pretty sure i would follow sleep. but there is a natural connection between me and dreamworks. i do not know if you know this, but my years were one of the inspirations -- my years -- my ears were one of the inspirations for shrek. [laughter] that is true, true story. [laughter] melody was being modest when she said she had a front row seat. she was one of my earliest supporters back when no one could pronounce my name. rogers helped to cochair some of my first fundraisers and then had to drag some scraggly group in, kicking and screaming, to write a check and listen to these young

senator who have a lot of ideas but not necessarily any realistic prospects to win. she went through a lot of ups and downs with me and my career and is just a great, great friend. i want to thank her publicly for all the support she has given us. [applause] we have got some folks here fighting for the people of southern california every day and i just want to acknowledge them. we have the mayor of glendale. we have three of your members of congress. they are all doing a great job. [applause] i want to thank all of you for being here and i want to thank your ceo, jeffrey katzenberg, for inviting me. applause]d

melanie, has been a friend and supporter through thick and thin. i think that his place in the entertainment industry is legendary. i do not need to pump him up too much. he has a healthy sense of self. [laughter] but he is a great friend and someone who has given counsel and advice that i value. i am incredibly grateful to be here at this wonderful institution that he helps to build. i have come here today because this is one of america's economic engines. not just dreamworks, but the that cluster of companies

we have grown up knowing. his knee, warner, universal, others. what the auto industry is to the midwest, entertainment is to this part of the country. spent a lot of time thinking about our favorite , but we dov shows not think about the entire infrastructure and industries behind it. hundreds of middle-class jobs. it is not always on the marquee. electricians, sound mixers, makeup artists, designers and animators. it depends on this incredible industry here in southern california. entertainment is one of america's biggest exports. every day you sell a product made in america to the rest of the world.

every time someone buys movie tickets, dvds, distribution rights to a film, some of that money goes back to the local economy right here. believe it or not, entertainment is part of our american diplomacy. it is part of what makes us exceptional. part of what makes us such a world power. you can go anywhere on the planet and you will see a kid wearing a "madagascar are" t- .hirt you can say may the force be with you, they know what you're talking about. hundreds of millions of people who will never set foot in the united states, thanks to you they have experienced a small part of what makes our country special. they have learned something about our values. we have shaped the world cultures through you. the stories that we tell

transmit values and ideals about tolerance, diversity, overcoming adversity. creativity. dna. are part of our as a consequence of what you have done, you have helped to shape the world's culture in a way that has made the world better. they might not know the gettysburg address, but if they are watching some old movie -- maybe guess who's coming to dinner, "the mary tyler moore "modern family," they have a modern seek -- they have a seat towards the march towards -- front row seat in the march towards progress. suddenly young people around the world make a connection and have an affinity to people that do

not look like them. maybe originally they might have been fearful, now they say -- this person is like me. that is one of the powers of art. that is what you transmit and that is a remarkable legacy. now, it is also a big responsibility. it comes to issues like gun violence, we have to make sure that we are not glorifying it. the stories that we tell shape our children's outlook in their lives. earlier this year leaders in this town sat down with the vice president to talk about what we safe do to keep our people in the wake of sandy hook. the stories that we tell matter. you tell stories more powerfully than anybody else on the earth. but i want to make it clear, even as we think long and hard about the messages we send, we

should never waver from our commitment to the freedom that allows us to tell those stories so well. protecting our first amendment rights are vital to who we are. it is also good business. because in the global race for jobs in industries, the thing that we do better than anyone else's creativity. that is something that cannot be copied. the reasons why even with new markets and new technologies, there is still a better place to make movies and music then right here in the united states. entertainment is one of the bright spots in our economy, the gap between what we can do and what other countries can do is enormous.

can find and get good middle class jobs that support a family and get ahead. than g is more important that right now. when i elanie mentioned came into office we were going through a severe crisis. later america has largely fought our way back. we've made the tough choices not just to help the economy recover but build on t more ew foundation for stronger and durable economic rowth and manufacturing and exports and today our businesses sell more goods and services han ever before and our manufacturers are adding jobs or the first time since the 1990s led by an american auto industry and american