mental and physical health. >> all right. good evening everyone. thank you for supporting your >> good evening everyone thank you for supporting your local independent bookstore. please make sure all phones are on silent also other events this month reading from

the new novel who are you kevin bledsoe. we also have some author events tickets are still available for the conversation and the author talk on wednesday tonight we welcome the author of liberty ai but to know that we are not on the doorstep of fully autonomous cars are super intelligent machines. taking the human mind to advance artificial intelligence to the next level we don't need to worry about the machine overlords. finally a book that tells us what ai is and is not and what it could become if we are ambitious and creative enough. the ceo of robust ai and founder and ceo of geometrics intelligence he publishes and

journals of science and nature the youngest professor emeritus at nyu. thank you. [applause] >> this is not good. we had some technical difficulties. i'm here to talk about this new book rebooting ai you may have seen the op-ed in the new york times called how to build artificial intelligence we can trust. people either building a lot of artificial intelligence we cannot d trust. it has a trust problem we rely on ai more and more but it has not earned our confidence and

we also suggest that there is a problem but a lot of ai is overhyped by people who are very prominent in the fields of one of the leaders of deep learningee which is an approach said that typical person can do a mental task in less than one second but we could automate it using ai in the near future that's a profound claim anything you could do in one second we could get ai to do if that is true the world would change altogether. may be some day i will try to persuade you it is not true now. but the problem is we have things like driverless cars they think they can trust but theyue shouldn't and sometimes they die in the process. this is a picture from a few weeks ago that a tesla crashed

into a stopped emergency vehicle for that has happened five times in the last year on autopilot has crashed into a vehicle on the side of the road. here's another example i hope this doesn't happen to my robots in the security robots committed suicide by walking into a puddle. [laughter] and so andrew says they can do anything in one second but a person can look at the puddle and say maybe i should not go in there but also bias that people have talkedav about lately. you could do a google search for the word professor you get something back they are all white mailed those of the statistics only 40 percent are white male look around the world is much lower than that. so systems take a lot of data but they don't know if it's good but they just perpetuate it back out under stereotypes.

the underlying problem right now is the techniques people are using are too brittle. everybody is excited about deep learning it is good for a few things, actually many things like object recognition. to recognize this is a bottle or microphone or my face distinguished from my uncle ted. deep learning can help some with radiology but all the things it is good at falls into one category of human intelligence is perceptual classification if you see a bunch of examples then you have to identify further examples that look or sound the same. but that doesn't mean that one technique is useful for everything. i wrote a critique a year and a half ago online wired wrote

a great summary that it is opaque and shallow and there are deep down inside so it doesn't mean it is perfect but first i will give you a real counterpart prick if you are running a business and wanted to use ai you need to know what can it do for you and what can it not if you are thinking of ethics and what machines could do soon then you realize there are limits system is of a typical person can do them mental task with less than one second of thought with the enormous amount of data that is directly relevant then as long as the test data that makes the systems work is different than what we taught the system on that's not too terribly different and if it doesn't change overan time.

so this is a recipe that says what ai is good at light games so the best player in the world is better than a human and that's exactly what they are good at the system i hasn't changed the game hasn't changed 2500s, years you have rules with as much data as you like for free. the computer can play itself for different versions of, itself and keep playing itself in gathering more data now compare that to a robot of elder care or you don't want that to collect the infinite amount of data through trial andd error. if you're elder care robot works 95 percent of the time and drops grandpa 5 percent you look at lawsuits and bankruptcy. that will not fly for the

elder care robot. when it works those neural networks fundamentally takes big data with physical approximations so you labeled a bunch of pictures of tiger wood woods, golf balls and then angelina jolie and then tiger woods and it can correctly identify this is tiger woods not angelina jolie. this is the sweet spot. . . . .

it's not able to do what you would be able to do which is first to recognize it as a silhouette. it's the generalization and people can't really do this so it's getting used in the systems that make the judgments about whether peopljudgment aboutwhetn jail or should get particular jobs and soer tfo forth so it ie limited. in a snow bank access with great confidence that is a snowplow and what that tells you is that

cares about things like the texture of the road and the snow is no difference between a snowplow and school bus or what they are for. at the right it was i made by people at mit if you are a deep learning system you would say espresso because there's foam that isn't super visible because of the lighting. but it doesn't understand it is a baseball. other learning system. in the sticker fo the deep learg top one tos from the the bottom on a toaster and it doesn't have a way of doing what you would do.

>> url rapist is starting to control the society you are not paying attention. >> to go without the slightest course because of technical difficulties. i will continue though. >> one second here to look at mn notes. i was next going to show a picture of a parking sign with stickers on it. presenting slides over the web isn't going to work. the deep learning system calls that a refrigerator with a love of food and drink.

then i was going to show a picture of a dog that is doing a bench press with a barbell. yes, something has gone wrong. picture a dog with a barbell. it can't tell yo can tell you ty weird how did it get to that it could lift a barbell. people are learning the concept of the things that is looking at. even more when it comes to reading so i'm going to read a short little story laura ingalls wilder wrote.

it's about a 9-year-old boy who finds a wallet full of money dropped on thet' street and then his father guesses it might belong to somebody named mr. thompson said he finds mr. thompson. he turns. to him and asks di tht you did you lose a pocketbook? yes, what do you know about it, is this it, yes. he opens it and counts the money and counts the bills and breathes a sigh of relief and says he didn't steal any of it. the boy hasn't stolen any of the money or where the money might be.

it wouldn't be there and so forth. all these things could make entrances about things like how everyday objects work and how people work so you can answer questions on what's going on. there is no system gets that can actually do that is the closest thinso the closestthing we havem released. they are going to give it away for free and that is what makes it so interesting so they gave away a book they made this thi thing. oey didn't want the world to have it people figured out how it worked and now you can use it on the internet.

he's found g a wallet and he is now super happy. you feed intot the story and it took a lot of time may be an hour to get the money from the safe place where he hid it and this makes no sense. it's perfectly grammatical but what is it doing it is just they are correlated in a database but it's different from the kind of understanding children do so the second half of the talk i will do without visuals is called looking for clues. the first clue as we develop further is to realize perception is just part of what intelligence is. some of you might know how this is so this verbal intelligence,

musical intelligence and so forth as a cognitive psychologist i would say things like common sense, there are many differentnt components. what we have right now is a form of intelligence is one of those and it's good at doing things that fit with that, good at certain kinds of gambling but it doesn't mean that it can do everything else. the way i think about it is it is a great hammer and we have a lot of people looking around saying because i have a hammer everything must be a nail. some things work with that but there's been much less progress on language so i would say there's been exponential progress in how well computers play games but there's been zero progress getting them to understand the conversations and that's because intelligence itself has many different components, no silver bullet to stop it.

there is no substitute for common sense. the picture i wanted to show you right now is a robot with a chainsaw cutting down the wrong side if you can imagine so it's about to fall down. this would be bad. we wouldn't want to solve it with a technique called reinforcement learning you would like a fleet of 100,000 robots and chainsaws making 100,000 mistakes, that would be bad. then i was going to show you this picture of something called a yarn feeder that is a little bowl with a string that comes out of the hole.ug you have enough common sense how physics works and what we would do with it to understand that i'm going to show you a picture of an ugly one so youou can recd is this even though it looks different because you get the basic concept, that's what common sense is about. then i k was going to show you a picture of a roomba and motel

and a dog doing its business you might say. the roomba doesn't know the difference between the two, and then i was conditioning is something that something that has happened not once but many times which is rome but that doesn't know the difference between a fellow that they should clean up and maybe dog swaste that is spread through people's houses it's been described as a jackson pollock of artificial intelligence common sense disaster. then what i really wish i could show you the most is my daughter claiming a chair like the ones you have now. 4-years-old, there's a space between the bottom of the chair in the back of the chair. she was small enough to fit through and she didn't do this reinforcement learning which is trying, she didn't give it by imitation, i was never able to come and for those who though

she never watched the television show dukes of hazard to get inside the window of a car. she just invented for herself a goal and this is the essence of how children learn things they rt a goal like can i do this or that, can i walk on a small ridge on the side of the road. i have two children, five and six and a half they make up games like what about this or can i do that so she tried it and learned essentially in one minute, she got a little stock and did a little problem solving. this is different than collecting data with labels the way that it's working right now and i would suggest we need to take some clues from kids and how they do things. next thing from harvard down the street she made the argument know that there are objects and places and things like that that you can learn about but if you

just know about pixels and videos you can't really do that. you need a starting point this is the opposite of the blank slate hypothesis. then i like to show a video nobody likes to think humans have anything in need other than their temperament. people don't want to think that they are built with notions of space and time and causality. i'm suggesting a i should do what nobody has a problem thinking animals might do this so i show planning on the side of the mountain a few hours after it's born but anybody that sees the video has to realize there is something built in from the minute it comes out it must do something about this at us.

the next video shows a bunch of robots doing things like opening doors and falling over or trying to get into a car and falling over. i'm sad i cannot show you this right now, but you get the point. if they are really quite ineffective in the real world. the video i was going to show beenhings that had simulated. everybody knew exactly what the events were going to be they just had to have the robots over the door and turn files and stuff like that. when it got to the real world, the robots fell left and right and couldn't deal with things like friction and windy and so forth.

when people would have been better off worrying about hygiene. we should worry about the limits of using print ai a lot so anyway on the topic i suggest a few things that you can do. the first one is just closed the door. robotsow right now can't open doors. if that doesn't work, walk the door so there isn't even a competition gets to have them block the doors and it will be like another seven or ten years before people start working on doors where you've got to kind of pol pull in the mob and stuff

like that so just lock the door or pick up one of the stickers i showed you and you will completely confuse the robots or talk with an accent in a noisy room. the second thing i want to say is deep learning is better than we did before and let us climb to a certain height. just because it is better to send me an it's going to necessarily get you to the moon. we have a helpful tool here that we have to discern as listeners and readers and so forth the difference between doing a little bit o and some magical fm that simply hasn't been invented yet. so, to close then we will take as many as we can. to build machines as smart as people we need to start by studying small people. human children and how they are flexible enough to understand the world in a way that ai isn't able yet to do. thank you very much. [applause]

i am a retired orthopedic surgeon and got out just in time because they are coming out with robotic surgery prominence and then the t replacement. the dream is the robots can do the surgery itself. like any other tool in order to get the robots to be able to the full-service, they need to understand the underlying biology of what they are working on so they need to understand the relationship between the different parts they are working with and our ability to do that right now is limited in the kind of reason i'm talking about so there would be advances in the next year but i wouldn't expect that when we send people to mars whenever that is if it is anytime soon thaanytime soon tha

sort of robot surgeon w like in sciencehe fiction. there is no reason we can't build such things into the machines better understanding, but we don't have the tools right now to allow them to caabsorb the medical training. it reminds me of a famous experiment in the cognitive development where a chimpanzee obviously named after noam chomsky was trained and raised in the human environment and the answer is no. if you said a current robot in medical school wouldn't work diddly squat or helpro it to bea robotic surgeon. >> another question.

>> it seems like maybe it is logically possible to build them, but the problem you get is a player cases if you teach a model system they are different han you see in the real world. so, the case of the tow truck sent firetrucks is probably in part because they are trained on kind of ordinary data where they are moving fast on the highway andd it doesn't really understad how to respond, so i don't know whether they are ultimately going to prove to be closer to something which we can get something to work in the current technology or language which seems completely out of sight of the rain that people have been working on it for 30 or 40 years

and this progress but it'sgr relatively slow. people solve one problem and it causes another so the first was a test love that ran underneath a semi trailer that took a left turn onto a highway so first of all, you had h a problem that it was outside of the training. i've been told what would happen is they thought that it was a billboard and the system had been programmed to ignore billboards because if it did and was going to slow down so often that it was going to be rear ended all the time so one problem solved. would havwhat happened this dris cars felt a lot like whack a mole. to my mind they don't have

general unique so people say i will use more data. right now they need human intervention about every 12,000 miles the last i checked. that sounds impressive so if you want to get it at this level you have a lot more work to do and it isn't clear the same techniques will get us there. this again is the metaphor it is and this is reall only going tot you tout the moon. do you think we are making progress on having machine learning kind of programs tell us how they are making real

decisions and details that are useful? >> there's a lot of interestst n that. right now they may change but there is a tension that are efficient in those that produce the results as i guess you know so the best techniques for a lot of the problems does this look like another asteroid that i've seen before, keep learning the best of that l and it's far as u could possibly imagine. people are making little progress to make that better but there's a trade-off now you get ritter results and interpretation. i haven't seen any great solution to it so far i don't think that it is insolvable in principle but here at the moment with a ratio between how the systems work and how we understand this extreme. we will have cases where somebody is going to die if

somebody is going to have to tell the parent of a child is parameter number 317 with a negative number and it's going to be completely meaning less but that is sort of where we are right now. >> we can't afford to have any misdiagnosis. can you use this stuff for medical diagnosist the answer is yes but it relates to the last which is how important is the misdiagnosis into the more it is we can rely on the technique. they have radiology in particular and they are pretty good atio pattern recognition. they can be as good as the

radiologist southeast and careful laboratory conditions. nobody really has as far as i know were the last i checked a working real-world system that has demonstrations i could recognize in this particular pattern but in principle it is an advantage over people there so it is a disadvantage in that it can't read the medical charts so there is a lot of unstructured type like doctors notes and stuff like that is just written in englishth rather than being a picture of a chart. if it relates to this accident a person had 20 years ago and tries to put together the pieces to have a story about what is

going on in current techniques just don't do that. i'm not saying in possibl impost it's not going to roll out next week. in the developing countries where there's not enough doctors and the system might not be perfect but you can try to reduce the false alarm to some degree. and you can get decent results where you maybe couldn't get any results at all. we will start to see that. it's going to take longer because we don't have the data. it hasn't been digital. radiologists have been digital for a while and then there are things like if you watch the television show, house, where you try to put together some complex diagnosis of the disease or something like that. so i made an attempt at that and it just wasn't very good.

so it was obvious to a first-year medical student and then it goes back to the data and understandings of the correlation you don't understand the biology and medicine so we just don't have the tools you have to do high quality medical diagnosis. that is a way off. >> i'm working as a data analyst, data scientist and part of what i'm -- in the organization part of what i'm working on is scoping what are small and discreet tests of the machine learning like forecasting and wider problems like you were saying with

current methods. i'm always interested in ways to explain or get the idea across and found it to be at the bounded the more limited the world is, the more frantic the can handle them and some are demandeopen ended on the usual . driving is interesting because in some ways it is close t closn like you only drive on the road we are talking about ordinary driving in ordinary one says this bridge is out and there are so many possibilities and it is open-ended and there's a lot of conventional data and they

looked very poorly when they were forced to go outside could say informally of their comfort zone. they go outside of it [inaudible] not that we are using it the wrong way by years of evolution what they did is both a genome if you look at the developmental biology, it ist clear that it isn't a blank slate. it's very carefully structured. we don't understand all the structure but there's any other experiments in all kinds of ways

and you can do deprivation experiments where they don't have any exposure to the environment and have to understand various things and so forth. so what it's done is to shape a rough draft and that is actually built to learn specific things about the world. you can think about ducklings looking for something to imprint on them the minute they are born. our brains are built to learn about people and objects and so forth but what evolution has done is give us a toolkit for assimilating the data that we get. you could say more and more data and time could i get the same thing, maybe, but we are not without replicating a billion years ofi evolution. that is a lot of trial and error and we could try to replicate that with enough time and enough graduate students and so forth, but there is another approach where you try to look at how it solves problems to take a clue from the way nature solves the

problem and that is a fundamentally what i'm suggesting is another name is we should look at how biology in the form of human rains were other animal brain manage to solve problems not because we want to build literal identical, we don't need to build more people, i have two small people and they are great they take the best of what machineswe do well which is to compute with the best of what people do which is to be flexible in their thoughts so we can do things like stuff like no other human being can solve right now this paper is published every day and no doctor can read them all. it's impossible for humans. right now machines can't do it at all. if machines could read and we could scale them the way we scale computers then we could revolutionize medicine.

i think to do that, we need to build in basic thing like time and space andpa so forth so they can make sense of it. other questions? yes. >> hohell are you thinking about fixing the problem building the new models and what form they willll take, is it going to be e same structure or something completely different? .. >> and then it always looks

englishtences in the language that represents correlations and poor with that type of knowledge and then we need a n synthesis so then traditionally those techniques and that works with abstract knowledge so you can teach us something by saying that apple is a fruit and we can do a tiny bit of this but we don't have systems where he have anyay way to teach something a while it occupies physical space we just don't even have a way to tell the

machine that right now and then we try but isn't a different game of whack a mole? and then we the to do allrr three of those. that all doesn't have to be hand t encoded. they can learn some of that themselves but there are some that enable other kinds of things. so if you don't know time exists but you see that

correlation they will not give you that. language it's like 50000 and then may be 100 pieces of common sense that goes of each of those words then millions of pieces of knowledge it's a lot of work to encode them all but maybe we could do that in a different way but it's a task people don't want to do because it's so much fun to play with deep learning and get approximations that nobody has the appetite but maybe there is no way to get there otherwise. that is my view.and there is a n to say the way you get into the game that means you can bootstrap the rest. i don't think it is whack a

mole but scoping. we have to bootstrap the rest that's true for babies they have some basic knowledge and then they develop more knowledge and then work on those and see where we are. other questions? if not thank you very much. [applause] [inaudible conversations]

>> we are back live at the national book festival professor malone from mit with