hopefully he will be back to talk aen mental and physical health.

>> all right. good evening everyone. thank you for supporting your local independent and employee owned bookstore. before we begin, c-span is filming this. make sure the phones are on silence. tomorrow night the author will be reading from his new novel. on tuesday poisener in chief's author will be joining us. we will also have a talk on wednesday. we welcome the author of rebooting ai which argues a computer being a human in jeopardy doesn't signal that we are on the doorstep of fully autonomous cars or super intelligence machines. taking inspiration from the human minds, the book explains if we need to advance artificial intelligence to the next level, if we are wise along the way, we won't need to worry about a

future of machine overlords. finally a book what tells us what ai is, what it is not and what it could become if we're ambitious and creative enough. it's also been called an informed account. the founder and ceo of robust ai and founder and ceo of intelligence. he's published in journals including science in nature. he's the author of guitar zero and rebooting ai. thank you. >> thank you very much. [applause] >> uh-oh, this is not good. um, okay, maybe it will be all right. we had some technical difficulties here. i'm here to talk about this new book "rebooting ai".

some of you might have seen an op-ed i had this weekend in the new york times called how to build artificial intelligence we can trust. i think we should be worried about that question because are building a lot of artificial intelligence they don't think we can trust. artificial intelligence has a trust problem. we are relying on ai more and more but it hasn't yet earned our confidence. we also suggested -- i also want to suggest there's a height problem. so a lot of ai -- to suggest there's a hype problem. one of the leaders of something called deep learning which is a major approach to ai these days. he says if a typical person can do a mental task with less than one second a thought, we can probably automate it using ai either now or in the future. that's a profound claim. if it were true, then the world would be on the verge of changing all together. it may be true some day, 20, 50,

to 100 years from now, but it is not true now. we have driverless cars that we think we can trust but we can't actually trust. this is a picture of a few years ago where a tesla crashed into a stopped emergency vehicle. that's happened five times in the last year that a tesla on autopilot has crashed into a vehicle by the side of the road. a systematic problem. here's another example. i am working in the robot industry. i hope this doesn't happen to my robots. a security robot basically committed suicide by walking into a little puddle. you've got andrew yang saying machines can do anything a person can do in a second. a person in a second can look in the puddle and say maybe i shouldn't go in there and the robots can't. we have other kinds of problems too like bias. a lot of people have been talking about that lately. you can do a google image search for the word professor and you

get back something like this where almost all the professors are white males even though the statistics are in the united states that only 40% of the professors are white males and if you look around the world, it would be lower than. that you have systems that are taking a lot of data but they don't know if it is any good and they are reflecting it back out and perpetuating cultural stereotips. the underlying problem with artificial intelligence is the techniques people are using are simply too brittle. everybody is exciting about something called deep learning. it is good for many things. you can get deep learning to recognize that this is a bottle or maybe this is a microphone. you can get it to recognize my face and maybe distinguish from my uncle ted's face for example. deep learning can help some with radiology, but it turns out that all of the things that it's good at fall into one category of human thought or human intelligence. the category that they fall into is they are all things we call

perce percentual -- perceptual classification. things that look the same or sound the same. that doesn't mean that one technique is useful for everything. i wrote a critique on deep learning, you can find it on-line for free. the summary of it says deep learning is greedy, brittle, opaque and shallow. there are down sides to it. even though everybody is excited about it. that doesn't mean it is perfect. i will give you some examples. i will give you a counterpart to andrew yang's claims. you were running a business and wanted to use ai, you need to know what it can do for you and what it cannot do for you. if you are thinking of ai ethics and wondering what machines might be able to do soon or what they may not be able to do soon, it is important to realize there are limits on the current sys m system. if a typical person can do a mental task with less than one

second of thought and gather a lot of data that's directly relevant we have a fighting chance to get the ai to work with that, so long the test data, the things we make the system to work on are different than the things -- are not too terribly different than the things we taught the system on and the system doesn't change much over time. the problem you are trying to solve doesn't change much over time. this is a recipe for games. this says that what ai is good at is fundamental things like games. so alpha go is the best go player in the world, better than any human. it fits in what the systems are good at. the game hasn't changed in 2500 years, a perfectly fixed set of rules and you can gather as much data as you like for free, or almost for free. you can have the computer play itself or different versions of itself, which is what deep mind did in order to make the world's best go player, and he can keep playing itself and keep gathering more data. compare that let's say to a robot that does elder care. you don't want a robot that does

elder care to collect an infinite amount of data through trial and error and work some of the time and not work others. if your elder care robot works 95% of the time putting grandpa into bed and drops grandpa 5% of the time, you are looking a lawsuits and bankruptcy; right? that's not going to fly for the ai that would drive an elder care robot. when it works, the way that deep learning works there is something called neural network that i depicted at the top. it's taking big data and making statistical proximations. taking labelled data, label a bunch of pictures of tiger woods, golf balls, angelina jolie and show it a new picture of tiger woods that isn't too different than the old pictures and it correctly identifies that this is tiger woods and not angelina jolie. this is the sweet spot of deep learning. people have got excited about it when it first started getting popular. wire magazine had an article saying deep learning will soon give us super smart robots.

we have already seen an example of a robot that's not really all that smart. i will show you some more later. this promise has been around for several years, but it's not been delivered on. there are a lot of things that deep learning does poorly even in perception. then i will talk about something else which is reading. on the right are some training examples. you would teach the system, these things are elephants. if you showed another elephant that looked a lot like those on the right, the system would have no problem at all. it would say wow it knows what an elephant is. suppose you show it the picture on the left. the way the deep learning system responds is it says person. it mistakes the silhouette of an elephant for a person. and it's not able to do what you would be able to do which is first of all to recognize it as a silhouette and second of all say the trunk is really salient and it is probably an elephant. this is what you might call extrapolation or generalization and deep learning can't really do this. we're trusting deep learning every day more and more. it is getting used in systems that make judgments about whether people should stay in

jail or whether they should get particular jobs and so forth. and it's really quite limited. here's another example. kind of making the same point about unusual cases. so if you show it this picture of a school bus, on its side, in a snow bank, it says with great confidence, well that's a snowplow. the system cares about things like the texture of the road and the snow, has no idea the difference between a snowplow or school bus or what they are for. it is fundamentally mindless statistical summation and correlation, it doesn't know what's going on. this on the right was made by some people at mit, if you are a deep learning system you would say it's an espresso because there's foam there. not super visible because of the lighting. it picks up on the texture of the foam because espresso. it doesn't understand that it is a baseball. another example is you show deep learning system a banana, and you put this sticker in front of the banana, which is a kind of

psychedelic toaster, and because there's more color variation and so forth in the sticker, the deep learning system goes from calling the top one a banana to calling the bottom one a toaster. in fact, it doesn't have a way of doing what you would do which is to say something like well it's a banana with a sticker in front of it. that's too complicated. all it can do is say which category something belongs to. that's all deep learning does essentially is identify categories. if you're not worried a this is starting to control -- if you are not worried that this is starting to control our society, you're not paying attention. let's get the next slide. maybe not. going to have to go without the slides i think because of technical difficulties. i will continue, though. there's just nothing we can do. all right. one second here, look at my notes. okay.

so i was next going to show you a picture of a parking sign with stickers on it. it would be better if i could show you the actual picture, but presenting slides over the web is not going to work, so parking sign with stickers on it, if you can imagine that. the deep learning system calls it a refrigerator, filled with a lot of food and drinks. it is completely off. it's noticed something about the colors and textures but it doesn't really understand what's going on. then i was going to show you a picture of a dog that's doing a bench press with a barbell. yes, something has gone wrong. [laughter] >> thank you for that. all right. we'll just shut it down. >> [inaudible]. >> yeah? i would need a mac laptop. i just couldn't do it fast. >> you need a mac laptop? i've got one. >> i don't think they will be willing to add it >> okay. >> just go on. you have a picture -- a dog with a barbell and it's lifting the barbell. deep learning system can tell you that there's a barbell there and a dog, but it can't tell you

hey that's really weird. how did the dog get so ripped that it could lift the barbell? deep learning system has no concept of things that it's looking at. current ai is even more out of its depth when it comes to reading. i will read you a short little story that laura ingles wilder wrote. a 9-year-old boy finds a wallet full of money that's dropped on the street. the father guesses that the wallet might belong to somebody named mr. thompson. he finds mr. thompson. wilder wrote he turns to mr. thompson did you lose a pocketbook? mr. thompson jumps, slaps his hand in his pocket and jumps. yes. i have. $1500 in it. is this it? yes, that is it. he opens it and counts the money. he breathes a sigh of relief and

says well, that darn boy didn't steal any of it. when you listen to that story, you form a mental image of it. it might be vivid or not so vivid, but you know -- you infer a lot of things like that the boy hasn't stolen any of the money or where the money might be. you understand why he's reached in his pocket looking for the wallet because you know that wallets occupy physical space and that if your wallet is in the pocket, that you will recognize it, and if you don't feel anything there, then it won't be there and so forth. you know all of these things. you can make a lot of inferences about things like how every day objects work and how people work. you can answer questions what's going on. there's no ai system yet that can actually do that. the closest thing that we have is a system called gpt 2. this is released by open ai. open ai is famous because elon musk founded it and it has the premise that they will give away all their ai for free.

that's what makes this story interesting. they gave away their ai for free until they made this thing called gpt 2. they said it is so dangerous that we can't give it away. it was this ai system that was so good at human language that they didn't want the world to have it. but people figured out how it worked and they made copies of it. now you can use it on the internet. so my collaborator ernie davis, my co author and i fed in this story into it. remember the boy has found the wallet, given it to the guy. the guy has counted the money. he now has -- he's super happy. what you do is you feed in the story and it continues it. what it said it took a lot of time, maybe hours this continues it for him to get the money from the safe place where he hid it. it makes no sense. it is perfectly grammatically. if he found his wallet, what is this safe place? the words safe place and wallet are correlated in some vast database. it is completely different than tunsing that children do --

different than the understanding that children do. the second half that i will talk about without visuals is called looking for clues. we need to realize that perception which is what deep learning does well is just part of what intelligence is. so some of you might especially in cambridge know a theory of multiple intelligences for example. there's verbal intelligence, musical intelligence, and so forth. as a cognitive psychologist, i would also say there are things like common sense, planning, attention. there are many different components. what we have right now is a form of intelligence that is just one of those, and it's good at doing things that fit with that. so it is good at doing perception. it is good at certain kinds of game playing. that doesn't mean it can do everything else. the way i think about this is the deep learning is a great hammer, and we have a lot of people looking around saying because i have a hammer, everything must be a nail. and some things actually work with that, like go and chess and so forth, but there's been much

less progress on language. so there's been exponential progress in how well computers play games, but there's been zero progress in getting them to understand conversations. that's because intelligence itself has many different components. no silver bullet is going to solve it. the second thing i wanted to say is that there's no substitute for common sense. we really need to build common sense into our machines. a picture i wanted to show you right now is of a robot on a tree with a chainsaw. and it's cutting down the wrong side, if you can picture that. so it's about to fall down. now, this would be very bad. we wouldn't want to solve it with a popular technique called reinforcement learning where you have many many trials. you wouldn't want a fleet of 100,000 robots with 100,000 chainsaws making 100,000 mistakes. that would be bad as they said in ghost busters. then i was going to show you a cool picture of something called the yarn feeder which is a little bowl with some yarn and a string that comes out of a hole.

as soon as i describe it to you, you have enough common sense about how physics work and what i might want to do with the yarn feeder. i was going to show you a picture of an ugly one. you can recognize this one even though it looks totally different because you get the basic concept. that's what common sense is about. i was going to show you a picture of roomba. you all know the vacuum cleaner robot. i was going to show you nutella and a dog doing its business, you might say, and say the roomba doesn't know the difference between the two. i was going to show you something that's happened not once but many times which is roombas that don't know the difference between nutella that they should clean up, and dog waste, spread the dog waste through people's houses. it's an artificial intelligence common sense disaster. [laughter] >> then whey -- then what i wish i could show you the most my daughter climbing through

chairs. when she was 4 years old, she was small enough to fit through the space between the bottom of the chair and the back of the chair. she didn't do it by imitation. i was never able to climb through the chair. i'm a little bit too big, even if i'm in good shape and exercising a lot. she had never watched dukes of hazard and climbed through a window and get inside of a car. she had never seen that. she invented herself for a goal. this is the essence of how human children learn things. they set goals, can i do this? can i walk on this small ridge on the side of the road? i have two children, 5 and 6 1/2. all day long they just make up games what if it were like this or can i do that? she tried this and she learned it essentially in one minute. like she squeezed through the chair. she got a little stuck. she did a little problem solving. this is very different from collecting a lot of data with a lot of labels the way that deep learning is working right now. and i would suggest that if ai

wants to move forward, that we need to take some clues from kids and how they do these things. the next thing i was going to do was to quote elizabeth spellke who teaches at harvard down the street. she's made the argument if you are born knowing there are objects and sets and places and things like that, then you can learn about particular objects and sets and places, but if you just know about pixels and videos, you can't really do that. you need a starting point. this is what people called the nativist hypothesis. i like to show a video. people don't want to think that humans are built with notions and space of time and causality as it's been argued, as i'm suggesting ai should do. nobody has any problem thinking animals should do this. i show this video. they have to realize that there's something built into the

brain that there has to be an understanding of three dimensional geometry from the minute that it is born. it must know something about physics and its own body. that doesn't mean it can't calibrate and figure out how strong its legs are and so forth but as soon as it is born, it knows that. the next video i was going to show you, you have to look at this on-line, robots fail. it shows a bunch of robots doing things like opening doors and falling over. or trying to get into a car and falling over. i'm sad i can't show you this right now. but you get the point. current robots are really quite ineffective in the real world. the video i was going to show were things that had been simulated. so it was a competition that was ran. everybody knew exactly what the events were going to be. they were just going to have to have the robots open the doors and turn dials and stuff like that. they had done them in computer simulation. when it got to the real world, the robots failed left and

right. the robots couldn't deal with things like friction and winds and so forth. to sum up, i know a lot of people are worried about ai right now. and worried about robots taking over our jobs and killing us all and so forth. there's a line in the book which is something like worrying about all of that stuff would be like in the 14th century worrying about highway fatalities, highway traffic fatalities when people would have been better off worrying about hygiene. what we should be worried about right now is not some vast future scenario in which aishgs is much smarter than people -- future scenario in which ai is much smarter than people. we need to worry about the limits of current ai, the fact that we are using it a lot in decisions about jobs and jail sentences and so forth. on the topic of robot attack, i

suggest a few things you can do. the first one is just close the door. robots right now can't open doors. there's a competition to teach them how to do that. if that doesn't work, lock the doors. there is not even a competition to have robots lock the doors. it will be another seven or ten years before people start working on doors where it is a little bit jammed and pull in the knob and stuff like that. or put one of the stickers i showed you, you will confuse the robot. or talk in a foreign accent in a noisy room, the robots don't get any of that stuff. deep learning is a better ladder than we've built before, lets us climb to certain heights. but just because something is a better ladder, it doesn't mean it will get you to the moon. we have a helpful tool here but we have to discern as listeners and readers and so forth the difference between late bit of ai that can do a little bit and some magical form of ai that hasn't been invented yet.

to close, and then i would love questions and we will take as many as you have. if we want to build machines that are as smart as people, what i think we need to do is start by studying small people, human children and how they are flexible enough to understand the world in a way that ai isn't able to do that. thank you very much. [applause] >> yes, question? >> i'm a retired orthopedic surgeon, and i got out just in time because they're coming up now with robotic surgery, prominent in the knee replacements. have you got any information about where that is headed and how good it is and snets >> -- and etc.? >> the dream is that the robot can do the surgery itself. most of that is an extension to the surgeon right now. like any other tool. in order to get robots to really be able to be kind of full

service, they need to understand the underlying biology of what they are working on. they need to understand the relation between the different body parts they are working with. our ability to do that right now is limited for the kinds of reasons we are talking about. there will be advances in that field in the next few years, but i wouldn't expect when we send people to mars, whatever that is, if it's any time soon, that we would have a sort of robot surgeon like you have in science fiction. we're nowhere near to that point. it will happen some day. there's no principled reason why we can't build such things and have the machines have better understanding of biology, but we don't have the tools right now to allow them to absorb the medical training. it reminds me of a famous experiment in cognitive development, where a chimpanzee was trained and raised in a human environment, and the question was, would it learn language? and the answer is no. if you sent a current robot to medical school, it wouldn't learn anything. that wouldn't help it to be a

robotic surgeon. >> thank you. >> you're welcome. other questions? >> do current limitations of ai also apply to self-driving cars? >> yes, i am sorry i didn't mention self-driving cars that much. it seems it is logically possible to build them but empirically the problem you get is what we call outlier cases. the training data, the things you teach the model system on are too different from what you see in the real world they don't work really well. the case of the tow trucks and the fire trucks that the teslas keep running into, it's probably in part because mostly they are trained on kind of ordinary data where all the cars are moving fast on the highway, and you see -- the system sees something it hasn't done before. it doesn't really understand how to respond. so i don't know whether driverless cars are ultimately

going to prove to be closer to something like chess or go where it's bounded enough we can get the systems to work with current technology or if it is going to be more like language which to me seems completely outside the range. but people have been working on it for 30, 40 years. a company part of alphabet which used to be google has been working on it far decade. -- for a decade. there's progress but it is relatively slow. people solve one problem, and it causes another problem. so the first fatality from a driverless car was a tesla that ran underneath a semi trailer that took a left turn on to a highway. so first of all, you had the problem that that it was outside the training center, whatever, it was an unusual case. and second of all, i have been told, and i don't have proof of this, but i've been told that what happened is that the tesla thought that the tractor trailer was a billboard and the system had been programmed to ignore billboards because if it didn't it was going to slow down so

often that it was going to get rear-ended all the time. so one problem was solved, the slowing down for billboard problems and another problem popped up. and so what's happened so far is the driverless cars have felt a lot like people make a little bit of progress. they solve one particular problem but don't solve the general problem. to my mind, we don't have enough techniques to try to solve the problem. people are saying i will just throw more and more data, more data, and it gets a little bit better, but we need to get a lot better. right now the cars need human intervention about every 12,000 miles last i check. that sounds impressive. it turns out humans have a fatality over 134 million miles -- every 134 million miles on average. if you want to get it to a human level, you have a lot more work to do. it is not clear grinding out the same techniques will get us there. this is the metaphor of having a better ladder is not necessarily going to get you to the moon. yes? >> my question is about machine learning and using it to do

right now -- i'm an astronomer, so we've started to use it in science. the problem is if you're just doing pattern recognition, you really don't learn anything. do you think we're making progress on having machine learning kinds of programs be able to tell us how they're making decision in enough detail that it's useful? >> there's a lot of interest in. that it may change, but right now there's attention between techniques that are relatively efficient and techniques that produce what we call interpretable results as i guess you know. right now the best techniques for a lot of perceptual problems. do i want to identify does this look like an asteroid that i have seen before? deep learning is the best from that. it is far from interpretable as you can possibly imagine. people with making progress to make it a little bit better. there's a trade off now. you get better results and you give up interpretation.

there's a lot of people worried about this problem. i have not seen any great solution to it so far. i don't think it's unsolvable in principle, but right now it is kind of at a moment of a ratio between how good the systems work and how little we understand is pretty extreme. going back to the driverless cars, we will have cases where somebody is going to die and somebody will have to tell the parent of a child or something, the reason your child died seems to be the parameter number 317 was a negative number when it should have been a positive number. it will be completely meaningless and unsatisfying but that's sort of where we are right now. other questions? >> what are your thoughts on application to healthcare diagnostics and the racial bias that the people are concerned about and also just the fact that we can't afford to have any misdiagnoses? >> well, i guess those are three different questions. the first is that -- i may forget one. you might have to remind me. the first is can you use this stuff for medical diagnosis? and the answer is yes, but it

relates to the last which is how important is a misdiagnosis? the more important it is, the less we can rely on these techniques. it's also the case of course the human doctors aren't completely reliable. the advantage that machines have right now for some things, radiology in particular, is they are pretty good at pattern recognition. they can be as good as radiologists at least in careful laboratory conditions. nobody really has as far as i know or last i checked a working real world system that kind of does radiology in general. they are more like demonstrations that i can recognize this particular pattern, but in principle, deep learning has an advantage over people there. it has a disadvantage though in that it can't read the medical charts. there's a lot of unstructured text in medical charts which is like doctors notes and stuff like that, that's written in english rather than a picture of a chart. and machines can't read that stuff at all or they can recognize key words and stuff

like that. so a really good radiologist is kind of like a detective. i had a radiologist explain to me, sort of like sherlock holmes, there's this asymmetry that shouldn't be there but is from an accident that happened 20 years ago. so they have to use interpretati interpretation, a story of what's going. current technology doesn't do that. it's not going to roll out next week. the first cases of having an impact on medicine are probably going to be radiology that you can do on a cell phone where you don't have a radiologist available. in developing countries where there's not enough doctors, the systems may not be perfect, but you can try to reduce the false alarms to some degree, and you can get, you know, decent results where you maybe couldn't get any results at all. we will start to see that. pathology is going to take longer because we don't have the data because pathologists haven't been digital.

they are starting to do it. but radiologists have been digital for a while. then there are things where if you ever watch the television "house", where you're like trying to put together some complex diagnosis of a rare disease or something like that. and systems aren't going to be able to do that for a long time. so ibm made an attempt at that with watson and it just wasn't very good. it would miss like heart disease when, you know, it was obvious to a first-year medical student. and there it goes back to the difference between having a lot of data and having understanding. so you're just doing correlations but don't understand the underlying biology or the underlying medicine, then you can't do that much. we don't have the tools yet to do high quality medical diagnosis. that's a ways off. >> thanks for coming. i'm working at a data analyst, data scientist and part of what i'm sort of working on -- sort of what i'm working on is

scoping small different tasks that automation, machine learning would be helpful for or also helping wider problems that aren't solvable right now with current methods. how would you -- i'm always interested in ways to explain or get the idea across between sort of -- like what you said -- >> i think the fundamental difference is some problems are a closed world. they are limited. the possibilities are limited. the more limited the world is, the more current techniques can handle them. some problems are open ended. they could involve arbitrary knowledge or unusual cases. driving is interesting because in some ways it's closed in, like you only drive on the roads if we're talking about, you

know, ordinary driving, ordinary circumstance, but it is open ended because there could be a police officer with a hand-lettered sign saying, you know, this bridge is out. there's so many possibilities in that way that it is open ended. what you end up finding is that the driverless cars work well in the stuff that sort of closed -- there's a lot of conventional data for it and they work poorly when they are forced to go outside their comfort zone. these systems have a comfort zone, and they go outside of it and they don't work that well. yes? >> you made a point about deep learning that -- [inaudible]. do you think that humans have an inherent advantage which is million years of evolution -- >> billion years of evolution. >> maybe we're not using enough data? >> well, i don't see it that

way. so the way that i see it that the billion years of evolution, what they did is they built a genome. they built a rough draft of the brain. if you look at the developmental biology, it is clear that that brain is not a blank slate. it is very carefully structured. we don't understand all of the structure, but there's any number of experiments that illustrate this in all kinds of ways, where you can do deprivation observation where animals don't have any experience with the environment, but they know things. you can think about ducklings looking for something to imprint on the minute they are born. our brains are built to learn about people, objects and so forth. what evolution has done is it has given us a really tool kit for assimilating the data we get. you could say by more and more data, more and more time, could i get the same thing? and maybe, but we're not very good at replicating a billion years of evolution.

i mean that's a lot of, you know, trial and error that evolution did, and we could try to replicate that with enough cpu or gpu time, enough graduate students and so forth, but there's another approach to engineering where you try to look to nature and how it solves problems and try to take clues in from the ways in which nature solves problems. that's fundamentally of what i'm suggesting. look at biology in terms of animals' brains or human brains, not that we need to build more people. i have two small people and they're great. we want to build ai systems that take the best of what machines do well, which is compute really fast with the best of what people do, which is to be flexible in their thought and to be able to read and so forth so that we can do things like solve problems that no human being can solve right now, such as,

integrating the medical literature. this 7,000 or something like that papers published every day, no doctor could read them all. it is impossible for humans. right now machines can't read at all. if we could build machines that could read and scale them the way we scale computers, then we could revolutionize medicine. but i think to do that, we need to build in basic things like time and space and so forth so the machine cans make sense of what they read -- the machines can make sense of what they read. other questions? yes? >> how are you thinking about fixing the problem, building these new modules, what form will they take? are they going to be the same structure that deep learning currently uses or something completely different? >> well, the first thing i would say is we don't have the answers, but we try to pinpoint the problems. we try to identify in different domains like space and time and

causality where the current systems work and where they don't. the second thing i will say is the most fundamental thing is we need ways of representing knowledge in our learning systems. so there's a history of things called expert systems. they are very good at presenting knowledge. so if this thing is true, then do this other thing, and, you know, it is likely that such and such is happening if these two things are happening. the knowledge looks a little bit like sentences in the english language. then we have deep learning which is very good at representing correlations between individual pix els and labels and things like that but very poor at representing that kind of knowledge. what we argue is we need a synthesis of that. learning techniques that allow you to be responsive to data in ways that traditionally the techniques aren't particularly responsive to data. learning that's responsive to data but that represents -- or that works with abstract knowledge so you can, for example, teach it something by saying that, i don't know, an apple is a kind of fruit, have

it understand that. we have systems that can do a little bit of this, but we don't really have systems where we have any way of teaching something explicitly like wallets occupy physical space and that a wallet inside a pocket is going to feel different than a wallet that is not inside a pocket. we just don't have a way of even telling the machine that right now. yes? >> i'm thinking if we don't take the deep learning approach and don't learn all this from data and we try to -- [inaudible] -- some of this knowledge, then aren't we playing a different game, like we're trying to uncode some knowledge about time and tomorrow we're like we need to -- about space and then tomorrow causality. >> i think we need to do all three of those.

there's a lot of knowledge that needs to be uncoded. it doesn't have to be hand coded. we could build learning systems that could learn it themselves. there's some core domains that enable other kinds of things if you have a framework for representing time then you could represent things that happen in time. if you don't know time exists then seeing a lot of correlation between pixels isn't really going to give you that. one way you could estimate things is you could think about the number of words in the english language that a typical native speaker knows. and it is something like 50,000 if they have a big vocabulary. maybe there's ten pieces -- or 100 pieces, let's say, of common sense that goes with each of those words. then you are talking about like millions of pieces of knowledge, but you are not talking about trillions of pieces of knowledge. it would be a lot of work to encode them all. it's been tried, and maybe we could try it in a different way. it is not an unbounded task. it's so much fun to play around with deep learning and get good

approximations, not perfect answers so nobody has the appetite to do it right now. it could be there's no way to get there otherwise. i mean, that's my view. that's spelke's view. there's a long tradition of nativists say is the way you get into the game is you have something that allows you to boot strap the rest. i think it is a scoping problem that we need to pick the right core domain so we can boot strap the rest f. if you look at the cognitive development literature that's true for babies. they have some knowledge and then develop more knowledge. i think we could at least start with the kinds of things that developmental psychologists have identified, work on those and see where we are. other questions? if not, thank you all very much. [applause] >> i think we will have a book signing if anyone --