go is an example of a machine algorithm that was to play against the ancient game of go. it's vastly more complicated back in 1997. there are more possible moves in the game of go then there are atoms in the universe and that means the traditional technique of trying to search exhaustively , even a tiny fraction of the moves is completely infeasible. for that reason, as recently as 2016, computer scientists were

surveyed when they thought machines would be able to beat humans at the game of go and they predicted probably around 2027. that was the median estimate. as some of you in the audience know, they did beat the chip world champion and gold last year end they did it by using a new technique by looking at the board and seeing patterns. that's the way humans do it. if you ask people how they play it, they can explain it. they say they look at the board when they see a move that seems right but they can explain how. they can't describe strategy or tell you exactly what they're doing. you can imagine, if you can't explain it, how would you code it. with this new approach to machine learning and successes

and failures, they can figure out the pattern on their own by applying reinforcement learning p they were babe able to very quickly learn the game of go to where they could beat humans at the game, and now human champions, when they see with the machines are doing, they say we have not even touched the edge of the game of go. we have been studying this for 3000 years but the machines are so far ahead of us that we realized we have no conception of what is possible. the reason we think we are still underestimating the impact in the power of machine learning, despite the fact that we all read about it all the time is exactly what he just said. here is the domain of intent human study for three millennium we built up this body of what we

thought was really exhaustive knowledge about how to play this game. this piece of machine learning technology comes out of nowhere and shows us how much more headroom there is in this particular domain. that's pretty amazing. we think about all of the other other incredibly complex things that were trying to get good at, whether that is solving the mystery of cancer or figuring out what to invest in or viewing the human genome, we now have a colleague for that work and it might be able to show us some ways forward. this last point is really key. sometimes it's called the second wave of the second machine age. that's the idea that the machine starts doing mental tasks. the fast past few decades we've programmed them. we've codified our own knowledge and included it so the machine

to do it. that tells us how too do tax preparation or spreadsheets but it's all stuff we knew exactly what needed to be done and we had to be very, very precise and the machine would do the same thing. that's great. there's so many problems that it sounds paradoxical. i wouldn't know how to recognize my mother's or ride a bicycle. these are just things we can do intuitively. they are giving them example of what works and what doesn't work this is the word yes or no in french or english in the machine

, if you give them enough examples, they will figure out the patterns on their own. it's a remarkable transition that has been important in the last few years. >> there was an example that went viral on twitter a while back. it was, could the machine tell if it was a blueberry muffin or a dog. >> there was another one that came out the same time. there was the lab or doodle or the kentucky fried chicken. >> this is a story that really looks works better if you look at the picture because if you don't see someone similarly looking at the pictures they can be hard to tell apart, but we humans can often do it and was remarkable is that seven years ago, machines had about a 30%

air rate and that was using the best technique. now it's down lower than 5% which, importantly is less than the air rate of humans on that same data set. we've kind of crossed a point. is not just a matter of degree but when you raise the temperature of water past the boiling point it's a fundamental transformation. if a machine can do a task better than a human, then if you're an entrepreneur or or manager and you have to make a decision, who do i sign, you're in the one i can do the job better. >> the one thing i was hoping we could talk about is the idea that in the future or now, people will be learning from machines in addition to teaching them. one thing i liked about the go story is that after the world champion was defeated, he said so beautiful, so beautiful, about the moon because it wasn't something he'd ever conceptualized before not something we could see in a lot of fields. >> and were seeing it over and over and you're right about people earning side side-by-side

with machines. you talk about the green guy with a widely televised matched. it's part of the reason there was a downward turn in the green economy for a while, ever but he to pay attention. >> he was beating about 75% of human opponents before he played apple go. after he had done and dealt with that technology for a while he was beating about 90% of his human opponents. the other thing that happened is the sales of go boards in korea and ages asia skyrocketed and little kids started playing the games and much higher volumes. one thing we hear is isn't it

terrible that were no longer the best of this activity, doesn't that reduce human value or dignity. i think that's dead flat wrong. if we do ourselves by the things that were best in the universe, that's a pretty silly thing to do. we now have this neat thing happening that people want to be a part of. if technology is part of that, i think it's good. >> what do you think. >> i think it's good. >> let me under one of the points he made. this machine came at the game from a fundamentally different way. when they saw some of the moves the machine made, the go expert said oh my gosh, that is idiotic every beginner knows not to do what the machine just did. a few minutes later, guess what. the machine one. then they were go, i see what it did there. that is really innovative. for 3000 years they've been

learning from the master and if they made that move their master would say stupid human, don't do that, but the machine came out with a different way of thinking when they came up with an innovative approach and it's not just true in the game of go but it's true in all kinds of problems. they're not just better in certain ways, but more importantly, they're different in certain ways and they come out with a different perspective that's hoping that human machines can work together. >> what's cool about that story, lots of things are cool. you probably needed to be a player somewhere near the level of lisa to even get the insight of why they moved the goal that they did. you have to be able to unpack to why the machine was behaving that way.

we need really high-level humans to look at that and say i think i know what's going on, let's go test that idea. >> where are some companies, parts of the economy where were arty seeing this, instead of a replacement of people were seeing like a mutual collaboration and learning. >> let me describe, is not in the book, it's something i learned afterwards but it's in the spirit of the book. one of our friends with the company you daphne, it's an online learning company that has a bunch of things you can learn and 90 degrees, 90 days so people come to the website and they have a with sales reps saying is this course right for me, what shall i be doing. so when they call the human sales representative, some of

them are better than others and some of them are able to answer the questions effectively, come for the people and commence them it's a good course to take, others not so much. it's a machine learning expert and along with a grad student they recognize if they take the transcript of the chat, that becomes the data for what's called supervised learning. that's basically giving examples of successes and failures to the machine and let the machine figure out what matters. they gave these transcripts and the machine a logarithm started seeing patterns that the successful sales reps had different ways of answering questions. at that point, they did not turn it over to the boss because there were still too many unstructured questions that the machine would be good clueless about but there are some very common questions that account for a certain percentage of the inquiries. they still have the humans doing the interactions with their incoming traffic, but the bot

would watch them, and one of these common phrases or questions came up, the bar would say here's a phrase you might consider using. it's worked really well in the past. then the humans would pick up on that. especially the not so good sales representative, their performance improved dramatically and sales went up to about 50%. it was a tremendous improvement and it was the ability to combine humans and machines in a way that neither one would have been successful on their own. >> and we should be clear, in some cases we are going to automate work and that's the appropriate thing to do, but it's naïve to think that that's the unidirectional, the only force going on. we love that story because it shows the ends of technology and automation realizing they still need humans involved in this process, but we could kind of amplify their abilities with a lot of smart technology in the background. >> in the broader lesson that's been applied is that each of those pairs, were not thing time

turn the dial all the way to one extreme, worsening thinking about the balance. sometimes it makes sense to go automated but in most cases you need to find balance in the balance is more toward machines platforms and cloud that i think more individuals realize, but it's not as simple. >> this question comes up a lot. what you see is areas that are most resistant to the advent of machines and computers? we were talking earlier about a part in your book for you mention this famous saying, would you like to quote it. >> andrew bird set a while back that the only thing, the only difference between a completely unlinked series of observation and a hit song is excessive confidence on the part of the

songwriter. >> i think that's a lovely way to look at it, but he's lowballing himself because it would be trivially easy to program a computer to spit out observations that might rhyme and might have words associated with love and lost in them. you can program a computer pretty easily but we are incredibly confident that the lyrics would probably not make a lot of sense and would not activate any emotions on the part of the rest of us. what great human lyricist do, they have a fairly linked observation that resonates with other human beings, and that's a skill we have not seen computers come up with yet. it's a common set comsat engine concept from linguistics. what it means is if i look at almost any length of can plex sentence in english, i can immediately tell if it's canonically correct or not before my brain has processed

its meaning or what it's trying to convey. my bet brain is trying to determine if it's kosher or not. i spent speak some french but i can't do that to save my life. english as second language individuals can't do that anywhere near as well as i can or any other native speaker. we have intuition about this physical world that we move through, trying to keep teach computers to have that, if we ever do that, i believe that's decades away. i have a lot of confidence that the human ability to invoke an emotional response via words and language, in fact it's automated quickly, i'm actually going to be surprised. >> but at the same time we actually have a logarithm's that are starting to compose music. it's hard to tell apart from the music of famous composers. >> not hard, actually impossible

>> so the interesting question is, what is the difference between music and lyrics and what about that makes it automated. >> this is getting a far away from my area of expertise, but the aesthetic of music is well known. we know what progression sounds good to our ear and which sounds like sirens on the new york street. we want to do more the former and less of the latter. you can encode that stuff and you know the rules, you know these kind of things so it's relatively easy to hit the button and send those things out >> and he goes beyond that. the machines that are doing a really good job, they're not just encoded with the rules of a few, but there are patterns that emerge. you don't have to know anything about the structure meaning of the music. machines are actually quite good

at the shallow pattern recognitions, but the lyrics, to really be proactive you need to understand the meaning. it's not just the two words that often appear next to each other in sequence. if you want to go beyond that you need to understand in the machines, although they can recognize patterns and words, they still do not really understand what it's doing. they can translate one language to another but they're not actually understanding what's going on. that's very different. and machines can be very good at certain kinds of video games that are quick reactions shooter games, great at that, but if they can't involve some kind of deeper planning and structure, they are hopeless at that. i think language is a little more like complicated and music is more like the pattern matching. we included in our book, and exert from a system that got

trained on jane austen novels. it's not even gibberish. it actually makes your head hurt it is' are really unpleasant too so if we can make that go 180 degrees difference, i don't think that will happen anytime soon. >> they've often said when you look at jobs and what job services are likely to become automated that it's not the heiskell low scale as much is repeatable and high-frequency test versus variable and unpredictable tasks. >> the broader lesson here is that machines are not just good, they're superhuman at certain types of tasks. they been superhuman at arithmetic for decades so we shouldn't be surprised. now they're getting superhuman that pattern recognition. when things are repeatable, you can get the data to be able to train the system. if they involve more complex

planning, creativity, those are things that are much harder to have a machine. one thing we want to stress as were talking about what's available 2017, there's a bunch of very bright people working on all sorts of things that may be breakthroughs in 2018 or 2080, were not sure because they haven't happened yet but based on current technology, they're basically taking a bunch of input and mapping them into a bunch of outputs and figuring out the mapping in the really good at that if there's not too much complexity between the act 's and why. >> -think that framework is rapidly becoming out of date. i'd never would've thought of medical diagnosis as any type of routine, repeatable work that he would hand off to machines. it's an incredibly important and difficult and subtle and complex pattern matching exercise.

machines are better at those things now that we are and make come out of essentially nowhere and have reached superhuman level of performance. i think medical diagnosis is squarely in the sight of these automatable tasks and one we should apply a lot more machines to but that did not appear to be the case five or ten years ago. >> if you look at image recognition, we talked about recognizing dogs and cats and faces, but guess what camille can also have some medical images. here's pictures of lungs. which have cancers in which don't. pathologists and radiologist took years to learn what it took for look for. you train a machine and they can barely very quickly learn the markers taught in medical school and other markers that no human has ever noticed before. >> what is the accuracy? you also talk about how machines are rapidly eclipsing human

experts in their own subject. >> this is a number that is changing as we speak. even since we finished the book there has been significant article published in february of this year. that example we gave of giving 30% was an example for me. in voice recognition, understanding speech, it went from 8.5% error rate to 4.9% error rate, it's not just the improvement but that improvement was done over the past ten months, 11 months since july 2016. as we speak, they are improving it more. a lot of it just comes from taking the same set of a and a lot more data and a lot more computational power. >> with medicine, the studies we've come across our little conservative. they generally say properly done

machine learning system will be at human level performance. i think that's probably lowballing it, i think they're probably better than the average doctor in a lot of cases today. >> so where are most people going to first really encounter this kind of technology. is it alexa, is she saying buy me whole foods. >> you do get those kinds of errors, but i think a lot of it, one example is voice recognition it's clearly not perfect yet, it still makes those kinds of errors but it's been measurably better to the point where i now dictate rather than type a lot of the texts that i write and there's a study of the stanford that dictating opposed to typing is about three times faster and more accurate for the applications they looked at.

we've encountered some of them, probably the leading example, the one most obviously profitable has been deciding which ads to serve up next to your searches and if they can figure out what you're likely to click done versus what you're not likely to click on, that is worth billions upon billions of dollars. that is one that we actually encounter quite a bit. >> the insights that we get are being translated over into all sorts of other domains. the main way were coming across these cutting-edge technology is via devices like these, speaking into them, having them understand what we want, getting decent quality translation, type out a few letters and give them to you. these are all artificial intelligence. >> you may notice, those of you who use facebook that it will

offer to label the names of the faces of your friends.

>>

>> that is in the capital market. and that can be a more e efficient way those who have cars most of them are sitting idle maybe 50 percent? that is 10 times improvement. >> one of the weird outcomes is said go home in cambridge because i have a big

expensive honk of metal now wishes to a colossal wave of opportunity to take bad infrastructure that is out there in the world to the warehouse capacity to make a much more efficient and you can be underwhelmed by that we should all care about efficiency if you want to make better use of our resources we will not walk away to be more efficient with their use but those are unparalleled assets to make greater use of those assets of the infrastructure up there in the world. >> teh's anyone have questions?.

>> i will ask a hard question we have known each other for a while and there is that dystopia claim. >> messines and i suspect you personally know the editorial did you see that?

and he ran various pieces of google or microsoft he said every economy will have to discuss when it comes to subsidiaries because it will require massive welfare that we the chinese will do this better so if we head into a world there was a headline

and if we do go there how did they deal with that rapid transformation?. >> great question but we did talk about those big changes every have been touching on that so far with that potential not just more millionaires or billionaires' but those from china and elsewhere. but we also talk about the spread that there has been a growing gap between the rich and the poll were overall equality and a big part of

the reason is the way of technology we think we have traces and and this will happen automatically. these tools can be used in a lot of different ways to take a more extreme position we're not in a world without work right now. very rapid transforming work but that what you see the as median wages and the people below that have done work and there is pressure there

is more than there ever were before so we need to do a better job listing the income of the bottom half i don't see a the robo apocalypse the that is the biggest challenge we have to face with more broadly shared prosperity i am convinced it is possible if we update the policies or institution is not something billed to to was and this was to provide guidance and how they can make that

adjustment marriage is a huge challenge of how to use these technologies but we should not think there is that robo apocalypse totally capable to do the best thing that ever happened to humanity. >> it is true those big tech companies with large impressive companies that is categorically different so how do we know this? they vanish it is a rounding error.

and then to be a most unprecedented with a meaningful share the global economy. we need to call down about the vital little bit. but they also or demand some rationale. >> i have been following your work over several years. there is a lot of concern i am interested to hear of what is emerging but the of

their question that caused at between centralization and decentralization that is a concept of the last couple of years so the organization or policy makers. >> the definition of a great question is devoted to two chapters of the book. so what are in the implications for radical

decentralization? block chaining is that ledger that keeps that point around the world. but that is say ledger that is accurate and appears to be unflappable not known by anybody and apparently not controlled by anybody even if the central bankers want to take over is probably impossible. this is weird we have not seen this before. writing the paper outlining the possibility that some of

those of the geeky people are going headlong into this new territory where it is completely autonomous we invented new currencies and that concept of a good old-fashioned corporation with a stable hierarchies in managers so with the people believe it is in their rearview mirror. i don't believe that. acid is intended to mimic the force of nature it fell flat on its face before it was launched that you all

have to abide by that. so this is so poorly written i can treat it like the atm to dispense money to siphon off one-third of that total currency. paper co then that is completely kosher to do. but actually they unilaterally would refocus all that but it got me thinking it is not negative autonomous as we thought it was just like companies have a management. we are not in that post organization in post company world and you have to trust somebody. >> and relying on research

as much as you like to write that complete contract the reality is at sun label they are in complete and that is one of the major reasons with that ownership of assets so we don't see that technology will involve with all possibilities in advance. and to draw on those types of insights and with those

strong claims. >> i vaguely are low on time >> so a general intelligence with is that? most artificial intelligence is narrow. it cannot play checkers zero were tic-tac-toe or talking tidies. one of the babes they are working on is general intelligence and also know what restaurants to eat but that does not exist yet and it is not like it will anytime soon because those breakthroughs have not happened yet. some are working on what will happen to society with

these profound implications for the economy, a security security, military but though we can do with the technology is available right now with an amazing challenge and opportunity with hundreds of millions that are transformed the dishes from narrow artificial intelligence that is more of the focus so food knows how far in the future?. >>. >> getting to those targeted issues and what systems we should set up? is as

defined by the majority? and following up on the last two questions. is substantially different. every have companies have decision rights organizations at all they get here that will change substantially. >> if we do that and with those tools of recreate if we allosaurs that value then shame on us.

and then to have at categorically backer. to say computers are useless. and that machines are great at doing that. but double down. even more important going forward. so what does that mean? you have to change the planet and the world. it is more important than ever before. we can blow things up. so we have to think part of

the global population. and maybe the values question is essential. and that is something to focus on. >> i just have the question i figure there's someone in your book how they relate to machine learning?. >> with us corer technology available -- of approaches. such a have rarely allowed for it neural network gephardt. is a broad category is

loosely modeled the way the brains connect to one another with one dozen or more layers. and that is immensely powerful it to fit any arbitrary function. and that is what we have never touched before. and to be driving a lot of that. >> i believe the book is on sale. [laughter] check that out. [applause]

[inaudible conversations] >> tuesday i highly is strongly encourage whether they are young or old to expand your horizons and learn and gain knowledge so you don't necessarily have to go on a long trip i would

point shot out a couple of books with the autographs i collected over the years it just turns out the first autobiography i ever read was called my early life by churchill. plus one of churchill's riding were hughes's a lot of terms so i tried to integrate that without much success because he is such a great writer this is a copy of the only photograph of lincoln when he was in congress in just about every biography i can think of it is a one volume and that is

a recommendation. and to be as part of the constitution to enforce the constitution with james madison as well. but again one of those early dedicated individuals with the foundation of our government but now to answer your question i finished a book called the gatekeepers and it is this book right here. but i think this is of very good book may be slightly better with the chief of staff himself and said maybe

there were half a dozen paragraphs he could have thought about taking out but it was still worth reading. my staff is nice enough that whenever i read a book this is an example of notes. this book called the gatekeeper's was wonderful because when the good idea is not enough. the new way to build coalitions. and that is one of the best books i have read.

and a couple other books there was one on climate this cesses leadership but it was a spiritual dimension with the increase of trusted and joey and engagement it was leadership more emotional and intuitive and uplifting than some of the other books. those that testified before the science committee he is not a climate denier but this makes the point on climate change that i a greek instead of high taxes whether we let talk -- technology solve our

problems and the innovation and reduce curve in the mission? -- reduce carbon mission? to solve those communication problems and transportation problems why shouldn't it solve energy issues as well? and for climate change this is the next but i hope to read despite what some people say this was not given to me by a wife but i tried to improve and i never stopped editing consternation to my staff but whether the the op-ed piece or a statement i cannot read everything i

would like to read a really didn't learn to love to read and co college and having said that the u.k. and read it too much and i think i am getting to that point i am loving what i read mosley is non-fiction and it is just as important to stop giving. you can think about what you have read, live, how to do your job better, so it is important and it is important to read for a purpose and to think about

what you have just read maybe that will change your reading habits to say i am a proponent of that combination is potent. >> todd is taking notes while you read?. >> i like to underline the points that our important because if i underline the my remember them better because then i read them once then i underline the passage so that is helpful to me and then the four in turn gets to type up my footnotes the help they are learning and they tell me they are and then i give

those book notes to my family members, senior staff on committees i am not convinced babydoll of them but i do take they read some of them but i underline those massive -- passages they think are important.

>> host: on the campus of ucla and we're talking with professors who are also offers teaching history professor stevenson would you teach?. >> this year on women's history and their dynamics. >> host: how long have you been here?. >> 26 years