it's also my pleasure welcome

yuval noah harari this evening to discuss his new book nexus a history of information networks from the stone age to a.i. harari is an academic, historian and philosopher for his ability to take complex history and present in an accessible manner and nexus he turns his attention to the existence crisis we face in the age of artificial intelligence, providing both the historical on theories of information, as well as a warning about. non-human intelligence. if, say, if we sapiens are so wise, why are we so self? he asks. harari is the author of five books, including the bestselling sapiens, and he's considered of the world's most influential intellectuals.

his a lecturer at hebrew university and a fellow at the university. cambridge's center for the study of existential risk. he has also co-founded the social impact sapiens ship. tonight he will be in conversation. nicholas thompson, the ceo of the atlantic. thompson is worked as the editor in chief of and as the editor of newyorker.com. join me in welcoming yuval harari and nicholas thompson. hello, yuval how are you? i'm fine, thank you. it's a real pleasure to be here with yuval. he is not only a great historian, as you all know, he is a very kind in the green room. among his many duties, signing

books, answering my about israeli politics. he also helped read a bedtime to my ten year old, who is a huge of unstoppable. so thank you, yuval? it's my pleasure pleasure. all right. so what i want to do in this book. is i want to go through some of the history, some of the stories you tell. i'm going to ask you a few questions about characters you introduce, the ideas they represent. we'll go through some of the arguments you make about history, about ai and about democracy. and then i hope there'll be some time at the end to go through some your apocalyptic thought experiments. does that work? yeah absolutely. and i have a whole series of questions, several thousand that people have sent. i appreciate that. we'll get to those as well. first question, what is that bird. oh, that's a difficult question. it's a pigeon. i mean, thank you. all these what does the bird represent and why is it on the cover of your book about the history of the world?

hmm. well for two main reasons. first of all, in hebrew, there is no difference between a pigeon and a dove. i know that in. in english, in the english speaking world, pigeons often called rats with wings. whereas doves are thought of as these white angels of. but in hebrew they are the same. and they'll know doves in the middle east. maybe this is why there is no peace in the middle east and. there are only pigeons. and in bible in the story. noor and the flood. so sends a pigeon, not a dove to save the flood is. so we are now living in the midst of a flood of information. and this is kind of my pigeon been sent to see if there if the flood is over. and the reason is that one of the chief characters of the book is a pigeon called the sher amin, which was hundred years ago, was basically most famous

bird in the world. and i think he is still on display. the smithsonian not far from here and until the story of shami because it's actually i think important to the to the arguments you make in the book. yes. so how did this bird become the most famous bird in world? so during the first world war, when the expeditionary force fought in northern france against the germans. so an american battalion was caught behind german lines, surrounded by the germans and american artillery, which tried to provide them with cover, didn't know their exact location, and actually dropped the artillery right on. the american soldiers. adding to the problems. and they try to send runners to division headquarters is to inform the commanders where the battalion actually is. but none of them could get through the german lines. so they turned the only thing that could, which was carrier

pigeon ceremony. and the the commander wrote this tiny note on a piece of paper and attached it to the leg of the pigeon. and they released it to the air. and the pigeon flew through the german fire. it was hit several times. it lost the other leg. luckily not the leg with the with the was the note. it was shot through the breast. but it nevertheless managed to get through and artillery barrage was lifted and help was sent to the right place. and the battalion, which was known as the lost battalion, was saved. and the pigeon, which called shir amin, was then hailed as the the bird that saved hundreds of american soldiers from death or, captivity at the hands of of the germans. at least this was story which was repeated again and again in army communiques, in the newspapers. there are movies, are children's

books. so if you want to read story, there is still a children's book about share army the brave pigeon. a recent historical research that delved into the raised a lot of questions marks about this whole story. first of all it now turns out that the headquarters learned about the right of the battalion before the pigeon arrived and. then it turned out that nobody is sure the pigeon was actually jeremy, but it could have been a completely different pigeon. but still, the the all the terror meme was displayed in the smithsonian for years and became a pilgrimage site for veterans of the first world war and was the most the most famous bird in, the world and this is part what the book is about. the power of information on the one hand.

and the tension between the truth and the stories we tell and the effectiveness of stories which are not necessarily always truthful. and that's that's why it caught my attention, because as you may have heard in america, we still have of these issues. and in fact, often with animals animals. and so i'm going to read quote that one of our modern philosophers dealing with the same issue, jane, events said yesterday, if i have to create stories. so that the american media actually pays attention, the suffering of the american people. then that's what i'm going to do. you are in favor of that, correct? i'm not. i think i'm in favor. i'm saying that this is what is happening throughout human history that again, most information is not the truth. the truth is rare and costly

subset of information. if you want to write a true story, you need to invest a lot of time and effort money. whereas fiction or fantasy, they are very cheap. you just write first thing that comes to your mind. the truth also to be complicated. because reality is complicated. whereas fiction you can make it simple. as simple as you would like it to be. and people usually prefer simple stories and the last disadvantage of the truth is that the truth is often painful, whether on the individual or if you talk about israeli politics, the on the national level, the many things people don't want to know about themselves, about donate and donation, about the world and, you know, fiction can be made as flattering as you would like it to be. so in a competition between

something which is cheap and simple and flattering, and something which is costly and complicated and painful, it's obvious who is going to win unless give the truth some help. this is the exact argument made to my board yesterday while i'm trying to money for our fact checking department. but let me ask you then about the question of a noble lie, because it's clear that history is determined by the people who tell the stories which have either truth in them or they don't have in them. and you are not as of the idea of a noble as i expected you would be when i began the book. explain what circumstances it is okay for someone who is telling a story to not tell it exactly truthfully, for the greater good of some kind. it is impossible for a story to be an exact replica of a

reality. you know, there was a famous bull story about an empire that wants to create a complete, totally truthful map, which will be exact li and exact represent of reality. and they end up producing map with a scale of 1 to 1, because this is the only map which will be 100% truthful and accurate and will not are simplified. anything will not change anything. it will have to be a 1 to 1 map. and so suppose the empire was covered by a map of the empire and the effort of of of creating this empire. is this map exhausted. the resources of the empire, therefore, collapsed. and we are now in a similar situation to some extent. there is a crisis of representation in the world, but

no representation seem good enough for us because no representation can actually be a 1 to 1 map of the world. this simply impossible and we are not sure what to do about it. now my my position. is that every story? to some extent is fictional and every story you can't tell the whole truth. this is simply impossible. and there are cases that, yes, you need to simplify as somebody who wrote, you know, the history of the world in 500 pages. i know sometimes you have to simplify, but and a fiction in itself is, not necessarily bad, you know, the rules of football or baseball, though they are fictional, we invented them. it mean that they are bad fictional literature is not all bad. the key is fiction should acknowledge fictional city and not pretend to be reality and to

to to. when you think about the cultural or political implications of that, if you want unite a large number of people you need to use some stories some mythologies and this is not necessarily bad as long as you acknowledge what you're. so if we compare for instance two foundational texts of human history. if you compare ten commandments and the constitution. so one text acknowledges its fictional city and the other doesn't. the ten commandments doesn't that it emerged from human imagination. it claims to be the product of divine intelligence to come down from heaven. and the downside of that is that it has no mechanism to admit and

correct its own mistakes. and, for instance, the commandments, as they were written sometime in the first millennium, bce, they endorse slavery. many people don't think about it or don't notice it. but the 10th commandment actually endorses slavery, says it's to hold slaves because the ten commandments says that you should not covet your field or ox or slaves which implies that god has no problem with people holding slaves. god has a problem only if you covered the slaves of neighbor. no, no, no. that's not okay. that may god angry and compare that to the us constitution, which also, like the ten commandment, have served as the basis for large scale cooperation for legal systems. political systems. whereas the ten commandments start with i am, your lord god.

the us constitution starts with we the people. we the people. invent this document we invented laws. and therefore, because it acknowledged, is that it emerged just from the human imagination. it's humans who wrote this document. it also acknowledge the potential that there might be mistakes in the document. and it has a mechanism to amend itself, which was eventually used. amend the us constitution, which again initially endorsed slavery and was eventually amended to forbid to ban slavery. whereas with the ten command ments because they claim to be just, you know, they came from heaven, there is no mechanism to change the text. there is no 11th commandment which tells people, well, if you don't like something in, the 10th commandment, if you have a

two thirds majority, you can change text. no, there is mechanism. i, i think you i would think it might be possible for pope francis to change the second commandment better than the u.s. congress could, change the second amendment. but that's where we are. all right. let's let's let's stick with religion. i want you to tell the story that i think comes up in at least three very consequential parts of the book is the story of the council, hippo and carthage and the consequences of selecting one timothy into the new testament instead of the acts of politics where you explain that story. i think it's a very important follow on to what you were just saying about the power of story. and i swear to god. we'll get to air in just a minute. but this is an important premise for that. i think we can actually with amy, which is very very relevant to the could the church council of carthage, which took place in what is today tunisia in 397 c.e. because 88 one of the first

big things that that we saw it to do that shaping human history is taking the power of recommender station that if go on social media what do you see is the result of recommendations made by social media algorithms and the power to recommend to people what stories to read or what videos to watch is extremely important. and one of the best examples in history the power of recommendation is the editorial process that created the bible in the new testament. the people who created new testament are not the author of the texts. they, the editors who decided what will be in and what will be out because know there was no new testament, there was no bible in the time of jesus or in

time of saint paul. they never read the bible they didn't exist in. the first four centuries of christianity christians produced an enormous number, enormous variety of texts. those stories about christ the prophecies, doomsday prophecies about the apocalypse, the by saint paul, by other church leaders. there are a lot of fake letters that people wrote things the name of saint paul, like 200 years after the men was dead. so christian communities were going flooded by very large number of texts and a question arose what should good christians read they needed a recommendation list the same way that today we flooded by tv series and we need a recommendation list. what to watch so in the late fourth century, a committee was set up a church council

theologians, bishops. they met first in hippo in what is today think algeria then in carthage in what is today tunisia. and they hammered out a recommendation list top 27 texts. every christian should read and this became the new testament they didn't try to text they went over again very large numbers of texts existed at the time and, chose what will be in and what will be out. and this was this shaped christianity and the views of billions of people, numerous issues until this very day. and to give you just one example out of many, so one very popular text with christians at the time was the acts of paul thecla paul is saint paul and thecla was one of the most. favorite saints the time she was a woman disciple of paul and she was leader of the community.

she preached. she performed miracles. she baptized and she was hailed as an example that women can be leaders in the church and women can preach and women can baptize and perform miracles so and so this was one popular text with one view of women. then there was another a letter allegedly from paul to timothy, which most scholars today think is a much later was not written by saint paul in the first century, probably forged his name sometime in the second century, and in this letter, a completely view of women and the role in the church it that women should be obedient, should be silent, should never be leaders they should fulfill themselves by doing whatever men tell them to do and by having children and raising children. this is their role in life.

and the committee in carthage decided exclude the acts of paul and thecla from the top 27. but include this letter timothy which is still part of the new testaments around the world. first timothy and this is shaped the views billions of christians about women in the church and also in general for more than 1500 years. this is the power of recommendation and now this and this connects to a i. this is the power which is increasingly held by algorithms we have now with this kind of huge debate about, social media and the spread of fake news and conspiracy theories and so forth, and social media. and you'll hear people like, elon musk or mark zuckerberg saying that we don't want to

censor anybody, that this is an issue, freedom of speech, but it's not the problem with spread of this type of information on social media is not human users producing certain lies or fictions or fake news. the real problem is parait algorithms deciding which stories to recommend which stories to promote, but then the power that was held by the bishops in the of carthage and the power was held by newspaper editors in recent generations. this is now the power in the hands of the social media algorithms and. this should be at the center of the debate, which we'll get to on regulation. it's not about the freedom of speech of it's about the responsibility, the corporate

algorithms, because if the corporate algore then decides to promote a certain conspiracy, this is not on the person who invented it this is the decision of the algorithm and the decision of its human corporate master was and this is what should be at the center of the debate. so what hearing is if the council of carthage had been slightly different, women would have been empowered much sooner. we would have had a feminist several centuries earlier. probably a.i. would have been invented and we all would be obliterated by now. is that correct. yuval, let's one possibility. i mean, you never i mean, history is extremely. yeah, no, i know. yes. so you can never predict the outcome. you can't unspool one thread from the tapestry. let's go to some of the concerns have about i want to go through some of the concerns quickly and then i want to go through your philosophy of how these algorithms should be structured. but very briefly, in just a word or two, explain modern a.i.

might destroy democracy. oh, very briefly. well, very briefly. democracy is a conversation, a dictatorship is one person dictating everything. that's a dictatorship. democracy is when a group people have a conversation in order to decide what to about any major question. now to have a conversation is not an easy thing and the technical difficulty if you have 20 people trying to have a conversation so they can all gather in in a room and talk with each other. but how can 20 million people have a conversation you need some kind of technology in order do that now until modern age there just no technology to facilitate large scale

conversation which is why there were no large scale democracy anywhere in the world. the only examples we have of ancient democracy since they are all small scale. there are 50 states like athens or republic and rome, just one one city. although even smaller tribes and bands and villages with many examples of small scale democracies, not a single example of a large scale democracy in the ancient world. all large scale polities are authoritarian we begin to see large scale democracies only after rise of modern, informal technology, the first crucial technology is newspaper and we have the telegraph and, the radio and the television. and suddenly it becomes feasible. all the other conditions that have to be met.

it doesn't guarantee democracy, just that you have a newspaper, you have newspapers and radio also in the soviet union. but it becomes possible for the first time in history to have large scale democracies and it's important to understand that because it means information technology, not a side dish that you have democracy. and on the side you have these all issues of information technology, no information technology is the basis of democracy. democracy is built on top of technology. so any major change in information technology is to cause an earthquake in democracy which is built top it. and this is what we are now seeing all over the world what we are seeing all over the world is the collapse of the democratic control. we have the most sophisticated

information technology, human history and people are losing the ability to talk with each other, and even more so to to each other. and in every country where this is happening, there are these explanations of what is in our country. why can't democrats and republican in the us have a conversation anymore? and you go israel to my country, you hear the unique explanation of what is wrong with israel? and then go to brazil. and then you have the unique explaining of what's wrong with brazilian society. but you see, the same thing is happening everywhere. the conversation is collapsing and this is not of some special feature in the history or society or economy of the country. it's universal earthquake, which results from the rise of this new information technology that originally the developers, the

technology promised to us, that it will spread the truth and bring tyrannies down and strengthen democracies. but is doing the opposite and very briefly one way to to visualize what is happening is imagine that democracies the group of humans standing and having a conversation suddenly a group of enter the and start talking very loudly very very emotion and we can't tell who is who who is a human and, who is a robot. this is what has been happening over the last ten, 15 years. and the result is that the conversation breaking down and again this is not a uniquely american phenomenon. it is happening all over the world, leading more and more to the rise of dictatorships, because dictatorships don't need

conversations again, is one person dictating everything? well, let's let's push on that assumption for a second. so i could have a different metaphor which is not a bunch of robots enter the conversation but a bunch of infinitely aides join me and they help me sort through the conversation and they help me prepare for what i'm going to ask. and then not only that, if you look at the last year, obviously venezuela tragic example, but we had elections are not the same as democracy, but we use them as a proxy for this. this hypothetical. we had reasonably positive elections. if you're in favor of democracy in poland, we had a little bit of progress in turkey, serious progress in india, in fact, where and illiberal democracy has actually been challenged. we had an extremely smooth election, the election of a jewish woman in mexico we had extremely swift and effective no deepfakes maybe don't like the outcome of the election in france similar election in the united kingdom i mean the

world's just doing all right on these despite the challenges of social media everything else and it's not that it's already a deal. it's not that democracy is collapsed. but if you look at the health of democracies today compared to ten years ago or 15 years ago, at least the momentum is very worrying. and again, it's not a i don't think what surprises me when they look at the example from around the world is that it's not about some huge ideological gaps actually ideological gaps between the different camps today, a place like the us are much smaller than 50 years ago. what worries is the kind of temperature of the of of of the argument and again this inability to have a reasoned debate, to have a reasoned conversation simply, having is important, but it is not enough

elections, not democracy. and even in of the examples you mentioned. so it's not about who wins, which 51% win. in the end, it's the relation between the 1% and the 49%. and democracy shouldn't feel like every election is a life and death struggle that if we lose this it might and even more so it shouldn't feel like a war between enemies. but if a country, a situation when people view their political rival as enemies, then democracy cannot survive for because then every election and again it's like a war you anything to win. if you lose you have no incentive to accept the verdict. if you win you only take care of

your tribe. what is happening in this situation that? a nation breaks down into tribes leading eventually either to tribal warfare and civil war or to dictatorship. that i think that the the key thing here also has to do not just with democracy, but also with nationalism and with the breakdown of national communities that many people think that democracy and nationalism democracy in patriotism as some of opposites. but they go together. they must go together. democracy functions as well. only when there is a national community. only when you feel that really care about. the other people in your country and that they really care about you. if a nation, which is the point when there is no longer a nation, the warring tribes and each tribe cares only about

itself, then it's only a question of before democracy collapses. and this is really the worrying that we see in many places around the world irrespective of the results of the latest elections here or there. and this, again, it goes back to the type of communique nation between people. can we, for instance, listen to people with different views, our own without thinking that, they are enemies. i learned in the room that one of the ways that you've all arguing with bots on twitter is he, as he says, as he is stuck in nineties and uses email and the telephone. let me ask you a question. all right. so we're headed into this age. i will certainly agree that democracy is at risk and i will certainly agree it is at risk for the reasons you give. and i will certainly agree that we will soon have extremely powerful algorithms that will underlie a lot of the decisions

we make and a lot of the thinking that goes on our own heads in. a very important section in the book you lay out for values that you think should be embedded in air systems, and they are benevolence right. they should air systems should be benevolent. it seems reasonable. decentralize and mutuality. you should understand it. what it understands about you and the ability evolve, much as you said about the us, the us constitution versus the ten commandments. so a question for you. what happens when some of these come into conflict each other? so when i read that section, i thought, well, benevolent is kind of his intention decentralization? because if you decentralize these algorithms and suddenly you have all kinds algorithms, you've all kinds of options you have all kinds of different companies, some of which will be benevolent. you define it. some will not be benevolent as you define. well, how is one supposed to weigh these four principles for designing future ai systems because weighing them correctly seems pretty important to getting the world you want now. was benevolent saying mean

something very, very narrow and. you know, it's something we've known for for centuries, it's something we have known for centuries and if you get ahold of my information you should use it for my benefit and not in order to manipulate me which is awhle basic principle we are to have. her doctor's and lawyers and accountants and our therapist and it should be no different with the people who provide us with these services like social media or like e-mail. >> facebook would not have said it's manipulating us even when i was in its heyday for giving us what we want. >> that's one way of putting it and the facebook algorithm has enormous power over us and facebook's business model and the business model of most of these social media companies, it

is based on increasing user engagement. an engagement is sounds like a nice thing, who doesn't want to being gauged by the for the companies that means we need to spend more time on the platform echoes the more time i spend on it the more money they make. they show me advertisements and commercials by collecting my data and then using it or selling it to a thirdd party. this is what i want or not that's a very big question. do i really want to spend more and more time on the platform? with their algorithms to is by trial and error they find my weaknesses and they use my weaknesses to keep me glued to the screen for longer. it's like hacking. how do you hack a computer or

program to look for the weakness that's in the code in the same with human beings. it is how you hack human beings. you use these algorithms to find their weaknesses. each person with their own weaknesses, what makes me angry, what do i r&d? what do i r&d feel for they give you more and more of this. it's like the food companies. they learn if you pump something full of sugar and people want more of it. whether it's good for us or not. so there is a question here and it's not an easy thing to solve but but this is the key of the dilemma the key of the discussion in the main message is it should ultimately not be about the profit of the corporation.

whether consuming all this information is really good for me or not. and why must point likening the example of food we have reached a point where most people need an information diet. the simplistic idea of more information is always good for us is distinctly wrong the same way that more food is not always good for us. and that's why everybody should subscribe to the atlantic. let me do a thought experiment because part of what i think is going to come for your system to come down to who defines benevolence and who defined by you i want to give you a thought experiment on a conversation i had not long ago. i'll change the details because i can't reveal who they are or what exactly they do but it was an engineer who works in ai and they make algorithms in their current job is that they work for the state of texas and they are in charge of ai algorithm

that would determine how someone -- how long someone could be sent to per prison to get to make sure any algorithm has the biases for the historical data. if it's you have to control for. just make sure it's proper and you control through it so having a long conversation and asking herw how do you control for tht in this and that and she said do you know what they do next? i break the algorithm and i break it in such a way that everybody will be sentenced to less time than the state of texas. the state of texas will never figure out that i have done that. is that something that an engineer should be doing? that's extremely dangerous. it's not only dangerous but it's a value that many liberals agree with andat they feel conservatie

states have since people for too long so it's embedding the value that person feels benevolent into the system leading to what they believe is justice and at the very least home we are talking about the law of the country it should be left to the citizen to the voters and not the dictatorship of an engineer. >> they have been given the power to hire whatever contractor they wanted to stay in the elected collected the governor of the state of texas and they have hired this person. the state is not texas where this is happening so don't google it. if we are seeing it in different places aguilla figured out but it's an example. >> the key point raised by this example is the issue of the unfathomably and to what extent we can still understand the system that controls our lives. i guess the question is what frightens me about ai and you have these hollywood scenarios

of the robots rebelling and they areel becoming pillars and thiss unlikely to happen anytime soon but what isal already happening and this is not a science fiction scenario for the future, what is art happening is we basically have millions of ai bureaucrats millions of bureaucratic algorithms making more and more decisions about our lives and what bank to get a loan. ai decides and they sent us to prison increasingly ai deciding for how long. and this could rise to the level of the key economic and financial decisions about what would be the rate of interest of the federal reserve. it's would increasingly be a decision made by ai and not human beings. for many g good reasons to give

this kind of power today i but what happenspp down the road whn eventually so many of these crucial decisions about us are made inad the way that we simply cannot understand. we don't know why the bank refused us alone. we don't know why they sent us -- sentence us to five years and not four years or six years imprisoned and we don't know why the adjuster is that 4% in a 3% and not because somebody is hiding it from us. it's far too complicated for the human brain. the advantage of ai and the good side of ai is that it can analyze much more data than any human brain to deal with mathematical complexities which are way beyond what we can deal with but the downside of all that is what is the meaning for

democracy if increasingly all the decisions for many of the decisionsy are made in a way which are not transparent and therefore not accountable? thee let's go to another win which is somewhat similar. when the companies that makes a systems that make products they use a system called constitutionalse ai and how they write their prompts and structure the mouth which is probably the closest of the major ai companies as i understand it the philosophy of yuval harari so it checks whether -- the u.s. constitution and the declaration of human traits and apples terms of service. f but in any event the biggest problem they have and the other ai companies is they don'tt knw why so they have been trying to understand what's called the interpreted ability --

interpreted ability. they said let's see what happens up with better training data and add extra weight to everything that have to do with the golden gate bridge and there's a picture of the golden gate bridge and mention of the golden gatea bridge and this box score about the san francisco giant and more than one about the philadelphia phillies. they do that and then they ask to tell me a love story and the story takes place on the golden gate bridge. if you mess with the data you get interesting outcomes. the question for you would be if we can do this why not go into the ai and not weighted towards the golden gate bridge. weighted towards love, passion benevolence. is that a good idea? thee i'm not sure what it means in the technical sense but when you look at h human history i kw quite often history you people

talk about love and you start with love and very quickly they get to into war. christianity which goes back to the conflict is itself as the religion of love and it's all a about love and responsible for all violence and the other religion or ideology in history and they somehow found a way how out of love we wage a crusade can rebuild in positions and burn hair tags -- heretics at the stake and they really believe it. the way i think it often happens is if you think that you are motivated by love and if you

think you are trying to build utopia whoever stands in your way must be. whoever stands in m your way mut be. the more good i am in any opposition is definition not just someone who thinks differently but they are so i'm not sure how to translate that. again wonderful lessons from history is that just thinking becausein we have these good waits, we equated our quota in favor of love. anything that has love has extra and somehow from that you get inquisition. so if you happen to humans i would be very worried about ai the basic thing again we learned again and again from history is

we need a self-correcting mechanism. we can't trust just because something has a good value and a good basis what will come out of it will necessarily be benevolent and compassionate. in the modern age we start with marxists which began with these wonderful ideas of equality and compassion and end with the gulags. if you are really convinced that you are coming from a good foundation and also if you think during the process of building utopia he gives you an open check to do the most things on the way to utopia. anything that stands in your way is then transformed from political rivalry to kind of possession. in some instances when of the scariest things you said because

the people who are building the aili models generally believe ty are leading us to the future and that's very dangerous because this is what gives them anan opn check to say we are building utopia so anything we have to sacrifice on the wait is worth it because when you look at the bottom line the basic argument people like lenin and stalin we have to merge these millions of people but in the end when we utopia real existing socialism here in earth it will turn out that these things were worth it. i had never thought of that. the khmer rouge topic. here wet are. in the next couple of weeks i think you were in tonto and you will probably move west and sit down with all these people because they all read her books. jeff bezos was in this interview and it got like four books behind him on his bookshelf and

plantsts so you are going to be talking so what you're telling me is we are trying to figure out and what we should train it on i'm thinking we should train it on positive uplifting factual stories. i don't think we shouldwe traint on things likers serial killers diaries in the say no, but but s serial killer diaries in their. don't just focus on your vision of love. so if you are arguing? thee no i'm saying i don't know how to train ai that no matter what kind of positive basis we give it no matter what positive intention you have the number one should be your number one the assumption should be this thing is not infallible. i'm not infallible. there's a high chance for mistakes and therefore i need to

leave room for correction. the most important thing when youg build it you build a mechanism for identifying and correcting a mistake if this is also the advice i wouldou have given to anyone that you are going on this huge, experiment and you are building utopia with the assumption you will make mistakesie including in the infrastructure the soviet union mechanism of identifying correcting the mistakes of the system includingst the mistakesf whoever is going to succeed you which is the one thing they didn't do. lenin should have read it. i've been image of him reading it on the train. let's talk about cocoons which is wanted extremely interesting parts of your book. you make an argument can you say their fundamental of mind and body and you tell the story of martinin luther and the way i've

never read before but you argue whatt could happen in an age of ai is you end up with some civilizations that have been totally different understanding of what is the mind of the body and totally different judicial systems. thisis is when of the most mindbending parts of the book and i'd like you to explain it to theau audience. a lot of explaining a try to keep it short. th' cocoon is the changing of the metaphor in this age of information that 30 years ago the dominus metaphor was the west -- the web the world wide web and the web was supposed to connect everything and everybody and over the years the web kind of closed in on us and now it's the cocoon that every person or every group is enclosed within an information cocoon and sometimes your next-door neighbors are in a different cocoonha then you and there's no

way to ask someone in an extreme example of where this could go today peopleop don't agree in te united states about who won the last election. this can go to a place where people don't agree what a human is or what a person is and what are the relations between mind and body. one of the recurrent arguments throughout history that we see in many traditions in judaism and christianity and hinduism and birgit -- buddhism is what exactly is a human being and what is the relation between mind and body or between soul and and body. beginning again we go back to the age of earlyy christianity judaism in the first christians which came out of judaism they

viewed humans as embodied entities. the body was central. we are bodies. biblical judaism doesn't talk aboutll the soul. the idea that there could be a soul that exists separately from the body if all about the body and the first christians focus on the body and that is incarnated in the flesh. the flash is at the center and after bodies crucified he supposed to come back in the flesh in the kingdom of is supposed to be a material kingdom of bodies on the earth. eventually for all practical

purchases the big problem for christians when you are persecuted you can have all these promises okay when we finally gain power then we will have the kingdom of than they have one of the biggest disasters that can happen to any religion to gain power. they become the religion of the inland empire. so where's the kingdom of? there's the kingdom of. you still have the same wars and corruption and civil wars and executions in human greed and it's all the same. so they say okay the kingdom of is on a different level of reality in heaven. and after you die your soul will go to heaven and many christians

have very different view of humans that my real soul isn't trapped in a material biological body with all these passions and all these lofts and the idea is eventuallybe the soul will be released from this earthly prison and get to a purely immaterial round which is heaven. for it will exist forever and ever and throughout the 2000 years of christian history you see the extension. they can never really abandony the body partly because the full

of -- alive in the flesh and there's little doubt the round in the itself. so there is this constant argument that often leads to and wars of religion in an early century when of the biggest arguments was about the nature of himself with one camp saying he was entirely a human made of flesh and another camp saying that he was entirely designed and nonmaterial spiritual beings and there was a third w campus said he was non-binary. this was eventuallyy the officil dr. and. he's a non-binary. he's both at the same time. the huge argument around these

issues. how does all this relate to ai? we are going to have another round of this mind-body space and we are already in the midst of it. what is your identity? what defines your identity? is your identity defined by your biological body or is your identity defined by what you believe, by your faith in people like martin luther say the only thing you can measures what you believe. we are living in a new round of this debate with some people saying if you go on line you can say whatever you want. the biological body in front of the screen should not limit the

identity that you can adopt. other people say no there's a biological body and this is your identity and you cannot ignore your biological body. everybody knows this right now and it also influences how we treat ai. ai has more bodies. they'll increasingly bailed and direct with us and to press our emotional emotional buttons in japan tend to have a motion themselves.ho people who give primacy to the body and the identity of the person will resist treating ai as a person. people who think identity is related to biology will have a much easier time giving or

treating ai as a person even though they have nobody. different countries can go different ways. arguments about human rights today between the united states and china what does it mean in 50 years when he perhaps that billions of entities which are considered personal in and one country but in another country they don't recognize them at all because they have no biological body. at least in the west intrinsically enough there's already a completely open legal part to recognizing nonhumans devoid of body as legal persons because corporations according to u. s. law or legal person. according to the supreme court inhi 2010. this is a legal fiction because

all of the decisions of corporations are made by human beings with a biological body so google according to u. s. law is a legal person but at the present all of the decisions of google have to be made by some humann being. what happens in the few years maybe when you start incorporating ai as a person. you can technically incorporate ai is a corporation. let's call a google and the interesting thing about google is it doesn't need a human employee to make the decision forr you. ai can do it by itself. the ai can open a bank account and can start earning money and you can go on to ask rabbit and offer services to people or

corporations and it could potentially become thee richest person in the west but think about the situation where the richest person in the u. s. is not a human being. it's ai and according to a anderson wanted to rights reserved for this nonhuman person is making political contributions. so ai could make billions of dollars for politicians in exchange for giving more rights to ai. these are the kinds of science fiction scenarios i think we should be more concerned with.

of the senior ai bot starts to hit on junior ai bot in this corporation what should the ai h.r. duplex we have a bunch of flesh and blood body here will have to get pizza and. i want to ask one more questionc about theso apocalypse and willo to the ai questions but i was reading your book one the subwy and wrapping it up before dinner the other night with my son and this is myou fair graph -- thiss my favorite favorite paragraph i read to you. we have now created a non-conscience but very powerful alien intelligence. if we mishandle at ai might extinguish not only the human dominion on earth but the light of consciousness itself turning the universe and to rome of utter darkness. a cheerful book here. it is the responsible of me to present to death prevent this a one-way to prevent this is to

prevent ai and another would be to create consciousness and send it to the universe. explain to me a question that this paragraph races is what's the thing that humansco could do that would allow consciousness tose exist even if weeks and was ourselves? i'm not sure what it is. the problem is we still don't understand consciousness and we don't know how it emerges in us. we don't know how it evolves and with regard to ai the big question at ai consciousness and i tend to be agnostic about it, i don't think ai can necessarily develop consciousness. i'm not sure that they will never develop consciousness. in this scenario that ai could destroy human civilization and takeover and maybe threaten

earth and other galaxies but in the process they never enveloped consciousness. this is the dark universe phenomenon. there's a huge confusion about these two terms because in humans they go together. intelligence is the ability to attain goals and solve problems on the way to that goal. consciousness is the ability to feel things like pain and pleasure love and. humans solve problems relying on our feelings and if you ask any other mammals are animals consciousness and intelligence go together. with computers so far we have only seen a huge advance in intelligence without any advance in consciousness as hard as we can tell. in some narrow fields ai is far

more intelligent than us and still has nos consciousness whe wintuk in the chest it's happy when it loses its not sad. doesn't feel anything. in many scenarios as ai becomes more and morell intelligent at some point against consciousness. but there is no reason to think it's inevitable. there could be different roles leading to super intelligence. mammals and humans have been traveling along roads for millions and millions of years and the roles involve the development of consciousness. computer ai might simply be traveling along a different road a much faster road which reaches super intelligence without passing through any developing

consciousness. if this happens and if it gets out of our control this could spell not just the end of the universe but the end of life itself. it can have a galactic empire without any feeling is just all dark. >> but why is that? explained to me why that's a worse outcome than just the obliteration of humans because they are plenty of bad outcomes. >> there are other entities, conscious entities in the world. their other animals. there's no reason to think the history of billions of years of life that sapiens are the last weather through biological evolution or whether through some kind of combination with ai. it'st' very likely if it survivs i don't think humans like us

will still be here in a thousand years or 10,000 years. the technology will be so advanced and there will be beings but they will have completely have different bodies and minds and different experiences. this is not necessarily bad. the same way the fact that we are here in the first human species that existed 2 million years ago are gone we don't think about this as a problem. most people with regard to their children they hoped those children will be a bit more involved than that. to think this would be completely wiped out it will still be intelligent. no consciousness. intelligence is overrated. the really important thing in

life is not intelligence. people do different things but it's about our consciousness. >> do you know who still going to be there? you are going to be there. you begin your book by arguing that humanityhu is cool -- i'm sorry has the most defeated quote hunger plight in war india still stand for that i.d. or do you consider that in your argument? >> i think humanity has the capacity to rein in famine plagues and four. whether we do it depends on the decision and we have been making decisions over the last 10 years which is why we are seeing a return of these calamities in these conditions.

we are now on the verge of a third world war which if it happens is likely to be accompanied by famine and potentially by a plague. and the key thing to understand is the decline for instance of war in recent generations was not the result of a change in the laws of nature and it was not some divine miracle. it was simply humans making good decisions and building the good institutions. if we start making bad decisions and neglecting institutions that than war returns with this question is from -- there's growing concern regarding the impact of al -- ai unemployment especially low-skilled jobs and low-skilled areas. what strategies are measures can be implemented to alleviate these negative effects? the safest thing is to slow

down. i think humans and human societies are extremely adaptable but in order to adapt you need time. it's 10% or 20% of people lose their jobs is a huge political crisis. if it's more drawn out over several years we have time to adapt. the most important thing is a global level because when i look at country and i say i'm worried many jobs will disappear in the coming decade or two and other emerge.l the question is whether people will be able to retrain themselves to fill the job and for that they will need thought and the countries that lead the ai revolution they will have limits results to support the retraining of theos population d

also to support those members of society that will not be able to go through the transition. the big problem will be in other countries that might face complete economic collapse and will not have the resources to retrain their workforce and to adapt to the new ai economy. >> what do you i think of the premise of the questions that ai will affect employment particularly for low skill jobs in low income communities. if you could argue both sides of that. >> it will definitely impact high income jobs accountants lawyers doctors engineers. there's no reason to think it will focus specifically on low-income jobs. and you will probably wipe out media ceos first. this is from mike pick my favorite thing about your rhetorical writing is it's not just how we live but how we

should live. if you were a lifestyle guru what message are recommendation that you have for your audience and i'm not a lifetime guru so i don'tow know. this man meditates two hours a day and takes a month-long retreats. sad say it but i don't tell all people to do it. i know what it means it works for me but it doesn't work for other people. for somebody else it's better to take a hike in the woods so do that instead. the idea that meditation works for me and not for all people. >> i tried it and didn't work for my employees are my three kids. how do you think ai and misinformation will affect minorities especially those in the ocb tq plus community. sega depends on the decision. work both ways. in my own life social media has been wonderful in many ways.

i met my husband 22 years ago in front of the first social media web site per day people in israel and it was really a revolution because if you think about minorities in history there two types of minorities. you have concentrated minorities and you have dispersed minorities and concentrated minorities you think of the jewish community. if you are born jewish in europe in the middle ages there aren't many around that you are surrounded by them. you were born into a jewish family in a jewish neighborhood of jewish community and you know a lot of jewish people so you have no problem finding other jewish people. i grew up in the 80s and the 90s in a very humble phobic

society and i was born in the suburb of haifa and i didn't know anyone who escaped. this is a dispersed minority. most boys are not born to a gate family negate community and sometimes it happens but it's very rare so the first question you encounter is how you find others. it's a question that jewish people don't have to do with the day people have to deal with that throughout history. into the large extent it solves they, problem and suddenly it's easier than before to find each other. i often criticized in the potential of criticism on information technology and social media but they also have policy -- if you could resurrect

one others species of human you could what would it be? the way that our species. ourselves just because we have a little bit of different language or o skin i would like to be ina minority in a homeless safety and world. spring shed aib raised with human parents of a curated developmental trajectory to create a sense of love and attachment in connection on the assumption that it's possible to fill ane moral code that humans should try? >> ai is inorganic. you're not where you can say i will bring a into the human family so become humanlike. this is the mers -- misperception we have about ai.

we have these people asking when will ai reach human levels text answer is never. it's not on the path to human intelligence and is not even organic. for me a ai traditionally did for artificial intelligence. i think it shouldi stand for alien intelligence. alien not in the sense of coming from another state but in a sense it's intelligence but it makes decisions it processes information and it creates ideas in this a fundamentally alien way but it's not organic in wondering important thing is organic things work bicycle day in the winter summer and sometimes we are active and sometimes we rest.ed when a problem is we encounter more and more is that now the

world is increasingly run by the non-organic t intelligence that never resurrects and they pressure us to be the same. instead of them becoming like us they pressure us to be the same and if you force an organic theme to be on all the time eventually it collapses and dies. you think even about something like a financial system traditionally the financial system is an organic system which sometimes a break. the market is open from 9:30 to 4:00 we the weekend before and christmas it's close. if you give ai greater control the system is always on in this puts pressure on humans and the

bankers to be always on which is humanly impossible in and the same thing is happening with journalists and to politicians. which consequently collapses. i often say the most misunderstood word in the english language release in the united states is the word excited. people overuse it and the good thing they meet you and they say i'm so excited to meet you. if you publish a book oh this is so exciting and they think excited means happy but it doesn't. excited means you're never system in your brain is fully on and if you could keep the never system from an art gamache entity on all the time it leads to collapse. so the whole system is filled with excitement. we need to relax and i'm relax to be here with you today and not excited at all and just

think how good it would be if politics was less exciting. [applause] i think the politics are politicians. i would never vote for politicians. >> i am so relaxed to ask the very last question. this is from misha. do you think that future reliance on artificial intelligence will impact religion of the source of thatua spirituality? >> quite likely. religions, many religions have always fantasized about havinger access to superhuman intelligence and we have it. think about tech. the idea is that this is coming from a nonhuman intelligence.

the problem with politics throughout human history and still today was they couldn't really talk back to us. there was something that we couldn't understand and the text could not explain it like what is the correct interpretation of the scripture? so even though in theory the highest authority in the religion was the text and process the human institution grew around it and the real authority will be the hand that the people who interpret it in the same way to have today the tech world fights in people who believe in open source and people as you know it's closed. the catholics they they should

interpret it meant protestants believe in open source. anybody can read the code at themselves. but what happens in for the first time in history that text can talk back. whether the text of the traditional book you canca train ai to read every single treaty written by every theologian or thinker in the third century or in the 11th century and had ai will understand the texts of christianity better than a human being. would be more authoritative than human theologians is one question and the other question what if you have a new religion with text written by a nonhuman intelligent -- intelligence and this is happening right now. with ai a new text on line which

creates the next big religion which will have text that can talk back coming from a superhuman nonhuman text. so i definitely think some of the most interesting developments in ai will be in the fieldd of theology and religion and maybe a recommendation that google and microsoft they should hire if you theologians because they will need them. that's a perfect note and on. thank you so much to politics and prose and thank you most of all to yuval noah harari. [applause]