so in the los angeles times for schoolbooks has also decided to push back there 25th annual festival to october. booktv will continue to bring you new programs and publishing news. you can watch all of our archive programs any time at booktv.org ...... >> and probably a bit for the better. but i think at the same time that some decisions have led us to better consequences as joy just said we live in a fragile world today and as a class of complex and difficult situations that we as a society have yet to

figure out. at a permanent level i make decisions every day. bought this car a while back. it's a hybrid. but i tell you i have no idea if the co2 i get from the 70 miles per gallon as a benefit to all of us,out weighs the cost of creating the car and the potential pollution impact of the battery after it gets stored. and maybe if we hadn't done that battery, we would explore new technologieses that have less of an impact. was it go for me to buy the car? i don't know. this is my mom. i bought her that scarf of christmas. if bought they scarf 100 years ago, fine, no problem. today, i don't know if that scarf was made in a sweatshop using cotton that had pesticides greg on it, and there were huge amounts of co2 as the scarf

traveled from a long way away to the place where i bought it. my choices have actions at a distance and i have no idea what those impacts will be. that's what i'm talking about today. you have a new super power. it's not all negative. the choices you make have ripple effects through the world. i think it's sort of like -- i look 0 think of it as a fish in a upon. maybe there's 200 fish the size of traditional human societies s and the fish are in the pond. everything is go okay. then a storm happens. and a river opens up. connecting that fish pond to the ocean. my goodness, new fish swim upstream, our first swim downstream, the environment is fundmentally different because they're salt water. i think we're in that situation

today as a society. things are changing radically. our actions have outcomes as a distance and that's what "link is bows," the actions we take and the decisions we make as we consider those actions, and the path through which the actions become reality. how can we be responsible for the outcomes of our actions if they're invisible. if we can't see the outcomes of our actions, irthere's fish in the larger ocean no longer in a small pond, we simply didn't evolve for this situation. now i'm luckie went through the computer revolution, and when i was a kid computers were big, old things that nobody could use and my mom didn't know the difference between software and hardware. had to explain that to her. we have again through a massive

democratization of computing technology. woe in the sale spate with a.i. and data and that new technology stack. a.i. is done to us, but we don't have control over it. data is overwhelming at a distance. at best we get a data visualization but i've had the honor of interviewing hundreds of people as a technology analyst and asking them, what are you frustrated about? if technology could solve one problem for you, what would it be? over and over and over again i heard a similar answer. it looked like this. oops, this is why i'm pretty sure i know what i'm dog. background for me. i've been building machine learning, applied systems, a really long time. over 30 years. so i was funded by the human genome project and graduate school i built $100 million budgets for the government. i've built thousands of machine

learning models. mostly supervised learning, back propagation who would have known it would still be with us so many years. my machine learning friends know what i'm talking about. so i believe that this background has given me an insight that is key. there's something been missing all of these years. and that something that is missing is we have been coming up from the technology instead of putting humans at the center of the equation. i'm honored to sit in doug's chair. my backside is honored and may is here who worked closely with doug. look that he called it intelligence augmentation but a you can think of a.i. flipping upside-down to the. c-span: a., putting humans at the center of the equation again and when i interviewed all of these people, i found what i called for a while a decision archetype. what is a decision in it's an

action, thought process that leads to an action. that action in a complex world learns through some stuff and i dent in the what buying that scarf or buying that car is going to do to the world. it's going to have some impact, but honestly i don't feel very motivate because i can't see that impact. it's not visceral and present. doesn't grab my primate brain in a way that makes me think i really need to buy that hybrid car. i can't see it and the data stack today and the a.i. stack today isn't giving that to me. this is my dog. his name is bowie. i'm training him to be a service dog. and i've had him pretty much his whole life. he's 11 months old. and i have this awesome thing happen to me. have a trainer who is teaching me to train the service dog, and she taught me but abc, antecedent behavior consequence, and my head explodedded of.

that's what i heard from the humans i've been interviewing, talking about a antecedent, which i the context. we're in the kitchen if say sit. the behavior. my dog sits. the consequence. he gets a cookie. okay. so this is a universal archetype. isn't just one way to think but how we might use a.i. and data and i'll tell you how that fits in, in a moment. what i think, i'm pretty certain, this is the way to think but because it has the lowest friction to how humans think. lowest friction to how humans naturally think. busy people live in come mex environments. they don't have much brain power to learn your optimization or inference methodology or those fancy things. we have to meet them where they're at and the fact we're not created a giant cultural barrier between people at the head of governments, at the head of businesses and even me, as

they try to make decisions and try to use evidence and dat and a.i. to help make sure that those decisions have a rim effect that is -- ripple effect that is good. so farmers working at, have to decide what crop to plant. down the road they don't know if that crop will make them productive or what will happen because they fewer migrant workers the situation has changed. businesses might decide, where to acquire a company or about product to launch or what this to change, as they tack that it through you hear what my dog hears, ant seed den, the behavior, going to launch this product at this price, and then town the road, for my dog it's immediate. he's aing to. for us and what makes us special is we can think through long chains chain consequences but that's limited and we need computer help. a decision is an imaginative process in our heads, as we

think through the actions in some context that will lead to some result. if you remember nothing else, remember this template and what is cool but decision intelligence, is because we start with humans, i can teach you some d.i. today you can take home and use immediately. that's my promise if you stick with the talk. okay. how do we make decisions today? i'm sorry to say -- and i only recently learn this. i had the idea thing happened but going back no a millenia of human evolution we don't think through the consequences of our decisions very deeply. we are much more likely not to think through things rationally and instead to use social signaling. we look for someone who looks successful in our society, who is dominant or prestigious and we'll copy the decision they've been making. that's very effective. it has been hugely successful

for the human race and in fact it's what operates us from -- separates us from other species. we're great copiers and cultural evolution theory says we develop behaviors and patterns that any individual can't understand but that the society, like kind of the unconscious process of genetic evolution, we use cultural evolution to couple up with these behaviors. this is what we're programmed for littler wore programmed to look at some prestigious or dominant person and do what that do others opposed to talking through the consequences. that was great for a few millenial but the situation has changed. if there's a bad actor here or here, and they tell us that would do and they're smart, they can subvert our behaviors. they can influence to us make decisions that benefit them but not us. if they're smart about the situation. second, the context is rapidly changing. we need to be developing new ways of coping with this big

ocean that's fundamentalry different than our pond because it keeps changing. the water is flowing back and forth, they're new issue and the old ways of thinking through problems at a societal and crowd level nor longer working. these have complex system dynamics. they're nonlinear there's feeback phoenix, winner take all, winners take all patterns where large companies or large artistes 90% of the benefit there's massive inequality. action at a distance. intangibles are important, too. anybody who has worked with data, we tend to focus on the things we can measure easily. money. size, price. we tend to overlook reputation, happiness, more morale and yet i've never built a decision model that didn't have at least one feed back loop that involved a tangible, soft factor.

we maas start talking to sociologist and others to understand the soft factors. the decision intelligence creates a road map how to do that. the other thing didn't say is the future is no longer like the past, and so if you in the work, the black swans, the problem when we assume the past and future are the same and modeled are based on the past and we don't realize the situation is changed, sudden lid the swans used to be white and there's a black one. what do i do? i believe that a. immigration and d.i., decision intelligence, can solve this problem. i grew up in a period of technology optimism. we were all sharing all of our code and the internet was going he democratize reality. we had a dream but he don't think we have realized that dream. i things decision intelligence

will help douse there. i think we have created a number of links in the chain, data machine, learning collaboration, the internet, social media and there's one more link that we need to start to make a dig difference to have a nonlinear impact and that'sd.i. how do we do d.i.? we start with people. we don't say, where is the dat? discover a da to is great but there's a huge amount of human knowledge in that is no no data simple. we're good nothing how our actions threated outcomes and your homework today is to go home, ask a friend who didn't come to this talk, how they think about a complex decision, i promise the recall talk but actions and the actions lead to enter meetat effect and that will lead to outcomes outcomes d they'll talk but to condition text. i sit down with a diverse group of experts and diversity is

great, old, young, gender, race. say what are the outcomes you're trying achieve? i know so many candidate that have massively big projects, who have never sat down and brainstormed how to the outcomes. okay? i go and i consult at fairly senior levels with many organizations and i say what are you trying to achieve? and the list of outcomes is different for each person. let me tell you you don't need technology to get better. you just need to have a brainstorming process where you think through the outcomes we're trying achieve as a team. is it higher revenue? is it net revenue after two years? is it some kind of a military advantage? do we want a military advantage that doesn't create a backlash that will hits ten years later in terms of the psychological reputation of cower country. what the outcomes we're trying

achieve? make sure you ask that question. second, brainstorm through the objection. many folk folked don't take the time have an open brainstorming session where they've allow bad ideas and funny ideas for the actions we could take that might achieve the outcomes. do that. move all the blood to the creative side another your brain because when the blood is in the an lit tall side of the brain -- analytical side of the brain you don't have room for the creative seem. so separate them. spend time being creative and then spend time being analytical. these triangled here are where a.i. fits in and most decision models i believe as we democratize a.i. this its the pattern. so let me talk but a decision i'm facing today. i saw greta on tv. and she was so compelling. she said, we've get a climate crisis and the way we solve

this, it's really simple. stop worrying about analysis. at the very least, pay for some trees. the organizations all over the world that will take your money and buy trees and those trees will go and there will be more bio mass and that will sequester some carbon and if enough people do this, she said it would make a big difference. i haven't sent any money to a tree organization yet. i can't visualize the money i might send leads to a khan of events to some outcome. if i'm going to use a.i. to benefit me i want a visceral, interactive fun experience. and so this is what i think is the future of a.i. it's going to look like a video game and i hope weening do that's in the base. because we can do that's in v. r.

we are expecting we asks we might take and we're leafing the computer help us understand the chain of event that sets in motion to lead to the consequences. valuable at a personal level, also highly valuable at an organizational level. let see if this works. yay. i've been having a lot of fun coding. i really like it. there's an institute for the future ball that had no purpose whatsoever except to show you there's a physics engine run. we're trying to make a decision how much money i'm going to pay each year to sequester carbon. as i change the decision, this data here is telling me about the future that puts in motion. i spend some money, i change my decision, it changes the number of trees'll purchase. here's the bio mass. here's the car been sequestered, he's the toldat atmosphereishing

carbon. there's barry hayes, each expert has an opportunity to say how the actions domestic outcomes here i can change my decisions but i can also change whether sam is right and who i can tryst? i can see that different people, different experts claim different things. ultimately i can also click on their name and go to a site where i can see where they're making their case. their model for how buying trees leads to outcomes can help us and that's like wikipedia, a site that is curated and we canny to understand the situation. this sort of looks like a business intelligence dashboard. sort of looks like stuff we have been building for a long time. it's not. we're not looking at a data set

here. we're looking at the future. we're only summarizing it in the bar chart so you can understand it. the background, depending on choice's have a sis fixes engine generating implications of the choices. a lot of investment. a lot of forests and then as i change my investment or use linda richie i can see how my decisions interact with the situation as they characterize it in order to impact the outcomes care about. this is a universal pattern. this example here is an example of something that you do in your head 500 time as day. and that large organizations really struggle with, understanding the impact of today's decision tomorrow. i'll give yaw couple of examples. these are machine learning models here. these little blue triangle and we might have bit a machine learning mod that can detect

where a computer system has a current intrusion. machine learning widespread used with an intrusion detection and gives us a score. might be 20%, 20%, up to 59%. it's pretty sure there's intrusion happening right now. another model that says here's the type of intrusion. it's a ddos attack or something else and this picture is typical of many decision model situations. it's got some spa getty in it but -- spaghetti but that is not how people think. if that is how you think about things it's better to have it on paper than 0 keep in your mind and explain when use invisible mechanism, words and text which is linear to communicate decisions need. a blueprint like this which is an architecture diagram for a machine learning system that shows how machine learning fits in and then we have choices.

can investigate the intrusionful that's going to benefit and impact outcomes. if i call the police every time it might be pretty costly. need to try to call the police only when it's necessary. this is the decision model i built with a bunch of farming experts as part of that big nsf project. it's got 23 ph.ds and seven institutions and just finishing up the proposal. please cross your fingers we need it. a national center for excellence in a.i. and greg. what -- agriculture. what is important is i didn't have to explain decision intelligence to anybody if sat down with my diversion tome of experts and said what a thin cal farm are trying achieve? at first they said they want to be profitable and i said is that all? they said, they wouldn't want to take any actions that will put them out of business in a few years. that's a second goal they need

to balance against that. and then we talked but the actions. that those farmerred might take, make choices howl they spray their crops, their spraying credit all, the choice of crop and hybrid and cult. this gives us a map to understand how all ot and ai and machine learning technology filths together because there's a couple of models tell us how precision spraying women impact the yield. another model says the amount of type of diseases or contaminants you might have based on spraying, and here's some i.o.t. sensors that farmed have an 0 drone that they can use with a.i. to interpret the data know is a pathogen starting? we want as early warning as possible so we can spray as little as possible. both from a cost and long-term available and a climate and pollution point of view.

so it's kind of spaghetti but having it on paper is better that in people residents head. this becomes an artifact. we design a decision in design thinking. a decision is something we can design. that's pretty radical. when we did, we realize we can bring all of those engineering best practices to decisions. we can qa this thing, continuously improve it. act as a blueprint that ex-cozy end user -- my stakeholders built this. their mental model. but connects them to the a.i. people so they know where to fit in. my team has been working together for a month before we had the diagram and nobody knew how to hold things together. we now have a map. we know hough it all fits together and we know where the a.i. will go in. so what's going on with decision

intelligence? those of you who have known me a few years, know that it's been quite the slog. i'm flying all over the world, trying to talk, sometimes there's throw people in the audience, right? trying to tell people the story. i'm really happy to the extent this matters, gardner has decision intelligence on its a.i. hype cycle new. and i can consider that a big accomplishment. there was an article lost week by ibm. going trained 20 town people in d. inc. and she is the big eadvantagist in this space. she's awesome. then there's a butch of companies that started to identify as d.i. companies, mose of them in specific-year-old. purr tech is a water company, these guys are hr, curriculum labs, medical devices, automatic people who recognize of they go

beyond building a machine learning model and imbed entity a decision model, that sort of surrounding of the machine learning model and connections actions to outcomes that's going to help them be more successful. it's cool. this is how kathy defined d.e. it's the disminimum of turning information into bert actions at any scale. we have slight live different way of saying it but it's essentially the same thing. i say d. immigration an this question, if i make this decision today which leads to this action what will be the outcome tomorrow? lots ofpeople have different approaches to d.i. mitchell i call the decision gram as mine. other people have other approaches. some people don't even come from tech. they're sociologists, economists and more. mat is common to all of us is we have taken seriously the action to outcome path as the core way the technology and science

interact with humans. we also made it to hollywood. who knows what show this? it's the good place. this is ted danson. i'm not going to do the spoiler. if you haven't seen season the, this is episode ten. recommend you take a look at it. he's basically discovered t. immigration by the end of the series are it's the big reveal. it's the mystery. which is so cool. right? and what does ted have -- who knows who this is? that's janet. is she human in no. she's an a.i. so here's ted trying to understand why there are all these negative unintended consequences and here is egypt who is helping him. how kyle is that. i don't know if they read my book. that would be fun to keep tis going because it's so important to the future.

with great power comes great responsibility. you have the ability to take actions as an individual and as an organization that have a giant impact. i'm an optimist. believe we're in the midst -- or the beginning actually of a solutions renaissance where it's not just a.i. but all of these technologies will come together under a common bankrupt, system dynamics, complexity, computational neuroscience, cultural revolution, all of these fields, many of them traditionally viewed as soft fields which are in their little silos. a thousand years of being specialists. we're entering the age of synthesis and those who are in generalists know what i mean by this. we're the experts who focus on the outside of these boxes. it doesn't matter if we know the math with operation research we know what goes into it and what comes out, and we start to know

using some version of d.e. how to glue that together to crystallize solution and solution to water impacts poverty. a solution to poverty impacts the status of women. and the impacts hit all of us. they hit governments, hit democracy, hit companies. we call these the sdg the sustainable development goal. i think they're all the same probable limp don't think we can solve them separately. and i think the only way we can solve them is to have a new approach to understanding how actions bounce around through the whole world and ultimately lead to the best outcomes. thank you. [applause] >> thank you, loren. >> do we have a chair

transporter? can you. >> that's great. just shocked you actually finished under the time. i need to mitt my little thing. >> do you have a particular case study or story to tell us that shows how this works in practice? >> i think the example i showed with the farming is a good one. we have also built an initial model for a government decision regarding reducing conflict in the sub saharan african country. the model showed -- it was a preliminary study put showed you need to take action whose to places at one. do some work with the rule of law and you had to stimulate the placements if you did those two thing that was enough force in the system to take a visual cycle of conflict to virtual

cycle. >> how long was this test case. >> the causal chain were five or five links long. less complex than this one but just an initial study. it took few weeks because i spend time interviewing expertss and they were busy. >> right if wonder how much you feel this could be extended to metaphorical think income people think in metaphors and models our things are like things, and whether you can actually imagine it being extended in that way. >> yeah. think the future -- part of the future of di is to understand there's systemic pattern that happen in one place and happen in another. one of them is the pattern where we'll talk an action we think is good and it's good in the short term. i call this the lobster claw. then the inaction the short term leads to a negative consequence

in the long tomorrow but the visible horizon done tell us that. we put vending machines in the schools. makes the kid happy but down the road maybe diabetes and obesity aren't such good things. >> i'd like to bring in several other people to talk as well. i asked a couple people for comments to kick everything off. bob, she's the ceo of inquirer. would you like to ask a quick question? >> i have a question. >> hi, jill. >> go to see you again. >> good to see you again. >> i have a background in climate science, and right now we're -- an interesting time when everybody is kind of weighing in on climate science and i find it interesting

because you have a lot of people naive lie want to pink to linksle. hey have facts facts and don't e they're putting -- it's solar flares or i have an idea and having studied earth science you want to see spaghetti, and yet i think about your metaphor and like this challenge but teaching science which is the links to mechanism. would you be able to not have to teach a science but demonstrate the complexity of thinking as a way to start to appreciate? i feel like they aren't seeing the complexionty and i -- complexity and just to be able to show this. anyny idea to make that's tool and almost infliction people's thinking without them having to painfully go through it. >> i use a metaphor to once your question. can drive a ferrari without

seeing what is under the hoot but certain people are really impressed when you open 'the social media see all the come mixty. so some of us don't want to hear about that at all. we just want someone who we trust to tell us if we take this action, it will lead to this other outcome. some people we want the ability to open up the hood and we have to have a multilaird approach. we have to have beautiful, immersive video game which grab your attention, and then have to be able to check on the expert so see at the thought process that led to the mechanism of that model. >> people don't know if you've been there they don't know there's something pause we make everything look easy, which you spending too. if they looked inside they didn't think they could build a ferrari. >> we need bowl. as technologists were obsess its with under the hood and haven't paid enough intention to the

user,. doug would be happy we're talking about it. >> do you have anything to say pout this? steve is a scientist and at a.i. brain. >> yeah thank you for a thankful talk and a great book. very great insights. i loved when you were saying the vision we have of the internet is just get remember started and it would be all a net positive and bring humanity together and you can see people of the world, understand them, and increase empathy and connection, and i think the reality in the last few years has been much more divisiveness and tribalism and breaking abuse pieces. i'm trying to imagine a world in which t. immigration is everywhere and everybody has tools to make decisions in better ways and i wonder of you think that will help people understand one another, bring people together, or contribute to the divisiveness? what do you think impact will be. >> i think it will increase our

sense of agency. a lot of people who feel overwhelmed be the information etch executive is talk to say this in particular. don't want to talk to those guys. i can't even understand a word they say. and if they have interfaces like this, they will engage with the evidence in more solid ways and with the data and the a.i. and all of this great, janet assistance. if they got the janet assistant they can do better. i think at individuals, we have gone, it's too complex. so i think the biggest initial i initial impact will be our sense of agency we'll bundle out the inequity that comes when technologists dominate the world and make us do things and click thing and we start toite for our own needs and personal needs. think we'll hit equality -- i'm an optimist. we. democratize the computer, didn't we?

late democratize a.i. >> steve jobs launched the mack 36 years ago last week. black and white. >> wow. let make this do that for people. >> the metaphor, go for the mind and i think this technology potentially really empowers people, and enables the kind of mine democratization of what is today a very complex -- >> let me repeat myself because it's so important. you guys don't need to read the become or learn text. just make sure you brainstorm your outcomes and action. if you do nothing else, there's huge value in that, and we're not doing it. we're so lost in our specialties. >> yes. >> i recall dialogue which became an argument, what engle barr said was you're designing a

tricycle and what i'm designing what he was designing is a mountain bike, and it was a real attention because the teach jobs thing was just too simple for doug. wanted it to be muss more robust the good news from what your talking about and one of our rules an the ten-year forecaster is most really big changes take 30 to 50 years to be an overnight success. >> yes. >> so in a real sense you may be on the verge of that number. when was the tomorrow a. eye coined. >> 1956. d.i was 2012. >> a long time ago, and my recall is that debate at that time was there was a debate whether to call artificial intelligence or augmented intelligence. >> that was the darkness in set 56. >> the wrong side won issue think. a.i. won and to me i think that vote up the evolution of the field by years. the good news is now i think it's possible that it's coming together and can actually work

and you can have both a tricycle and a mountain bike. >> i think what jill asked fits well into that. she's a brilliant technologist -- i'm speaking on your behalf youch might real estate naval with the engle bart view. it's a testimony. you need to be mow sophisticated and. we build the engines -- it's like a.i. feels leak we have been building car engines for 50 yours but nobody built a car. every engines has different driver koles. we need to build common control that is easy and then be classy under the hood. it's computer science principles. another metaphor. thank you. that's a great question, been. >> just fascinated by think about the possibility of all of these worlds within worlds, and really the story building that is impress sit in understanding

all of these different fields and also wondering beaut bias in implied or perceptions of boyas. when there's a wikipedia entry but one of these people there are rules that are set up for how people can edit those entries or not. how is it -- would it be possible or should it be possible to have multiple visions of these different characters rather than having only one set of perception of their outcomes? >> this is the challenge and i think this will happen -- begin over the next year or. so there will be a new kind of wiki speedy which has rules that maximizes as jimmy wills figure out. how to get reasonably accurate information from crowd sourced critiqueing. but instead of giving facts, this is what they call warm

data. we get connections. we'll say when you do this, it cases this. when you give this money to this tree charity, they'll by ten trees and then we'll have someone who just opens the link and curate that and say here's all the reason is think when you spend $10 with this you get one try. someone else will have an opinion. no, when you spend $10 with the charity you get 20 years and somebody else does the next link. each throw has this byow mass and another expert says no, it's this biomast. we are creating what nora talk body -- talking boat a link, something leveling -- wick bid yaes great but it's static fact. and that connection between things that cause other things which cause other things, what we have been missing for years. that will be what is new. >> the intersections a little more transparent. >> exactly.

that's where bias is helped. one thing that is hang with a.i. is we're getting unintended consequences because wore nod modeling the context in which the a.i. happens. so when i start a new a.i. project -- they want to say with hey the data and we're ready for day a.i. i say put the data eye side. tell me what the digs is this a.i. will be used for. unless we understand the context and actions that lead to some outcome i might build the wrong a.i. system. they say you can do this without dat? i said data might help but if you don't -- this is back to software engineering. don't understand the requirements you'll code and build things that brake ask there's research saying nine out of ten a.i. joe expects fail. this is part of why. >> anybody necessary. >> i like your point about the car. i think that's really

interesting point, one topic that came up i'm abare of there's a strong correlation to making women -- their access to being control and do family planning, very self-le link. humans are intelligent and women can note the side of the family they'd like to support. this is like a missing link most are not aware of. they tout boat. what are we going do it overpopulation, educate women, make their moment economic viable. ask that's the secondary effect because you're all the primary link. i'm curious when you envision what technology can do i'd loaf of love to see the technology to reself-when you have multiple interpretations and somehow you see a we too build a system that can somehow mend that in a way people could see that. that it a bigger link, a fatter link. >> i don't think we have a lax of research that demonstrates

these kinds of things. i think science has been give us randomized control kyles. we have a lot of of results. i think what is missing is taking the individual links and connecting those two actions that i could take today. let's say believed you and i care about the status of women globally. what can i do about that? show me some evidence that -- puck tours. not evidence in at the boring sense but evidence in the virtual game, augment real, mate ick fun, diversesive, it's cool, i seek when i move the letter and i pick my expert. maybe jill, and we have other people curate ago links. i can see women's happy unless. how doie design women's happiness? we need to have these deep immesssive displays that show these futures immediately as they play out. so, that's the piece that is

missing. i think we have great science, great a.i., great data. we don't have this last piece where we democratize a.i. through chiefs evented that give you a sense of agency. >> i feel like shy call on people on this side. >> we're. [inaudible question] -- fox news, twist and turn and it's all behavior that irpredictable, and the biggest thing to behave on that. and that's the different sort of model than having a brain controlling a body over an organization or chat involving a company. you can certain of see it a bit like culture or religiouses or other people which people don't actually know what their decision will do in the big scheme of things. just do it and it's quite a powerful adaptive model.

>> i think so too. >> how your information and intelligence works and i'm thinking -- >> unthinking level. we have been through 2 yours years and a.i. was the extreme examplele etch can only be intelligent if we are verbalize it and represent is as inference and logic but there's massive amounts of subconscious rationality that is very smart and this emergent intelligence where individuals in a group, none of them can explain what they're doing. none of them even know what i they're doing on their own. instead somehow the signals are giving each other gives us emergent intelligence. cultural evolution knows that, too, and humans have been doing that. but i think we're in a new fish pond. i think we're in a situation where our natural instincts for huh we behavioral like birds done work anymore. we're in the ocean. you asked how d. immigration fits into this. i think is surfaces assumptions and it creates a challenge to the cultural evolutionists to

really study the emergent behavior of humans as they make decisions. and it's an invitation to that part of the world to work together with this a.i. and data people of the world in a coherent way. it's a great point. we do have emergent behaviors and have to model complex systems and my friend, alex lamb, knows this better than die and he builds simulation models that help us understand how we get the emergent behavior. it's very nascent. hopefully you can help. [inaudible question] -- the book i am starting to read. certain sense i hear you articulating much more clearly something that i've been spending most of my life trying to bring to people so it's very familiar. guess i'm curious, what have you found effective in terms of when

you go into an organization and provide consulting or coaching, maybe teaching in a class, how do you get these ideas across more effectively or -- also, as a particular tool you find that are useful. >> i go in -- should i say maked but that's probably not good. i go in with not a lot of studying because i don't want to bring my bias to the picture. go in with an methodology. it's chapter six -- it's simple. brainstorm the outcomes and the actions and then elicit people's innate sense of the causal change that gets you from 2001 the other. then only after we have done that guy back and usually this really ugly whiteboard picture, just degrees. clean it up and we look at it together. i haven't used any special technology. only of we have an agreement that picture would would i bring in an immersive visualization

process and we can have these decision dashboards which not just show data from the past and present but say given the decisions we're making and the situation we're in and the data feed we have from climate and gdp that is live, here's where our decisions are going. it's all new. i don't use a lot of tools yet. >> a lot of machine learning more than anything. >> i think there's a real art to how you present this material to people, especially in the visualization of the information and the type of, let's say, graph or whatever you want to use and it's not very clearly done yet. we should partner with people her and i more to make that sort of an art and a craft that has some positive opportunity that people can be shown, because die

think you can there change people's understanding bit different types of graphic and they get seduced by some ways and put off by other ways and it's not sometimes very clear what to do. >> just want to double underline what you just said um i'm no a user experience children don't have the huge career that you have in this space, and so i'd like to make an invitation to you as welling a anybody in the user experience base, i'm not claim these spaghetti diagrams are to place to go. thats untested. and i'm not the foreign do this. those who know this space, and i hope we do get to work together -- they need to be invited invited into the picture and that just hasn't happened yet. yes, over here. >> i have so many things crowding my brain right now. could use one of your intelligence to figure out how

to get to it. so, my name is annika and i've been working on a idea, responsible innovations and the last talk i gave issue said i'm going to stop using responsible a.i. because what are you proposing? some always slot you in a.i. for good. and are you making irresponsible? -- that's the only reason why. and i think a lot of things you talk about, really calm together for pieces of pourings we're working on. i had a technology where we are saying instead of using responsible a.i. we're saying just use -- this is really good engineering practices, and good decisionmaking. and the question is, how do you decide? that's why i was very interested in your talk because i've been thinking how do businesses make

decisions. so when you made the comment but technologists who have taken over, really i actually find it's a business, the way we have designed our economy, the economic model and how do we make debit in that struck -- make a dent in that structure. you hand these amazing tools to the same people who have the same desire for the same outcome, macmore money so just go with outcome and action and the outcomes don't change and people dent get the place at the table to change the outcome, then just given an amazing tool to people to create the same outcome in a much more delicious even maybe way but a larger way, and any that is what i've been every day think pout. how do we take this amazing intelligence we have for change. >> so, there's a lot there and you're doing amazing work. want to respond to one point which is the reason i believe in

this is because it's making the invisible, visible. no matter what it does, even if we have bad actors using this, it invites them and insists that they draw a map. i would love, before we made any big government decisions, we insist on somebody drawing a decision model and we have diverse team with multiple ages, gender races and who all agree to the model. at what's happening ought now the executives talk to the loudest guy in the room gets his way or who can tell the best story. i think we have to combat that and the way we do is by taking the invisible and make it visible. >> one last point. i think guys just mentioned something and what you're saying, the challenges, i love -- i really want to work -- figure out how to figuring this out. the challenges when you have

five perspectives. one approach we took, we talk to five different people and put it in. it becomes lick an overload. you talk but decision intelligence. we get overloaded by all these variables and different perspectives, and then how do you kind of say, well, -- the loudest to the richest or whenever person doesn't end up saying we give you all the platform, we said your piece and now this is what we're going to do. >> really good points. let me speak to just one aspect of it, which is a principle. the main thing is read the book and that will answerrure question. there's the methodologies there. one of the key best practices in this space, i the outside of the box vs. inside of the box. you talk about this thing, this technology, this agent based system or complex system model. people talk but the math inside it. that will overload your braid.

information hiding, another good compute are architectural principle. focus on the interface. our goal is to glue the technology together etch we don't knee insights or answers of we need more science but an awful lot of scientific results sitting around unused because nobody connected them to the action to outcome chain. so, yes we can get overwhelmed but if we keep using best practices that keep people focuses on surfacing their mental model we can overcome the overwhelming. that's the exciting thing i go into rooms talk thing to 20 different directs and then i say we'll draw a mop how your thinking and the moment they see is there's a relief because the department have to keep net heir head. a its no a picture. it's just a mapping technology in many ways. let's talk. >> a couple more questions,

please. >> few for at the talk. >> thank you. >> also had a chance to look at your previous learning to know. >> oh, wow. that's $140 book. thank you. >> yes. i had a chance to look at it. so, you have -- [inaudible] -- what comes from the normative side. >> what side. >> the normative side of decision -- you can coming -- [inaudible] -- almost cal approach and being a technologist i get bias towards that, but then there is the descriptive side where psychological behavior and economy is offering. do you see any intersection between them? do you seed the need for though worth together? >> yes. >> how is that. >> let's take a behavioral economics thing. and let's look at a cognitive

bias we might make. that behavioral economist fits into this gram somewhere and will help us learn how when we do this step or when this is true, that will cause that too be true. the way i see these expertise areas fitting in is they inform the links. so the picture is just an integration for how we pull different understandings of how people work some the picture. so whether we're modeling in a game theoretic way, how 0 another company or person will behave that becomes pretty important. [inaudible question] i often see that -- make the wrong decision. which is when i realize that you need probably -- [inaudible] -- not necessarily inside this approach that you're taking, and

therefore that is very -- helps you to understand how to execute. >> what that book, weapons of mass destruction? talks about an algorithmic model. chapter two where this woman is fired who is a great teacher and it's an a.i. system and it's opaque. they don't know why. and if they had built a decision model that said here's the variables this thing is using and how this prediction will be used in this larger context i think they could have surfaced that unintended consequence much faster. with understand the context in which a.i. is become used, it makes it visible and subject to credit speak to weening continuously improve it and all the engineering best practices around the context of a.i. and also technology solutions we use. thanks for the question. that's great. >> the last question.

[inaudible question] -- i'm wondering if you could talk about conflict and how that sort of -- the role that might play in setting outcomes. it depends so much on who is involved in making -- who is at the table making the decisions to define the outcomes and if you're -- in your experience or methodology if you thought through what you do when outcomes are in opposition. >> i believe in many situations, outcomes that appear to be in conflict actually aren't, and that if we do that's kind of maps -- i'm not saying this is mature. the stuff we africa, the whole purpose is to resolve conflict. you have two people with different opinions on the actions and outcomes and capture

those on a common map then a.i. and some of this technology in many cases can find a holy grail, a set of actions that help you achieve everybody's outcomes. because we're better at doing this, because we understand these links and we're not at the same time we're arguing with somebody, we're having to keep the complexion links in our head but we can't and don't want to admit we can't so we fall back on arrogance and assertiveness and yelling', and we don't have to do that are we have a map and it's not you against me and our opinions, it's you and me working on a common model, and so it's not this conflict. it's less drawing together to get a shared view. and this kind of a map facilitates that process. great question. thank you. >> yeah. i think people like being heard. >> they do. and when you draw a puck tour of

their mind, it's great. >> very scary that thing. >> [inaudible] -- makes you more open to the future. >> the one addition. >> thank you for that addition. any last comments from anybody? >> thank you all for the decision to be here today if hope the positive outcome succeeded your time with negative outcome of your time and effort to drive here or get here. >> people and how we make decisions together as you putted out we did just make this decision to be here today and i often think now, so difficult to get people out or their houses and engage with each other. we do have food for you to continue this conversation with

each other and thank you so much for having us. thank you bill for bringing us together every quarter, wherever he is and thank c-span for beina reporter and being very much in the background. thank you very much. >> maybe give a round of place for gay. >> yay, gay. [applause] >> on our author interview program "after words," jim mckell have i, cofound ore square, offers thoughts on innovation. here's a portion of that interview. >> i'm a glass blower. i make stuff that nobody needs. i make art. ...

i was angry. i just lost this great windfall and i was talking to the lady. i was second hurt on one of these devices this device is a magic device. if i wanted to be a television it will be a television. literally tomorrow turn into that book if you want it. it did not turn into a credit card machine. i was angry but i was also motivated to fix that. i said let's make our iphones turn into credit card machines.

that is what happened and became square. an innovation stack is not something we know about it is probably the most powerful phenomena i've seen and business. and we stumbled across it is simply a way of interweaving inventions together. sometimes very simple inventions. they create new industries. if you look throughout history at the great industries that have started almost always there is an innovation stack at the beginning. i know idea that any of this was happening. as a matter fact i wrote this book and i had been having people review it like yourself and one of the greatest compliments i got was from a

very successful entrepreneur. he has a painting on the wall that is worth more than my house. i'm all intimidated. he finally said i wish i had known this when i was 20 years old. see mac i said me too. it turns out that there is this thing that happens this process that can happen when you start to solve a perfect problem something that has not been solved before because most of what we do with copying and most of our tools in training and comfort with solutions that exist. you can build something that is truly different but the process is different and it creates this thing called innovation. >> if you build the innovation stack your company will dominate the world.

it will just run whatever business you are in. >> to watch the rest of the talk and to find more episodes of afterwards visit our website. [indiscernible] please welcome carl hiassen. [applause]. [inaudible conversations]. we will wait a second to put away the flask you and just ha s