and that something that's missing is we haven't coming up from the technology instead of putting humans at the center of the equations. we work closely with doug.

i like that he calls it intelligent augmentation because you can think of ai: think upside down to the ia. when i interviewed all of these people, i found what i called for a while a decision architect what is a decision? .-dot process that leads to an action. at that action in a complex world flows through and i don't know what to buy that scarf or car will do to the world. it will have impact, but honestly i don't feel very motivated because i cannot see that impact. it doesn't grab my primate brain in a way that makes me think i need to buy that hybrid car. i can't see it in the the data today in the ia stacked today isn't giving that to me.

this is my dog. i'm in training him to be a service dog and i have had him pretty much his whole life since he was 11 months old and i had this awesome thing happened to me. i have a trainer teaching me too train a service dog and she taught me about abc and it's like my head exploded, that's what i heard from the humans i've been interviewing, the executives always talking about in a seated which is the context. we are in the kitchen and i say sit, behavior, sits, the consequences he gets a cookie, so this is it universal archetype, not just one way to think about how we may use aim data and i will tell you bit about how that fits in a number. i'm pretty certain this is the way to think about it because as the lowest friction to how humans think. lowest friction to how humans naturally think, busy people

live in complex environments. they don't have much of brainpower to learn inference about the dollar to you or any. we have to me too them where they are at in the fact we are not has graded a cultural barrier between people at the head of governments, at the head of businesses and even me as i try to make decisions and use evidence and data and ai to make sure those decisions have a ripple effect that is good, so farmers i'm working with had to decide what crop to plant. down the road they don't know if that crop will make production or well have been because that you are migrant workers made decide where to acquire a company or what price to change and as they talk that through you hear much of what my dog hears, a situation, a behavior

we launch this product at this price and then consequence and for my dog its immediate, but for us at what makes us special is weak can think through long chains of consequences, but that's limited and we need computer help so again a decision is an imaginative process of her as we think through the actions in some context that will lead to some result. if you remember nothing else, remember this template and what's cool about decision intelligence which is what the book is about is because we start with humans i can teach you the i today that you can take home and use immediately so that's my promise if you stick with the talk. how do we make decisions today? i'm sorry to say and i only recently learned of this, especially in a complex-- complex world, but going back to evolution we don't really think

through the consequences of their decisions deeply. we are more likely not to think these rationally and instead use a social signaling. we look for someone who looks successful in our society, dominant or prestigious and safely copy the decision they have been making. turns out that's effective. it's been hugely successful for the human race and that's what separates us from many other species. we are great copiers and cultural evolutions as we develop behavior through pattern that any individual can't understand, but the society like the unconscious process of genetic evolution we use cultural evolution to come up with these behaviors. this is what we are programmed for, to look at prestigious or dominant person and do what they do as opposed to thinking through the consequences and that was great for a few millennia, but the situation has

changed. first of all, if there is a bad actor here or here and there they tell us what to do in their smart they can subvert our behaviors. today can influence us to make decisions that benefit them, but not as if they're smart about the situation. second, the context is rapidly changing. we need to develop new ways of coping with this big ocean that's different than our pond because it keeps changing. water flowing back and forth, new fish and the old ways of thinking through problems of a societal level are no longer working. these have complex system dynamics, nonlinear, feedback effects, winner take all patterns where large companies are large artists get 90% of the benefit in this massive inequality, action at a distance we have talked about and intangibles are important. anyone that's worked with data,

we tend to focus on the things we measure easily, money, price. we tended to overlook reputation, happiness, morale and yet i've never built a decision model that didn't have at least one feedback loop involved something intangible, a soft factor. we must talk to the sociologist, cultural evolutionist and all those other disciplines to understand the soft factors. decision intelligence creates a roadmap for how to do that. the other thing i didn't say is the future is no longer like the past and is so the black swan is a problem when we assume the past in the future of the same and models are based on the past and we don't realize the situation has changed. suddenly all the swansea used to be right now there's a black one what we do? i believe that ai and di i'm a decision intelligence, can solve the problem.

i grew up in a time of technology optimism. we are sharing our codes and the internet was going to democratize reality. we were going to collaborate. remember, steve? we had a dream and i don't think we've realized that dream. i think decision intelligence will help us go there. i think we created a number of links in the chain, collaboration, internet, social media and there is one more link we need to start to make a big difference to have a non- linear impacts and that's di which we will talk about practically now. how do we do that i. we start with people. we don't say where's the data. we don't say we can't do this ai without data. i'm sorry, data is great but there's a huge amount of human dollars-- knowledge that is in no data set.

we are good at knowing how our actions lead to outcomes and your homework is to go home and ask a friend that didn't come to the talk how they think about complex decision and i promise you they will talk about action, actions will lead to intermediate affect and ultimately outcomes and then they will talk about the context, so i sit down with a diverse group of experts in diversity of gray, old, young, gender, race and say what are the outcomes you are trying to achieve. i know so many companies that have projects who have never sat down and brainstormed through the outcome. i go and consoled and fairly senior levels and say what you trying to achieve and at the list of outcomes is different for each person. you don't need technology to get better. you just need to have a brain process where you think through what are the outcomes we are trying to achieve is a team.

is at higher revenue? is it net revenue after two years? is it some kind of military advantage? military advantage that doesn't create a backlash they will hit his 10 years later in terms of the psychological reputation of our country. order the outcomes we are trying to achieve? make sure you ask that question. brainstormed through the actions many folks don't take the time to have an open brainstorming session where they allow bad ideas and funny ideas or the actions we can take that may achieve those outcomes. do that. move all the blood to the creative side of your brain, because when the blood is on the analytical side of your brain you don't have room for the creative side, so separate those two and spend time being creative and then spent some time being analytical. these triangles here are where

ai is and most decision models i believe as we democratize ai is the pattern: this is how we will do it. when we talk about a decision on facing today. i saw greta on tv and she was so compelling. she said we have a climate crisis and the way resolve this is a simple, stop worrying about analysis. at the very least pay for centuries. organizations are all over the world that will take your money and buy trees and of those trees will grow so there will be more biomass sequestering carbon and if enough people do this might to send leads through a chain of eminent to some outcome.

if i'm going to use it ai to benefit me, i wanted this surreal, interactive, fun experience and so this is what i think is the future of ai. it will look like a videogame and i hope we can do some of this in the basement because we can do this in br and walk through these spaces. what are we doing these spaces? experimenting with actions we might take and delighting the computer help us understand the chain of events that the system motion could lead to consummate, valuable at a personal level and highly valuable at an organizational level. lets us see if this works. i have been having fun coding in unity. there is the for the future ball with no purpose whatsoever which -- just to show you there's a physics system running here. we are trying to make a decision as to how much money i will pay each year to sequester carbon.

as i change the decision, this data is telling me about the future that puts in motion. i spend some money. i change my decision. it changes the number of trees i will purchase. here's the biomass, the sequester and total atmospheric carbon so i can see the chain of events. linda ritchie is the expert in the background who had done research and by the way there's barry hayes and me-- he may have done research in each expert has an opportunity to say how those actions connect to outcomes and here i cannot only change my decision, but also change whether sam is right and who i can trust because i can see that different people, different experts claim different things. ultimately, i can click on their name and go to a site where i can see where they are making their case that their model for how buying trees leads to

outcome can help us and that will be like wikipedia, a site curated that we can use to understand people. sorry, to understand the situation. this sort of looks like a business intelligence dashboard like stuff we have been building for a long time. let me tell you, it's not. we are not looking at a dataset. we are looking at the future. we are only summarizing in the bar chart to understand and in the background depending on our choices we also have a physics engine generating the implications of those choices. i did a lot of investments, a lot of forests and then as i change my investment or use wendy-- linda ritchie's arrogance i can tell my decisions interact with the situation as they characterize it in order to impact the outcomes i care about. this is a you know personal pattern, this example here is an

example of something you do in your head 500 times a day and large organizations really struggle with understanding the impact of today's decisions tomorrow. i will give you a couple of examples. these are machine learning models. little blue triangles and we may have built a machine to detect if the computer system as a current intrusion happening right now. that's a common machine learning model. gives us a score, may be 20%, 20%, goes up to 95% is pretty sure there's an intrusion happening now. yet another model that says here's the type of intrusion. it's got some spaghetti and it, but that spaghetti is mapping out people naturally think. i promise you if that spaghetti is how your thinking about things, it's better to have it on paper then get keep it in

your mind and explain it. we use invisible mechanisms today, word and text which is linear to communicate these decisions. we need a blueprint that is out architecture diagram for a computer learning system and then we have choices. i could send my computer-- might intrusion information to the police. it will flow through the cops and benefit and produce outcomes if i call the police every time it may be costly. you need to try to call the police only when it's necessary. this is the decision model i built with farming experts as part of that big project i talked about, 23 phd's and seven institutions and we are finishing the proposal this week. cross your fingers we will win. it will be a natural for ai and arboriculture. if i didn't have to explain decision intelligence 21, i

simply sat down with my diverse team of experts and said what's a typical farmer trying to achieve and a first they said they want to be profitable at the midyear and a sense we are brainstorming i said is that all and they said they wouldn't want to take any actions to put them out of business in a few years, so that's the second goal to balance. then we talked about the actions that the farmers might take to make choices about how they spread their crops, spraying schedule, choice of crop and hybrid and cultivar. of this gives us a map to understand how all the iot and it ai machine learning technology fits because there are some models that tell us out precision it will impact the yield and another model that says the amount of diseases are contaminants you may have based on strain and here's iot, sensors that farmers might have on the drone or somewhere in the

field that they can use again with ai to interpret the sensor data to know is there a pathogen the starting and as early warning as possible to spray as little as possible in order to achieve our goal from cost and also long-term viability and climate and pollution point of view so it's kind of spaghetti but i promise you have a spaghetti on paper is better than what's going on right now which is invisible and peoples head. itch becomes an artifact and we talk about design thinking. we design a decision. a decision is something we can design. that's pretty radical, but when we do it we realize we can bring all those engineering best practices through decisions. we can qa, continuously improving, it acts as a blueprint that connects end-users, my stakeholders built this. it next them the ai people so

they know where they fit in and there was a wonderful moment because our team has been working together and mothers or before the diagram and i don't think anyone knew how the whole thing fit together and we were like we don't have to explain in words anymore, we have a map and know how it fits together and we know where the ai will fit in. what's going on with decision intelligence? those of you that know me, know it's been quite the slog. i'm flying all over the world trying to talk sometimes with three people in the audience trying to tell people the story. i'm really happy to the extent this matters, gardner has decision intelligence on its ai cycle now and i consider that a big accomplishment. we are also-- ibm, google, he has trained a 20000 people in di she's the other big evangelist

in this space. she's awesome. google her. a bunch of companies have started to identify as di companies, or tech is a water company. these guys are a chart, curriculum labs, medical devices, all people who recognize that if they go beyond the building and machine learning model and in that it into a decision model that sort of surrounding of the machine learning model and connects from action struck him that it will help them be more successful so it's cool. i love we are starting to get notes. this is how kathy defines a di, discipline of turning information into better action at any scale. slightly different to way of saying it, but essentially the same thing. i say di answers the question, if i make this decision today which leads to this action would be the outcome tomorrow. lots of people have different

approaches to di, that my decision diagram is mine. other people of other approaches some people don't even come from text. they are sociologists, economists and more. what is common to us is we have taken seriously the action to outcome path as the core way, technology and science and are asked. we also made it to hollywood. knows what show this is? what is it? it's the good place. this is ted dancing. i won't give a spoiler. if you have not seen season three this is episode 10 and i'm recommend you take a look at it. he's basically discovered di by the end of the series. it's the mystery which is cool and what does ted have? who knows who this is? that janet. is she human? no, she's a ai.

here is ted tried to understand why there are all these negative unintended consequences and here is janet helping him. how cool is that? i hope they read my book. i hope they call me. that would be fun to keep the scene that going because it's so important in the future. with great power comes great responsibility. you have the ability to take action as an individual and as organization that have a giant impacts. i'm an optimist. i believe we are in the beginning of a solution renaissance where it's not just ai, but all of these technologies will come together under a common blueprint, system dynamic, complexity, complementation neuroscience, all of these fields, minimally-- many traditionally viewed as soft field in their little silo,

i think we are entering the age of synthesis and those of you who are doj and a list know what, i mean,. we are the experts who focus on the outside of these boxes. doesn't matter if we know that math, we know what goes into it what comes out of it and restart to know using i think some of the version how to cool it together to crystallize solutions where we understand a solution to water impacts poverty. a solution to poverty impacts the status of women. those impacts hit all of us. they hit government. they hit democracy. they had to companies. we call these the sustainable development goal. i think they are all the same problem. i don't think we can solve them separately and i think the only way we can solve them is to have a new approach to understanding how actions bounce around

through the whole world and ultimately lead to the best outcomes. thank you. [applause]. do have a chair transporter? thank you. >> you actually finished under the time. one question i had is the of the case study or story to tell us that shows how this works in practice? >> well, i think that example i showed with the farming is a good one. we have also build an additional model for government decision regarding reducing conflict in the sub-saharan african country

where the model showed a net was a pulmonary study, but it showed you need to take action in two places at once. you had to do work with the rule of law tree and also had to stipulate the police, but other things didn't have to happen. if you did those two things i was not forced to take the vicious cycle of conflict to a virtuous cycle. >> how long when the test take? >> may be or five links long, but-- less conflict than this one but it was pulmonary. it took a few months to get it done because i spent a lot of time interviewing experts. >> i wonder how much you feel this could be metaphorical thinking because one special thing about people if they think in models of how their like other things and whether you can actually imagine it.

>> i think the future, part of the future of di is to understand their systematic patterns that happen in one place and happen in another. one is the pattern where we taken actually think is good and it is good in the short-term. i call this the lobster claw in the action the short term actually knew leads to a negative in the long-term, but our visibility horizon doesn't tell us that. i see that pattern everywhere. i think of vending machines in kids in schools. makes the kids happy, but down the road may be diabetes and obesity and such good things. >> i like to bring in a couple other people as well. i asked a couple people for comments to kick anything off. bob? [inaudible]

would you like to ask a quick question? >> high, jill. >> i have a background in climate science and right now it's a really interesting time when everyone is kind of weighing in on climate science and i find it interesting because you have a lot of people went to speak to these links. they are naïvely putting in links saying it's solar flares or i have an idea or i have an idea and having studied earth science you want to see spaghetti and yet i think about your metaphor and like this challenge we have about teaching science which is the length and mechanism. or would you be able to not have to teach the science, but demonstrate the complexity of thinking as a way to start to appreciate?

i feel like they aren't seeing the complexity and just to be able to show it. ways to make this a tool to just influence almost like it was people thinking without having to painfully go through it. >> i will use a metaphor to answer your question. i can drive a ferrari without seeing what's under the hood, but certain people are impressed when you open up the hood and you see that complexity, so some of us don't want to hear about that at all. we want someone who we trust to tell us if we take this action it will lead to this outcome. some people we want the ability to open up and so we had a multilayered approach. we have to have beautiful immersive and video games that grab your attention, but the knee up to be able to click on the expert into the thought process that led to the mechanism of model. [inaudible]

>> we make it look easy, but if they look under that the day don't think they can build a ferrari. >> i think we need both. we are obsessed with the good stuff and we haven't paid enough attention to the user interface, the augmented intelligence. [inaudible] >> . >> give anything to say about this? a few scientists. >> thank you so much for a wonderful talking a great book. great insight. i love when you are saying the vision you had with internet was just getting started and how it would be all positive and bring humanity together we could sort of see people in the world and i think the reality in the last few years has been much more

divisiveness than tribalism and breaking up into pieces and i'm trying to imagine a world in which di is everywhere where everyone has tools to make decisions in better ways and i wonder if you think that will help people understand one another bring people together? what do you think the impact will be? >> i think the first thing is it will increase our sense of agency. i think there's a lot of people who feel overwhelmed by the information, the executive i talked to say this a particular. i don't want to talk to those quality guys. i can't understand a word they say and if they have interfaces like this they will engage with the evidence in a more solid way and with the data and the ai and all of this great-- if they get the janet assistance they can do better. not a sociologist, but it seems like we all say it's too complex i can't figure it out so the impact will be our sense of agency to balance out the inequities that come when

peptalk-- technologist sort of dominate the world and send us things and make us do things and click things and we start to use it for our own needs and personal needs so i think we will hit equality. , and optimists. we democratized the computer. let's democratized ai. >> [inaudible] >> lets make this do that poor people. >> bicycle for the mind. >> this technology potentially really empowers people and enables a kind of domain democratization of what today is very complex-- >> let me repeat myself because it's a so important. you don't need to read the book or learn tech, just make sure your brainstorm your outcome and your actions. if you do nothing else, there's

value in that and we're not doing it. we are so lost in our specialty. [inaudible] >> became somewhat of an argument. what angle bar said was you are actually designing a tricycle and what i'm designing what angle bart is designing is a mountain bike. it was of little attention because the steve jobs thing was too simple. doug wanted it to be more robust, but the good news from what you are talking about and one of our rules of thumb is that most really big changes take 30 to make 50 years to be an overnight success, so in a real sense you may be on the verge of that. when was ai termed? >> 1956. di about 2012. >> a long time ago and you may

recall the debate at that time was whether to call it artificial intelligence or augmented. >> that was dartmouth in 1956. >> the wrong side one. [laughter] >> ai one and i think that slowed up that evolution the field, but the good news is now i think it's possible that it's coming together and can actually work and you can have both a tricycle-- [inaudible] >> weigel asked fits into that. she's a brilliant technologist and i'm speaking on your behalf. you may resonate with the angle bart view, you know tricycle, more complex, but we don't have to be either/or. if we take seriously where low friction with humans understanding it and we build the engine, it's like ai feels like we've been building car engines for 60 years, but no one has build a car in their different controls. we need a common set of controls easy for everyone and then we

can be fancy under the hood. its computer science principal. >> another metaphor. >> thank you, great question, bob. >> the possibility of all these worlds and really the story building that is implicit in understanding all of these fields, but also wondering about bias and implied or perception of bias like so when there's a wikipedia entry about one of these, they are our rules that are set up for how people can edit those entries are not. how is it-- would it be possible or should it be possible to have multiple visions of these different characters rather than having only one set of perception of their outcome? >> this is the challenge and i think it will happen over the next-- begin or the next year or

so which they will be a new kind of wikipedia which has rules that maximizes as jimmy figured out, how to get reasonably accurate information from crowdsourcing. but, instead of getting facts, this is what nora called warm and data. we will give connections. we will say when you do this it causes this. when you give this money to this tree charity they will buy 10 trees and then we will have someone that just owns that link and they will curate that and they say here are all the reasons i think when you spend $10 you get one treat and someone else will haven't an opinion when you spend $10 with this charity you get 20 trees and the next says each tree will have this biomass and another expert will say it has this biomass, so critical is that we create again what nora would call a warm data version of wikipedia that talks about links, something leaving--

wikipedia is great, but it is static and doesn't say when you do this it causes that in that connection between things that cause other things that cause of the things is what we have been systematically missing for years. [inaudible] >> exactly, so that's where bias gets help. one thing happening with ai is we are getting unintended consequences because we are modeling the context in which the ai happens so when i start a new ai project. they want to say we have the data and women clinching the data so we are ready for ai and i say put the data aside and tell me what the decision is that ai will be used for because unless we understand the con plex context of the decision that will lead to outcome i may build a wrong ai system and they said you can do this without data and i said yeah data might help and we will build ai later, but-- this is back to software

engineering. if you don't understand the requirements-- there is research lately that says nine out of 10 ai projects fail and this is part of why. >> i like your point about the car. i think that's an interesting point. one topic that came up that i'm aware of is a strong correlation to making women having access to birth control said they can do family planning, a simple link. humans are intelligent all over the world. of this is like a missing link people are not aware of and yet they talk around it. what are we going to do about overpopulation? we need to educate women to make them were economically viable and you say that's a secondary effect. that's a secondary because this is the primary link so i'm curious if you envision what technology can do.

i would love to see that technology somehow resolve when you have these multiple interpretations and somehow you see a way to build a system that could somehow mended that in a way people could see that like wow, that's a bigger link? >> i don't think we have a lack of research that demonstrates these kind of things. i think science has been giving us randomized control trials in the situation with this intervention. we have a lot of results. i think what is missing is taking those individual links and connecting those two actions that i could take today. let's say i believed you and i care about the status of women globally, what can i do about that? show me some evidence, a picture, not evidence in the boring sense, but immersive video augmented reality, make it fun, immersive, cool that i can see as i move this lover and

pick my expert and maybe jill would curate that link and other people curating other links. i move my letter and i see women's happiness. i need some good game designers. we need to have this immersive display that showed these futures immediately as they play out, so that is the piece missing. i think we have great science, great ai, great data but we don't have this last piece where we democratize ai through the chains of the event that you a sense of agency. >> people on this side. >> we are getting all the people on the site. [inaudible] >> it's all behaviors that are predictable and others bigger things to behave on that and that's the difference of the

model than having a brain control and audio and organization or a big company. you can sort of see it cultural religions were other things that people actually don't know what the decision will do in the big scheme of things, so that's quite a powerful adaptive of model. i'm wondering-- [inaudible] >> i think we have been through 2000 years and ai was the extreme example. we can only be intelligent if we can verbalize and represent it as inference in the logic, but there's massive amounts of subconscious rationality that is very smart in this emergence intelligence were individuals in a group, none can explain what they are doing, not even know what they are doing on their own instead somehow the singles are giving each other this emerging intelligence and turns out cultural evolution knows that

and humans have a doing that, but i think we are in a new fish pond. we are in a situation where our natural instincts or well we-- how we babe like birds doesn't work anymore. you asked how di fits into this. i think it's surfaces of assumptions and creates a challenge to the cultural evolution is to really study the emerging behavior of humans as they make decisions and it's an invitation to that part of the world to work together with the fonts and ai and data people of the world in a coherent way. it's a great point that we have these behaviors and the model complex systems of my friend knows a better than i do and he builds a simulation models help us understand how we get that behavior. it's very-- [inaudible]

>> book that i have been starting to read. i hear you articulating much more clearly something i have been spending most of my life bringing to people, so it's very familiar. i guess i'm curious, what have you found effective in terms of when you go into an organization providing consulting our coaching or maybe teaching in a class. how do you get these ideas across more effectively and also as a particular tool that you found that is useful? >> i go in-- i should say naked, but that's not good. i go in with not a lot of studying as i do want to bring bias to the picture. i think it's chapter six. brainstorming outcomes and actions in the new list people sense of the causal change to get you from one to the other.

then, only after we had done that i will go back and it will usually be this ugly whiteboard picture, i mean, it will just be gross. i go back and clean it up and we look at it together. i have it use any special technology. only after we have an agreement to the picture would i bring in a immersive visualization environment and at that point they start to get their intuition going for how the actions need to be outcomes and that can be in process to have the decision desk to show data from the past and present and given the decisions we are making on the decision wherein in the data feed we have from climate and gdp that is life here's where our decisions will go. it's all new. i don't use a lot of tools yet. a lot of machine learning more than anything. >> i think there's a real art to how you present some of this material to people especially in the visualization, the information and the type of

graph or whatever that you want to use and i think it's not regularly done yet. as we talked about we should partner more to make that sort of an art to the craft that's positive opportunity that people can be shown because i do think you can twist people's understanding by different types of graphics and they get confused in some way or put off by the ways and it's not very clear sometimes what to do, so i just sort of-- >> i went to double underline what you just said. are not an experience person i don't have the huge career you have in this space and the so i would like to make an invitation to you as well as anyone in the user experience base-- i'm not claiming these spaghetti diagrams are the place to go, that's untested. i'm not the person to do this.

of those of you who know this space and i hope we get to work the other, they need to be invited and it's so new that it just hasn't happened yet. yes, over here? >> i've so many things crowding my brain might-- right now that i could use one of your decision intelligence to figure out how to get to it. i have been working on this idea on facial maps in the last talk i gave i said i'm going to stop using responsible ai because what's the opposite, what are you proposing because they always plot you and ai for good. are you making responsible innovation. i think a lot of things he talked about, really came together for pieces and portions

that we are working out so i have a technology saying now instead of using responsible ai or something like that we just say let's use best practices, good engineering practices and good decision-making and the question is how do you decide so that's why i'm interested in your talk because i was thinking about how. when you make the comment about technologists who have kind of taken over, i actually find the way we have designed our economy , economics and how do we make it then in that structure, so for example if you handed these amazing tools to the same people who have the same desire for the same outcome, make more money, so if you just go with outcome in action and the outcomes don't change and those people don't get that space at the table to change the outcome then you would just give in amazing tools to the people to create the same outcome in a

much more delicious even may be way, but may be a larger sort of way. anyway, this is what i have been like everyday thinking about like how do we take this amazing-- >> there's a lot there and you are doing amazing work. i went to respond to one point, which is the reason i believe in this is because it's making the invisible visible, so no matter what it does, even if we had bad actors using it, it invites them and insist they dry mouth, i mean, i would love before we made any big government decision that we insist someone draw a decision model and drama collaborative diverse team that all say they have agreed to the model so we surface. what's happening now, executives i talked to it's like the loudest kind of room gets his way or who tells the best story and i think we have to combat

that and the way we do it is by taking the invisible making a visible. >> one last point. mentioned to something and i think what you are saying is the challenges, i really want to work with you to figure out-- figure this out to the challenges, when you have five perspectives and your pack about the approaches and you talk to five different people and put it in, it becomes like an overload. you are talking about decision intelligence and we get overloaded by these variables and different perspectives and then how do you kind of say well, synthesize this synthesis now to a point where again the loudest, richest, whatever person doesn't end up saying we gave you all platforms and you got to say your piece, but now this is what we are doing. >> again, good point. lets me speak to one aspect of it, which is a principal peer to

the main thing he has read the book and that will answer your question. there are methodologies there, but one key best practice in this space is that outside of the box versus the inside of the box. you talk about this thing, technology, agent -based system model, don't let people talk about the math inside it. you will get overloaded. separation of concern. information hiding. a focus on the interface definition. our role is to go a level of abstraction up and to glue the technologies together. we don't need any more insights were answers. we need more science but we have a lot of scientific results sitting around unused because no one connected them to the action to outcome the chain, so yes, we can get overwhelmed, but if we keep using best practices keeping people focusing on servicing their mental model we will overcome that overwhelm and that's the exciting thing. they are these rooms i go in and they talk and 20 directions and

i let them go on per day and then i say we are going to try map in the moment they see it there is relief because they don't have to keep it invisibly in their head. it's rendered in a picture now, a mapping technology in many ways. yeah, let's talk. >> a couple more questions, please. >> over here. thank you. [inaudible] >> 140-dollar book. thank you. so, you have talked about approach,. [inaudible] >> what side? [inaudible]

do you see a section between them? >> yes. >> how is that. >> let's take a behavioral economic thing and look at a cognitive bias we might make sure to that behavior economists that's into this diagram somewhere and will help us learn how when we do this step or when this is true that will cause that, so the way i see that expertise area fitting in is that it informs the link, so the picture is just an integration for how we pulled different understandings of how people work in the picture so whether we model in that theoretic way how another will behavior how a person will behave, that becomes pretty important.

>> i often see they still make a wrong decision, which is grand and i realize you need probably a leader, not necessarily inside this approach you are taking and therefore that helps you to understand how the outcomes are linked. >> what is that book, weapons of mass destruction, talking about an algorithm of model like in chapter two where he will misfired a great teacher and it's a ai system. they don't know why. if they had built a decision model that said here's the variables at this thing is using and here's how the prediction will be used in this larger context, i think they could've surface that unintended consequence of much faster, so i

think if we understand the context in which our ai are used , it's not perfect but at least it makes it visible and subject to critique and we can qa and continuously improve it and all the best practices around the context of ai and also the technology solutions we use. thanks for the question. that's a great. >> any other questions? [inaudible] >> i'm wondering if you can talk a little bit about kind of conflicts and how that sort of-- the role it may play in outcomes it depends on who is involved, who's at the table making decisions to define the outcomes and in your experience or in developing the methodology if you thought through what you do in outcomes are in opposition? >> i believe in many situations outcomes that appear to be in conflict actually aren't and

that if we do these kind of the maps and i'm not saying it's mature, its preliminary work in the fact the stuff we did in africa was to resolve conflict. if you have two people who had different opinions of the actions we could take in the outcomes we are trying to achieve and you capture those on a common map, then ai and some of the technology in many cases can find a holy grail. it can find a set of actions that help you achieve everyone's outcome. because we are better at doing this, because we understand these links and at the same time we are arguing with someone we have to keep these complex links and are headed we cannot, but we don't want to admit we can keep the links in our head so we fall back on eric knit-- arrogance and its assertiveness and yelling and when we don't have to do that and we have a map and it's not you against me in our opinions, it's you and me

working on a common model and so it's not this conflict, it's a let's join together to get a shared view and in this kind of a map will take that process. great question. thank you. when you draw a picture of their mind, that's great. they keeled really heard. [inaudible] [laughter] >> makes you more open to the future. >> wonderful addition, bob. thank you for that edition. any last comments from anyone? >> thank you all for the decision to be here today. i hope the positive outcomes exceeded your time with negative outcomes of your time and effort

to drive here. >> people and how we make decisions together as we pointed out, making the decision to be here together and i often think now it's difficult sometimes to get people out of their houses ended to engage with each other. we do have food for you to continue the conversation with each other. thank you so much, bob. thank you, bill, for bringing us together. also, thank you c-span for being a recorder and also being very in the background, so thank you very much. >> may be a round of applause for gabe. [applause]. >> at the ronald reagan presidential library in simi valley california donald trump junior discussed his book "triggered" here's a portion of

the programs. >> all-time low, unemployment. [applause]. for everyone, african-americans, for hispanics, for women. [applause]. all-time high, startup businesses. [applause]. how about even his haters say i don't like the guy, but he's doing all the things he said he was going to do. [cheers and applause] i mean, trevor noah not exactly a fan is a saying this and i go, isn't that supposed to be what politicians do? when did we get to a place where that's no longer the norm? and how sad is it that it isn't, so they don't talk about that. we sought the last debate was how can we take donald trump's

twitter account away. [laughter] you know what was not mentioned, china. since we are at the reagan library, china sort of your 2000 version of what was then the soviet union. and they didn't even talk about it, but they spent a lot of time trying to limit the president of the united states free speech. if that doesn't tell you all we need to know, i don't know what does, so again i had to sort of get all of this down and put it on paper and have some fun with it and it was sort of nice because most people are used to me of doing the short form bomb on social media. i have mastered that art. i have learned from the best. [cheers and applause] buck, to be will to put it in the long form, to take my experiences, to really spell it

as a people can see, again, it's actually much less of a political book than it is a book on our culture because that's what i believe is at stake. there's never been a greater divide between the ideology of both parties. there's never been more you know oppression and suppression of free speech. we get into that as it relates to social media and the hypocrisy of mainstream media and so it was time to do it and it was my honor to be out there continuously fighting for the beliefs that we all had, for the america ronald reagan wanted us to be able to live and i want to make sure that extends it to our children and our grandchildren and for generations to become-- to come because it's worth fighting for. >> to watch the rest of the program does our website, book tv.org and surge donald trump junior or the title of his book "triggered".

>> book tv is television for serious readers all weekend every weekend. join us again next to saturday beginning at 8:00 a.m. eastern for the best in nonfiction books .. >> historians and scholars explored african american history from 1776 to the present day at an