no agri cultural and and because of the desert group along with it. on the trail and have you been on i-5, another trail and corys -- quarrys out in different things that we would gather like red oak or the paint that would be used for trade.

now, we're looking at the major languages, do y'all see that? okay. bun of the things about california were the diversity of language here. california had more language diversity than any other place in the world. so you could go like ten, 20 miles to the east and you'll run into a group that speaks a completely different language two miles north, different language, so what do you think happened, are people -- our people learned to speak more than one language. how many of you here speak more than one language? doesn't that give you warm and fuzzy. i mean, you know, to be able to communicate. communication is important. that's what we do.

that's what we do. and how many of you notice in the different languages you speak, you may speak something in one language that's not -- it's not easy to convey at times those -- the way of thinking in another language, that's what makes diversity so beautiful because of the different ways we look at things. it's like that diamond, it has all those facets. mike is going to go over a lot more of this. he's going to go, really deep into this. koz molg -- how many have you been over to the museum of man. natural history of museum. you see the thing on the koz molg, it talks about many of the

different things our creation story takes four to six days to be told. i'm not going to tell you about our creation story, it will take way too long. it teaches us where we are today and where we're going in the future. now how much time do i have left. let me check. half hour more. okay. thank you for letting me now. i'm going to tell you something real quick, just a little thing,

just one part. i want to tie this in with main street thought. we say before there was a universe there was something else. and there was a creator, the name that we call. had a younger twin. and this, the creator decided to make no one knows why the creator decided to do what he did, but the creator did it. the creator decided to make this universe as we know it. the creator pushed the younger one through it because the younger one was not as strong. the younger one was scared, closed his eyes and came into this and the older one said, are

you ready for me to come and the other one said yes, but keep your eyes open. the way that it is talked about is that other place was like the difference between here on dry land and salt water, being underneath the ocean how many ever had a younger brother or sister. and put it into it, to us, our people tobacco is a sacred land. it's a connection.

and do this. >> this is the different between an old person and elder, how many have you ever no old person, oh, you better do this or that. just yells at you. and expachbded and expanded. and at the universe. and some people. however, if you ask businesses today, they will tell you that the universe, the known universe as we can understand it is like an expanding bubble. as we say, so.

so with that said they decided to make all they needed light. so the younger one again said, i'm doing to make this. white clay. and it's dull. and it

. >> the word is called motaw, which means a fire within our body. so when the creator made the universe and made us and our belief, part of the creators in each and every one of us, so ladies and gentlemen, when you come together and you talk about strategy building, you're honoring the creator. that's part of the creator coming to you. when you talk about building something, that's part of the creator coming together. when you make something at home and you cook and you clean and when see all of these different things, this is the creator inside each and every one of us. so i just wanted to say, we celebrate our diversity, but

we're all still the same, too. we're all human beings. we all came from the creator and i think -- i believe as this knowledge we come together and we talk about this and inclusiveness, bringing in all of these different cultures, we grow. i talked about hunting earlier. it's a word we use for an arrow and any of you ever went bow and arrow shooting. an arrow can hurt but it's easy to break. each and every one of us are arrows. when we come together with all of our knowledge and put it together, we try to break that bundle. it's a powerful thing. all of us coming here together. we are a powerful vehicle of movement. when we talk about social justice throughout the world, change, equality, equity, all of

these things come with this knowledge, these library ries t we talk about, and inside each and every one of you is a library, too. so talked a little bit about cosmelogy, michael will talk much more on that. observe to observatory, with our people we're governed by the season. we have many different things that we've learned in our different areas. how many of you are from the northeast, any of you. pretty dry over there.

>> we have a lot out there and things in the desert and i don't get too much into that. it shows that we have been, therefore, for thousands and thousands of years. there use to be a great lake over there. we move on real quick.

california has all these missions that goes into the state. and i've seen pictures of mission indians taking care of the crops and they're all sitting together.

and they have what is third grade and justification to the people and to build an infrastructure as a prelude class. it znt happen well. and 1775.

it was burned to the gund, father was killed. on the other side of the border in bah ha, in between there's a place called -- it was named after a mission. they were looking for the priest, and spanish seniority and they lifted up the skirt and the priest was hiding unr under

there. we resisted that. because our believes and our love for our ourselves. >> well mexican government said they were going to secularize the missions, in other words they were going to give the land back to the native people, right? believe me.

new set of revolts all up and down, california specially and it almost fell three times. a lot of people don't know that. and then. united states government in mexico. it spilled over into what? a war? it spilled over into a war. well, guess who came to california mexican soldiers

general cerny came down. the only battle mexican american war was here in san diego. guess who won.

and i knew what happened before. and warriors. what they did? are you enjoying your accommodations? yeah, that was it. or you can stay here in prison, pardon and we're going to give you a gun and you can be part of an army. what would you pick. they said, i want to go. i'm going to serve my country. this became what's called the cholo army and it helped to defend california. but then they were defending san

diego, but they weren't getting paid. if you're not getting paid, what do you think is going to happen. oh, yes, what they would do at night they would rape the restaurants, cantina, all the different things there. any way, all of these things don't happen. so after that in the end who won the war. so then this is where -- i'm sorry, i'm skipping all of this, because there's a lot to this. but then something happened up north, the center for it, gold was discovered in california.

the people that came through, the atrocities that were committed, especially central and northern california. when we talk about the holocaust, there were 251 tribes in california prior to the gold rush, after a few years of the gold rush, there were only like 51 tribes. most of the tribes were just annihilated. they would take whole groups of people and wipe them out. there was a policy of extermination that was going on and there was a bounty for the scalps of native people down south, remember the mexican american war that we were

talking about, well, one of the things that the mexicans did they armed the people, so a lot of them had weapons. it's a lot harder to annihilate a gloup roup if they're just as armed as you are. there were still things that were being done. they had a process of so in 1851 or 1852, january 7th, the treaty of isabel was made and it was going to grachbt nt us a reserv all the way to the mexican border to river side county and from east of alpine all the way to the desert, that was going to be one large reservation. this was one of the 18 treaties that was made with the surviving california tribes.

remember the gold rush, the things that were happening, people were just flooding into california, a lot of people were coming in from the carolinas, georgia, this was 1852, take a look at that date in american history, what was happening then. what was that? and this was a prelude to the civil war. this was the prelude to the civil war, what happened, these people were coming in from georgia, carolina and places like that and were saying, if you ratify these treaties, we're going to side with the confederacy. well, guess what, do you think the federal government wanted to have another group of people fighting with them. they lost those treaties, they were never ratified. this is one of the reasons why california has such small reservations. in san diego, we have more

reservations than any other county in the country, we also have the smallest, which is six acres. so as all of this started happening, laws were made, you could not leave the reservation unless you had a permit from somebody you worked for. you could be arrested and kept in service for six months, all of these things were happening there, too. when we talk about gold, let's move down a little bit farther south. let's go to peru, what was happening in peru, what happened to 1561 or 15 -- and the leader of peru, he was captured and ran

so somed for a huge room filled with gold, once and silver twice, the largest ransome ever made and he was executed. huge, stretched -- i mean, one of the largest empires in the world. but did you know more gold was taken out of california than was extracted over there. this is how bad it got. so these were some of the things that happened. this was some of the things that contributed to the loss of life and the policies of native people here in california. and there's a lot more to it. native religion was made

illegal. do you know when it was legalized. when do you think it was legalized, when we could practice it. 20th century, august 10th, 1978, the freedom of native american religion act was passed, prior to that, we didn't have the same protections in this free country. . it wasn't ratified until clinton was in power in 1990s. it was struggle against that. the boarding schools in an effort to stamp out native culture and language, started off in the 1870s, the famous thing in order to save the man where they would take children away from their homes and they would raise them in boarding schools and cut their hair. a lot of times it would say to clean out the lice. actually, in our way, our hair

is part of our religion. we only cut it when somebody dies. all of these things were attempts to eradicate culture. even the termination acts and the 20th century, indian relocation, all the things, there are a lot of things that have happened, our people still survived. on the other side of the border in mexico, the mexican government didn't harass the native people in baja as much, possibly because mexicans are filled with indians, you mess with one, you'll have to mess with all of them, however, they did have policies of forcing them to speak spanish. they would bring in mexican teachers and the teachers with,

very similar to what was going on here. the different organizations that mexican government works with for the native american or the native people in mexico, i would say at best it's marginal. mexican government, the tribes over there, they don't have the same protections that we had over here. we have reservations, over there the -- the indigenous lands, mexican government could come in and say, if you don't -- if you're not working all the land to our specifications, we'll just take it and give it to somebody else. these are some of the things that people have been struggling with also. how many more minutes i've got.

five more minutes. okay. i'm going to shut up pretty soon. before -- i want to sing a song real quick, but before i say that, our culture has been like a pot, that's been thrown on the ground and shattered into many different things, how many have you ever worked with pottery? i talked to an elder one time and she said, that pot is all of our culture, but we put it together and we grind it up, we grind it up and makes temper. we put new clay in there, which is today and we build it. we make a new pot and we fire. and this is who we are. i think it is our responsibility to remember this, i'll call it the sacredness of every

different culture and to honor that, because with that, comes a way of knowledge and way of being that cannot be replicated and i just want to say to all of you, we praise you for your work. and now before i shut my snoot, i wanted to really welcome you here. i'm going to tell you a quick story, okay this is a blue light special version and this is a camera blue light special version, and this story talks about when the sun and the moon were going to get married.

>> the te male. she said her stomach is growing. the guy looked and she jumped into a pawn. she looked in there and it was filled with -- and she says, oh, gosh, look what happened, what happens when the sun and moon get married. we've got it and they started hopping up to the top of the mountain and they got there and the sun and the moon were

waiting for them and they said, you're late. and two frogs said, son, moon, you cannot get married. and they said why not. they said because when you get married, and look, you'll have -- and they looked at the pawn to all these little frogs in there, singing, they said, sun you're sacred, moon you're sacred, there's only one of each one of you, if you two get together the sky will be filled with suns and moons. they said you're right, we're in love with each other and they agreed the sun will be up while the moon was asleep and the moon would be up when the sun would go to sleep and that's why it is what it is today. but every now and then you'll see the sun and the moon up at the same time. every now and then you'll see a solar eclipse

[ singing ] thank you for everybody from coming here. we wish you all the success.

[ applause ] with donald trump elected as new next president, melania trump becomes the second unborn american first lady. learn more about the c span's book first lady. it's a look into the personal lives and influence of every presidential spouse in american history. it's a companion to c span's well regarded biography tv series and featured interviews with the historians, biographies of 45 first ladies and the photos from each of their lives. first ladies published by public affairs is available wherever you buy books and now available in paper back.

in the morning we're live from the museum for conference on foreign policy, we'll here from house armed services. >> thanks for bringing us together. i think it's very very timely. the session this morning, if you look into your program, is called securing the car for vun

blt testing and coordinated disclose sure programs. some of you are probably thinking what does that mean. that and specifically what does bounty, crowd sourcing of security talk about their role and what they're doing and let's start over to the right. casey ellis, founder and ceo. casey. >> good morning. pleasure to be here. it's amazing to see such a turn out for this topic and this event. i think we're seeing this conversation evolve at an incredible space. it's really good to have you all in the room. so my backgrounds by and clearly

from america, started back around in 2012, actually founded the company and really what it was, was a combination of two things, was realization of the fact that there's this incredible group of good guys think like bad guys and girls that are already at the table and wanting to help. what we're looking at two groups of people that really need to have a conversation, but historically terrible of getting along. need to adjust that and improve that. the other side of it is, i've been in the security industry for become i've become a entrepreneur and did all the things that led today. random penetration company and looking at basically the deficit and how we're discovering vun abilitier abilities and -- vuner abilities to remove the stuff that's there. they can get better at waiting at the next time around. what we're doing today is we've got automation that gets 80%

solved. we try to fill that out, which is where all the bad stuff happens with consultants. 210,000 unfilled cyber security jobs in the u.s. right now. so you put all of those things together. you've got one person being paid by the hour. they asked to come -- being asked to compete to find the vun ability first against the crowd. you've got lots of hackers, lots of different motivations and lots of time. and their incentive model was based around result. when this all started, it was really feedback from a bunch of organizations that i worked with that were more traditional that saw what the technology leaders were doing, google, facebook and so on. this makes sense. this is a pretty logical way to level the playing field and get to a better position where we can win the race, so, yeah, it's a pleasure to be here today. >> thank you so much. titus is the senior manager for fcaucs, titus.

>> my name is titus, i have the least interesting accent here in the stage. i just learned, but just to tell you more about my role, i'm actually in the i.t. organization over at fca and what we're doing as far as the vehicles security program is being sure across multi functional discipline. it's exciting to have my team and i have a team of security experts and they're part of my organization we have a seat at the table, that we listen to and that our input is valid.

when you know your vuner ability, you can fix it. as a result the companies or customers are among most secure in the industry. we're working with car mapping service companies, we're working with general motors, uber, a large giant program here in the industry. as a company, we were hand picked to run the pentagon program, which you have heard of where secretary of defense announced a program where hackers were invited to hack the pentagon. and in just a few weeks we had 1,400 hackers who discovered 138

seve seve severe vuner abilities, they had paid 5 million over 3 years and then they reached out to the community and paid 150,000 and within a few weeks, 138. the first report came in after 13 minutes of opening the program. that's how fast these 15-year-old kids hacked it. >> it's true. >> yes, my name is marta, i'm from finland, originally. i've been in california for the past 13 years, mostly in open shores and now in the securities space. >> great. right back to you real quick, can you describe for us, how does a bug mounting program work, what are the guts of bug mounting program. >> it's like a neighborhood watch you're traveling and ask your neighbors to take a look at it when you're going, you cannot protect it against everything. you ask the world around you to

help you and the program or more broadly, to make disclose sure, that's exactly that. you ask the world around you to come in and look at your software systems and see if they find something. then you put it, just look and then reporters don't do any harm. the cases say these people think bad but act good. you invite them to come in and when they have reported something useful to you, you usually award them for the results. you pay them and the bounty can be as little as $100, the biggest we have paid through our platform, is $30,000 for a single bug that somebody discovered. it was so severe that the company decided to pay so much back to the hacker and the result is then that the hacker is more committed to the company will go back and look for more and you will get more and more more and more and more vulnerabilities found and it is actually good for you. it is as good as it is going to be doctor and doij doing all

those check ups that you don't really like to do but it is much better to know your weaknesses than not to know them. >> it is not always just hackers looking for this programs. we're talk about the vehicles. and people have been tuning vehicles for a long time. trying to get as much performance and coolness out of their vehicle as possible. when you made the vehicles connected suddenly you had people wanting to figure what can i do with the mobile app and the website and these other things? and they are finding. as they are getting additional functionality they are seeing vulnerabilities and saying i wonder if these automotive companies intended to work this way. and some have been reaching out to us saying hey i saw something. and after a few discussions we said we really need a coordinated program for this. to make sure we're communicating with them and those that want to do this understand the parameters. if you are going to do research this is ow you do it safely. this is how you don't break the law and this is how we reward

you for that research. >> so why the chrysler fca starting the bug bounty program now? >> just an evolution of the program. as i said before we've been working with them. there are a lot of passionate people out there. people that like to hack or people who like to test and break things they do it because they love it and they want to communicate and get recognition for what they have done. so they have found ways to reach out to us and we said, you know, what? let's go forward. while it is unusual for a automotive company at this point it shouldn't be. we should have be doing this. and find things that so people report them do and address the risks and also think about it in the future so make sure those are considered in the designs of our vehicles. >> there have been a couple of articles recently since the announcement that $1500 was the headline may not be enough. been some good and bad criticism of some very very positive response. how would you respond if someone

said well $1500 isn't going to be motivating enough for a hacker. >> based on submissions i would say it is a motivator. but with that said i understand the comments and the criticism here. but we had too start somewhere and that is where we are working with our friends at bug crowd that have done this for many other companies to give us an idea where should we start. and we may evolve that and have bigger payouts but at this time we wanted to start. and we'll revisit ive. it is going evolve over time. >> i can add to that. the way these programs work and one of the key mistakes early on was organizations that went out with a number that was interesting to the press more than it was necessarily a commitment that we're willing to uphold to the community. and, you know, what we've seen, you know, we've been running programs for technology companies right through to a lot

of ofrrganizations in more traditional verticals taking up us on the idea. including automotive manufacturers, some public and prooiftd private. the ie deer is that okay. start at a level a this sane. and we're at a point now where we can start to collect data and figure what the a sane starting point and really the number, from fca's standpoint because i responded to some of those comments as well, is more about -- it is not about putting out this flashy number that is not going to be upheld. it is about aligning fca as an organization with starting this community and doing with people who are going to participate and doing it in a way that can be upheld and establish trust over

time. and what you see in these programs is you start at a particular point and basically you reach a stage where the velocity of vulnerability submissions drops below a certain level. and at that point we generally go to the customer and say hey congratulations. you have now graduated from, you know, the level of security that you are going to be able to get feedback on at this level of reward. it is time to, you know, think about upping your game and driving the researchers deeper. >> wouldn't you also say there are other motivations besides money for the hackers? discussion we had last evening -- and any of you can respond to this. but for a young hacker in college or computer science major or someone if they can get that on their resume? are there other motivations besides finance? >> definitely. the initial and preeminent is hackers are going to hack. and we heard that before. and these are people that are just absolutely fascinated and compelled to understand the true nature of how things work and then try to get them -- you know, try to be able to

manipulate them to do things maybe they shouldn't or maybe they weren't necessarily designed to do in the first place so there is that intellectual assent and curiosity is the preeminent feature in the community. beyond that we're seeing a lot of people get employed base off reputation they build. i think the main reason is it is purely on merit. it is this person hacked this company. here is evidence of that. that is actually proof of their skill in the real world and of course cash is king. so, you know, i think as things normalize over time that is going is it steady and consistent motivation that will be applied to the community but the others still exist. >> and to add on it. think about automotive security. how many automotive sus cybersecurity experts out there? there are some names with i know but this allows us to identify those people. say in the future we do a closed

bounty program? these are the people we want to work with because they have a history of finding things. >> the benefit of the disclosure programs are vast. we heard a couple this morning and this opening comments but why are some companies vendors still resisting programs? what are some reasons why some companies are not adopting bug bounties? they must not care about security. that's not the --. the fact is of course i tried to provoke you who --. it has been proven to be not just the best but the only way to detect vulnerabilities in live software. it is such that when human beings create problems, only human beings can find them. and not the same human beings. and not a small group of human beings. we've seen this effect in open source software why i've spent the last is15 years.

and i remember when we came with this database to people they said no i can't use it it's dangerous. and companies decided against open source because it was a cancer and a risk they thought. today if you don't run on open source software you are doomed. and there is a the similar shift happening now in security where the principles of openness of sharing and acting fast are taking over security. and soon we'll look back and say how can we ever had a time we didn't do this? just a question how was people's mind changes and i see evidence it is changing much faster this time. because here we have the secretary of defense launching a bug bounty program more the department of defense which is the world's most powerful organization. they are sitting with nuclear weapons yet they need help from 15-year-old kids. and then one of the presidential candidates starting arguing for bug bounty programs for everybody. so i think it will happen

faster. but it is a shift because you must have the courage to face yourself and say tell me about my vulnerabilities and then in return i'll share my experience with all of you. and that takes some confidence and not every company has it. >> if i can add to that just quickly. completely agree with that as the preeminent issue here. the two others that i believe are in the mix, you know, we talked about good guys that think like bad guys before. i think most people think that the folks that can do these types of things to a computer are bad guys. ha it is preeminent perception of a hacker and in reality it is not true but it is more interesting to talk about crime than it is about good things. that's a part of the reason why i think it is the way it is. the other component is operational over head of dealing with a community trying to give you input. they are at the table, the

process overall has efficiency issues because trying to basically drink from this hire hose. so a lot of these considerations before launching programs can be a blocker and sometimes that can be a part of what we we've tried to make easy. particularly where they don't have a massive tech team, they are just happy to deploy on this type of activity. >> great questions from the audience. so keep them coming. i'm going to go to the audience questions in a few minutes. i'm going to titus and ask you what else is being cone? what are auto makers doing to change the vehicles based on the issues you are seeing? >> i think you have already heard it discussesed here which is considering security at the design phase. including all the other disciplines the other experts and making sure everyone

understands these vehicles are interconnected. they all talk to one another. we engineer as best we can but the threats are evolving and we need to make sure we can respond very quickly to that. >> great. thank you. boy some great questions we're getting from the audience. i want to jump to one real quick. casey or martin, where are researchers offended by the word "responsible" versus "coordinated"? people here may not understand the difference. >> if you don't mind i'll take that one. >> it is a term that very easily gets a moral loading attached to it. that is main reason. i think the term "responsible" has been abused in times where, you know, the reality is the idea of this conversation it's actually been happening the last fifteen years or more. so this is not a new thing that is happening here. it is just starting to pick up a lot of steam and you have folks like martin and i jumping in trying to help it go faster.

but that wasn't always the case. and i think, you know, the kind of counter to responsible to the idea of a hacker being irresponsible, that's been kind of basically thrown as the researcher community in a lot of instances and not a lot are justified. i think there are a lot of cases where there is the element of you are basically getting someone coming along and calling you baby ugly with these things and not everyone is prepared to have that conversation and react to it sanely. so this whole idea of no i don't like that, you are being irresponsible. that is part of the precedent for that. the reason i still like that term to some extent is the responsibility is not just on the hacker side. i think the thing that's becoming more of a feature of basically companies becoming proactive about how they message what the nature of the conversation they want to have is, that actually sets up their responsibility to hold up their end of the bargain. so, you know, there is the kind of -- it is an age old debate

within the hacker community. do we use this word or coordinated disclosure which is more accurate but people don't understand what it means outside of the community. so that is part of why. there is a rich history in the debate. >> i would go back to why isn't it happening yet and here a little blame with you guy who is have been in security for 15 years. you created the world's most complicated terminology. it is impossible to popularize this with such word monsters as responsible disclosure. so collectively we should come up with much easier words to make this an every day part of what everybody is doing. just like in my view the automotive industry always did with safety. embedded it into the design process without making much noise i think. and that is what we need to learn. we need to start from the beginning of the life cycle and give it simple understandable names and then it will work. >> i'd like to apologize for the

language in the -- >> [ laughter ] >> so we have about five questions coming in all around the same topic of black hat, white hat. how do you know that good guys aren't bad guys? how do we make sure ham cker go guys think like bad guys? there is a number of renditions. but starting out with how do you vet who you're talking to? how do you know it is a good guy and he's not going to somehow do evil? >> first of all if you are a bad guy. and guy here means man or woman, young or old. if you are a a bad human being you are already hacking you don't want for any program to start. so what we are doing is just adding good guys to the mix. the second thing is the programs we run reward you only for good results. it is like the skult movement. a gad deed every day and that is the only thing you get rewarded if. if you have a -- inclination,

why would you spend time there? because you get no benefit. and that is the basis. and in sociology we know that bad guys are maybe 1 one in a thousand or so. and i so we know --. and they have good intent. they want to do good. they are a little too intelligent to fit into society. and so they are sitting at home. they are wondering what to do with their lives. and when you give them real work do, they will do wonderful things that are good. so that is how you make sure that the platform is positive. then of course special programs like the hack the pentagon. we did a vet thing and we checked they were u.s. taxpayers and had a good background so we can do that for the specialized groups that we distill out of this giant community. but i would throw it back to you and say how do you know that

your employees are all good actors? and i would say you don't score them the way we do. we keep track of everything they do. we know more about our hackers than you about your employees. >> i couldn't agree more. these people are earning a reputation. inviting these other programs and also are given the parameters. every one of these sites. if you go to the page you are going to see this is the parameters. this is the only place we want you to look, for example. or this is where we want you to focus. don't do this service. don't look at other people's pi.i. we don't want you to go to jail. that way they know hey this is what will keep me out of trouble but allow me still to experiment and maybe have a finding. >> i want to shift gears a little and go back to casey and get a little more broader perspective here as well. bug crowd recently issued its

research on a stage of bug bound boundings. how does the industry compare to others? >> just to quickly tap that, this is a hard pill to swallow but i think the people in this room are the maturity to get it in this topic. you can control where you are vulnerable if you know where it is. you can't control the behavior of an adversary. so are we inviting good guy, bad guys to do these things? is that the right question to be asking? you can't control ultimately the behavior of someone who is intent and has the resources and skills to attack you properly. they are just going to do it. so the task is how resilient are you when they come along? and this is a good way to do that. to speak to the question around the report. what we've seen is just incredible acceleration in adoption. you think as a spectacular you

have facebook and google and the crazy bay area tech companies that really launched this thing and are more aggressive when it comes to their adoption of technology risk and these things. then at the other thing you have folks like the d.o.d. and western union. there is a bunch of conservative companies that are in this mix as well. you kind of consider that as a spectacular of risk tolerance or marketed option really. the consistent trend is it's just moving a lot quicker than we thought it would. and that is driven by the results. it is driven by the efficiency. it is driven by the just severe need to get better at this quickly. given how to consumer demand for newest features and greatest convenience is accelerates. but having to find a way to have security be a part of that. the whole idea of make security easy and insecure obvious for the people building your

products. it is driving demand. they are looking at the precedent being set by the tech companies and saying okay that is kind of scary. that is a novel idea. that is new concept for us. it is going to make some of us uncomfortable but ultimately we're kind of stuffed if we don't do this. so they are stepping in and starting to do it. the other thing is a really strong adoption that we're seeing. we put a fair bit of focus on the vetting piece. so within the community of researchers at the table there are those that understand sometimes you have to wear a suit and tie to work and there are those who don't. and ultimately if you are running a private program or a program in which you are trying to give an elevated level of trust to the people that are participating, you have to trust them more. so we've got a bunch of stuff that we do around that to make sure that's possible. the adoption of that as a way that people are thinking about augmenting or even replacing the things they are doing today when it comes to consultive pen

testing or automotive tools, that is spreading more rapidly and the adoption of that. for every public program we do there are another five, at least, private programs where the customer is thinking about it in that way. >> so do you see the auto industry, do you see this bug bounty go across all in the industry? do you see this -- how many bicker in the next couple years? >> my, as a founder part of my job is to kind of predict the future and see if i can skate to where the puck's going to be. and so far we've done okay with that. which is good. in terms of how it looks moving forward i see five years time in the room everyone is going to be this in some fashion. and it is not going to be because it is cool or because of, you know, any like social pressure or anything like that. it is going to be because you will realize that this is the most efficient way to get things done. and given the asymmetry between what the adversary has at their

disposal and what we're trying to do to compete is we're actually going to be if a poor position if we don't adopt it. i see it actually as inevitable. >> when you think of your role at chrysler fca, what do you think in terms of insider threat versus outside threat? how do you think about that? it could be bug bounty or it could be broader. just the cyber security. 50/50? >> i think 50/50, quite honestly. there are always going to be bad actors and those internally in the system have greater access than those on the outside. but i also think the insider threat isn't purposely trying to damage the company. more they are clinking on that link and responding to e-mails they shouldn't be responding to. i wish we could patch stupidity but it's not happened yet.

but it is going to be a lot easier to identify those making poor choices either intentionally or unintentionally some time. >> software --. >> we see already that everything of value to human beings is being governed by software today. and we love it because we can have mobile apps and everything and self-driving cars. the problem is all software is vulnerable. and when the software eats the world this way, software needs to change. i come from the software industry so i'm guilty as the. and my point is that the auto industry learned early on to start building safe cars. early on i can remember all

kinds of arrangements to keep my life safe. that is mechanical safety. but we need the same principle where security starts at the design phase. and bug --. we have to reflect the knowledge back to the designers and coders so they start developing code that isn't as vulnerable. you can never get to 100% security. never. but in total we can get closer to it. so this whole thing of the future where everything is secure will not happen until we create a software development life cycle where security san every day consideration at every step of that chain. and we need to feed back what we find in life software back to the designers so they reduce the numbers of simple injections or possibilities for overflows and all kinds of things that happen there. and there is a job for the software industry but now everybody is in the software industry so it becomes societal

challenge and problem. >> there is one question from the audience here. do we need to safety safety critical systems to open source based on your earlier comment? >> yes. >> okay. >> i think we've shown that transparency trumps everything when building something you can trust. the essence of security must not be based on secrecy. it was logical flaw to think secrecy leads to security. it is the opposite. when you drive openness you get more security. the more eyeballs looking at the code the quicker you can fix it. i certainly believe so. and the world hasn't shifted 100% to occupy source software yet and the real world things don't happen as beautifully as we would like but we are on a good path. >> todd back to you. a question from the audience. do you see aspects of

vulnerability testing rolled out to dealerships? repair shops as in extension to the 19 point or 30 point vehicle inspection they do now? >> i don't have any insight into that? but i can tell you the tools used for our dealers and mechanics to manipulate the car, to enhance it, patch it and work on it that is part of our information security program. that is something we realize is a possible point of attack. and so it is something we're tackling together with the product development and electoral engineering teams. we take it extremely seriously. >> thanks. over the casey. is there sufficient anonymity enforced in bug bounty programs? any comments on that? >> basically the precedents that out there for how identity works

is a pseudonym -- however you say that word is the -- anyway. hackers have a tendency to use handles. that goes back a million years. not really but you get what i mean. and what it comes down to i think for programs, again, it comes down to how much trust do you require in your interaction with these people? because i think for a public program, a pseudonym will suffice. because ultimately you are getting the vulnerability. you can do something about immaterial you have ultimately a payment flow set up at which point the payment of the person who submitted the -- >> so thank you dark lord duder for all of your submissions. >> yeah. exactly. there are some crazy names out there. but in terms of the behavioral analysis of the researchers we have other tiers that involve

proof-positive identity verification and background checking on top of that. and funded like from an ideological perspective and looking at it overall, i don't think that should be necessary because ultimately it should evolve towards being an open conversation where it really doesn't matter who's involved. you are just transacting data and actioning what you find. but the reality is where we are very far from a stage where this is normalized, as a concept. and we're very far from a stage where everyone is perfectly comfortable from w that idea. so what we end up doing is saying okay, if your paradigm is to require background checks or require proof-positive identification we can provide that to get you other your trust hurdle and get this thing going. nine times out of ten is the customer comes back after the first enkbajt and says okay. we get it now. that helped us get started but it limited the pool as well. so now that we understand how this works and we're starting to

develop more trust in the model we're going to start to relax those things as we go along. as the complicated subject but what it comes down so optimizing for the level of trust that, you know, whichever vendor it is, what they require to get the thing going. because the important piece. >> martin, does the bug bounty program only focus on systems related to risks? or is there also a review of business operational risks that impact security? so is it only system risk? i'm sorry. is it also business operational risk? >> that is a great question. we make sure that our platform anybody can submit vulnerabilities. anybody can receive them. so if you come on board just like that, that is what you do. customers who ask they get additional service where is we go in and right long reports with recommends for them. we participate in the risk assessment and go as deep as they like to go. of course we want customers to develop their own skill and

practice here because it needs to be an intrinsic function in every company. but many of our customers say martin we have just two security people here. we have 200 engineers and just two or three in the security team. we need your help. so like you said there is a shortage of security expertise but we need to make sure it really happens inside the companies and there are certain steps you take. one is you make sure it has attention from the top level. not just from the c suite but from the ceo and the governance committee on the board. we have customers who report to the boards of directors once a quarter how the bug bounty program is going and then you have a ceo who loves the stuff like mary barra who said that cybersecurity is an issue of public safety and it gives the mandate to who's ever in charge. and then you have to make sure the security teams sits close to the engineering team. because it is the engineering team that produces all the problems, right? let's remember that. and the engineers like to be

focussed on the opportunities and security are focussed on problems. and it is difficult to make those two groups work together. so there is a lot of work do in the organization and we do it with our largest customers. >> i want to give 15 seconds to each panel. one take away. lot of great sessions today. but something you would want the audience attendees to think about maybe a week from now. what would that one takeaway you could highlight for us? >> in a week's time it will be interesting for everyone in this room to revisit the thought of how am i going to get started with this? like is this something that -- not even is. but it's like how? what does this look like for my organization? if it's true in five years time my entire industry and most industries are going to be doing? how am i going to be a part of that? >> and the security researching

community is an awesome resource and you need too find a way to harness them and engage with them and bring them out. >> listen to mary barra's keynote once again or two times or three times. take note oaf every single word she said. especially cybersecurity is a matter of public safety. >> thank our panel for a great session. appreciate that. [ applause ] >> all right. good morning. thank you very much for the opportunity to be here today to talk about the fbi and what we're doing and seeing with regard to cybercrime. i feel morally obligated to start off by saying i know i am the last person between you and lunch. i will keep that in mind. aye got 15 minute, give or take for comments and then time for

q&a. i'll hold up my side. you've got to hold up yours with the q&a. four things. first the overall cyberthreat stream. and how we see this affecting the automotive industry, what the fbi is going to prevent and respond to cyberattacks and the public sector and collaboration. and what you can can expect from the fbi if you suffer breech oar victim of an attack. i'm going start with a little story. everybody loves a story, right, about a meeting i went to in march of this year at intel corporation. and at this meeting was a commercial futurist panel. and there are three individuals on this particular panel. one was marc andreessen from andreessen horowitz. the second was peter fenton from benchmark and the third jim gets

from sequoia capital. all very successful and prominent venture capital firms in california. and the question asked was where do you see future growth in the next ten years from a technology perspective? and they weren't all consistent in their responses, but one or two of their responses were consistent and they were mobile. they were quantum computing and out mous autonomous driving systems. these are where venture capitalists see growth over the next ten years so that gives us a good idea what we have to consider moving forward. the big question from us from a bureau perspective and industry perspective is what are we going to do about that today? the current cyberthreat landscape, in general, more complaints, more intrusion, more victim, more losses. and the bad guys are getting more sophisticated.

so we've got that going for us. right? who are the players? nation-state sponsored computer intrusions. usual cast of characters to include china, russia, north korea and iran. we're dealing with multi national cybersyndicates who are stealing information for sale to the highest bidder. the hacked-o-viss are still out there motivated by a number of things. and the bureau we still consider from a cyberterrorist perspective. so we know terrorists are highly proficient at you using the internet for recruiting, propaganda and executing attacks and also know they aspire to gain access to our systems. we know they are not there yet or don't think they are there yet but it is still very much a concern for us. so how to do they operate? increasingly complex attacks on larger targets. combining multiple techniques

and inside knowledge and using social engineering and social media to target employees and also have to be, or i would remiss if i didn't mention the insider threat. it is not just limited to hackers on the outside. it is an insider threat is also a significant problem. talking about disgruntled ploepploep employees who are targeted and employees willing to sell to the highest bidder. what are they after? pretty much everything and anything from an a information perspective. they want access, anything that gives them an advantage. but today we're not so much concerned about the loss of data. a after the sony case it became also an issue of lack of data of lack of access to our own information. ransom data is a performance example. why does this matter to everybody in the room? it is more than attack on your

infrastructure. these are attacks on employees and customers and they are attacks on your reputation and they are attacks on our economy and our security. real quickly i'd like to talk a little about the automotive industry at least from the fbi perspective. most of the folks in this room are all too floor with the cybersecurity risks to the automotive systems. we've heard many panels on that this morning. from our potential the vulnerabilities are the network and autonomous systems, obviously. because new cars and the systems are increasingly connected to networks an attack could prevent vehicles from communicating with each other. obviously autonomous vehicles are especially vulnerable to this type of attack. in my previous job i spent a lot of time working with the california highway parole commissioner joe pharaoh who was very interested in these particular issues and he comes

at it not from a negative way. but he was constantly asking when it comes to autonomous vehicles, who is thinking about these issues? and who is asking the hard questions? and in the wake of the recent tragic accident involving a tesla using auto pilot this month, safety as we heard earlier on the panel is obviously front and center. but it is also critical that security, and particularly cybersecurity will be in the design stage rather than as an afterthought. and it is not just tesla and google pushing the envelope. i'm sure you have heard of george hots who's built a self-driving car in his basement in san francisco. what could possibly go wrong with that? supply chain, i know we're going to talk about supply chain again today. one of many possible scenarios we are thinking about involving

million ware bei mall ware to updates. transplantation infrastructure. hackers could compromise gps and send drivers to the wrong place. ort bad actors could use ransom ware for exchange to get them to the right place. here is what the fbi is doing about the threat. direct comey has recognized the severity of this particular risk and made combatting it a top priority. we're constantly going how we handle hour priorities. and department of justice inspector general recently pushed out a report talking about how the fbi is looking at the cyberthreat and giving us some areas for improvement, all of which we will take very seriously and implement if possible. so for the last hundred years the fbi has worked cases primarily the same way. we've assigned those cases to

investigators that are either where the bad guys are or where the victim companies are. it doesn't work in cyber. so wooep he've had to change th model and it's not been without some pain but now we make case assignments based on technical expertise and where that expertise resides. we've created cyberaction teams. taking our best trained, technically trained agents and computer scientists and deploying them to locations when necessary as a fly team. we are maintaining a constant focus on recruiting, training and retaining cybertalent. obviously we know we need to hire more, just as everybody else does. but we are constantly thinking how about this differently and how to go about it in different ways. the fbi generally would place technically-trained folks in two different job families. as agents or we put them as what

we call professional support employees. which is what a computer scientist is. we're taking a look at whether that is a good idea and the sense is generally i think it is not. so bringing traditional computer scientists on board and data scientists on board and expanding the subject matter expertise we have we know we'll need moving forward. we're trying to provide additional clarity on the lanes in the road. i know it can be confusing to the private sector in terms of who will respond to a particular event and who will do what following an intrusion. we've been working very hard with the interagency to come up with additional guidance and clarity for the private sector. this effort is still ongoing. you can imagine how hard it is to herd the proverbial cats. but we're close. we expect an announcement soon and i would think in the next week or so there would be

additional guidance coming from the federal government. imposing costs. we are doing our very best to impose costs. we're getting better at attribution, fieg uruguay out who the bad guys and are prosecuting them when appropriate. when we can't touch them, reach out and touch them we kmpz them publicly. i was skeptical of at first but it really has had a chilling effect. in march this year we did this with regard to seven iranian hackers. it can be embarrassing for a country if the activity is state-sponsored and it also has consequences to the individual in the event they would like to travel with their family or otherwise. and lastly the fbi is helping our local and state law enforcement counterparts be more effective in dealing with the computer-facilitated crime. providing them with training, equipment and expertise and we expect to continue to do so for the foreseeable future. so what can you expect?

what can industry expect from the fbi if you suffer an intrusion? and where you should be at with regard to engagement with the organization? as vehicle technology continues to evolve the fbi and automotive industry must continue to engage on cybersecurity issues. this is a no brainer to me but if you haven't done it or haven't heard it please develop a relationship with your local fbi office if you have not already. the time to do that is on the front end of before something happens as opposed to after something bad happens. the fbi will do everything we can to share all of the relevant information that we can share with you. but frequently push out flash reports to allow us to share threat indicators and tactics and malware systems to you. and in the event you provide was information we will provide you with feedback and analysis on what you have given us.

so the bottom line is we need your help to allow us to better address these threats. we know the private sector owns almost all of the infrastructure. it is the primary target. and all of the information and evidence that we would need to move forward resides on your networks and serves. but unfortunately, more often than not, law enforcement is not notified when an intrusion occurs. and the estimates are about 20% are reported. so another 80% out there. we understand there are a multitude of reasons a company would not want to report an intrusion to law enforcement but we've got figure a way to get past and and work together. we need to make it routine to companies to turn to law enforcement for help. why? well first and foremost we need to find out who's behind the attack to prevent them from doing it again. that nay not be a company's

first concern which is typically to get back to business as normal but if you we don't find those responsible like i said they will continue to attack. speed matters. the faster you identify, the faster we can go after leads and get you on the right course. the fbi understands your concerns about competitive advantage in the marketplace. loss of investor confidence, public perception and reputation. disrupting operations and dealing with regulatory agencies and potential liability. but the bottom line -- and this is what you can expect from the fbi -- is you will be tweeted as a victim. we'll minimize the disruption to you and your employees. we will protect your privacy. we will not share data about your employees or operations. we will do our best to provide clear rules regarding the information you share with us. what happens to it and how it can be used. and we will share as much information as quickly as we can. let me just wrap it real

quickly -- i think i'm doing okay on time here. thank you again very much for the opportunity to be here today. i applaud the efforts of the automotive industry to date to recognize the mitigate the risks associated with more connectivity. more is coming and i look forward to continuing to work with you on these issues. i'd be happy to answer anies questions th -- any questions that you may have before lunch. >> thank you very much. if people have note cards. questions. please do write them down and we have folks who can pick them up. one question. if the cyber incident does occur and the supplier calls the fbi. what practically can the fbi provide not only in cyber reach response but in areas like handling the media, etc., given the fbi's extensive experience in dealing with incidents across

sectors? >> so a couple of different thoughts. first off -- i'm going the wander if i can. so on the media front we have office of public affairs. each field office has a media coordinator and in the event it is determined that that may be something that a company would want to help you effectively engage with the media. the bureau would, i would think, would be more than willing to provide you with a plan to make that happen. some of the other things that we provide is we have an office of victim assistance. so each field office has a victim's specialist. and if your employees or employees of a company are potential victims, the victim specialist can sit down and talk to your employees about ways to

mitigate those risks and help them get back on track. if it is likely a nation state sponsored intrusion we would be the interface between the intelligence community and other outside agencies that would have that visibility, to the extent we could if we had cleared individual, if there were cleared individuals in the company, we would be able to share that information as quickly as possible. if we can. and lastly, you know, i talked a little bit about the preexisting relationship. right? so you want to have that in place before something happens and the reason why is so you engage in these conversations on a regular basis. you can learn more about what the fbi can bring to the table, what dhs could bring to the table, what secret service could bring to the table so in the

event of something you will know exactly what you can and cannot do or should do. >> any other questions from the audience? okay. >> actually i had this one question but then i thought of something else while you were talking. so i can see that maybe some organizations may be hesitant to report this information. we know reputational risk and all those things are reputational damage i should say. but also is there any connection with your reporting an investigation to the federal regulators, for instance. so there could be concern that they would be under more

scrutiny if they are reporting those things to the fbi. >> a fair question. i was talking with jim trainer before i came out. we were talk about the sony and how the bureau responded to that and how it -- we have tailored our response since that investigation. the answer is the company will be treated like a victim. and the fbi is not going provide opinion or commentary to regulatory agencies about conduct or omissions or otherwise. that is just not in our lane in terms of what we would do and how wie would respond. >> so i get to read this. so i get to make this any question i want.

right? what are we having for lunch today -- no. what measure does the fbi employ to protect the anonymity of a company that reports a cyber incident. so internally we don't. we don't protect the anonymity of companies. when it comes to pushing information out that is relevant or maybe be relevant to other law enforcement agency or to other intelligence community members, we don't identify the company that has either suffered a breech or has -- you know, we have information or intelligence about. so we may refer to a company in a report that's going out to the community as company a or company b. so internally we don't. i'm unaware of any instances

where we had to. but externally we don't refer to -- well i -- sony is an example, right? but we don't, absent prosecution. >> thank you. i think that -- that should wrap it up. thank you very much for your excellent keynote. thank you. [ applause ]

with welcome to today counsel on foreirelations meeti.

i won't go over their long biographies. you have them and the main thing you might want to know is they have all served as chief of staff in the white house. and we're so lucky because i can't imagine people who know more about presidential transitions at a moment when we have a very uncertain presidential transition. i'd also like to welcome cfr members around the nation and the world participating in this meeting through the livestream and we'll hear from them during the question and answer session. and i'ms also asked to let everybody know that the next meeting is on domestic security in the age of isis on november 28th. this panel is about navigating the transition but there really are a lot of transitions that are going on at once. there is this handing over of so many institutions from one set of hands to another.

there is the transition that the president elect has to make from being a candidate to being somebody who governs. in this case that is a big transition for somebody who's never been in government before. and there is also the transition that all the people around him have to make and that we as a country have to make from a moment of very intensely fought campaign to the moment of government when the choices are different. and not everybody may want to make that transition. mack, since we never had a transition quite like this, but of the three of you i think your experience is the closest. in early 1993 you and a governor from a small southern state arrived in the white house after 12 years with the other party in power. talk about that. how disorienting and

transformative is it? and what do you wish someone had told you? >> well i survived number one. amy, first of all it is good to be here. always a pleasure to work with cfr and of course with chief bolten and chief daley. i think you hit it just right. the real key to any transition, of course it is the hallmark of any working democracy, peaceful transfer of power. is pivoting from a campaign -- and this was certainly lay hotly contested one to say the least -- to governing. there is a the key pivot that a transition entails and so much do and so little time to do it. you really have a number of stakeholders in terms of getting a government in place. that is what you eluded to. the cabinet, the white house staff. you have the press to immediately engage with. you have to be sure you remember those that brung you, as the old saying goes, your supporters.

then you need to reach out and broaden that base. and then there are the members of congress and each of whom think they are a pretty important individual. and in many cases you are stepping on the world stage for the first time, and that is an important first step. so all these things to do with so little time to do it. less than eighty days. >> what is the one thing you wish someone would have told you? >> i kept wishing baker would give me details and finally he said you have to just be there to fully understand it. >> josh, you are -- i have to say you are kind of a transition legend, that -- the transition -- the handoff that you managed from the george w. bush administration to the obama administration is considered to be one of the -- very smooth.

how much of that is something that would have been the case if not for your experience on september 11th? >> well, first thank you -- thank you for having us. and it is a privilege to be here with bill and mack. 9/11 had a lot -- lot to do with it. the tenor and the substance of the transition that we worked on, as president bush was leaving office in 2008 was very much a product of the terrorist attack of 9/11. in the sense that president bush called me in, probably a year before the inauguration, maybe even a little more and he said he wanted to be sure that whoever was elected president, that his white house executed the best, most effective, deepest, most cooperative

transition in modern history. in large part because this was the first transition in modern history during which there was a really keen sense that the homeland itself was under threat. and this period of transition is a real point of vulnerability for the country. those of us who have either left the white house on january 19th, or arrived on january 20th and -- and i've done the former twice and the latter once. you know that the west wing is completely empty on the night of january 19th. there is nothing on the walls. there is nothing on the desks. there are computers. but their hard drives have been taken. there is no one in any of the offices, except some watch people in the situation room and

the navy people who serve the food in the mess. otherwise it is completely blank. and so the people that come in to take over our government are doing it with, at least as far as the white house is concerned and many other places are concerned, are doing it with a completely new team. and if the outgoing white house doesn't help the new one get as prepared as possible, there is a real moment of vulnerability for our country. >> do you think that window of vulnerability has closed somewhat compared to where we were on 9/11? >> yeah. it's closed a lot. there have been two waves of legislation that have made it much, much easier for candidates -- the two major party candidates -- to begin the transition planning, which used to be considered measuring the drapes. it is now basically legally

required, which is excellent. and as soon as the nominee has the nomination of his or her party party, the gsa makes office space available and there is money available to pay staff. and so now in our law, thanks to some really good work by a lot of groups, including the partnership for public service, former senator ted coffman and former governor mike levitt. there is the coffman/levitt act which puts a lot of this into law. so now there is standard operating procedure to get prepared and have the resource hopefully without all the excuse of measuring the drapes. >> do you think the resources have been used this time? if you have all these offices are they being productively occupied? >> well -- i mean -- yeah, we'll

see. bill says we'll see. i interacted with both the clinton and the trump transition teams several months ago at events sponsored by the sponsorship for public service. and they have a whole book and all this kind of stuff. and i was impressed that the trump people seemed to be paying even more attention than the equilibra clinton people. i think the clinton people felt like oh you know we've done this dozens of time before we got this. the trump people were very serious, were well prepared. i was optimistic about both. the concern that was in my mind is that two days into the transition, the trump -- the president elect's team basically decapitated their transition team. they purged certain number of

folks associated with governor christie and so took out some of the leadership of the transition that had been going through all off this training. so they lost several steps. my sense now is they have regained their footing. they have time. it is as mack was saying it is an enormous amount of work do. it is a big scramble. but hopefully they are well-positioned to do it proper. >> i >> you have had the experience of going through confirmation as a cabinet secretary for commerce. talk a little bit, both about what that's like to have your life examined, and also how you as chief of staff saw the confirmation process playing out for other administrations. for other people. >> well i doubt there are many people in this room or in this town who don't think the nomination process for so many spots has gotten totally ridiculous. the amount of information, the

number of people, you know, the general counsel of commerce having to go through a full nomination, all due respect to the lawyer. you know, this has gotten crazy. so the whole process is unbelievable time sconsuming. gets people you have a -- offer track. and can destroy someone's reputation for something minor. and there is always one or two people who suddenly get the prize of being the target that somebody is after whether the press or -- >> always going to be somebody. >> there is somebody. and you always hope there is somebody that gets identified real early before you get queued up. so i went as i went through -- when i was announced on december 13th and the hearing, i kept waiting every morning when i get the paper to hopefully see an article on somebody else.

that they were going to be the target. but -- and it ends up almost being like an oral bar test. you know you go before the members at least -- i had never done it so it was kind of strange. and i remember one of the senators finally saying to me, why -- why do you seem so nervous," as i was going up so see them. they don't really care what your answers are. it is a question they want to ask. and i kept thinking it was some exam i had to know everything and all these briefing books and all about the department. and basically i think it was john mccain who said to me. they don't want to know what you know. they want to know who you are. and suddenly the light bubbulb on and it was a very different process through the hearing. it is no fun. if it sounded like it was fun it is it was not.

trust me. >> a couple of names mentioned in this transition. generally mattis for example, who would need a waiver, statutory waiver because he's not been out of the military all that long. what is your perspective on that. >> first of all i have enormous respect for him. obviously, his service to the country and talents and remarkable. i personally think that is -- that's a bit -- i think it will be more of a fight than people think for that waiver. i think the last time donald and harry truman recommend forward bradley marshall. >> the lincoln -- >> so it's not something that's done often. and very often presidents or president elects, you know, think about a military defense because often times you get highly visible, highly talented people. but they kind of step back because there seems to be somewhat of an inherent conflict in that. so i personally think that it may be too soon. but i think he'll -- i assume

he'll go for it. nominate him. and probably win the vote. just the political dynamics of it. i don't know if that is good precedent. that is my personal opinion. >> interesting. one job that doesn't mean confirmation is chief of staff. i wonder if you all have the impression if the chief of staff job as you experienced it is even going to exist in this administration. we have reince priebus, but also steve bannon who was a also simultaneously announced as the chief strategist, senior advisor and in the sense it is not really clear the balance between those. is that normal? is that -- is that a good way to run a white house? >> josh, why don't you start. [ laughter ] >> i was going to say yes.

but the shorter and more correct answer would be "no." and you know -- well, let me answer that. no, it doesn't sound right. it sounds like a big mistake have have them announced to have the senior strategist and senior advisor and chief of staff announced as co-equals in the white house. that on its face sounds like a real mistake in the sense that you -- you need the chief of staff to be the emissariry below the president and setting the strategic course for the white

house. imagine a situation where the president says something to his senior advisors, all right. go do it. and steve bannon stays it means bomb iran. and reince priebus says it means he was, you know, mouthing the words to a beach boys song -- >> is this more of an issue between of who steve bannon is? >> it would -- you know i it would be an issue anywhere having -- the provocative past steve bannon has maxz it worst. but let me add a note of caution to everybody in assuming that is how it will work out.

you could say they will be co-equal. and so you mean they are co-equal advisors to the president. i donald trump as president will listen to these people equally and give their advice equal weight. that is not a problem. there are plenty of people in a white house who as advisor, as private advisors to the president can have equal weight. i served as chief of staff as the senior advisor in that position, in what sounds like a similar position, was carl rove. president george w. bush has no closer, smarter more astute and effective advisor than carl rove. and i'd be the first to say that if i were the president i would probably listen more closely to carl rove's advice than i would to josh boulton'lten's. but there was no doubt when it came to chief of staff when it came to running the white house

and interpreting and executing the president's instructions, president bush empowered me do that and not carl rove. if they mean co-equal in the sense of advice, fine. if they mean in sense of equal authority within the white house, potential disaster. >> let me just add. i firmly agree with the way josh answered that question. i would say it really does depend -- and we don't really know his management style yet, the president elect. and it is one thing to -- you could dismiss the announcement of co-equals as i'm satisfying my base by appointing priebus, announce priebus. instead of just doing priebus as chief of staff where he would have gotten a lot of blowback from the alt-right crowd and the very conservative crowd. so he satisfied that political problem he could have had. and my guess is that's more likely the case. where as josh said somebody's really got to be responsible for the day-to-day.

i don't know mr. bannon. >> a grace note to build on what josh and bill have said. i think josh articulated how kind of the co-equal could be viewed.