the current approach, i see a stronger version among too many political nato leaders to take political risk or spend a lot of money to actually deal with the threat we're facing. as a result of that, every time russia acts and creates a provocation to the west, we have a very muted response. too often this means a bilateral response. there is a lack of political deterrence within the alliance. when russia pushes one of our allies or partners in europe, i think there needs to be a multinational diplomatic response. this will reinforce our military deterrence. likewise i think nato is taking too long to resolve the decision making problem. i think it needs to remember it has already delegated in the past authority during the cold war and secretary-general during balkans conflict.

nato needs to return to these things and not think it's reinventing the wheel. also i think we need to see a change from basic defense budget planning among alliances. there's too many free riders in europe, but at the same time i feel washington is enabling this because we're doing too many unilateral actions. i feel already two rounds of e.r.i., the united states has put money on the table. according to the nato secretary-general's report, nato defense spending grew in the past year by over $3 billion, over european and canadian allies. e.r.i. itself is 3.4 billion, which is a significantly large number. so they are spending more on their national defense but not committing more to nato missions. i feel that before a third e.r.i. is approved or recommended by the next u.s.

administration, it needs to be a multi-lateral e.r.i., one in which both the united states and our allies both put capabilities on the field. thank you very much. i look forward to your questions. >> thank you, very much, for your assessments. the imbalance as you mentioned and vulnerabilities and particularly your maps, and we'll come back to it chlt i would like to develop a discussion first among the panelists and then with the audience. i have one general question. i think it was already referred about the next administration, let's say the first 100 years -- i'm sorry days. first 100 years would deal with the challenge. but anyway, i would like the panelists first to respond to that or respond to some of the other perspectives in regard to

the agenda of the next administration. secondly, the agenda of the many summit in brussels in early 2017. we have at least two dozen challenges or issues that ken weinstein and ambassador volcker and other speakers mentioned. can you cite, for example, some five top concerns that the administration should consider and then, of course, the nato mini summit in brussels in early 2017 should consider. we'll start with you, maybe. >> why don't i defer to my colleagues that are not in the government. >> thanks. you and i did include that in my opening remarks.

just to recap, for the 100 days i think it's important -- dan hamilton stressed this point, too, can't wait for a summit. the new u.s. administration needs to come out strongly on collective defense in article 5. i'm calling on nato to join with u.s. in doing crisis management with these major crisis surrounding nato territory, to renew a call for building europe whole, free and at peace that's inclusive for all europe's democracies and to offer to work together with russia. the post-1989 security order. when you come to the nato summit then, i think it should be an affirmation of all allies on those things. and i would add to that as what i said and as others said, we need to have a greater emphasis on capability development for the future. nato's behind. >> well, i think i think i try

to outline some of that. i think the main point about the political statement, political affirmation is the most important. i think there's an agenda as was underscored here about developing capabilities to move forces to forward defense. i do think the resilience agenda is quite important and it's new, it's different for nato. it's not only for nato, but it's important piece now within that framework. and the south needs huge amount of attention. at which nato will fit somewhere. it won't be the answer. it's not going to be the answer. but there are many things that nato can do. but i think overall, especially first 100 day type of thing, it is this political statement of affirmation. europe free, that's been our focus, but today, europe is fractured and anxious. and unless we sort of get that discussion going again about how

we work on these things together, not just how governments do it, but how populations think we're working on it together, that's probably the more important job, as public diplomacy part of this, getting government i think is going to be very hard. >> i agree with what dan and kurt said. having a strong political declaration at the outset of the next u.s. administration is extremely important. not least because of what dan noted about the burden sharing debate and the extreme criticism of nato at some points in this presidential campaign. so i think, you know, addressing the way forward with our nato allies is extremely important. and keeping up the momentum on resources which has been built up so effectively over the last couple of years. and is, you know, bearing fruit perhaps slower than some would

like, but is moving in the right direction. keeping that moving, it would be important as well. >> i agree with the good ideas that have been proposed already. i'll just reinforce too that i've made and one i didn't get to a chance to include in the presentation. i think it's important right off the bat that we change to these unilateral and align these bilateral responses to russia by having multi-national political and diplomatic responses to russian provocations. if russia threatens any one of our nato members ever again, there needs to be a unified nato response from all 28 perhaps we should all dispel some of the members from their embassies in the national capitals at a sufficient level, but i think there needs to be that kind of a message, that where ever russia pokes any of our nato members, they will face political will he, diplomatically, military,

and even economically, they will face unified nato response. i also feel in the same wie that the next level of eri that the president, administration should say, start off working with the new summit by saying that we expect there to be a multinational eri with a proportional and tangible european contribution to it as well. and then lastly, i'll also say that part of these things that are consistent with these two things is there is a lot of talk about the a2 threats in europe and the u.s. military are particularly moving and having to deal with it. but, i am concerned that there is very little attention and very little activity being seen publicly about a european contribution that nato is facing, and that should be one of our friers going into the next summit. >> before i turn this over to richard, i would like to ask the questions of the panel, obviously we focussed on nato, nato membership, do you see any

role for the partnership for these countries, all the way from finland to sweden and the ukraine and so north and the istanbul cooperation initiative on the gulf states, for example, and the mediterranean dialogue, some of the countries like egypt and israel and morocco, tunisia and so forth. in other words, non-nato members who can also participate in the strategy whether it is in europe and elsewhere. >> i will start off on this one. i think you've touched on something that's very important. and i think nato partnerships have greatly extended the reach of the alliance. the great thing about the partnerships is they are demand-driven. no one is required to be a nato partner. nations decide to become nato partners because they think it's

in their own interest and in turn the allies also see the interest in having a partnership. so in different ways because the partnership activities are tailored to each nation many countries have drawn closer to nato. we have very close relationships with finland and sweden. also a close relationship with jordan. we have increasingly an increasing tempo of activities in kuwait and in tunisia, so there are a lot of countries that see it in their own interest and security terms to draw closer to nato, and this is a real resource going forward. >> just to make the obvious point that those partnerships, the last few that you cited down in the middle east are tremendously helpful going forward as to my area of interest, the counterterrorism area and the more we can foster that and translate that

partnership into actual operational coordination and cooperation with others. >> i would say that the partnership of peace efforts offer a tremendous opportunity to blend them into the overall strategic thought process. discussions like this don't get anywhere, unless you have a master tragedy. you can't tell the nato countries to do this or that if you don't have a nato strategy, for example. i think some thought ought to be given to that. >> exactly. and again, the bridges the communities try to develop one is the partnerships for peace review that cherokee was -- turkey was mandated by nato to combat terrorism and beyond

that, and i would say they tried to reach out, of course, now the situation is uncertain how things will develop in the region. but at any rate, we did produce journals in this particular area, partnerships for peace review as well as terrorism. anyone else? >> just briefly. i think, i mean, i think partnership for peace sort of needs a re-think. it has worked well for 25 years. but in a different, very different context, why it was created. for all the reasons jorge said, we're facing new kinds of issues, and many of the partnerships have now become nato members, and others have sort of graduated. but it is not really coming together in any way that i think is as attune to the challenges as it could be. for instance, the enhanced opportunities partner, now we

have the five countries that joe mentioned, it is kind of a grab bag, and you know, it is good do that, but you know, let's take that further. finland and sweden are frankly in a class by themselves in terms of the value added they could provide, and we could take that even further. i have argued for a whole other tier of partnership that would basically get them as close to being a member as they can. i think the strategy frankly give the swedish and finish debates is to get as close as you can, so by the time the question is really asked in sweden or finland, everybody knows what the answer will be. instead of trying to push it too hard, have it backfire. but also, that means incremental partnership, get them to be value added contributors. i mentioned resilience. that's a whole other area. why wouldn't we do partnerships in the resilience, forward resilience. we need to know how resilient ukraine is. all sorts of way it could be

disrupted. otherwise, a lot of our he have for -- efforts won't be that useful. we have proposed resilient support teams, nato support teams, go in, experts, work with a country on where they need help, with their problems. that's applied at the moment only to nato space or eu space, but i don't know why we don't do advisory support teams like ukraine. it gives us a new plank. japan and south korea wanted to be partners, they have a he been rebuffed. we have an article 5 commitment to those countries as well. we should see how we should develop that so they could also provide active toos. australia is an enhanced opportunity. i think there is a rich area for new thinking in the partnership

project. beyond where it is. >> anyone else? okay, richard, do you want to -- >> do you want to open it up to the audience to have questions? i think we have colleagues with -- >> please identify yourself and make your questions brief. we'll go a little over the -- >> yeah daphne, patomac institute. in view of the upcoming elections, where do you see the support, the u.s. support fortuna for nato going? in any case, where would the pedestri pendulum swing? >> could we take a couple of questions, sir? >> thank you so much for your panel and insights.

mr. weinstein, i guess how you can fit in this panel, you mentioned, but there is -- being from the russian embassy, definitely support as main threat in the world as part of terrorism, which our countries russia, better position to eliminate in this world if you work together. speaking of advantages in 2001, september 11th, you might remember, president putin was the first one to call president w bush and offered assistance and federally, sympathy for the people who were killed in new york. also, russia supported the information, isis information in

afghanistan, security council in new york, providing logistics. now we don't have it. nato suspended all the military cooperation very much intel cooperation. i have hardships to go through pentagon and so on. it doesn't contribute to security. that's the situation i have now. i understand the electoral mood within the belt way and definitely russia, it is kind of supports, nyou know, to put russia in front of this campaign. war mongering, opportunity and draw circles and say what will russia do next, threaten united states. that's -- i understand it is a

fading political campaign election, electoral campaign. >> sorry, is there a question? >> yeah. to be short, i don't approve the messages that my country, you know, broke the trust and did everything wrong, and, you know, doing hardships for the united states and other world. let me remind you that nato was the first one to break trust by building up personal around the borders. and provoking arms race. what kind of resilience does the alliance project forward, like in ukraine? supplying arms there and many here go for more robust measures. so is that the kind that you call stability, peaceful

alliance, which doesn't dictate new one. >> that was the question? >> the question, do you see any -- what exactly can nato do to resolve the -- resolve the craz crisis in ukraine and stop building up military, build up from this -- on its eastern, secure of europeans, russians, americans. thank you. >> thank you for the questions. the two questions are on the new u.s. administration, and what's the view of the panel, vi vis-a-vis -- >> sir, one more question, right here. take the third question. >> right here in the front. sorry. >> yeah, just two very quick, just building on the -- yeah,

sorry. embassy of canada. just building on the question from the potomac institute, if we have a trump administration who has already talked about japan and south korea and needing to better support themselves, how does that translate potentially to allies, especially those who doesn't necessarily live up to the 2% defense spending commitment? and then just about resilient, we saw a pro russian president, nato has done a lot of work with them. how do we go forward, not just them, but other countries in that area who may be lean ago way from europe? thank you. >> i'll take one more question in the back. >> the defense, well, thank you for representations, and there will be a lot of questions, time

limits, there probably are, so i'll be very short. one of the nato abilities is the time which is used to decision-making for decision-making process. and some of the aspects have already been improved, and eventually, it could be one of the aspects of the appointment of the new secretary general for intelligence. due to the fact that it is one of the most important issue for a lot of planning. but the -- do you see any other possibilities to increase the speed of the decision-making process in nato, even in that situation? thank you. >> thank you for your question. turn it over to the panelists if they want to tackle the questions. >> i'm happy to touch on these quickly and unfortunately i have to run as well, too. first of all, u.s. support, i

think that the american public has a great reservoir of support fortuna for nato, and the countries that make it up. has a degree of frustration that we don't want to feel like we're doing things for the european security that they don't want to do themselves. that situation gives the u.s. president to define the issues about what the u.s. interests are with nato, and to lead a path forward that affirms that commitment, and also tries to tackle that sense of frustration. so i think there a lot of running room for a u.s. president, given the state of the public. on the question as to what nato can do to increase stability in ukraine, it is important to remember the reason there is a crisis in ukraine is because russia invaded an continues to support separatists there, has troops there, has annexed part

of the country. what's important is that nato provide additional support, political, economic, financial, military, to the government of ukraine, so that it is able to resist this more effectively and then create some stability. as far as moldovo goes, unfortunately, there isn't a whole lot we can do in the short-term about reversing this. one of the things that we do cling to is the ability of countries to have their own political systems and make their own decisions. what we see is through means like russia influencing that, and then producing an outcome such as we had. i think what we can best do is to continue to support our own values and do that in a very visible and public way, which i think public will resonate with public over time. and to make sure we're communicating that effectively. and finally, speeding up decision-making, that's an excellent question, and jorge

was right to put that into a slide. to me, the logical thing always would have been to make a clear distinction between political decisions and implementation of political decisions. and it is possible, i believe, to preserve for consensus at the north atlantic council, decision-making on political issues. but then once decisions at a political level are taken, even if it is well, well in advance, it ought to be able to then be implemented through the nato authorities, both civilian and military, without having to go back for additional political decision-making. and that is the part that always slows things down. i remember in the kosovo,

afghanistan, it always came back for the knack for further decision. we taken initial political decisions, future big political decisions, but in between, we need to let nato authorities get on with their work. >> just, i just wanted to mention on the moldovo and the point, i mentioned the book we have out, "europe's gray zone." we're facing a vast space of europe, non-nato or eu that is turbulent, it is violent. there are lingering issues, not only russian engagement, but corrupt elites, legacy issues, those trying to block it within many of these countries, and these -- the people of that region don't know where their future lies. i would argue, we haven't been all that engaged to help them

find that out. so i don't -- this is my other point. i don't know that nato leads on that. frankly, the european union should be having a real re-think of their eastern partnership policies. the eastern partnership has been the way the eu has engaged these countries, particularly the ones we're talking about. i think people in the region don't understand because this is a process like many of the eu processes, a country like moldovo has to do all this stuff in order to really, noy, move forward. it is not going to become lux luxenburg tomorrow. so far removed from public attention, that people in moldovo, people in ukraine, people in these countries, they don't know what the eu would mean to them. there is nothing tangible. i think a tailored approach to the eastern partnership would be

attuned to the basic needs of those countries, and how you deal with it. and be a bit more willing to deal with the corruption issue. in fact, how we enabled the construction. the banking fraud in moldovo, all the money funneled out of moldovo ended up funneled through latvian banks. there are thousands of examples of this. we don't even enforce our own laws to allow this to happen. things we can do to ourselves to help moldovo, we don't need to go there. but we don't do it. so i think there is a lot of stuff that we're going to be facing this unstable east for some time to come. the european commission said no enlargement under this current commission. i would bet there is no enlargement with the next commission. we're facing a decade, at least, of uncertainty and instability. we to create new tools by which we engage, an nato is one, but not the only one. we have to be realistic this is

the europe we're facing. it is not fixed. it is not a stable place. it is in fact the potential for violence continuing is actually very high. particularly in part of europe. if there is one lesson of his tore we've learned, i hope, whenever we turn away from the gray zones of europe, we always pay a higher price later. we need to engage up front. think about these things now. >> if i could just add, with respect to the public support question, the pugh center has done a lot of public opinion research in the united states and nato countries on this. if you look at their data from april of this year, among people who lean republican, 75% of them said nato was good for the united states. among democrats, it was 81%. independents, 78%. they even broke it down for whom you support. supporters of donald trump, who has been the most critical of

nato, even among those, even among his supporters, 64% said nato is good for the united states. that speaks to the reservoir of understanding and support in the u.s. public. with regard to the question on ukraine, i agree with curt voelker. don't agree with the facts you laid out. i think the facts are different. if you take one bit about the allegation that nato is moving its military closer, if you look at the numbers of forces that nato has placed in the east, they are in no way a threat to russia. and it was only done after a long and careful evaluation chlt thanks. >> i agree with what was said about the other answers. i'll adjust believe focus on the decision-making. i think it is time fortunate tow

to, for the north atlantic council, the political leadership to realize it is not in a military implementation business. that it has created these forces t can create whether these forces enter combat. but i think it should defer to sacu and other military commanders the authority to train these forces, to have snap exercises for these forces, to move them two to three times a year across the alliance, it will improve readiness, which is a key need of the alliance and it will improve deterrence in europe to have these forces both more readily deployable and quickly deployed by the military commanders. >> just a quick note, also on the decision-making question. first of all, let me be the first to freely admit that of course, there is room for improvement in nato decisions making, having spent many of hours of my life in windowless rooms like this.

but having said that, it is our experience that, and of time of real crisis, nato can take important political decisions very quickly. i was at the u.s. delegation at nato on 9/11. and within 24 hours of the subject being put on the table of the declaration of article 5, the nac agreed to declare article 5. a couple of days, when the united states requested certain support from the allies, within 24 hours, everything the united states requested had been granted by the allies. so when there a crisis, sometimes the nac can move quickly. doesn't mean we can't do better. >> once again, we're way over time. so that's why you've asked me to talk. i want to thank our distinguished panelists for a very, very discussion here. i would only add that -- of course i'm an old-timer now.

when i have the privilege of commanding the nato strategic reserve forces in the late '70s and early '80s, we invoked, we didn't worry so much about central so much about central europe. our whole effort really was on the flanks of nato, and we mad a maritime thought process and a maritime strategy. we had sink land and sack land, as you would call it, and the like. and we talked about a much larger kind of a thought process and a much larger kind of an integrated and adaptable strategy. i go back to what i said earlier. you should have -- there should be a nato strategy, and there should be a free world strategy. we don't have to worry so much about russia, i don't think, if we just put our thinking caps on and remember that it's in nobody's interests, russia's interests or anybody else's interests, to get too fancy to

start a problem. we're in an era of globalization. technology is a part of it and of course the military. we simply don't seem to have that kind of not only national, but international thought process going on. so if i were asked to advise the new president, i would think along those kind of lines, but don't worry they're not going to ask me. thank you all for being with us. [ applause ] the election is in four days and coming up today live on our companion network c-span at 7:00 p.m. eastern donald trump is holding a campaign rally in

hershey, pennsylvania. this weekend on "american history tv" on c-span 3 saturday night at 8:00 eastern on lectures in history collin callaway, history professor at dartmouth college on native american history from the colonial era through expansion. >> they presented themselves as allies and friends for our future are clearly our enemies. they're occupying our lands with troops, which was the one thing we were fighting against. and at the same time by cutting off and refusing to give gifts, limiting trade with us, that's essential a declaration of hostile intent. later at 10:00 on real america, we look back to the 1966 campaign for california

governor between edmond pat brown and republican ronald reagan. >> my experience has turned me inevitably toward the people for answers for problems just instinctually. i put my belief and faith into the private sector of the economy. >> and in every single solitary category that tells whether california's economy is good has proven that we have done a good job. >> at sunday morning at 10:00 eastern on road to the white house rewind. >> next tuesday all of you will go to the polls, will stand there in the polling place and make a decision. i think when you make that decision it might be well if you would ask yourself are you better off than you were four years ago. >> our proposals are very sound and very carefully considered to stimulate jobs, to improve the

industrial complex of this country, to create tools for american workers, and at the same time would be anti-inflationary in nature. >> the 1980 debate between president carter and former california governor ronald reagan. a realist would not have devoted his life to fighting slavery or said this a disillusion for the cause of slavery would be followed by the war. so glorious would be its final issue that as god shall judge me it will not be desired. author of "quincy adams, militant spirit," and another

discuss if he was a realist. they talk about the foreign policy views and the legacy of the sixth president. for our complete american history tv schedule, go to c-span.org. the carnegie endowment for international peace and carnegie melon university cohosted a series about artificial-hosted series about artificial intelligence and consumer privacy. this is an hour and 45 minutes. good morning, everybody. welcome to the carnegie

endowment for international peace. together with david brimley, we're delighted to welcome all of you here in person and for those of you joining us on the live stream online. the hashtag for this event is #carnegiedigital. i have the pleasure of introducing ambassador bill burns to you. thank you very much. [ applause ] >> good morning, everyone. welcome again. let me begin by congratulating tim and david and their carnegie endowment and carnegie melon colleagues for putting together this collqium.

as president of the carnegie endowment over nearly the past two years and as a diplomat for 33 years before that, i've had the privilege of welcoming heads of state, military generals, foreign ministers, university presidents, and distinguished thinkers and doers of all stripes, but i've never had the privilege of introducing a robot, let alone several. it is a pleasure to welcome them to today's event. like all of you, i look forward to getting a glimpse of our robotic future later in today's program. robot's are not today's only first. today is also the first of two events we're holding for the first time with carnegie melon university, one of the world's premiere universities and a fellow member of the impressive group of institutions founded by andrew carnegie more than a century ago. the foundations of the

international order that had prevailed for most of the 19th century were beginning to crack. catastrophic war and disorder loomed. and the last great surge of the industrial revolution was transforming the global economy. the carnegie endowment together with a numb of its sister organizations sought to help establish and reinforce the new system of order that emerged from the two world wars, a system that produced more peace and prosperity in the second half of the 20th century than andrew carnegie could have ever imagined. it's hard to escape the feeling that the world is once again at a transformative moment. the return of great power rivalry and the rise of conflict after many years of decline, the growing use of new information technologies both as drivers of human advancement and levers of disruption among countries, and

growing pressures of economic dislocation and stagnation, and the rejection by societies in many regions of western-led globalization and the embrace of an angry fortress like nationalism. here at the carnegie endowment we're trying to meet these challenges head on across our programs and six global centers. we focus this and our partnership with carnegie melon on one of the most significant of these challenges. technology's capacity, as all of you know very well, to simultaneously advance and challenge global peace and security is increasingly apparent. in too many areas the scale and scope of technological innovation is outpacing the development of rules and norms intended to maximize its benefits while minimizing its risks. in today's world, no single country will be able to dictate

these rules and norms. as a global institution with deep expertise, decades of experience, and nuclear policy and significant reach into the some of the most technologically capable governments and societies, the carnegie endowment is well positioned to deal with these gaps. working quietly with government officials, experts, and businesses in key countries, our team is developing norms and measures to manage the cyber threats of strategic significance. these include integrity of financial data, systemic corruption of the communication technology supply chain. our partnership with carnegie melon seeks to deepen the

exchange of ideas among our scholars and the global community wrestling with the range of governance and security issues. tim and david have cure rated an exceptional set of panels with diverse and international perspectives. on december 2th, we'll reconvene in pittsburgh for an equally exciting conversation on governments and cybersecurity numbers. our hope is this conversation will be the beginning of a sustained collaboration between our two institutions and with all of you. there is simply too much at stake for all of us to tackle this problem separately. we can and must indeed tackle it together if we hope to sustain andrew carnegie's legacy. i'd like to thank the carnegie corporation of new york for

making this possible and for everything they have done and continue to do to contribute to a more peaceful world. let me now thank and welcome to the strategy an extraordinary leader of an extraordinary institution and a terrific co-conspirator in this endeavor. thank you so much. [ applause ] >> thank you, bill. i also want to thank tim and david for all their efforts. welcome to the inaugural carnegie colloqium. first and foremost, i would like to thank ambassador bill burns for hosting this event today. as two organizations that reflect a strong legacy of

andrew carnegie, carnegie melon university and the carnegie endowment for international peace have formed a powerful partnership to examine technology and diplomacy across a set of emerging areas critical to our collective future. it's my sincere hope that this event as as well as the follow-up meeting which will take place at carnegie melon university on december 2nd form the basis of an even broader and closer relationship between our two institutions. let me also add my thanks to the president of the carnegie corporation of new york who provided support for both of these events and in fact is based on a conversation that ambassador burns and i had a few months ago.

dr. gregoryian was very supportive and enthusiastic of this effort. to understand carnegie melon's importance, we must first recognize cmu as a place where pioneering work in computer science and artificial intelligence took place decades ago. ever since herbert simon and allen novell created the ingredients of artificial intelligence in the 1950s, before the terminology was recognized broadly, cmu has remained at the cutting edge of this field. carnegie melon created its software engineering institute, which has served the nation through the department of

defense and served industry by acquiring, developing, operating, and sustaining innovative software systems that are affordable, enduring, and trustwort trustworthy. designing safe software systems and attempting to create the learning abilities of the human brain where natural progressions toward two of the modern world's most pressing concerns -- cybersecurity and privacy. to meet this challenge, carnegie melon, cybersecurity, and private search is encompassing a broad range of separate disciplines. it incorporates faculty across the university in areas such as policy development, risk management, and modelling. our aim is to build a new

generation of technologies that deliver quantifiable computer security and sustainable communications systems and the policy guidelines to maximize their effectiveness. tmu's premiere research center on the subject is a visionary private/public partnership that's become a world leader in technological research, education, and security awareness among cyber citizens of all ages. by drawing expertise of more than 100 cmu professors from various disciplines, the lab is the world leader in the technical development of artificial intelligence, cyber offense and defense, and is a pipeline for public and private sector leadership in organizations as varied as nsa

and google. the work of a lab professor, for example, was featured in nova program and "60 minutes" report on machine learning and many other aspects, in particular his facial recognition programming helped match a very blurry surveillance photo with the boston marathon bomber from a database of 1 million faces. you will have an opportunity to see the professor's work in action today during the lunchtime demonstrations downstairs. today, you'll also hear from the lab's director david brimley, who led a cmu team just a couple mont months ago that won this year's super bowl of hacking, darpa's

$200 million cyber challenge. congratulations. thank you, david. just a week later, a week after that, david took a team of cmu students to deathcon in las vegas where they won again in another hacking competition. finally, you'll hear from andrew moore, the dean of our school of computer science who was also recently featured in a "60 minutes" report on artificial intelligence. i would also like to acknowledge dr. jim garrett, the dean of the college of engineering at carnegie melon university who joins us along with rick sieger who played an important role in helping put together this event between carnegie melon and the carnegie endowment. cmu's advancements in cybersecurity will be

highlighted in the meeting today, which is an outgrowth of the partnership of our two organizations. you'll learn more about this in the two panel discussions today. we hope that these discussions on the future of consumer privacy and autonomy and military operations will lay a strong foundation for future meetings, and they will better inform ongoing thinking and technology and diplomacy in these critical areas. i would like to welcome you to the colloqium today and close by again thanking ambassador burns. thank you. [ applause ] >> so we will now get started with the first panel discussion. before we start, let me briefly outline the two key ideas that have been driving this event and when david and i started with the planning for this, the first one was essentially to bring together the technical experts of carnegie melon university and

the policy experts from the carnegie endowment. that's why each panel have a presentation with one of the experts for carnegie melon university and then followed by discussion. the second idea was to draw on carnegie melon's policy. speaking of pittsburgh, if you're interested to join the event on december 2nd, drop your business card off outside or send us an e-mail. i would now like to introduce andrew moore, who is the dean of the school of computer science at carnegie melon university and the computer science school at carnegie melon university has been ranked as the number one school by "u.s. news" repeatedly in the past four years for the grad school program. andrew, prior to becoming dean,

was vice president at google for google commercial and was named as a fellow for the association of advancement of artificial intelligence. >> thank you very much, tim. so this is a really interesting and exciting time in the world of artificial intelligence for many people. for regular consumers, it's got great promise for them. for companies, it is a differentiator. for societies in general, we have options here to make the world a much better place through carefully application of technology. what i would like to do to set the stage here is talk about two things, which at first sight sound like clear goods,

personalization and privacy, two extremely important issues. then i'm going to run through a series of cases where these two great principles start to bump into each other, and they will get increasingly sophisticated. and by the end of this stage setting, i hope to have everyone squirming in their seats because it is so annoying that two important things, privacy and personalization, which seem like clear goods, lead us to very different societal and technical challenges. all right. so let's begin with privacy. it's a clear right, and we would almost all of us agree that anyone who intentionally violates privacy by revealing information which they gained in confidence is doing something bad. and there's laws in our international legal system and

domestic legal systems that deal with that issue, so that's important. personalization. personalization is probably one of the most critical features of a world based on artificial intelligence and machine learning. i'll explain some places where it is obviously good. many great institutions, including carnegie melon, are pushing very hard to understand how we can help children learn more effectively. if it turns out that i, as a child, have problems with understanding when to use the letters ck while i'm writing, it makes a lot of sense for an automated tutor to personalize their instruction to me so they can really practice that particular issue with me. no doubt about it. that seems like a sensible thing. if i'm a patient in a hospital and it becomes pretty clear that unlike most patients i cannot

tolerate more than a certain amount of ibuprofen within 20 minutes of a meal, as we learn that, of course it makes sense to personalize my treatment. so that is good. in the moment, there's no difficulty involved. here's where it gets interesting. some aspects of personalization, like for instance how i'm likely to react to some liver cancer medications, it's not like we can personalize it as we look at what's happened to me over my lifetime because probably this is the first time i've ever had those medications. when you're an artificial intelligence engineer building a personalization system, the way you power it is to find out about me and then ask the question, to make things good for andrew, what should i do and what can i learn from other people like andrew?

that's suddenly where you begin to see this big conflict. other people like andrew is something which can help me a lot because if it turns out that everyone who is over 6'3" with a british accent is virulently opposed to the electric light orchestra, it's an extremely important thing to know so i can make sure that's never recommended to me. so it makes sense to use other people's information in aggregate to help personalize things for me, and in many examples that can really make things better. recommendations for movies is an obvious one. then when you start to think about information on the web, for example if i like to browse news every day and we know i'm typical of people who perhaps in the mornings are very interested in policy-related news but in

the evenings when i'm relaxing i like technology-related news, then that's useful information. this is the upside of personalization. personalization uses machine learning. machine learning is exactly the technology that looks at data, figures out the patterns to usefully say what would other people like andrew want. it's the thing which powers ads in g-mail. it's the thing which powers movie recommendations, and it's the thing which helps the personalized medicine initiative figure out how to treat you who probably need different treatment from someone else. now i'm going to go through four examples of increasing squirminess of why this stuff is hard, why privacy and

personalization actually start to conflict with each other. the first case is one where i don't think we actually do have any trouble with policy. it's a simple case of what we'd like to think what society is going to do. if someone published unauthorized data about me, they are breaking the law and that should be remedied. that is the simplest case. and the responsibility of a good company or a well functioning company is you have the legislation in place, you have clear rules, and if somebody does look up the bank account of a famous celebrity just so they can blog about it, that person is going to get fired. in some cases if the consequences are serious, there's more significant penalty. cases two, three, and four are ones where it starts to get a little fuzzier. case two. someone uses your data in a way that you didn't expect, but it

turns out you kind of agreed to it. a famous example is a firefighter in everett, washington, who was suspected of actually starting fires, and one of the ways in which the police really got to understand that this was a serious risk was they then went to their grocery coupon supplier and looked at the things that this particular person had purchased in the last couple of months, and they found a huge number of fire starting kits in that case. in another case where someone was suing a supermarket for a slip and fall accident, the supermarket produced sales records showing that they were buying excessive, in their eyes, amounts of alcohol. both of those are not actually illegal. both of those were covered in

terms of service. that's difficult. at that point we have already hit something where the general public is going to be very uncomfortable, and it's the thing which means that we all feel uneasy when we sign these terms and services. those are difficult ones, but now i'm going to get to the ninja difficult ones which are just beginning to emerge and make things very interesting for artificial intelligence engineeriengineer i s who are trying to do good, but could accidentally do bad. but this next example is where we're using machine learning to really help people, but inadvertently, accidentally, the machine learning system starts to look like a bigot or make decisions which most of us would think a reasonable human would not make. a good example of this is a member from jim garrett's

faculty, who showed a little experiment with google's advertising system where he looked at the ads, which were shown in response to a query about job searches. and he used google's personalization system to give exactly the same queries to google when the revealed identity of the user was male and when they were female. horribly, it turned out that the ads showed when the person was revealed to be female were for jobs with lower pay. which you look at that and anyone would think if that machine learning algorithm was a person, they're both a jerk and doing something illegal. this morning google using an

ethnic terminology system has fallen afoul of a certain issue. why would a machine learning system do this? none of the engineers were -- i would very much assume that none of the engineers had any intent of causing harm. the reason was the machine learning system had just observed in the data prior to this that all else being equal, which is a very dangerous phrase to use -- it was seeing the women who were clicking on ads tended to click on ads for lowering paying jobs than men. so this machine algorithm has a kind of defense. it can really say i'm just showing people what they're most likely to click on. it's not my fault that society is set up in such a way that my data is showing that women are clicking on lower paying ads. now this is complicated, and i don't have the right answer for

you. if it helps, i should notice that this experiment is particularly unrealistic in the sense that it's very rare a machine learning system only sees an identified gender. usually the machine learning machine sees more information about the person. so it will actually -- you will find there are other features of that person much more important than gender or race for showing their particular interests. it still makes us feel uncomfortable. so that is what i would regard as the most difficult part of machine learning and personalization at the moment. it is very hard -- and i do not know of a piece of research that i fully trust to prevent these things from being, if you like, bigo

bigots. finally, i'm going to mention the ninja hard case. this is pretty simple. it is the case if you really want to preserve privacy, you can cost other people their lives. there are examples of this in many law enforcement situations, but another simple one is in medicine where if you're involved in a drug trial and suppose you had 20 hospitals all trying out some new drug treatment on 20 different patients, then it is difficult in the interests of those patients of the hospitals to pool their data, to actually share data with each other, so one central body can do the machine learning with a large end for statistical significance to find out if in the system is working or not. if you decide not to do that, you're just so worried about privacy that you're going to not let the hospitals reveal details

about the patients to each other, then you can still actually get some statistically significant results as to whether the medication is effective or not. it is just going to take longer, more patients will have to be in trial, and you'll have to wait longer to get the results. a professor has shown some very clear cases of the actual analysis of privacy levels versus lives saved or years of lives saved. unfortunately it's a tradeoff. there's a tradeoff curve there. it's almost certain in my mind that we don't want to be on either end of that extreme tradeoff curve, but we do have to decide where we are within the center of it. so hopefully we're squirming. i've tried to show you that no extreme position on

personalization is good screw privacy, or privacy is good, screw personalization. neither of those extreme positions are useful. we have to use technological smarts and our policy smarts to try to find the right place in the middle, and that's the set up for this panel discussion. [ applause ] >> thank you. at this point, i would like to introduce our panelists. yuet tham is an expert on cross border compliance and international agreements regarding data use. if you want to come up to the chair. paul timers, the director of the sustainable and secure society

directorate. we have experts here from asia and from europe who are helping us discuss this issue. next i'm pleased to introduce ed feltem, a computer scientist and the deputy director of the white house office of science and technology policy who has been meeting a bunch of extreme intense thinking about artificial intelligence over the next few years. i would like to introduce our moderator ben scott, who is the senior adviser to the policy technology institute. >> thank you very much, andrew, for that introduction. we're going to jump right into a discussion with our expert panelists, who as you see, strategically represent different regions of the world,

so can offer perspectives on these questions from across the globe. if i may quickly summarize the policy conundrum that sits beneath the cases that andrew set out, it is this. machine ai benefits from the personalization of data using algorithms. personalization requires large data sets to compare individual cases to a lot of other cases. that raises two key questions. one is what are the rules of governing data for commercial uses of ai and raises key issues for what are the rule for collection and processing of data for government use of ai. underneath that sits the basic question of algorithmic capability. if you decide it is unacceptable to have an algorithm that reflects gender bias in employment practices, how do you

regulate that? and if you decide to regulate that at the national level, how do you coordinate that at the international level when data markets are global? these are the problems that we are all facing in government, and i think it's fair to say that the technological reach of machine learning and artificial intelligence has exceeded the grasp of our policy frameworks to contain and shape these new forms of digital power in the public interest. so what i'd like to do is start with setting a baseline of where different parts of the world are coming down on these issues and what the building blocks look like at the regional level. there have been lots of efforts in the u.s. to address these questions. there have been lots of debates in the european union to address these questions. i would say less so in asia, although i would be interested to hear more from yuet about

what's happening in asian markets, but i want to allow all our panelists to speak from their own perspectives about what's happening in this field in their region. what is the approach to regulating or establishing a policy framework for these most difficult questions, big data collection, and the application of artificial intelligence? maybe i'll begin with you, ed. >> okay. well, first i should start with the disclaimer i'm not a lawyer, so do not treat me as an authority on u.s. law on this issue, but i can talk about the policy approach that has been taken in the united states. and it is rooted out of the longer term policy approach that the u.s. has taken with respect to privacy, and that involves generally regulation of certain sectors where privacy is

particularly salient, whether it involves things like health care or practices related to credit and employment and so on. and it also involves a broader consumer protection framework around privacy that is rooted in notions of notice and consent. and so we have a framework for privacy which the u.s. has used and is continuing to use and that involves both laws and also enforcement of those laws. when it comes to the particular issues that are raised by ai and machine learning, there are a bunch of things that have been done. and i point to in particular over the last few years the work that the administration has done on big data and then more recently on artificial intelligence. on both of those areas -- and i think they're tightly

intertwined -- the administration has engaged in a series of public outreach activities and then published reports. the idea being to try to drive a public conversation about these policy challenges and to try to move -- both to move the debate about making rules and policy in a fundamentally positive way, but also to heighten the attention to and interest in these issues and to try to drive a public debate, because i believe strongly the institutions, the companies, that are collecting data and using it in this way almost universally want to use it, collect it, and engage in ai activities in a way that is responsible and positive and sustainable. because i think people recognize that if you push the envelope too much, the public will not

allow that to stand. and so we've really tried to drive a public discussion. we've tried to raise the level of dialogue, and that's been fundamentally one of the areas in which the administration has worked. we also recognize the importance of -- we also recognize the ways in which these issues operate across borders and the need to work with international partners to make sure that as data flows across borders and as citizens everywhere encounter the companies, the institutions, of other nations that we can work together reasonably and we have an international system for dealing with these things. >> thanks, ed. paul, what's the view from brussels? >> the view from brussels. perhaps i should put another kind of disclaimer in. if you look at what is happening in policy development, whether that is engagement with

stakeholders, public debates like here, or whether you go in the direction of official public policy or law and regulation, you have to put it actually against the reality of what is happening around technology and around the use of technology. so i think the examples that andrew gave are really interesting and challenging. the case where, for example, you don't -- machine learning doesn't have access to your personal data, even if it would be good for other people. well, it's a very interesting case because you have to look at it. how could you apply today's frameworks, including law, to that? to a degree, law is very strong in the european union based upon fundamental rights. we would look at fundamental rights, but also fundamental rights are not absolute. so public health is one of those reasons that you can actually start using someone's personal data, also individual data, but with appropriate safeguards.

that may mean you put a challenge to technology. can you encrypt sufficiently? can you use new technologies after the fact, after it has been used? it is, i think, that dialogue that we're also very much looking for in the european scene. must be said, fundamental rights are very, very important in the european setting. privacy, we split it into the privacy from the perspective of the protection of your private life and the protection of your communication versus personal data. there are differences. there's more than one fundamental right that is at play there. based upon that, we have law, but we have also policy development. and it's a very actively moving field. for example, at the moment we are working on a policy initiative around the free flow of data and around platforms and precisely those are being put to the test by machine learning, ai, precisely by the questions we have here on the table.

>> yuet, how does it look in asia? >> i'm not a computer scientist. i'm a lawyer. i think one of the challenges about asia is that it's not even bifurcated in terms of the laws and regulations that are coming out of the region. in fact, when we talk about asia, what do we really mean? different people have different views about asia as well. i think when you talk about privacy laws in asia-pacific, i think the countries that come to mind as being at the forefront of regulations would be japan and korea and then to some extent australia and new zealand. following that would be countries such as singapore and hong kong, taiwan, and the philippines where they've got fairly new laws. some of them were actually put into place in 2012. singapore is a country where --

they are progressive, but the fact is they implemented the privacy laws for the first time in 2012. that again gives you some idea as to the importance that they place on privacy. then in the last category you've got countries such as indonesia, vietnam, and china. so these are the countries where we call them privacy laws, but they're not really based on privacy. not individual privacy anyway. and i heard today a lot about human rights, about how privacy's a human right. for a lot of these countries, these laws emanate not because of a motivation to protect human rights, although a lot of it would be consumer rights. some people would argue that consumer rights would be to some extent human rights as well.

a lot of these laws that come into place -- i mean, the last category of countries what is challenging about them is they don't have a single data privacy regulation. i tell my clients sometimes the more laws a country has -- i mean, i do a lot of corruption investigations, for example. in the cause of that we take a lot of e-mails throughout the region, which is why we need familiarity with data privacy rules and regulations. i joke with some of my clients to say look at how many laws they have. the more anti-corruption laws they have, the more problematic corruption in those countries tend to be. you find little bits and pieces of information. we refer to how privacy is a

right of all citizens, but they don't really tell you how that's going to be enforced. that is a regulation that you see in china. so i think some of the challenges in asia is just trying to harmonize the regulations for a lot of companies that are trying to operate and transfer data across borders. and you have a lot of -- so japan, for example, has got a new law that's going to come into force in about two years. that's probably the first time where they actually talk about data. in terms of all the other countries, i think the idea of artificial intelligence is not even something that the countries have seriously considered. there are things you might see. guidelines chosen by regulators, but these are just guidelines and there's no teeth to any of them. >> let me pick out a point which

i think is implicit in what you said. you all have described the approach of the united states, europe, a variety of asian countries from a commercial data privacy perspective, commercial actors providing algorithms to produce core outcomes. when you collect a lot of data and you begin to produce these outcomes, that is of interest to government. government access to data is inextricably intertwined the protective regulations. in the united states and europe over the operation of american technology companies in europe has to some extent been about commercial data practices, but ultimately it is rooted in u.s. government access to the commercial data that is collected by american companies. so my question is, do you believe that even if we were

able to find a harmonization, is it all undermined at the end of the day by individual country's national security interests and their unwillingness to give up any kind of access to that data to a government for national security and law enforcement purposes? >> i can give a very quick example before we go to europe and the u.s. china has got a provision where -- if any information that relates to health, the medical information, the health of the citizens has to be stored in services in china. another example is the singapore data privacy provisions. the singapore government and all entities are excluded from those

provisions. that's a good example of where the states' rights come first. >> perhaps building on that, i think this whole question about national security and sovereignty perhaps you have to generalize a little bit and the interests that are certainly governmental interests or should be interests for society, which is safeguarding democracy. i think one of the concerns -- if you look at a speech last week that talked about the transparency of algorithms of live platforms. this is in order to keep consumers properly informed, but it is also what is the kind of bias that may creep in through the algorithms of the provision of news. that has everything to do with the way you execute democracy. there's an underlining debate where democracy gets polarized into chambers and we don't have a real debate anymore.

that's also a serious interest, i think. you're talking about essential norms and values and to what extent are they shared nationally and internationally. if we talk about data protection, we have after all been able to make an agreement between europe and the united states even if we do not have exactly the same starting point as regarding data protection, let alone regards to national security. the privacy shield i know is going to be put to the test, and that's how it should be, but nevertheless we have a lot further than we have at the time of safe harbor. because we started to describe that area of access to the government and the safeguards. it is possible if you negotiate to make an agreement on certain types of issues. whether you can do that for everything across the world i think is very doubtful. there are many places where norms and values don't work. if we bring it to the field of

cybersecurity, that's where we clearly see it. we do negotiate internationally about ai issues and machine learning. i think it's not a single type of answer to this question. there is a degree of progress between let's say those that have a degree of li like-mindedness, but there are many areas where we should be reserved or perhaps pessimistic. >> there are plenty of areas in which government access to data for purposes of national security or law enforcement is relatively is uncontroversial. i think we don't want to forget those. of course, the international discussions around this issue have been going on longer than

the conversation about ai. these are not -- these issues are not simple, but i think if you look at privacy shield, for example, it is an example of the way in which it is possible for us to engage internationally and to get to a point where we can work together. as to these issues about fairness or nondiscrimination, i think this is another area in which there is a broad alignment of interests internationally and in which i think there's a lot of progress we can make by working together. >> let me present a more pessimistic vision and ask your responses to this, which is to me it stands to reason as the private sector grows more sophisticated with machine learning technologies, collects more data, it will be irresistible for government to reach into those companies for

legitimate reasons in many cases, but also perhaps for illegitimate ones to gain access to power. the example you raised of the firefighter buying arson kits. i don't know where you buys those or whether you have coupons for them, but the idea that law enforcement may not only tap your phone calls or e-mails, but look at your purchasing records and know your health data and compare you against others and calculate the probability that you may have committed the crime is an extraordinary development, and one in which i think many governments in many cases would legitimately want to use. but what that says to me is ultimately every country wants to control that data for themselves in their own sovereign territory. my question is, number one, are we headed for a global data sovereignty movement where everyone tries to have data

localization rules, where the ai operated by domestic companies is used as a geopolitical assess. second, if the countermeasure is algorithmic transparency, to what extent does that get you an outcome? is that an effective solution? if facebook turned around and said, okay, we'll show you how our algoe rrithm for news feeds does that effect the problem? it reflects back to them the things they're most likely to click on. do you then regulate that algorithm and tell facebook you have to change that algorithm? then how do you hold them accountable? are we headed towards a hard power regime of data localization at the local level?

if we're able to use transparency to push back against instances of ai, does it even work? >> let me start by taking the second part of that about the value of transparency, which i think really goes to a desire for governance and accountability. and one way to try to get there, to increase accountability, would be to say open up everything, tell us everything about what your algorithm is, tell us everything about what your data is, but here i think is a place where we can apply better technical approaches to try to provide accountability, to try to provide evidence of fairness or nondiscrimination, or certain accountability along certain dimensions without unnecessarily needing to reveal everything. i think one of the traps we can fall into in thinking about this

issue is to think that this is a problem caused by technology which can be addressed only by laws and regulations. but i think it's important to recognize, as i think the discussion today has, that technology can be an important part of addressing these problems, that there are technologies of accountability, and that we need to think in a creative way about how to put those things together. we also need to think, i think, about the ways in which forces short of legal prohibition can constrain the behavior of companies and authorities when it comes to the use of data to the extent that what is happening is known to the public, to the extent that there's an opportunity to provide evidence of fairness, evidence of accountability. that in itself credits a dynamic in which companies and

authorities may -- will often voluntarily provide that kind of accountability. we have seen that to some extent in privacy, where companies would like to make compromises to consumers for comfort, but knowing they will be held to those promises. to the same extent, if we have technologies and mechanisms of soft accountability, that can lead, number one, to a competition to provide a service in a way that's more friendly in order to bring people in and it can also lead to a kind of accountability that occurs when some bad behavior is revealed. so i think there's a lot more opportunities there to use softer forms of governance and to use technology to try to work on that issue around fairness and governance. >> well, do you think the general data protection regulation is a sufficiently flexible instruments for softer forms of ai?

>> absolutely. well, i found that really challenging because i think indeed technology needs to be invited to make things work really like regulation. if you talk about informed consent about automated processing, that's a challenge for technology. it's impossible because we don't know ourselves what's happening inside. i think you are referring to approaches where you can measure things like fairness, things like did you actually understand what is happening and the decision making? also, i must say, a little bit away from the assumption or the notion. no, there's an interaction you can continue to have and that's where the technology can be. you are talking consent as to the use of data evolves. i'm kind of optimistic about the

reach that the opportunities that are in technology. when you talk globalization, again, probably an approach to that is necessary. there is a real risk that localization happens. itis happening today and actually, you don't necessarily get it by country, but an internet by region. at the same time, we have initiatives. i referred to the shields as a way to avoid data localization and we are talking personal data. we have a free throw of data going on. actually remove restriction to localization of data. and, i think, we probably want to differentiate which domains are we talking about? when we talk about public health

problem like the rise of the zika virus, i think we have a localized report to that. we have w.h.o.'s and the collaboration in the field of health that allows us to do big data, analysis on the data we are getting from zika all over the world, as a matter of fact. for me, to end this debate, we need to involve the governance that exists, any kind of institution that we have in the world that works, will be exposed to the question of what do you do with data? make use of those institutions, too. that may be in a different way, domains. work, for example for the data you have coming from self-driving cars, i'm not sure. >> perhaps a necessarily complex but therefore differentiated

sector by sector approach. >> i think you learn for what you have one sector for. not necessarily showing it is impossible to come up with governance. there's a strong plea, i think in europe to come up with new governance approaches and say not all governments approaches will work. the realtime threat of cyber incidents may not be compatible with the governance we have set up between people and organizations that could be slow and we will also have to review the type of governance we have. >> for asia, it sounds self-serving, but i think we need regulations. so many countries still don't have something that will be taken for granted in the rest of the world. so, for those jurisdictions that are the laws in place, the question is how is the enforcement and policy positions in terms of the guidelines

issued by the regulators. so many other countries still don't have very basic privacy laws. i think, at the end of the day, you need those to be in place for the framework, at the very least. a lot of asia follows the notice and consent principal that is adopted in the rest of the world. i think in terms of data localization, a lot is done for various reasons. they are usually not because of privacy. for example, indonesia, they talked about localizing data. the reason for that is because they thought this was a misguided belief that was going to help them improve the economy localizing data. what the government didn't realize is that was going to put off a lot of multinational corporations from investing in the country. it brings me to my, you know,

last point in terms of -- i think we have seen with all the, you know, international, mul multinational companies that have set operations in asia, they did bring along with them regulations that they have to follow because, for example, they are dealing with data from the european union or the u.s. and because of that, they tend to follow the standard that's set the highest. so, when you have consumers in asia who see, hey, this is the way my data should be treated and this is, you know, the way an international corporation would deal with my data and my privacy, then, you know, they start expecting that from the other institutions within the country. so, i think there has been a lot of that where the cascading of privacy even though the regulations aren't in place. you have the economic pressure, to a large extent.

>> i want to put one more provocation to the panel. before i do that, i want to invite you all to start thinking about questions you may have for the panelists. we are going to reserve the last section of this panel for audience questions. so, start thinking about that while i put this question to the panel, which is about all three of you have raised notice and consent. it is the basis of privacy law across the world at the moment, yet, even before a.i. was under fire, already under attack about whatever is sufficient, for various reasons. it is an argument that the consent is a sham because, you know, you are presenting a consumer with a 15-page document in nine-point type for a service they want to buy and no one wants to read it. they have no idea what they have consented to. once you click that box and click i agree, all the rights you have up to that point are

gone. not all, but many. second, as we collect more and more data and companies become diversified across platforms, it may not know exactly what it is they are going to do with your data. and they may not know to give you notice at that point. at what point do you build in multiple points? i recently had occasion to talk to a number of founders of silicon valley start ups, not just in silicon valley, but europe where i spent the last several years in berlin. data is the new value property. people are building companies based on the idea they are collecting lots and lots of data. what they will do it with, how they will monotize it and pull that resource with other resources, acquired and integrated into another larger enterprise, big question mark. undeniably not a deterrent for

venture capitalists. it draws into question the basic notion of notice and consent. if we come into a world where data is pooled intentionally to maximize the utility of personalization, it might not be reasonable to ask a company to predict in advance all the ways that data may be used. they may not be the only ones that gain access to the data and use it for purposes that may benefit or harm the user. my question to the panel is -- >> because of the audio problems with this event, we are going to leave it now and take you to a discussion from today's washington journal. at the table is jeffrey rosen, president and ceo of the national constitution center. good morning. >> good morning. >> so, we asked you to come on to talk about this thing called the electoral college. so much is made of the 270

electoral votes. we want to learn about how folks get there. i want to start by explain whag the electoral college is, who put it together and when and why? >> wonderful. we have to start with the constitution. what i want you to do is read the constitution. you can get it, you can have a pocket constitution like the ncc, a great pocket constitution or an incredible new app that i told you about. download interactive constitution. there's an amazing tool co-sponsored by the federal society. the liberal and conservative lawyer organizations in america where the scholars write about every clause of the constitution describing what they agree and disagree about. you can click and read it and see what agreement and disagreement there is and what it means. let's read the text. in the electoral college, article 2, section 1.

clauses two and three says each state should appoint in such a manner as the legislatures may direct the number of electors equal to the number of senators and representatives which the state may be entitled in congress. no senator shall be appointed an electorate. it says how many electors each state gets. d.c., thanks to the 22nd amendment gets one -- gets three, basically, two senators plus the one congressperson that would have gotten or otherwise entitled. sorry, that's the 23rd amendment, the d.c. e welcome trat, not the 22nd. they shall give their votes and that should be the same. where did this come from? basically, some framers, alexander hamilton, the rap star of the moment thought this was

the most perfect part of the constitution, the part everyone would agree is the best part. they rejected the direct election. james wilson was the greatest of popular sovereignty, why not have the people elect the president? they were afraid of a demagogue being chosen. they chose direct democracy. a wise group of elites, former office holders, you know, people who could filter popular passions, these electors would choose the president. what's interesting is they expected that, in many cases, the electoral college would be unable to reach a majority decision and the election would be decided by the house of representatives because the electoral college says if there's no majority the house gets to decide. that led to a mess. if you know the musical of "hamilton." under the first system, the choice becomes president and

second choice is vice president. in 1800, thomas jefferson and his running mate tied. they each get 73 electoral votes together. it has to go to the house. incredible drama. thanks to the interaction of hamilton. it led to the dual that killed hamilton. that was a mess. the final part of the constitutional story, the framers amend the electoral college in the 12th amendment. go to the interactive constitution viewers. it's interesting. you can read an essay from the university of texas. the electoral, the 12th amendment is so uncontroversial both the federal society said a single scholar could write about it. click ton 12th. there are interesting parts about it. one is, if it goes to the house, according to the 12th amendment, the house picks from the top

three votes as president and the senate picks from the top two vote getters as vice president. that was important in the election of 1824 when henry clay might have been part of the mix. he was excluded because he wasn't the top three. the 12th amendment takes out a weird system where you could have a president and vice president tieing, acknowledges the existence of a party system and says you have to vote for president and vice presidents. >> let me jump in and put the phone numbers on the screen. jeffrey runs the constitution center. we have lines and calls, we hope to come in in a couple minutes. we'll get right to them about the electoral college. so, take us further through history then because this electoral college process, while still in place, has not necessarily worked perfectly, correct? >> that's a nice understatement. despite the hope of alexander hamilton, does not work perfectly. two occasions when the election has been decided by the house

rather than the electoral college. four occasions when the loser of the popular vote or the winner of the popular vote lost the electoral college. let's talk about those. the famous election of 1800 and 1824, which was an incredible battle where andrew jackson won 99 electoral votes, john quincy adams 44, crawford 41. clay is excluded because he's not in the top three. the house chose john quincy adams who got fewer electoral and popular votes than jackson, infuriating jackson supporters. jackson won the next time around. then we have four dramatic times when the winner of the popular vote loses the electoral college. you remember bush v. gore where gore won and the supreme court stopped a recount and as a

result george bush was awarded the votes and won the election. that happened three other times in 1876, which was an amazing configuration of 2000. a dispute between electoral votes out of florida. the democrat wins the popular vote. hayes, the republican, claims he's won florida. so, congress creates an electoral commission split by republicans and democrats and the tie breaking vote is cast by republican supreme court justice handing the election to the republican candidate. it was a dramatic story that led to a congressional law being passed, which determines how to resolve disputes today. two other occasions, 1824, which we talked about, where adams wins even though he's lost the electoral and popular vote. the final one, i forgot this, fwu national constitution center, you have to visit in

philadelphia. it's a beautiful museum of we the people on independence hall with gorgeous copies of the declaration and being an education center, which you can learn about online. we have the great exhibit called "headed to the white house" which talks about the disputed elections. the fourth example was 1888, the democratic incumbent is grover cleveland. he wins the popular vote by 1%-1% 2%. he wins the election. i guess the electorate must have been angry. they retaliated re-electing cleveland making him the only president to serve two non-consecutive terms. >> before we get to calls, one question. who are the electors? where do they come from? how are they chosen? what are their duties? >> great question. we know they may not be any

senator or representative or person holding an office of trust or profit under the united states. otherwise, it's sbeerly up to the states to determine who the electors are, the rules to be chosen and how they are alloc e allocated. state office holders, former officials, they can be local citizens. most states, 48 of the states have a winner take all system. if you win the state in the popular vote, you get all the electors. two states have an allocated system where you can get a portion of the entire thing. that is maine and nebraska. and, as a result, there's a huge bonus to winning a particular state. now, there have been a bunch of proposals to either require

allocation of proportional votes across the board, there have been proposals to can the electoral college and have a national popular vote. there's been an interesting proposal that all the states agree to vote for the winner of the national popular vote, which would have the same effect. those reforms people contest. >> a lot more to talk about. carl, you are up first from florida. question or comment about the electoral college? >> caller: aside from the electoral college, the board of governors is not known by anybody in the country and yet they own the republicans and the democrats. the second thing is, hillary said not too long ago, to control the negro population we would have abortions. we didn't like it when hitler

killed 10 million jews. we slaughtered 60 million babies in this country. we should be very proud of ourselves. >> let's hear from mickey from milwaukee. we are talking about the electoral college. anything about the process, the formation of the present day? >> caller: good morning. thank you mr. rosen, i always follow you on c-span and the constitutional center. it's a great asset to the country. thank you for your hard work. >> thank you. >> caller: you partially answered my question. there are 535 elected officials, 100 in the senate and 435 in congress. there are 539 delegates. you mentioned washington, d.c., had two congressmen and one senator. where is the extra one electoral that is missing so it's 539 versus 538, possibly. the second thing, as you mentioned, the fact that they

instituted this electoral college because they didn't want a demagogue. if that was the intention of the founding framers, why do people go to washington, elected officials and say we are acting on behalf of the people but the people don't know right from wrong, why doesn't washington behave in a way that's best for the u.s., not a small minority of the population. >> thank you, mickey. >> it's a great question. my math isn't good and i don't know the answer. i'm going the ask the prep team to figure that out. if you e-mail me, jrosen@constitution.org. we will have the answer by the end of the day. as for the fears of the demagogue, what the founders made of this system is interesting. they didn't trust democracy in any form. they had filters from the original election by state legislatures, which was changed

by the 17th amendment to the electoral college. they also didn't anticipate the rise of a party system. that's a huge part of this story we need to talk about an the reason the election of 1800 was a mess is the rise of the democratic party and the federalists for adams. wise, elites choosing people based on the public interest rather than partisanship is elusive from the beginning. it might be great after this election is going to host a national conversation about what the founders would have made of our current democratic system. what are the forces, technological, institutional and structural that may have diverged from the vision and what to do about it. i'll get you the answer by the end of the day if you e-mail me. thank you so much. >> jim from royal oak, michigan. hi, jim. >> caller: hi. i have been following the constitution and i have been

following -- i have read the minutes of james madison of the convention and he was for the popular vote. but, since they needed nine states to ratify the constitution and there were four slave states that never would have voted for popular vote. that's why they had the electoral college. it was to satisfy the four slave states and get them to vote and my question to the gentleman is why hasn't the electoral college done away with it? it was based on one theory and has no place in today's world. >> that's an excellent question. you are absolutely right. you identified an important dynamic. the supporters were the slave

states like virginia where madison was from. they were able to get lots of electors even though they didn't grant the votes to slaves because of the three fifths clause because they count slaves as three fifths a person. it was that advantage the whole slew of first presidents were from the south, washington, jefferson, madison, monroe and so forth. so, that leads to your second question, a good question. why wasn't it eliminated? well, the smaller states have a big incentive to keep it and also, states like ohio, florida and pennsylvania and the swing states like to keep it, too. basically, because congress has been reluctant to end the advantage for the small states because the senate likings it, then congress has been reluctant to propose a constitutional

amendment. the only other way is 3/4 of the state legislatures demand congress propose one. that's a high bar to reach as well. for all those reasons, the practical reason is where the states agree informally to cast their electoral votes for the winner of the popular vote. some claim it impacts the clause. a treaty that required a higher bar of approval but it's at least a possibilitpossibility. the other problem is, after a contested election, one side is hap happy, the other isn't. thinking far ahead and coming up with a bipartisan reform is hard. >> we have stewart. >> caller: good morning, gentlemen. >> morning. >> caller: i suggest everybody read a book by john marshall, the chief justice that saved the united states. >> who is the author of that

book? that's a great suggestion? >> caller: getting to your topic, i disagree with what people have been saying, popular vote. if we just go by the popular vote and do away with the electoral college, then we hold the elections, let new york city vote, let california vote. whatever the popular vote is, that's going to be the president. it's very important that we have the electoral college, otherwise, the country is completely forgotten about. have a good day, gentlemen. >> those are two excellent points. i asked for the author because i agree, you should read about john marshall. the greatest chief justice who came up with strong power and allowed congress to regulate and unify it. another is gene edward smith's biography. itis a great place to start. you raise a great point.

there are good arguments for and against the electoral college. if it was a popular vote, the candidates spend their time in california and new york, not in the rest of the country. so, just trying to ensure geographically that every part of the country is attended to in an election is crucial. now, flyover country is not attended to on an equal basis because ohio gets a lot of love. there are a couple swing states that get the attention. your notion that a popular system would not do geographic justice for the entire country is a good one. thank you for that. >> you said who the electors are, the states get to determine who they are. are people able to know who the electors are? are they matters of public record? >> i think they are. you said you had a clip from pennsylvania deliberation or something like that? >> we do. >> we can actually see them. their names should be online,

you would hope. we definitely know, i won't run through all of them, there's a history of faithless electors who pledged to vote for one and voted for the other. 1796, samuel miles a federalist pledged to vote for john adams, said he voted for jefferson. the vote didn't affect the outcome. pennsylvania yans were so furious. they wrote, what, do i choose miles to determine? no, i choose him to act, not to think. the idea is, we want our electors to carry out their instructio instructions. miles and a series of six or seven other famous electors who refused to vote for the candidate they were elected to vote for. >> at what point after the election do the electors get to work? >> the deadlines are important

and i want to make sure they are exactly correct here. they are -- and this is all amended fwi 12th amendment and the 20th amendment, which is very important. we can go to the 20th on the institution. essentially, the 20th amendment says, what's the fastest way to get it. i need my constitutional reading glasses on whether i'm going to get it online or the interactive constitution is important. >> we are talking with jeffrey rosen. we are going to look in a minute at a bit of the 2008 pennsylvania electoral college meeting. we are looking to get a sense of timing first about how all that works. >> essentially, it is the wednesday after the second monday in december. the electoral college meetsan