have the machines been secured in a better way? do we know? >> i definitely think we are in a better state than we were last decade. three out of four voters will cast a ballot using a paper ballot or on something that creates a paper trail. in addition to the geographic distribution, we talked about, you can be confident three out of four people have a paper record. it is different on the audit side. some of the audit styles people are doin-- thehole reason to do an audit is computer tally of the voting computer against a manual tally looking at the actual paper records. that's a way of sort of arriving at ground truth. i think i would like to claim a little bit of the responsibility for fact that voting machines and the procedures around them have gotten considerably better since the last decade. the election systems commission

deserves a lot of credit. that's the federal agency that is tasked with helping local election jurisdictions run secure, robust, useable elections. over time, their testing procedures have gotten better and more sound. the trick is any computer security person will tell you testing only gets you a certain level of confidence. there is always going to be ways to get around it. some of the things i'm not so confident about are things like tamper-evident tape. we put these numbered seals over seams of the voting machines and if you try to hack your way in and get in there and mess with the brains inside, you have to pull that piece of tape up. it will -- it looks like it has been messed with. it says, void, void, void. typically, a heat gun, something that anybody that's been in a shop class, is all you need to lift that tape without disturbing it. plenty of examples like bad keys that are not as good.

i think the fact that we have been a little rigorous about saying, please don't ever do networking on these machines. virginia is one of the few places i don't even want to state it. there was something you could do from like across the street to one of these machines. do you want to talk about that? >> you could use a pringles can attack, it is an effective antenna for wi-fi. you can log in from across the street and manipulate the votes. >> that's unique. t typically it was the only machine in use in the united states that had wifi. >> you didn't even need that. as i recall, any voter in the precinct who had a smartphone could connect to the wifi that the voting machine was using and get access. >> there are these concepts of voting like one we call software independence, which is the notion that any undeticketable

error in the software of the machine should not result in an indy textable error in the outcome. that's why we have things like paper trail and a set of crypto graphic voting methods that aren't widely used. some of us worry about them in other ways. those can provide some sort of hard check against the software being bad ideas. >> we have two ways of hacking, remote hack if the system isn't connected to the internet, doesn't have to be directly connected and proximity hack, if you can access the ports while you are in the poll booth or if you can access the machines while they're in storage before theye distributed. many of these sit in schools and feeders overnight or they sit in an insecured warehouse for years in between elections. >> you hear people say these voting machines aren't connected to the internet.

that should give you a little bit of comfort but not a whole lot. as dan wall lec said recent letter, there was malicious hacking before we had the internet and the network. there were things called -- very few people may see or know what they were but they are viruses that are transmitted by floppy discs, media that you can put in and out of the machines. when i was at princeton, people designed a virus for this machine used throughout the entire state of georgia. this was a machine where the default password was 1, 2, 3, 4. not even spaceball's combination, 1, 2, 3, 4, 5, that's not much better, and they designed a virus that would do this, in one election, say the primary election, you would get access to the back of the machine and stick a usb stick on

there and this would install. between the two elections, that would have the opportunity to go from that machine to the election management system, the one computer that tells all the other machines, here is what the ballots are going to look like for the next election. the general election. that's a way of installing a piece of malware on a device that over a longer period of time spreads itself. that's the kind of thing if you think about what kind of attackers would do that kind of thing that's not something two months ago woke up said ooh, man, maybe we want to hack the vote, so to speak. what you are going to have is much more sophisticated and long-term entities like nation states that are more likely to employ techniques like that. georgia is not a swing state by any stretch of the imagination, but there are platforms that may be susceptible to these kind of slower proximity hacks.

>> it is interesting that georgia did that sort of test. georgia actually had a problem with certified software in i think it was 2008. they had all these touch screens in the warehouse and using officials at georgia tech who were helping them. about two weeks before the election, these helpers went in and upgraded the software on these debolt systems and no one had oversight over the software. they simply said these machines needed to be upgraded and they installed on thousands of machines. that's sort of a problem of process. you can have an external actor that gets access to these machines and you can have a problem with upgrading machines at the last minute in a way that software is not examined or certified. if you are doing something intentionally, you can design it in such a way it disappears once it has caused its problems so

that someone examining the code afterwards will not be able to see the malicious code's presence there. >> there was an interesting case in iowa a number of years ago that's very much like that where there was an upgrade to the windows operating system on -- i don't remember which brand of machine. it turned out it had a certain feature which meant that each voter when they stepped up to the voting machine, it would prehighlight whoever the previous voter had worked for. it was a feature of how windows worked, nothing intentional. nobody recertified it, because they didn't think changing the version of windows would cause a change in the voting bemay have behavior. doug joans talks about that at doug jones from iowa is the one who talked about that in his book.

>> we are talking about remote and proximity hacking. internet voting, while it is not a huge problem right you know, election officials are very keen for internet voting. they'll throw out a lot of reasons for why this is crucial for going forward and why the young generation of voters with smartphones, this is what they want and they do. they seem to think there is nothing wrong with it, why shouldn't they be able to vote online for more convenience? how widespread is internet voting right now and some of the issues around it? >> there are more than 30 states that allow internet voting for military and overseas vote certificates. in the case of alaska, any voter can vote online. there are no standards for internet voting systems the national institute of technology that is charged with writing these standards has declined to

provide standards saying, we don't know how to do it securely. therefore, we are not going to provide a standard that says how to do it securely. we don't think it can be done at this point. however, as i say, more than 30 states are doing it, thence states are doing so and doing it on their own. it's unclear what security measures they are bringing to bear in terms of external reviews, anything like that. there's been nothing public i've seen from anywhere other than here in the district of columbia with the infamous case of the university of michigan hack against a test system that they provided. that was the one where they made all the robots win the election. there was a sample election and not a real election of course, and they played the michigan fight song for every voter. >> how many votes were accounted from internet voting in this election then? >> it's pretty hard to find that

information out. most states do not break out the source for the votes. they'll tell you this many votes from this county. they don't tell you this many came over the internet and this many were mail-in, absentee ballots or in person absentee and so on. it's hard to tell. in virginia i was on the commission that looked at this and i think that there were somewhere in the range of 10,000 voters that were eligible if the internet voting passed, there were 10,000 people that would have taken advantage of it out of the voting population of about 4.5 million. >> i want to talk about audits and bring you massimo in about election monitoring and influencing. let's discuss auditing.

how many states have auditeding at this point and why is virginia particularly opposed? let's talk about what exactly an audited involves in terms of 1% and all that. >> absolutely. if you're going to capture a paper record, you want to do something with that paper record. count all of them maybe, which is the standard definition of an audit is basically a recount. these are extremely expensive and time-consuming. there had been efforts since 1964 to try to get aspects of counting the paper to check the computer results without having to do a full recount. california passed the first statewide paper audit law in i think 1964. it's morphed over the years to be a 1% manual tabulation of voting machines from certain polling places, certain precincts. they used random numbers to pick

a set of polling places in every county that they recount every ballot in the voting places and compare that to the election results. that is the simplest way to think about an audit. this is the number that's put in the statute, 1%, 2%. some say if it gets below of half of one% in the margin, you go. there's a new flavor of audits. the australian ballot, a secret ballot transformed elections in the lastentury, this thing i'm about to talk about is going to transform elections around the world in the next century. they are called risk-limiting audits. this is a statistical way of counting the paper to assure that you have a way of capturing if you misstated the outcome.

if the outcome you reported to the press and the thing everyone waits for election night, that's incorrect, they have a high probability correcting that bad outcome. they are very different than 1%. you look at the margin of the race and other factors. and then you tune the sample that you pick to be and to give you the confidence that you need. so for example if the race is extremely close, say the bush v. gore race in florida, you may have to do a recount. but for most cases where there's 5% margin, 2 or 3% margin, you count as few ballots as you need to to confirm the actual result. if you can't confirm the result, you enlarge your sample or do a full recount which corrects your misstated outcome. so this is what you need to do to make sure. having the paper is one thing but counting it correctly is

another thing. that's one thing we are lagging behind. for example, there's almost half states, 24 states have no mandatory manual audit of their paper. 13 states have some post election auditing, but it tends to be 1%, 5%, things that aren't tuned to the margin of the race. they're not very well designed. there are a whole bunch in mexico and california have provisions in the law to bring them in and do a sample and compare them and decide if you need to count them all. >> david becker said this morning, if jill stein starts winning south carolina by a wide margin, that might be a sign that something's off. the ironic thing he didn't say is if jill stein wins south carolina by 5%, probably there's nothing legally that can be done about it because it won't be within the margin of a recount, so an audit is the only way we

would discover it. not that i have anything against jill stein, but it would be a surprising result. to address the question of virginia, as far as i know, i've talked to a bunch of people, virginia seems to be the only state that it's illegal. i say illegal because it's with an asterisk. the asterisk is if every race on the ballot was decided by a margin of at least 10% and after all the results had been certified by the state, i.e., it doesn't make a difference any more, then you can do an audit. you can't do an audit any other time. it's not an audit to have an opportunity to correct the outcome. they will say you have screwed that up. don't do that again. >> right. they tried this out. they did some sample audits after this. it used to be they were totally illegal. this was an improvement, very small improvement. and the outcome of the audit was the results were pretty similar. of course an audit result will never be

exactly the same as the original. you have to know that there's going to be differences because you're going to have human differences in how they look at the color circles or the machine differences. the conclusion was it was very close, it was close enough we were confident in the results, and therefore we never needed to do an audit again. that was the conclusion of the study. it's like if you file your taxes with the irs and they audit you and say, everything's okay. you will never be audited or considered ever again because you obviously know how to do everything perfect. the irs doesn't work that way, yet that was the outcome of that audit examination. >> there was an election in california in a small county. i think this was 2008. where the audit wouldn't have caught the problem. a small county. they discovered -- this was an optical scan machine. they had a paper trail. they discovered that the machine

had dropped about 167 ballots. the votes were there after they did the initial canvassing of the ballots after the election. they disappeared some time after that. it would not have been caught in that 1% because the 1% takes a random number of precincts in order to do the audit. and this would not have been included in that 1%. they decided to do this new radical transparency where while they have their diebold optical scanning machines, they bought a fujitsu printer. they scanned all the same ballots simultaneously in their scanner. when they compared the results, they discovered the diebold system dropped 167 ballots and the other caught all of them. what do you trust? the risk assessment or risk management -- >> really quickly. there are materiality audits and process auditing.

materiality audits check the numbers, check that you arrived at the right result. process auditing checks that everything that goes into making the result was done correctly. that's where it's very hard to do that right. you don't get up to be a process auditor when you go to work and stuff. it's not the best job in the world. those are the things where we need just as much rigor to that as we do with all the processes that arrive at the actual results themselves. >> let's switch it for a moment. i want to come back to security of voting machines a little later. let's talk about influence hacking. we have this sort of unprecedented election in the u.s. we've never had this situation before, or have we? it's quite common erseas and the cia has been successful influencing elections in the past. give us an idea of sort of where we are contextually for this kind of

situation in the u.s. is it true that we haven't had it before? >> if we place it in the wider context of electoral processes worldwide, there is a growing experience by election observers in dealing with electronic wording. you mentioned a few examples in the u.s. states that adopted it. outside of the u.s. there is, i would say, consolidated experience in india, in countries like the philippines and brazil, for example. >> you're referring specifically to the machines again. >> i'm referring to machines again, but they are also experiencing internet voting in some cases. although that is not the rule for the latest political elections in those countries. talking about audit, i think you

can get what does that mean in terms of auditing a system, so to speak? if the focus is on election day, there will be a lot of interest in looking at the numbers and focusing on the statisti techniques if you had thmeans that are basically related to the availability of a paper trail that can be used. but auditing system, auditing the process before election day, three, four months before election day is also practice that election observation nations are trying to introduce in their best implementation of the guidelines adopted internationally. there are examples from the osce in europe and also the carter center. both developed handbooks on observing electronic voting.

these are actually drawn from general principles that belong to the election observation per se, not necessarily focused on electronic voting. they have specificity that are related to the medium, to the technique. just to give you an example, you mentioned the risks related to internet voting, but if you assess them, you would look at how data that are produced through electronic voting machines, for example, are processed by computers that may be hooked to the internet. so that is a form of indirect access to the internet that is not normally considered part of the label category internet voting, and still presents risks of

manipulation. let me just support the russian security firm ceo of kasperski labs that identified last friday on an italian tv he was interviewed. he was asked, what is the biggest threat to democracy in your view? he said it is internet voting, in his view. unless the environment and procedures and the systems are safe enough there will be a growing tendency towards using these means of voting, and there would be risks, high risks of reaching manipulation, before, during and after elections. in fact, that applies also in different ways to the history of voting, also to paper ballots. so what we had to learn was how

observing elections can introduce elements of independent assessment that can help election management bodies to make the environment for voting safer, and to increase also the confidence by voters in the system, which is one of the challenges of the u.s. system now. just one, not the only one. in the recent paper from harvard university, identified five challenges to the integrity of the elections in the u.s. these ones, the risk of hacking is only one of the five. you have the regulation of campaign financing, issues about the polarization, and therefore trust among political parties in the electoral procedures. you have issues of, of course, lack of professional standards in electoral management, especially in highly fragmented

environment where elections are managed. i would say the most important one is lack of public confidence in the electoral process. all of these are interrelated, although we focus now on just one of them. i think this should address all of them at the same time. >> let's talk about some of those latter ones influencing and things. there have been reports and concerns that for example the associated press might get hacked. the associated press is, of course, what we are relying on for the results. what is the potential and what are the possibilities when monitoring election for preventing that kind of influence hacking? the results are not all in, especially in a country as large as the u.s. where you have multiple time zones you're dealing with and you've got partial results being reported on one coast while another coast is still voting.

how do you address that issue of false reports coming out other than sort of securing ap's computers and trying to monitor that? >> let me answer in a simple way. what is an election observer mission? it is a group of people who visit a country. you may have in that group expertise that focuses on the technology. you have other experts. you have media experts, for example. you have legal experts who looks at how the system is defined, designed in order to internalize processes of electronic voting. and you have long-term observers coordinator that gathers and analyzes the long-term observers. those in the field. not only during and around

election day. and can monitor how the media are reacting and influencing public views about the election. this is a combination of expertise in a process that is not just a week or month but covers the entire electoral cycle and can provide avenues for addressing together with the electoral management bodies, the bodies that are responsible for the management of the elections, i think issues like public confidence as a result of, i would say, manipulating behaviors by media. >> another response to that is you need to be really patient on election day. this is one thing working on elections you learn quickly . one thing you learn is election officials get no days off six to eight weeks before election days. please thank them for doing what they do because it's hard work. on the other side, you're not going to have a very high

confidence number on election night. a.p. works very hard, 5,000 reporters, whatever. a great story from 2014 ukraine can illustrate how you really need to be careful what you rely on on election night. in 2014 in the ukraine in march of that year, a russian affiliated hacker group started poking around the ukraine's central election commission site and may 21st four days before the end of the election day, the entire central election commissions server infrastructure was hosed. someone had gotten there and run rampant with destruction in terms of pulling things apart and speaking of software, 12 minutes before the close of polls, the main website that reports the results, reported the ukrainian right leader had won. instantly, the russian state

television station started reporting that was the outcome of the election. this was exactly a hack designed to influence the hearts and minds of ukrainians and russians. in ukraine they have paper ballots they delivered to a place in kiev. they were able to count them. it took a while to get their servers back up, but two to report what was the high confidence result from the paper ballots. you should see anything on election night being potentially the right answer, but that -- understand it takes in a lot of states a couple of weeks, even three weeks to get at what is the official correct answer, which is called the canvass. so don't be so concerned. it's not going to be the end of the world if we get weird results on election night. i have a feeling the likelihood of that is small. >> did you want to add anything? okay.

in addition to sort of hacking potentially media outlet, there are some issues about the election results coming from county websites. we had some issues about ohio at one point where the website that was delivering the results was being maintained by a third party company who had ties to republican party. so there was this issue raised of even if you had the voting machines covered and security voting machines you felt confident in, those results that were then being fed through this system that was telling everyone, especially in a state like ohio which is always a crucial state in presidential elections. so i guess -- you're talking about being calm and waiting for the final results, but really on election night we call the president, we call the winner. i understand what you're saying about back tracking later on, but -- >> that is what happened with

bush v gore. cnn called it and had to back track. >> it is the academic consensus that election ended up being called incorrectly. >> that can influee subsequent voters. >> this is the problem having election day as a natural experiment where we run it on one day, is it's very susceptible to tacks like denial of service attacks. for example, there are things called e-poll books. when you go and vote, you sign in and they check your name off on a list. for the longest time those have been paper. increasingly those are moved to laptops or tablet computers. we had cases in the past where the computers wouldn't start or crashed or something or didn't have a network connection. they didn't have a way of making sure you didn't vote at one place on the other side of the city and vote on this side. >> which is almost unknown.

there is very little evidence of what we call voter impersonation fraud, which is why that one candidate dan talked about should step up and give us evidence of what he's talking about. these poll books, things can happen to them. i'm confident, because i've heard from many election officials say we have contingency plans and we have paper copies of these. i worry, if we have congressional staffers that have been there over 10 years, you experienced this. in the state of maryland you have this thing, the poll book crashing or not being able to work. people can't sign into vote. you get in these lines and the polling place can't open. that may delay the opening by a couple of hours. for those who have to commute a long distance, it may be the only time you have to vote. maybe only certain people have the luxury coming back later in the day. the parties will sue to keep the polling places open longer.

those are a critical element here we've been worried about. you can ask your election official, do you have a contingency plan? what are your contingency plans? hopefully they involve printing out the roster you used to use. you may not be saving money anymore, which is often part of the impetuous to move to these kinds of things. you are making sure even if those things don't work, you can start voting right then and there. you have a contingency plan. >> we'll be opening up for questions in a minute. have in mind if there are questions you want. i just want to elaborate on that a bit. you talked about this is a way to disenfranchise voters if you wanted to. in georgia it was a huge problem. e-poll books. county after county was reporting e-poll books was down. it wasn't just that problem. a lot of voters were showing up

and their name wasn't in this electronic data base. it may have been at the central data base but they hadn't updated it. if they were registering at the last minute, it didn't get in the updated database that actually got on the poll book. a lot of voters were disenfranchised. usually the process is you can vote on a provisional ballot. you get a paper ballot. that vote is sort of, that ballot is set aside until they can verify you are a registered voter. then your vote will be counted. but often times they don't have a contingency plan. if the e-poll book breaks down, you don't have enough provisional ballots for everyone that's going to show up at the polling place. again, voters won't come back if they've been there in the morning, they are not necessarily going to come back in the afternoon to recast a ballot. huge problems. not just voting machines, e-poll books, registration data bases. >> we should mention briefly voter registration systems. >> yes. >> we spent the whole 2000 as academics and hackers worried about vote flipping attacks, changing the vote.

now this year it's specially becoming apparent how much we neglect it. >> the registration year. >> not to sound like a broken record in always talking about virginia, even though it is where i live, the voter registration system in virginia crashed on tuesday -- i think it was tuesday. the last day you could register. so now there are lawsuits about extending the date for voter registration. there is no reason to believe this was malicious. it was an overload like what used to happen on e-commerce sites the days before christmas where everyone tried to place their orders and they couldn't handle it. now they learned how to do that for e-commerce. we haven't learned yet for voter registration. so some people will be disenfranchised. because they couldn't get their voter registration in because the system crashed. >> we're seeing amazing die unanimousics. facebook will nudge you and

say, where you live, the last day to register to vote is this day. i have a feeling, i can't prove it, but i have a hunch these lead into these things. like jeez, oh, man, today is the last day. >> everyone gets an alert at the same time and they rush to their computer. so blame facebook. >> on the contingency plan, it is in fact one of the criteria, one of the usual questions used in assessing internationally elections, but it is not the only one. having a plan is good but not enough. the other question very important is, is there a training for the election management body offices to implement that plan? having a plan on paper is really good, not enough. >> remember, the people manning those polling places are often volunteers who have just been recruited in the last 24 hours to man those polling places. usually more earlier than that. i've been a poll worker and i find out three people call in the morning sick and you're the only poll worker that day.

>> this is a good emphasis of things we do in the larger realm of cyber security we don't do as much in election cybersecurity which is for data breach types of scenarios, if you're an enterprise that doesn't actually sort of have a fire drill around what you do when you had a data breach, you're not doing it right. this is exactly the point. we want to see election officials and bodies actually running drills as if, hey, it's election day. this happened. what do you do. put people right there in that situation. sorry. >> a number of years ago i was giving a guest lecture at an university. i asked, how old is the average poll worker? one student raised his hand and said old. i said how old? he said really old. i said how old and he said like 35. so the actual average age of a poll worker in the united states according to the election

commissions system is 73. think about that when we ask them, and i'm getting there, i'm not there yet but getting there. when we ask folks to set up complex technical systems on election day and run them securely. we are asking -- >> and trouble shoot them. >> and troubleshoot them. >> we're asking a bunch of folks who are not i.t. specialists who grew up before they had computers and networks and all that sort of stuff. we are asking them to be our i.t. experts for the most important thing that happens in our country. we have to recognize these things are hard to do. >> really good point. >> volunteer to be a poll worker. >> i'm one. >> let's take questions. do we need a mike for them? >> thank you. irv chapman. the congress, as you said,

passed an appropriation in 2002 to update voting equipment. i don't think there is anybody in this room that has a computer or smartphone dating from 2002. >> that's old. >> so was our dysfunctional coress derelict not putting more money into updating? and if i may, you mentioned the ukraine experience. does putin's hacking crew have the ability on election night in this country to put a finger, a thumb on the scale in favor of either donald trump or just general disconcerting the american public? and for that matter, there are elections in european countries of significance to us in the next couple of years. could he do the same for a right wing candidate in france, germany, wherever? thank you. >> i'll address the first part of it about the finances, which

is that there have been proposals in front of congress. there's actually a bill that was just introduced recently to provide federal level funding -- johnson -- and it historically prior to the help america vote act, it was a state responsibility from a financial perspective to purchase voting machines. in a way it's not surprising after that one shot we're done and congress isn't inclined to put more funding. in virginia there was proposal by the governor to provide funding in localities to replace machines like this and that did not pass. the legislature -- there was just no incentive. nobody got elected, whether to congress or the presidency or even the city council by saying i'm going to spend more money on better voting machines. it's the perennial stepchild.

when it comes to money, elections don't have a constituency. assist as ironic as that might seem. >> when any government official has a choice between filling a pot hole or putting money in elections, they are going to fill the pot hole. you hear about elections maybe two or four years, you hear about the pot hole every day, that that person is hitting the pothole, which is unfortunate. we do need more regular federal funding of elections. it would have to be structured like highway funds. which means you are going to have all the shenanigans that happen in terms of things you have to agree to do to accept those funds. that's just how politics works. as to your question about putin having a thumb on the scale, the very honest technical answer is, we're not sure. i say that meaning you could think of elections as a meadow that hasn't seen a lot of predators. so we have a lot of entities, beings, election officials optimized for a place that hasn't seen a lot of predators.

i actually think it's not a bad thing we have predators now because we have to adapt to work in this environment. something we didn't talk about is the u.s.'s history, the cia of directly influencing elections 1948 with the christian democrats in italy, the chilean elections in 1964, 1970 that resulted in a coup. it's strange we are crying human foul when we've done it over 100 years. [ inaudible ] >> i would say the technical thing about putin is hacking the presidential election is probably the hardest thing to do in this election. you're much more likely to see your proverbial tony soprano hacking one county to ensure that a waste management bond was passed. i think that's where you are going to see the first detectible evidence in the u.s. of a vote hack. i don't think something that crosses that many states is as

attractive. but then again, if you can get suburban philadelphia and it's close enough, there may be places that don't have paper trails that are, like for example dauphin county, i tweeted about this, dauphin county uses machines from 1985. i guarantee very few of us have computers from 1985. i remember those computers. they were a lot of fun, monochrome, oh, boy. election officials said, i could drop this off in the red square and russians couldn't hack it. the only response i've seen close to correct was, it depends on how high you drop it. >> i would like to add one thing to this interesting question which is about a recent bill that has been introduced to the committee on september 20th on

election infrastructure and security promotion. this aims at designating election -- electronic infrastructures as infrastructures of critical strategic importance to the nation. now, these would allow a response that could even be military response in the worst scenario including by engaging in this response allies in nato. and my comment on these is, cyber security is not only to do with cyber defense. cyber security is about striking the balance between cyber defense and cyber offense. so is there enough consistency in our intelligence systems between the two? because the weaknesses of the

systems that should be addressed in order to protect, to make our electoral environments safer are also potential entry points for attacking other systems. so is there consistency in our societies when they deal with intelligence in cyber security between the defense and the offense? i think there is not enough clarity on that. it would be interesting to see whether this bill could actually imply imposing some restrictions on the cyber attack side of these conundrum of cyber security. >> so this has been a very frustrating thing for me and probably all of you, as well. you see one story in the news that says that putin could hack the presidential election. then another story says it's not going to happen. that's where you're falling.

so it's really an unknown. we won't know until we know, essentially. >> we may not know. the scariest part. something i say a lot, these systems aren't designed to keep the kind of evidence you want to detect those kind of attacks. they are not designed to be resistant against nation state kinds of attacks. even then, if you're going to attack one of these machines, it didn't work, you would make it fail to look like a garden variety computer error. like the blue screen of death which you used to see on windows machines. you see it now, too. that's something i'd keep an eye out for. if we see marked uptick in errors and strange kinds of things, that could be the only evidence we see of mischief. we'll never know. it's hard to really bound that kind of stuff. >> of course, machines are

specifically designed so it doesn't say who you voted for, kim, and who you voted for, joe. to give you privacy. the side effect of that is if there's problem, it's like, i'm kind of surprised -- if i knew who you voted for, maybe i could say i could have sworn you told me blah, blah, blah. because we can't do that, it's very hard to detect strange things. it's like in 2012 there were comments, precincts in philadelphia where obama got 100% of the votes. was that tampering? there are allegations there was. if you look at the history, no, that's a precinct that is really realize reliably democratic. similarly there were precincts went nearly 100% for mitt romney in other places. things that look strange isn't necessarily wrong. it's hard to tell the difference between something that looks strange and something that is

strange. >> good point. okay. >> is that an "i voted" sticker, sir? >> yes. >> way to go. >> i voted this morning in virginia. it's a little bit disconcerting now learning that even though i asked about whether there was a paper track kept of my vote, to learn they can't audit the system even if they do keep a record of it. but my question relates to your discussion has been primarily as to the capability of hacking, and taking into account what you just said about detecting hacking in the past, other than in chicago in the 20th century, which had a reputation for fraud, is there much evidence or

any evidence as to how much fraud or hacking has been done in the 21st century up to this point? >> there's a lot of evidence of accidental bugs. sometimes it's hard to tell the difference between a bug and a hack. because the symptoms are the same. pot awhat a -- pottawatamee county in iowa had a case where their machine was misprogrammed. i think you wrote an article. >> i probably did. >> if you and i are living in the same precinct, the order of the candidates on your ballot and my ballot will be different so no one has an inherent advantage of being first. turned out they misprogrammed the ballot rotation so the totals came out wrong. there was a bug. we have lots of evidence of things like that going wrong. we don't have evidence of hacks. >> miscalibrated machines.

that's the problem with touchscreen machines. >> there are terrestrial, noncyber hacks. maybe that's not the right word. i believe this wasn't in chicago but was someplace in illinois where we had poll workers running the same ballot through an optical scan machine a couple hundred times. they didn't think they might actually compare the total number of votes on the memory to what's in the basket, the number of ballots. people were like, hey these don't match, we'll just actually count the ballots over again. we have seen that. there's a whole bunch of absentee ballot fraud. someone will go to a nursing home, have everyone sign their ballots blank, take them and fill them out and do stuff with them. there's examples of that. there's very little, if any, maybe four instances, i can get you a paper that cites this stuff, of what we call voter impersonation fraud which one candidate seems to be concerned with. it's hard to tell. which is

coming to a polling place representing you're a different individual and doing that a number of times in other places. we don't see a lot of that. >> in terms of voting machines, we have problems where after the election they find memory cards from voting machines that haven't been accounted for and they're in the trunk of someone's car, a poll worker's car, something like that. or you have, again, is it intentional or not? it's hard to say. did someone forget to take in the tallies? >> there is a known bug in the diebold gem system, which is one of the systems that tabulates where under certain circumstances even when you put the memory cards into it to say give me the grand total for the whole county, it loses certain ones. i don't know exactly when it loses it. it's only certain versions of the software. this happened in california a number of years ago and happened last year or maybe this year in tennessee in a local election where three precincts out of 100

or five out of 100 weren't counted until they actually went back and did the canvass joe was talking about. got the final results and discovered, oh, we don't have the results for the xyz church and abc elementary school so they went and found them. >> it's hard to know what is a glitch and something intentional. i'm not sure if this was a diebold machine, but there was a superintendent school race where every one out of 100 votes for this one candidate got dropped by the machine. >> that was in virginia. on windows. >> that's right. the candidate lost the race by 2% of the votes. 2% of the votes was about 1,600 and she lost by 1,600 votes. and 2% of the vote was 1,540. so again, intentional for a school superintendent? probably not. but maybe it was a test run to

see if it actually works, right. and to see if anyone notices before you do it in a more serious election. >> organized crime in our school district. >> maybe that was our tony soprano moment. more questions? >> thank you. john nicholson at the british embassy. >> can't hear. >> john nicholson at the british embassy. so the system we use is very different from what's been described by the panel. it is paper and pencil, hand counting. sort of the equivalent books are sort of printed off and scored off when you vote. standardization across the country. clearly, that doesn't eliminate the possibility of voter fraud and there are historical examples. >> how many choices does a voter

have to make? >> in an a general election, it's one choice. i wonder if you were designing a system from scratch what you would design at this point given what you've been talking about. >> that opens up my question about the los angeles county vote system. >> there's a couple counties in the u.s. that have been so fed up with the united states is the only country that seems to think it can buy voting machines on a free market and that's going to work out. everyone else puts out specifications and a request for bids and you buy it as a an country. we're huge and our federal government can't tell our states what to do in elections. but two counties, l.a. county, california, los angeles county, california, and travis county, texas, have decided they're so fed up with what's available on the market they're going to build their own. travis it's mostly on paper, it's a design concept and software that works. in l.a. county, they've spent the past five years working with a storied design firm ido with about $15 million to produce a new voting machine and they have

five prototypes now. this is essentially what we call a ballot marking device. it's a big pen. you walk up to it with a bank blank ballot, it sucks it in. you interact on a touch screen top cast your vote. it fills out your ballot for you. it doesn't keep information, how many votes or anything like that. it shouldn't know how many votes are run. but it will pull the data off it to do the count. you put that the ballot in a ballot box. those are later scanned en masse at a central facility. you have to keep the chain of custody very, very secure. then it's the most secure voting system i have seen that seems to be at the state of development that it's in right now. it uses a dual chip trusted computing architecture. each piece of software for each device is cryptographically signed, so the county is the only one who can put a given piece of software on the machine.

so the reach around the back and stick a usb stick that wouldn't work sheer unless you did some complicated stuff. they had two goals, one was redesigning the interaction of the voter so it's as smooth as possible. there's a bunch of neat things they've done with accessibility. it is trying its best to replicate the security of like an optical scan system with a touch screen system. so basically you can think of it as a very expensive like million dollar, very expensive pen that fills in your ballot for you. it's going to be open source. that means all the software, all the hardware, you can take all the autocad files to your metal bender and they'll stamp out a bunch. the idea is to have a system that anyone around the world could build off of. if you don't like the hardware, you can change that, too. i'm hopeful that this sort of -- this effort to you know, have a more open way of designing things would go deliver, you

know, one, increased unit, usability, but also the kind of security that we expect and something like linux and other kinds of things which aren't by their nature open source systems we use elsewhere aren't innately secure but can be more secure if used in certain ways with certain kinds of tools used to analyze them. >> just one small correction. i believe travis county, texas, put out their rfp. >> cool. >> this week, last week? >> last week. >> joe would know. >> joe would know. >> do you have a question back here? did that answer your question? i'm sorry. >> hi, david. >> i wish we -- just one very small comment. so los angeles, the average election has 100 things on the ballot to vote on. >> 11 languages. >> that's why as much as i love the simple hand counted paper ballot concept and there are people in the united states who think we should do that, i don't think it's workable.

>> for a large county. >> for a large county unless we completely rethink how we elect our government. it's not just the voting machines. the voting machines are a side effect of these complex elections. >> can we borrow a parliament? >> where you've got a 25-page ballot. >> david touretski. isn't a big part of the problem that we try to set this up in a way that is magnifying them rather than minimizing them, by trying to do this in one day, between 6:00 a.m. and 7:00 p.m. so that any problem causes the most chaos possible? you were talking about electronic poll books, talking about what in cyber world we talk about as an incident response plan. there's so little time to react. and what happens at a polling place is if the electronic poll books start working and you're starting to check people in on

those and then they stop working, even if you had paper poll books, you don't know who's already voted in the electronic poll books. so you're in an extremely chaotic situation. they're going to try to fix the electronic poll books first. what happens is they try to do that is the lines grow longer if it's near a peak part of the day. what happens when the lines grow longer is people don't move their cars, and months there's no parking and there's utter chaos on the roads. and by trying to jam all of these voters into such a short window, which means the problems are harder to prepare for, solve, and resolve quickly, isn't that the biggest risk to the credibility of our elections more so than the outcome being challenged through -- changed through hacking? >> i just want to point out that the trend now is towards a longer election cycle. we now -- it used to be in order

to do absentee ballots, right, you had to be out of the country and you had to prove that you were going to be out of the country years ago, this is the only way you could vote outside of the actual election day and you could vote on a paper ballot. states have relaxed those rules now. i don't know what the percentage is about the number of voter who now vote from home using a paper ballot. >> depends on the state. >> you have to be willing to give up the privacy of your vote because it's on a paper ballot, it comes in with your name on it. and they verify that it's your ballot. it's no longer a private vote. >> there are processes in place in the election offices so they check the envelope, make sure you're registered and there's an inner envelope. they put the unopened inner envelope in a ballot box and don't open that till later. there's very strong process security to ensure that your ballot is secret. however, it doesn't always work, if you're the only absentee voter in your precinct, someone was

telling me about a story about that, we know how many absentee ballots voted for whomever, we know who it was. >> a lot of jurisdictions have been moving towards vote centers. so having -- you can vote over this two-week period at five different facilities around the city or the county or whatever. for example, i think colorado and maybe washington, one of those two is doing something now where they send everyone a ballot in the mail and you can return them. you can surrender it and vote in person somewhere else if you want to. those are -- i could go on forever about those because there's some things that are not so good about those kinds of models but that tends to alleviate this what we call the load or the scaling problem with having everything on one day. it's like a tuesday when everyone has to work even though you should be able to get time off for it. >> do we know how many voters are voting prior to an election these days as opposed to the ones voting on election day?

>> i don't have that with me, but it's considerable. some places permanent absentee in california is the like 60, 70% of all voters. >> it's going to differ a lot by state. some states still require excuses. you can only do it if you have one of these eight different reasons. so you can't generalize from one state to another. >> okay. >> there is also the aspect, if i may, of getting the immediate results. getting results immediately, the rush toward having results as soon as possible. that is also interesting. i had the opportunity a couple of weeks ago to meet with a high level litigation firm from a country that had elections recently and they used e voting to a large extent. and he stated that the results were well accepted but he also added they were immediate. just a few minutes after the closing of the polling stations.

and then his comment was also they were all well accepted but because we hadn't a closed election. so if you have a closed election, that is the political risk. in the highly polarized political context when you have a close election, irrespective of the technology, eventually, there will be room for disputes and possibly also in some cases even violence for not accepting the results. and that is irrespective of when you get them. >> so i voted on yom kippur. don't tell my rabbi. and what if something exciting happens in tonight's debate and i say oh, i really wish i hadn't done that. this is the downside of having an extended polling time is it's hard for candidates to decide when to do their advertising, the logistics and the strategy of advertising and when to drop those bombshells and so on

changes. so it's not just the mechanics of voting that changes. it changes everything about our elections. >> how many more questions do we have? >> we can go for a couple hours. >> there's people behind you, too. >> i completely missed that side over there. let's go with this question first. >> that's the left wing. >> thank you. i'm dave rabinowitz. i live in oregon which has had totally vote by mail for years. and basically, you get your ballot in advance. you can fill it out anytime you want. you can mail it in as long as it's going to arrive by election day. or you can drop it in dropboxs, and it has a secret envelope so your vote is still secret. dr. hall seemed to indicate there are problems with that. i'm wondering. >> i use vote by mail. i can't do my job without -- i can't go to the polling place, just the nature of what i do. so i'm a big fan in terms of convenience. it is one of the deep dark

secrets in terms of voting security that to put it another way, before about 1900, election day was a payday for most americans. 85% turnout. they could actually observe you putting a ballot in a ballot box and there were very distinctive colors. it was easy to figure out how it went. that's why we adopted the australian ballot which is a secret ballot printed by the government where all the candidates look the same and no one has big letters. when we adopted that uniformly in the u.s., turnout went down to 20% because there was no longer this incentive push or pull, they had to have civic reasons to want to vote, not compensatory or monetary reasons for going to vote. so that's the thing that concerns me is you can have coercion, buy-in, you can say hey, i know a person who does the opposite which is really strange where they will sign their ballot and give their

ballot to their lawyer with about 500 bucks and say vote who you think i should vote which is very strange and is totally illegal at least where this person lives, west virginia it wouldn't be illegal because the constitution in west virginia allows you to show anyone your ballot befe you cast it, after you've marked it, which is the only state that allows you to do that. the reason i'm down on vote by mail is it reduces the secret element or privacy element. secrecy is i'd really like you not to tell anyone else how you voted. unfortunately, vote by mail is not going anywhere. it does have the level of enfranchisement in highly rural areas is unparalleled. it's better than internet voting because you have a permanent paper record that can be audited. >> two quick things. one is for people with disabilities, vote by mail makes it harder for them.

they have to have someone else fill it out for them. unless they have a marking device on their computers, it just reduces someone who has motor impairments or visual disabilities. the other thing is, it increases the level of undervotes because there's no machine to check, did you correctly color in the circles or did you put a check mark or people do amazing things, they circle the name of the candidate they like -- >> a circle instead of filling out the circle? >> you can't read those. >> i had an election official tell me the other day had he someone who came in who said i don't need your instructions. i'm a p.h.d. and proceeded to circle the circles and it got kicked out by the voting machine and he was like -- i'm a p.h.d. but i can't follow instructions. >> they rarely can. >> modern day literacy. >> there was a question back here. then i come to you guys, sorry. >> you've all mentioned a couple different scenarios that have happened. >> hold it closer.

>> is this better? you mentioned in ohio where there was a third party that had access to the networking. > that's very typical. >> third party company that was sort of managing the aggregation of the votes from the different counties and reporting them to the county website. >> got it. there was also some discussion of hacks of voter registration rolls. could you illustrate what the outcomes could result from the access or compromised networks or databases that would result from these activities? >> sure. so for example, in ohio, in 2004, we had a machine that, and this is when we were using modem -- telephone modem-based transmission of results from the polling place to the central facility. we had a machine that phoned

into the central facility and basically reported twice as many votes as it is had actually recorded. this was at a time when they weren't using encryption on a connection, they were using something called a crc which is not appropriate for this kind of thing. we don't know what happened. it could have been a cosmic ray. but you could imagine with access to the network itself, you can fiddle with the bits in realtime because this isn't hard to change stuff that flies by. unfortunately, there are a lot of -- instead of saying unfortunately, there is something like anywhere from 8,000 to 10,000 election jurisdictions in the country. most of those don't have a full fte, full time staffer for their elections operations because that person has to do titling, clerking, all sorts of other stuff, too. so you better believe they're going to write a contract that says, hey, you take care of as much of this as you can. that's why i think there's a great opportunity for some sort of like cloud provision for these kinds of folks, something

that could at least, i don't know who should do that. i don't have all the answers. that's the kind of thing you can imagine trying to abstract a way that either way you don't need to trust the third party which is what i prefer, using end to end cryptography that will ensure i think we're a little far away from that, but having you know, having maybe someone else run infrastructure for folks so that they don't have to sort of either, you know, pay someone out of their pocket who may not do it very well. so that's a not very satisfying answer. >> was that sufficient? >> [ inaudible question ]. >> it's very easy to imagine this. so voter registration data is the most useful data for reidentification attacks. if you heard of deidentification of data in health or other kinds of things where they remove certain kinds of identifiers to be able to share that data more widely. voter registration data because

many people are in it and it has very specific types of key data like the last four of your social, your date of birth, your home, your phone number, in some southern states your ethnicity and other things like that there's already a motive to get access to that kind of data and to have that kind of stuff. so illinois voter registration hack where 90,000 individual records from exfiltrated from their staging system that's a good example of something you do. in terms of influencing the vote you can imagine an attack that would remove 5% of the voters from the registration rolls from one particular party and given how close our elections are, check out wikipedia, duvenger's law, the only law in political science basically said if you have a system that's first pass the post, a system where the majority wins you regress to a two-party system with very close margins.

removing 5% of the voters from one party or the other could be a perfect attack for influencing the vote. >> we're actually out of time, but i promised you guys that you would get your questions in, so if you can ask them quickly and keep your answers brief, please. >> so i will try and be brief. i'm concerned mostly about the perception thing because we have this copy of the report that i know joe was here at the atlantic council two years ago on the stage with my boss, i work for congressman langevin, he was skyping in to talk about that. so i think that, you know, it's really great to see all this focus on election cybersecurity, but a lot of it is tied to the dnc hacking. i mean, there were voter registration databases that were getting dumped online and people were finding them for years before arizona and illinois. so how -- you know, i think the focus is good, but i think that the biggest concern is the

perception that people will have that the election is illegitimate. how do we build resiliency in the electorate to deal with that fact? one of the great threats to our voting system from where i sit is rain. that drives down turnout like nothing else. i mean, you can disenfranchise 5% of the population that would show up or the voters that would have shown us but it was raining, we try to build resiliency. how can we build resiliency in the populous because we know the ap has been hacked, the dnc s been hacked. even if we don't have the specific examples and it's hard to figure out in terms of the voting system itself the idea that dissemination about something like that is going to be hacked is we have past evidence that these things have

been hacked so how can we build residency in the american populous to deal with that fact? >> three word answer, evidence-based elections. so being able to demonstrate to folks that this is how it's supposed to operate, this is how it operated, don't worry, any mischief, you know, i'm going very -- it's extremely complicated how you do that, but to the extent we can base trust on the evidence that these systems are resilient that's as good as we can do. people can still worry and that's their problem. >> i would agree. i would also say for today part of the answer is diversity or complexity, depending what you want to look at it can be our friend and i think is our friend. it's just too damn hard because every state does things differently, every county does things differently. it's awfully hard to have a

large-scale impact as joe says, it's easy to fix the wastewater treatment bond but it's really hard to change big things because there's just so many different things that get cross-checked against each other. and our election officials do a fabulous job with not nearly enough resources so we need to not blame them when things don't go the way we always hope because they are doing a fabulous job given their limitations they have. >> i also agree. the technology is just a fragment of a mosaic that has to be consistent and that is the system. the people should be confident in. >> quickly. [ inaudible ]. >> please. >> i promised you, sean. >> thanks so much, i really appreciate it. dr. hall, you referred to elections as a meadow with very few predators. i was curious other than nation states what some of the other malicious actors look like and

if there's any intelligence that leads you to believe that tony soprano is trying to hack an election in suburban philadelphia. >> so i don't have any evidence that there is any organized criminal that is actually trying to do this. i do think that we see nation state influences in the guccifer kinds of stuff, but i'm a little wary to attributing that directly to fancy bear and whatever. it seems reasonable. i do think that there are folks who just recognize how valuable some of the data is and that there are people who are, you know -- as i forgot your name, i'm sorry, but as he was stating there have been -- security researchers have found, you know, databases that campaigns have not decommissioned of like all voters that include not only the list of their names but appended commercial data, so even if you have had the addresses removed for all the federal judges in a voter registration list if the campaign adds the addresses back

on all of a sudden that federal judge's address is publicly available and is a very sensitive piece of information. there are people who -- and there is also sort of the -- i don't know what to call them, the lols folks, the folks long there's fun to be had here. people poking out all the scripting vulnerabilities on donald trump's infrastructure which is apparently legion. most of the folks are sitting around, hey there's a website, hey, i can play around with it. [ inaudible question ]. >> i think the second we put a kaept that's attractive to a ballot that's feasible to internet voted, they will be elected. >> it's worth mentioning it's hard to predict. the one election that i know of that was fixed was the u.s. rowing association. why would someone fix an election for the u.s. rowing association? i don't know. but somebody did. i don't remember if that was an insider or outsider attack.

i don't remember. but it sometimes boggles the mind what people think is worth their effort and so i'm not going to presume who is or isn't, whether it's lolsic or nation state or a campaign that has gone off the rails and is willing to try whatever they want, whatever they can in -- you know, in a neighborhood election. there was a case in one of the colleges in southern california where a student put keen loggers on the machine in the student union building because they really wanted to be elected to the student council. it's like, you know, come on, it's not that important to have that on your resumé, but -- >> how many federal crimes but just commit? >> and the fbi caught them and locked them up for trying to hack the student council election. >> so we need to wrap up but before we go i do want to sort of throw this out there as a public service announcement about calling in on election day if you discover problems.

there is a group run by a legal coalition and i will let joe talk about it. it's very helpful, it's been in existence for i don't know how many elections at this point. >> since 2002. >> people can report in problems, long lines, toucher machines not recording what they are intending to report, e poll books not being up and running, any problems you can report to this legal group and they can provide assistance. >> there are partisan efforts but this is a nonpartisan effort, check out 866-our-vote.org or call 1-866-our-vote and you can ask questions, you can get help, they will even send a lawyer out if you're having serious problems that require legal intervention. >> that multiple jurisdictions. >> nationwide. >> every place you can vote in the united states of americas he will be on the ground and available. >> thank you. join me, please, in thanking all of the panelists here for a great discussion.

see pan where history unfolds daily. in 1979 action cpap was created as a public service by america's cable television companies, and is brought to you today by your cable or satellite provider. >> coming up on c-span3's american history tv a discussion on the civil war and reconstruction. topics include the history of gettysburg, the united states after the civil war, freed people's refugee camps, and ulicysses s. grant.