. . .

there's plenty of examples like bad keys that jeremy was holding up, other things like that that are not as good. i think the fact we've been a little rigorous about saying please don't ever do networking on these machines. virginia was one of the few places, and i forget -- i don't want to state it. there was something you could do from like across the street to one of these machines. do you want to talk about that a little bit? >> you could use a pringles can attack. it's an effective antenna for wi-fi. >> that's unique.

>> it was the only machine in use in the united states that had wi-fi. >> you didn't even need that, as i recall. any voter in the precinct who had a smart phone could connect to the wi-fi voting machines were using. >> without a pringles can. >> exactly. >> but still, these concepts in voting, one we call software independence, which is the notion any undetectable error should not result in an undetectable error in the outcome, the final count. that's kind of an academic way of thinking about it. there's a whole set of cryptic voting methods widely used. some of us worry about them in other ways. those can provide hard check against the software being bad. >> so we have two ways of hacking a system, a remote hack if it is not connected to the internet or connected to a system that is connected to the

internet. then we have what you were describing as proximity hack if you can access the ports on the machine while in the poll booth or access the machines while in storage. many of these machines sit in schools and theaters overnight. or they sit in an unsecured warehouse for years in between elections. >> absolutely. a really good example. you hear people say these machines aren't connected to the internet should give you a little bit of comfort, but not a lot. there was malicious hacking before we had the internet. there were floppy disks very few people in this room may know what those are, but there were viruses transmitted by floppy disks could you put in and out of a machine. when what is at princeton, i didn't directly work on this, they are such smart people that designed a virus for specifically this machine which

is used throughout the entire state of georgia right now. the default password was 1234. it's not even the spaceballs combination which is 12345, though that's not much better. they designed a virus that would do this. in one election, say the primary election, would you get access to the back of the machine, stick a usb stick that would install malware on that one machine. between the two elections that would have the opportunity to go from that machine to the election management system, the one computer that tells all the other machines, here's what the ballots are going to look like for this next election. >> the programs. >> through that spread all the other voting machines for the next election, the general election. that's a way of installing a piece of malware on a device that over a longer period of time spreads itself. if you think about what kind of attackers would do that kind of

thing, it's not going to be someone two months ago that said maybe we want to hack the vote, so to speak. what you are going to have is a much more sophisticated, much more long term kinds of entities like nation states that may, are more likely to employ techniques like that. they don't always rely on it being a perfect way of doing these things. georgia is not a swing state, but there are other platforms that don't have the kind of security that may be susceptible to these kinds of slower proximity hacks. >> it's interesting that georgia did that sort of test because georgia actually had a problem with certified software in i think was 2008. they had all these debold touch screen machines in the warehouse and using officials at georgia tech helping them with the machines. about two weeks before the election, these helpers went in and upgraded the software on these debold systems. no one had oversight over the software. they said these machines needed

to be upgraded and installed on thousands of machines. that is a problem of process, as well. you can have an external actor who does something but you can have a problem with simply upgrading machines at the last minute in a way that software is not examined or certified. then you don't know what it's doing. again, if you're doing something intentionally, you can design it in such a way it disappears once it's caused its problems so some would be examining the code after is not going to see malicious codes present there. >> there was an interesting case in iowa a number of years ago that is very much like that. there was an upgrade to the windows operating system on, i don't remember which brand of machine it was, and it turned out that it had a certain feature which meant that each voter when they stepped up to the voting machine, it would prehighlight on the ballot whoever the previous voter had

voted for. this was a feature of how windows worked. it wasn't anything intentional, but nobody had recertified it because they didn't think changing the version of windows would cause a change in the voting behavior. in this case there was an interesting interaction. doug jones at the university of iowa is the one who talks about that in his book. >> interesting. okay. we looked at remote hacking, we are looking at proximity hacking. i want to address internet voting. while it's not a huge problem right now, election officials are very keen for internet voting. they'll throw out a lot of reasons why this is crucial for going forward and why the young generation of voters with spart phones, this is what they want. they do. they seem to think there is nothing wrong with, they do everything else online, why shouldn't they be able to vote online for more convenience. how widespread is internet voting right now? and some of the issues around it? >> there's more than 30 states

that allow internet voting for usually military and overseas voters, in the case of alaska for any voter can vote online. there are no standards for internet voting systems. the national institute of standards and technology which is charged by the help america vote act with writing these standards declined to provide standards saying we don't know how to do it securely. therefore, we're not going to provide a standard that says how to do it securely because we don't think it can be done at this point. as i say, more than 30 states are doing it. they are basically doing it on their own. it's unclear what, if any security measures they're bringing to bear in terms of external reviews or anything like that. there's been nothing public that i've seen from anywhere other than here in the district of columbia with the infamous case of the university of michigan

hack against a test system they provided. that was the one where they made all the robots win the election. it was a sample election, not a real election, of course. they played the michigan fight song for every voter. >> approximately how many votes would be accounted for from internet voting in this election then? >> it's pretty hard to find that information out. most states don't break out the source for the votes in their results. they'll tell you this many votes from this county or even this precinct, but they don't tell you this many came over the internet and this many were mail-in absentee ballots and this many were in person and so on. it's hard to tell. in virginia, i was on a commission that looked at this and i think there were somewhere, i don't remember the exact number, somewhere in the range of 10,000 voters in the

state who are eligible if internet voting had passed, which it did not. there would be about 10,000 people who might have taken advantage of it out of a voting population of about, i think, 4.5 million. >> i want to talk about audits for a second. i want to bring you massimo in about election monitoring and influencing. let's discuss auditing. how many states have auditing rules at this point? why is virginia particularly opposed? let's talk about what an audit involves in terms of 1%. >> absolutely. if you're going to capture a paper record, you want to do something with that paper record. you want to count all of them maybe, which is sort of the standard definition of an audit is basically a recount, right? those are extremely expensive and time consuming. there had been efforts since 1964 to try and get some aspects of counting the paper to check

the computer results without having to do a full recount. california passed the first statewide paper audit law in i think 1964. i'm pretty sure. it's morphed over the years to be a 1% manual tabulation of voting machines chosen randomly from certain polling places, so they actually use various way of generating random numbers to pick a set of polling places in every county they then recount all the ballots and compare them to the election result. in fact, that's sort of the simplest way to think about an audit. some number put in a statute 1%, 2%, some places condition it on if the election was particularly close like pennsylvania, for example, has a mandatory recount law if it gets below 0.5% in the margin you go.

there is a newer flavor of audits which the australian ballot which is a secret ballot transformed elections in the last century, the statement i'm about to talk about is going to transform elections around the world in the next century. they are called risk limiting audits. this is a way of counting the paper that ensures you have a large probability of catching the fact that you misstated the outcome. to say it in plainer terms, if the outcome you reported to the press, everyone, the thing everyone waits for election night, if that is incorrect, these mechanisms have a high probability of correcting that outcome. they are very different than 1%. the idea is that you look at the margin of the race and a few other factors, and then you tune the sample you choose to be, to give you the kts would you need. for example, if the race is extremely close, say the bush v gore race in florida 2000, you may have to do a full recount. in most cases where is there a

5% margin, 2%, 3% margin, the way you do these calculations ensure you count as few ballots as you need to to confirm the actual result. if you can't confirm the result, you end up enlarging your sample or doing a full recount which corrects your misstated outcome. this is what you need to do to make sure -- having the paper is one thing, counting it correctly is another thing. that's something we are still lagging behind. for example, there's almost 24 states have no mandatory pan ul audit of their paper at all. nine states -- no, no, no. 13 states have some post election auditing. it tends to be 1%, 5%, things that aren't tuned to the margin of the race. so they are not very well designed. new mexico, california and colorado have provisions in the law to allow people to do these actual risk limiting audit.

bring the statisticians in, do a sample and decide if you need to count them all. >> david becker said if jo stein starts winning south carolina -- if jill stein wins south carolina 5%, probably there is nothing done legally that can be done about it because it won't be in the margin of a recount. an audit is the only way to discover it. not that i have anything against jill stein, but it would be a surprising result. to address virginia, i talked to a bunch of people. virginia seems to be the only state where it's illegal. it's illegal with an asterisk. if every race on the ballot was decided by a margin of at least 10% and after all the results have been certified by the state, i.e., it doesn't make a difference any more, then you can do an audit. you can't do an audit any other

time. >> not an audit that has an opportunity to correct the outcome. you screwed that one up, don't do it again. >> right. >> they tried this out. they did sample audits after this. this was a new audit. used to be they were totally illegal, this was an improvement. the outcome of the audit was that the results were pretty similar. an audit result will never be exactly the same as the original. you have to know there's going to be differences because you're going to have human differences how they look at the colored circles or machine differences. the conclusion was it was very close. it was croce enough we were confident in the results, and therefore, we never needed to do another audit again. that was the conclusion of the study which was sort of like if you file your taxes with the irs and they audit you and say, oh, everything's okay. you will never be audited or considered ever again because

you obviously know how to do everything perfect. the irs doesn't work that way, yet that was the outcome of that audit examination. >> there was an election in california in a small county. i think 2008 where the audit would not have caught the problem. small county, they discovered that it was an optical scan machine so they had a paper trail. they discovered the machine dropped about 167 ballots. the votes were there after they did the initial canvassing of the ballots after the election. and they disappeared some time after that. it would not have been caught in that 1% because the 1% takes a random number of precincts in order to do the audit. and this would not have been included in that 1%. they decided to try this new radical transparency where while they had their debold optical

scan machine scanning the ballots, they bought a fujitsu printer that scanned the same ballots. they discovered the debold system dropped 167 ballots and the other system got all of them. what do you trust? risk management. >> they are what we call materiality audits and processes auditing. materiality audits check the numbers, check you arrived at the right result. processes auditing checks everything that goes in making the result was done correctly. it's very hard to do that right. it's not the thing people get up in the morning -- you don't get up to be a processes auditor. those are things we need as much rigor to that as we do in all the processes that arrived at the actual results. >> let's switch for a moment. i want to come back to security

voting machines later. let's talk about influence hacking. we have this sort of unprecedented election in the u.s. we never had this situation before, have we? i'm wondering, it's quite common overseas in elections where you've got this problem. the cia has been successful influencing elections in the past. give us an idea where we are contextually for this kind of situation in the u.s. is it true we haven't had before? >> well, if we place it in the wider context of electoral processes worldwide, there is a growing experience by election observers in dealing with electronic voting. you mentioned a few examples in the u.s. of states that adopted it. outside of the u.s. there is, i would say, they experienced in india in countries like the

philippines and brazil, for example. >> you're referring specifically to the machines again. >> i'm referring to machines again, but they are also experiencing internet voting in some cases. although that is not the rule for the latest political elections in those countries. talking about audit, i think you can get what does that mean in terms of auditing a system, so to speak? if the folk why us is on election day, there would be a lot of interest in looking at the numbers and focusing on these statistical techniques if you have the means that are related to the availability of a paper trail that can be used. but auditing system, auditing the process before election day, three, four months before

election day is also practice that nations are trying to introduce in their best implementation of the guidelines adopted internationally. there are examples from the osce in europe and also the carter center. both developed handbooks on observing electronic voting. these are actually drawn from general principles that belong to the election per se not necessarily electronic voting. they have specificity that are related to the medium, to the technique. just to give you an example, you mentioned the risks related to internet voting, but if you assess them, you would look at

how data that are produced through electronic voting machines, for example, are processed by computers that may be hooked to the internet. so that is a direct access to the internet that is not normally considered part of the label category internet voting, still presents risks of manipulation. let me just support the russian security firm of a lab that identified last friday on an italian tv he was interviewed. he was asked, what is the biggest threat to democracy in your view? he said it is internet voting, in his view. unless the environment and procedures and the systems are

safe enough there will be a growing tendency towards using these means of voting, and there would be risks, high risks of reaching manipulation, before, during and after elections. in fact, that applies also in different ways to the history of voting, also to paper ballots. so what we had to learn was how observing elections can introduce elements of independent assessment that can help election management bodies to make the environment for voting safer, and to increase also the confidence by voters in the system, which is one of the challenges of the u.s. system now. just one, not the only one. in the recent paper from harvard university, 85 challenges to the

integrity of the elections in the u.s. these ones, the risk of hacking is only one of the five. you have the regulation of campaign financing, issues about the polarization, and therefore trust among political parties in the electoral procedures. you have issues of, of course, lack of professional standards in electoral management, especially in highly fragmented environment where elections are managed. i would say the most important one is lack of public confidence in the electoral process. all of these are interrelated, although we focus now on just one of them. i think this should address all of them at the same time. >> let's talk about some of those latter ones influencing and things. there have been reports and

concerns associated press might get hacked. of course we are relying on them for the results. what is the potential and what are the possibilities when monitoring election for preventing that kind of influence hacking? the results are not all in, especially in a country as large as the u.s. where you have multiple time zones you're dealing with and you've got partial results being reported on one coast while another coast is still voting. how do you address that issue of false reports coming out other than sort of securing ap's computers and trying to monitor that? >> let me answer in a simple way. what is an election observer mission? it is a group of people who visit a country. you may have in that group expertise that focuses on the technology. you have other experts. you have media experts, for example. you have legal experts who looks at how the system is defined,

designed in order to internalize processes of electronic voting. and you have long-term observers that i think gathers and analyzes the long-term observers. and can monitor how the media are reacting and influencing public views about the election. this is a combination of expertise in a process that is not just a week or month but covers the entire electoral cycle and can provide avenues for addressing together with the electoral management bodies, the bodies that are responsible for the management of the elections, i think issues like public confidence is a result of i

would say manipulating behaviors by media. >> another response to that is you need to be really patient on election day. this is one thing working on elections you learn quickly election officials get no days off six to eight weeks before election days. please thank them for doing what they do because it's hard work. on the other side, you're not going to have a very high confidence number on election night. a.p. works very hard, 5,000 reporters, whatever. a great story from 2014 ukraine can illustrate how you really need to be careful what you rely on on election night. in 2014 in the ukraine in march of that year, a russian affiliated hacker group started poking around the ukraine's central election commission site and may 21st four days before the end of the election day, the

entire central election commissions server infrastructure was hosed. someone had gotten there and run rampant with destruction in terms of pulling things apart and speaking of software, 12 minutes before the close of polls, the main website that reported the results reported the ukrainian right leader had won. instantly the russian state television station started reporting that was the outcome of the election. this was exactly a hack designed influence the hearts and minds of ukrainians and russians. in ukraine they have paper ballots they delivered to a place in kiev. it took a while to get their servers back up, but two to report what was the high confidence results from the paper ballots. you should see anything on

election night being potentially the right answer, but that understand it takes in a lot of states a couple of weeks, even three weeks to get at what is the official correct answer, which is called the canvass. don't be so concerned. it's not going to be out of the world if we get weird results on election night. i have a feeling the likelihood of that is small. >> did you want to add anything? okay. in addition to sort of hacking potentially media outlet, there are some issues about the election results coming from county websites. we had some issues about ohio at one point where the website that was delivering the results was being maintained by a third party company who had ties to republican party. so there was this issue raised of even if you had the voting machines covered and security voting machines you felt confident in, those results that

were then being fed through this system that was telling everyone, especially in a state like ohio which is always a crucial state in presidential elections. so i guess -- you're talking about being calm and waiting for the final results, but really on election night we call the president, we call the winner. i understand what you're saying about back tracking later on, but -- >> that is what happened with bush v gore. cnn called it and had to back track. >> it is the academic consensus that election ended up being called incorrectly. >> that can influence subsequent voters. >> this is the problem having election day as a natural experiment where we run it on one day is susceptible to attacks like server attacks. for example, there are things called e-poll books. when you go and vote, you sign in and they check your name off

on a list. for the longest time those have been paper. increasingly those are moved to laptops or tablet computers. we had cases in the past where the computers wouldn't start or crashed or something or didn't have a network connection. they didn't make sure you didn't vote on this side of the city and then vote. there is very little, if any evidence of what we call voter impersonation fraud which is why that one candidate dan talked about should step up and give us evidence of what he's talking about. these poll books, things can happen to them. i've heard from many election officials we have contingency plans and we have paper copies of these. i worry, if we have congressional staffers that have been there over 10 years, you experienced this. in the state of maryland you

have this thing, the poll book crashing or not being able to work. people can't sign into vote. you get in these lines and the polling place can't open. that may delay the opening by a couple of hours. for those who have to commute a long distance, it may be the only time you have to vote. some people have the luxury coming back later in the day. you can ask your election official, do you have a contingency plan? hopefully they involve printing out the roster you used to use. you may not be saving money which is often part of the impetuous to move to these kinds of things. you are making sure even if those things don't work, you can start voting right then and there. you have a contingency plan. >> we'll be opening up for

questions in a minute. have in mind if there are questions you want. you talked about this is a way to disenfranchise voters. in georgia it was a huge problem. county after county was reporting e-poll books was down. a lot of voters were showing up and their name wasn't in this electronic data base. it may have been at the central data base but they hadn't updated it. if they rer registering at the last minute, it didn't get in the poll book. usually the process is you can vote on a provisional ballot. you get a paper ballot. that vote is sort of, that ballot is set aside until they can verify you are a registered voter. om tultz think doulgd haupg

om -- often times they don't have a provisional plan. again, voters won't come back if they've been there in the morning, they are not necessarily going to come back in the afternoon to recast a ballot. huge problems. not just voting machines, e-poll books, registration data bases. >> voter registration systems. >> we spent the whole 2000 as academics and hackers investigatorying about vote flipping attacks, changing the vote. this year it's especially becoming apparent -- >> the registration system. >> always talking about virginia, even though it is where i live, the voter registration system in virginia crashed on tuesday -- i think it was tuesday. the last day you could register. so now there are lawsuits about extending the date for voter registration. there is no reason to believe this was malicious. it was an overload like what

used to happen on e-commerce sites the days before christmas where everyone tried to place their orders and couldn't handle it. now they learned how to do that for e-commerce. we haven't learned yet for voter registration. so some people will be disenfranchised. >> facebook will nudge you and say, where you live, the last day to register to vote is this day. i have a feel, i can't prove it, but i have a hunch some of that leads into these kinds of things. man, today's the last day. >> everyone gets an alert at the same time. so blame facebook. >> it is one of the criteria, one of the usual questions used in assessing internationally elections, but it is not the only one. having a plan is good but not enough. the other question very important is, is there a

training for the election management body offices to implement that plan? having a plan on paper is really good, not enough. >> remember, the people manning those polling places are often volunteers who have just been recruited in the last 24 hours to man those polling places. usually more earlier than that. i've been a poll worker and i find out three people call in the morning sick and you're the only poll worker that day. >> this is a good emphasis of things we do in the larger realm of cyber security we don't do as much in election cyber security which is for data breach types of scenarios, if if you're an enterprise that doesn't actually sort of have a fire drill around what you do when you had a data breach, you're not doing it right. this is the point. we want to see election officials and bodies actually running drills as if, hey, it's election day.

this happened. what do you do. >> a number of years ago i was giving a guest lecture at an university. i was asked how old is the average poll worker? one student raised his hand and said old. i said how old he said really old. i said how old and he said like 35. so the actual average age of a poll worker in the united states according to the election commissions system is 73. think about that when we ask them, and i'm getting there, i'm not there yet but getting there. when we ask folks to set up complex technical systems on election day and run them securely. we are asking -- >> and trouble shoot them. >> we're asking a bunch of folks who are not i.t. specialists who grew up before they had computers and networks and all that sort of stuff. we are asking them to be our i.t. experts for the most important thing that happens in our country. we have to recognize these things are hard to do.

>> really good point. >> volunteer to be a poll worker. >> i'm one. >> let's take questions. do we need a mike for them? >> thank you. the congress, as you said, pass passed appropriation in 2002 to update voting equipment. i don't think there is anybody in this room that has a computer or smart phone dating from 2002. so was our dysfunctional congress derelict not putting more money into updating? and if i may, you mentioned the ukraine experience. does putin's hacking crew have the ability on election night in this country to put a finger, a thumb on the scale in favor of

either donald trump or just general disconcerting the american public? and for that matter, there are elections in european countries of significance to us in the next couple of years. could he do the same for a right wing candidate in france, germany, wherever? thank you. >> i'll address the first part of it about the finances, which is that there have been proposals in front of congress. there's actually a bill that was just introduced recently to provide federal level funding -- johnson -- and it historically prior to help america vote act, it was a state responsibility from a financial perspective to purchase voting machines. in a way it's not surprising after that one shot we're done and congress isn't inclined to put more funding. in in virginia, there was a

proposal by the governor to provide funding to localities to replace machines like this and that did not pass. there was just no incentive. nobody got elected, whether to congress or the presidency or even the city council by saying i'm going to spend more money on better voting machines. it's the perennial stepchild. >> when any government official has a choice between filling a pot hole or putting money in elections, they are going to fill the pot hole. you hear about elections maybe two or four years, you hear about the pot hole every day, which is unfortunate. we do need more regular federal funding of elections. it would have to be structured like highway funds. you are going to have all the shenanigans that happen in terms

of things you have to agree to do to accept those funds. that's just how politics works. as to your question about putin having a thumb on the scale, we're not sure. i say that meaning you could think of elections as a meadow that hasn't seen a lot of predators. so we have a lot of entities, beings, election officials optimized for a place that hasn't seen a lot of predators. it's not a bad thing we have predators now because we have to adapt to work in this environment. something we didn't talk about is the u.s.'s history, the cia of directly influencing elections 1948 with the christian democrats in italy, the chilean elections in 1964, 1970 that resulted in a coup. it's strange we are crying human foul when we've done it over 100 years. [ inaudible ] >> i would say the technical

thing about putin is hacking the presidential election is probably the hardest thing to do in this election. you're much more likely to see your proverbial tony soprano hacking one county to ensure waste management bond was passed. i think that's where you are going to see the first detectible evidence in the u.s. of a vote hack. i don't think something that crosses that many states is as attractive. if you can get suburban philadelphia and it's close enough, there pay be places that don't have paper trails that are, like for example dolphin county, i tweeted about this, dauphin county uses machines from 1985. i remember those computers. they were a lot of fun,

monochrome, oh, boy. i could drop this off in the red square and russians couldn't hack it. the only responsive's seen close to correct was depends how high you drop it. >> i would like to add one thing to this interesting question which is about a recent bill that has been introduced to the committee on september 20th on election infrastructure and security promotion. this aims at designating election -- electronic infrastructures as infrastructures of critical strategic importance to the nation. now, these would allow a response that could even be military response in the worst scenario including by engaging

in this response allies in nato. cyber security is about striking the balance between cyber defense and cyber offense. is there enough consistency in our intelligence systems between the two? because the weaknesses of the systems that should be addressed in order to protect, to make our electoral environments safer are also potential entry points for attacking other systems. so is there consistency in our societies when they deal with intelligence in cyber security between the defense and the offense? i think there is not enough clarity of that. it would be interesting to see whether this bill could actually

imply imposing some restrictions on the cyber attack side of these conundrum of cyber security. >> so this has been a very frustrating thing for me and probably all of you, as well. you see one story in the news that says that putin could hack the presidential election. then another story says it's not going to happen. that's where you're falling. so it's really an unknown. we won't know until we know, essentially. >> we may not know. the scariest part. something i say a lot, these systems aren't designed to keep the kind of evidence you want to detect those kind of attacks. they are not designed to be resistant against nation state kinds of attacks. even then, if you're going to attack one of these machines, it didn't work, you would make it fail to look like a garden variety computer error.

like the blue screen death. those kinds of things -- that's something i'd keep an eye out for. if we see marked uptick in errors and strange kinds of things, that could be the only evidence we see of mischief. we'll never know. it's hard to really bound that kind of stuff. >> of course, machines are specifically designed so it doesn't say who voted for kim and who you voted for, joe. to give you privacy. the side effect of that is if there's problem, it's like, i'm kind of surprised -- if i knew who you voted for, maybe i could say i could have sworn you told me blah, blah, blah. because we can't do that, it's very hard to detect strange things. it's like in 2012 there were comments, precincts in

philadelphia where obama got 100% of the votes. was that tampering? there are allegations there was. if you look at the history, no, that's a precinct that is really democratic. similarly there were precincts went nearly 100% for mitt romney in other places. things that look strange isn't necessarily wrong. it's hard to tell the difference between something that looks strange and something that is strange. >> good point. okay. >> i voted this morning in virginia. it's a little bit disconcerting now learning that even though i asked about whether there was a paper track kept of my vote, to learn they can't audit the system even if they do keep a record of it.

but my question relates to your discussion has been primarily as to the capability of hack iing, and taking into account what you just said about detecting hacking in the past, other than in chicago in the 20th century, which had a reputation for fraud, is there much evidence or any evidence as to how much fraud or hacking has been done in the 21st century up to this point? >> there's a lot of evidence of accidental bugs. sometimes it's hard to tell the difference between a bug and a hack. because the symptoms are the same. padawadume county in ohio had a case where their machine was misprogrammed. i think you wrote an article. >> i probably did.

>> if you and i are living in the same precinct, the order of the candidates on your ballot and my ballot will be different so no one has an inherent advantage of being first. they misprogrammed the ballot rotation so the programs came out wrong. there was a bug. we have lots of evidence of things like that going wrong. we don't have evidence of hacks. >> m >> miscalibrated machines. >> noncyber attacks. maybe that's not the right word. i believe this wasn't in chicago but was someplace in illinois where we had poll workers running the same ballot through an optical scan machine a couple hundred times. they didn't think they might actually compare the total number of votes on the memory to what's in the basket, the number of ballots. hey these don't match, we'll just actually count the ballots over again. we have seen that. there's a whole bunch of

absentee ballot fraud. someone will go to a nursing home, have everyone sign their ballots blank, take them and fill them out and do stuff with them. there's examples of that. there's very little, if any, maybe four instances, i can get you a paper that cites this stuff called voter impersonation fraud which one candidate seems to be concerned with. it's hard to tell. coming to a polling place representing you're a different individual and doing that a number of times in other places. we don't see a lot of that. >> in terms of voting machines, we have problems where after the election they find memory cards from voting machines that haven't been accounted for and they're in the trunk of someone's car, a poll worker's car, something like that. or you have, again, is it intentional or not? it's hard to say. did someone forget to take in the tallies? >> there is a known bug in the debold gem system which is one of the systems that tabulates

where under certain circumstances even when you put the memory cards into it to say give me the grand total for the whole county, it loses certain ones. i don't know exactly when it loses it. it's only certain versions of the software. this happened in california a number of years ago and happened last year or maybe this year in tennessee in a local election where three precincts out of 100 or five out of 100 weren't counted until they actually went back and did the canvass joe was talking about. got the final results and discovered, oh, we don't have the results for the xyz church and abc elementary school so they went and found them. >> it's hard to know what is a glitch and something intentional. there was a superintendent school race where every one out of 100 votes for this one

candidate got dropped by the machine. >> that was in virginia. >> that's right. the candidate lost the race by 2% of the votes. 2% of the votes was about 1600 and she lost by 1600 votes. and 2% of the vote was 1,540. so again, intentional for a school superintendent? probably not. but maybe it was a test run to see if it actually works, right. >> and to see if anyone notices before you do it in a more serious election. >> organized crime in our school district. >> maybe that was our tony soprano moment. more questions? >> thank you. john nicholson at the british embassy. >> can't hear. >> you john nicholson at the british embassy. so the system we use is very different from what's been described by the panel. it is paper and pencil, hand

counting. sort of the equivalent books are sort of printed off and scored off when you vote. clearly, that doesn't eliminate the possibility of voter fraud and there are historical examples. >> how many choices does a vote ver to make? >> in an a general election, it's one choice. i wonder if you were designing a system from scratch what you would design at this point given what you've been talking about. >> that opens up my question about the los angeles county vote system. >> there's a couple counties in the u.s. that have been so fed up with the united states is the only country that seems to think it can buy voting machines on a free market and that's going to work out. everyone else puts out specifications and a request for bids and you buy it as a an country. we're huge and our federal government can't tell our states what to do in elections.

but two counties, l.a. county, california, los angeles county, california, and travis county, texas, have decided they're so fed up with what's available on the market they're going to build their own. travis it's mostly on paper, it's a design concept and software that works. in l.a. county, they've spent the past five years working with a storied design firm ido with about $15 million to produce a new voting machine and they have five prototypes now. this is essentially what we call a ballot marking device. it's a big pen. you walk up to it with a bank ballot, it sucks it in. you interact on a touch screen top cast your vote. it fills out your ballot for you. it doesn't keep information, how many votes or anything like that. but it will pull the data off it to do the count. you put that the ballot in a ballot box. those are later scanned en masse at a central facility. you have to keep the chain of

custody very, very secure. then it's the most secure voting system i have seen that seems to be at the state of development that it's in right now. it uses a dual chip trusted computing architecture. each piece of software for each device is crypt tote graphically signed so the county is the only one who can put a given piece of software on the machine. so the reach around the back and stick a usb stick that wouldn't work unless you did complicated stuff. they had two goals, one was redesigning the interaction of the voter. there's a bunch of neat things they've done with accessibility. it is trying its best to replicate the security of like an optical scan system with a touch screen system. basically you can think of it as a very expensive like million dollar, very expensive pen that fills in your ballot for you. it's going to be open source. all the software, all the

hardware you can take the ought cocad files to your metal bender and they'll stamp out a bunch. the idea is to have a system that anyone around the world could build off of. if you don't like the hardware, you can changing that, too. i'm hopeful that this sort of -- this effort to you know, have a more open way of designing things would go deliver, you know, one increased usability but also the kind of security that we expect and something like linux and other kinds of things which aren't by their nature open source systems we use elsewhere aren't innately secure but can be more secure if used in certain ways with certain kinds of tools used to analyze them. >> just one small correction. i believe travis county, texas, put out their rfp. >> cool. >> this week, last week? >> last week. >> joe would know. >> joe would know. >> do you have a question back

here? >>. >> did that answer your question? i'm sorry. >> hi, david. >> i wish we -- just one very small comment. so los angeles, the average election has 100 things on the ballot to vote on be. >> 11 languages. >> that's why as much as i love the simple hand counted paper ballot concept and there are people in the united states who think we should do that, i don't think it's workable. >> for i a large county. >> for a large county unless we completely rethink how we elect our government. the voting machines are a side effect of these complex elections. >> can we borrow a parliament? >> where you've got a 25-page ballot. >> the david turetssk y. isn't a big part of the problem that we try to set this up in a way that is -- between 6:00 a.m.

and 7:00 p.m. so that any problem causes the most chaos possible be? you were talking about electronic poll books, talking about what in cyber world we talk about as an incident response plan. there's so little time to react. and what happens at a polling place is if the electronic poll books start working and you're starting to check people in on those and then they stop working, even if you had paper poll books, you don't know who's already voted in the electronic poll books. so you're in an extremely chaotic situation. they're going to try to fix the electronic poll books first. what happens is they try to do that is the lines grow longer if it's near a peak part of the day. what happens when the lines grow longer is people don't move their cars and so fleas no parking and there's utter chaos on the roads. and by trying to jam all of

these voters into such a short window, which means the problems are harder to prepare for, solve, and resolve quickly, isn't that the biggest risk to the credibility of our elections more so than the outcome being challenged through -- changed through hacking? >> i just want to point out that the trend now is towards a longer election cycle. we now -- it used to be in order to do absentee ballots, right, you had to be out of the country and you had to prove that you were going to be out of the country years ago, this is the only way you could vote outside of the actual election day and you could vote on a paper ballot. states have relaxed those rules now. i don't know what the percentage is about the number of voter who's now vote from home using a paper ballot. >> depends on the state. >> you have to be willing to give up the privacy of your vote because it's on a paper ballot, it comes in with your name on it. it's no longer a private vote.

>> there are processes in place in the election offices so they check the envelope, make sure you're registered and there's an inner nfl. they put the tun opened inner envelope in a bat box and don't open that till later. there's very strong process security to ensure that your ballot is secret. however, doesn't always work if you're the only absentee voter in your precinct, someone was telling me about a story about that, we know how many absentee ballots we know who it was. >> a lot of jurisdictions have been moving towards vote centers. so having -- you can vote over this two-week period at five different facilities around the city or the county or whatever. for example, i think colorado and maybe washington, one of those two is doing something where they send everyone a ballot in the mail and you can return them. you can surrender it and vote in person somewhere else if you want to.

those are -- i could go on forever about those because there's some things that are not so good about those kinds of models but that tends to alleviate this what we call the load or the scaling problem with having everything on one day. it's like a tuesday when everyone has to work even though you should be able to get time off for it. >> do we know how many voters are voting prior to an election these days as opposed to the ones voting on election day. >> it's considerable. some places permanent absentee in california is the like 60, 70% of all voters. >> it's going to did i have a lot by state. some states still require excuses. you can only do it if you have one of these eight different reasons. so you can't generalize from one state to another. >> okay. >> there is also the aspect, if i may of getting the meet yad results. get going results immediately. the rush toward having results

as soon as possible. that is also interesting. i had the opportunity a couple of weeks ago to meet with a high level litigation firm from a country that had elections recently and they used e voting to a large extent. and he stated that the wrults well accepted but he also added they were immediate. just a few minutes after the closing of the polling stations. and then his comment was also they were all well accepted but because we hadn't a closed election. so if you have a closed election, that is the political risk. in the highly polarized political context when you have a close election, irrespective of the technology, eventually, there will be room for disputes and possibly also in some cases even violence for not accepting the results. and that is irrespective of when you get them. >> so i