washington dc campus. this is one hour 15 minutes.

>> good afternoon, everyone. is this on? is this good? all right. thank you. good afternoon. welcome to the brennan center all,ustice, and, first of i just want to thank nyu's brademas center center and the great team here come in white teen in d.c. who always make us -- team in d.c. who always make

us feel welcome. i'm larry nordern, deputy director of the democracy program at the brennan center, and just to repeat because i was told i can't say too much, entrees discussion of how you can keep up with her work, you can find this online at brennancenter.org. follow us on facebook and twitter, watch our videos on youtube, and listen to our podcasts on itunes and sound cloud. for those who don't know, the brennan center is a nonpartisan law and policy institute the -- that works to reform and revitalize and when necessary defend our systems of democracy and justice. we have been deeply engaged in addressing the many issues surrounding the election security to protect our constitutional democracy for over a decade. so i'm sorry to disappoint folks we will not be discussing rod , rosenstein or robert mueller or judge kavanaugh today, but we are here to discuss a somewhat related topic, election security. and as we hurdle to election day 2018, in fact, voting has already begun for military and

overseas voters, and, in many places early voting has begun , for all voters, it's impossible to escape a discussion of the vulnerability of our election infrastructure, cyber attacks, foreign meddling and there's really no doubt for the cause of all of this discussion in 2016 -- discussion. in 2016, the russians attacked our elections and our intelligence agencies have told us that we should expect more this year and especially in 2020. much has happened at the local, state, and federal level to reinforce our election infrastructure and election security since 2016, and we're are going to talk about that today. at the same time many have , criticized the lack of action on the part of congress and the president, i include myself among them, to take dramatic

action since 2016. and, in fact, at this point , there's little more in terms of big actions that the states can do in the remaining days before the end of voting. so that brings us to our conversation today. where are we on election security? what is being done to protect, detect, and recover from any cyber attacks this fall? and importantly, once we get through this election, what do we want to see our federal and state policymakers do to protect -- to keep us safe in 2020 when the stakes may be even higher? so to discuss these pressing issues and questions i'm , grateful to have an exceptional panel here with me today.

invite you all -- i'm not going to go through everybody's bio. we have very impressive bios for all four of our panelists. you can find the full bios on the brennan center website, on our events page. i'm just going give a very brief introduction. to my immediate right is laura grace, the election protection manager for the lawyers' committee for civil rights under law. and, to her right, and i guess for those facing your left, is edgardo cortes, the former commissioner of elections for the state of virginia and i'm happy to say is also an election security advisor with the brennan center for justice. to his right we have brenda cabrera who is a general register and director of elections, voter registration and elections for the city of fairfax. and, finally, last but not least, noah praetz, director of elections for the office of cook

county clerk, david orr. and as we were discussing before we started, many of you probably would guess, cook county is one of the largest and the top five of local election jurisdictions in the country. so, i am just going to begin with a kind of lightning round of questions that really nobody that works in our space wants to answer. so i will give them to you, choose whichld ones you want to answer. first question is what are you most worried about? would you even know if the russians or chinese or some other hacker hacked into our election system? would you know that in real time or would it take a while to find out? and so that we don't leave everybody in the panic, what are

you doing to prepare for these things? laura. >> ok, thank you. thanks to the organizers for bringing this all together today. the lawyers' committee leads coalition called election protection, the nation's largest nonpartisan voter protection effort in the country. so, from our viewpoint, we are worried about anything that prevents voters from accessing the process can weather has been -- whether that is being able to register, vote or have their votes counted ultimately. obviously the are a number of cybersecurity issues that are of concern but we're also concerned , more broadly about election technology, mishaps or problems that can occur that really put some real barriers up for voters trying to access the process. in terms of what we're doing, through our nationwide coalition and our state coalition partners , we are advocating election officials to prepare as much as

possible for anything that might happen on this front, making sure contingency plans are in place and that officials are trained and what to do when something goes wrong, when a machine breaks. and making sure that those , solutions are really voter centered at a a reasonable and centered and reasonable and not create additional barriers for voters. we ran and 866 hotline the voters can access if they do face these types of process at the polls and we deployed thousands of nonpartisan monitors around the country to help on the ground. >> thank you, larry. i think, heading into november, my biggest concern, equipment place, or the polling venting individuals to be able to cast their ballots. i think for me on the side i thinky

statewide databases, the potential for incorrect information or some sort of impact to the statewide database that will create all sorts of confusion and chaos at the polling place. i think the focus, laurie mentioned, just kind of working with election officials to look at making sure there are some low-tech contingency plans to deal with potential high-tech problems and so making sure that election officials have done that and have things like sufficient provisional ballots on hand and backup paper, paper poll books. things that will keep the process going and allow people to vote, even if we do end up in kind of a worst-case situation. that would be my greatest concern heading into november. >> so, at the local level i , think what we are always concerned about is what we don't know could happen. that's always concerning. this year there is a greater emphasis on election security, so we do a lot of contingency

planning and we are fortunate to have a paper balloting system so the advice is to order enough ballots for everyone. we have enough provisional supplies for everyone to vote provisionally. we download things and have things available ahead of time to make sure that if the system , were to go down that we could , continue to service the voters. >> thank you. can we hear? okay. it's important to talk about things like election systems and an umbrella term which covers all of the digital services we rely on. one of the services is a voting system which is generally way harder to get to than the other digital services. but we have websites to put up election results, websites a put

up voter administration to week that electronic poll books, command centers. each of these has a stronger internet presence. an easierore likely target of our adversaries. i think everybody in the election community now has accepted the admonitions of our intelligence community that there are actors with the capability and intent to disrupt our business. sure there are nationstates but there are also nonstate actors and individuals so the entire election community is focus a lot on security. it is important to reframe security because it can't just be about defending systems. in the corporate world defending systems is difficult and you can look at maybe uber, equifax, sony, companies that have hundreds of millions of dollars defending systems. in the election sphere, we rely a heckuva lot on securing our systems through building resiliency.

that is the ability to detect when something bad has happened and to recover from it. that's what we are spending a -- that is where we are spending a tremendous amount our time and that's what about of the recommendations like the one the brennan center put at recently are focused. not just on defending systems but on continuing strong , business practice, even in the event of successful intrusion. that's where we put most of our efforts. >> so for people in the audience who may not spend the days think about elections, particularly from our election officials, we heard a lot of contingency plans and resiliency. could you give a concrete example of -- and it doesn't because if you have something that might go wrong in one of these connected systems and what the plan is to recover from them? >> sure. >> do you mind? >> i will hop in real quick. so things like one official

election things that get put up election night. they are not real results. they are not coming from your system but you can imagine how frustrating it would be if all of a sudden your website, and counties with that was good episodes that ultimately didn't match what the true results were certainly think about ways to hardened defenses for them. one of the great things, the private sector is stepping up in this area both google and the , company called cloud flair have offered free the dos protection, some additional hacking protections. that is helpful and the department of homeland security , especially in this area offers , cyber hygiene or scanning for this got the bandwidth to protect all 8800 local election officials websites across the country. to let us know what our vulnerabilities are so that we can quickly putting mitigations. you do have two kind of role play the idea that the websites are taken down and you start to build a list of okay, can we get our results through twitter? what if our twitter account is

taken down? they can harden the accounts of the election official in the -- and the administrator. you start thinking of backups. luckily, and our field, we all do that by our very nature, i think what we're all doing now is sort of building 300 page playbooks so in the event of something you flip right to the , sort of symptom and it comes, we have our answers right there in these of the five people you call, these are your vendors. this is law enforcement you call. this is your fbi contact. it's a lot of our efforts have been spent making sure business is reestablished pretty quickly. >> does anybody else want to add to that? >> a lot of what we do relies on information that we get and relies on the internet, and so as much as we can keep a copy of that information on file so that we are not dependent on the internet and if something should go down, which happens even

outside of this emphasis on security we do scant our , computers. we have a relationship with dhs to do that. we are in constant contact with our i.t. department for things that they will help us with but butter -- help us with, what i think having a paper system, doing paper poll books so that is information were to not be available to us, and we could re-create everything we needed to to conduct the election. >> i want to change directions and talk about we're in washington, d.c., the role of the federal government at all of this. as i said in my introduction, congress and the president have gotten some grief about not doing much in the face of a hostile power attacking our elections in 2016. i think that may give some folks the impression that the federal government is doing nothing. noah, i know you've been involved with some of the steps

the federal government has been taking since 2016. maybe you could talk a little bit about that? >> sure. the first thing that happened was in january 2017, secretary jeh johnson of the obama administration named elections as a critical infrastructure sector. meaning as a level of federal involvement, federal care that is automatically triggered in order to support that. the homeland security department quickly pulled together a set of government stakeholders. so the national association of secretaries of state, the national association of state election directors and then some of us local have been working together for nearly a year to try to drive dhs' policy in this industry. the secretary is -- secretaries of state zealously guarded the election institution. from the beginning election run at the state level. state election directors bring the idea that they need to institute and protect the

institution of elections. they've got a slightly different flavor. us locals where the rubber meets the road almost everywhere, have got a different set of priorities and we all have been working together to make sure that dhs is working properly and not overstepping their bounds. i think it's working really well. much to the credit of the leadership of the secretaries of state and of the state election directors. and dhs. it's important to point out that the folks that are tasked with this regardless of the sort of , top-level administrations approach, the people from dhs are working on this are working -- are working really hard on this to defending the -- defend the institution. >> someone else want to come up with enough they found the -- come up with what they found the federal government involvement to be helpful, not

helpful over the past couple of years? >> i was in virginia election to measure until the beginning of the shooter last year is involved in a lot of the discussions and kind of interactions between dhs and the state and local government. there's been a lot of progress especially on the communication front and one of the biggest obstacles we encountered in 2016 as we started getting information about the potential threats to the system is that there was really no communication process in place between the federal government and the state and locals to even know where to get information from or to direct potential concerns to. a lot of the focus last year and this year has been improving that. also the establishment of the , election infrastructure information sharing and analysis center under dhs two kind of set up those parameters and getting all the states signed up, a lot of local governments signed up for the participation and that and for all the kind of services they get offered. has been huge step forward so there has been kind of a lot of interaction and discussion on

movement around improving that communication. there are still disagreements about the extent the federal government should be involved or can be involved, but, aside from those kind of more policy related questions there have , been some actual concrete steps that have been taken over the past you have to really prepare for the 2018 elections and get everybody in a lot better position. >> you mentioned policy. one thing that congress has not done is pass any legislation related to election security. there's this bill, the secure elections act which i believe i i read this morning there will , be an even newer version of this we introduced by senator lankford, who is republican from oklahoma. this bill has i think the support of at least 14 senators,

seven democrats and some republicans. could you talk a little bit about what the bill is, at least in its previous forms and what -- whether you think it would be helpful or not? >> sure. i think again, there is at least there was reporting that towards the end of this week maybe early next week there will be a new version introduced, but it unfortunately hasn't gotten action yet. it was scheduled for a committee vote and that kind of got pulled. the bill does a couple things. the first is that it really formalizes legislation documentation process that i was talking about between the states and the federal government. it makes it a federal requirement under federal law that information sharing and kind of communication flow is happening going forward. so regardless of the administration, parker this of who is in charge, that continues

on. a couple of other key things is it requires things like election vendors to report cyber incidents. right now there is no requirement that if a vendor that provides election technology, if they have a breach in their system, there is no requirement for them to report that either to their customers or to the federal government, and so this really would set up a requirement for reporting those incidents so that everybody can address them properly. one of the biggest things i think in the act has been a point of contention is the requirement for effective postelection audits, risk limiting audits. basically the original version had very strong audit requirements. the current version has reduced that a little bit, but the idea behind postelection audits and

the move to paper voting systems is that you have some independent verifiable record of the votes cast and then you go back after the election and confirm those ballots were counted accurately. it's having people take a look at the paper after the election to make sure that the machines were tallying correctly. there's kind of this whole we will see what the next version comes in. i think it's unfortunate congress hasn't taken a lot of action. there was some money that went out to the states earlier this year for election cybersecurity. it certainly is not everything that is needed in the states. i think be helpful and has been helpful to the states as we move forward, but i think it kind of speaks to the need that is out there. the passage of the secure elections act or something similar along those lines will come with some continuous line of funding for state and local officials because i think

cybersecurity is a risky, expensive undertaking and elections have become much more expensive and resource intensive to run and so we need that kind of federal involvement to make sure that there's some money there to put in these minimum requirements. >> does anybody else want to comment on either the secure elections act specifically or what they would like to see, what more they would like to see from the federal government? may be nothing but -- >> i will certainly take it. in our mind that are really two priorities. this is me personally. one is to make sure there are voting systems with paper artifacts that can be hand counted for an audit. to me, it is simply because our institution has two virtues, put out election results that are true and trusted. even if the technology were bulletproof, which it is not

today and may never be, trust and virtue are hard to establish in the political realm. so the idea that an eighth grader could verify the election results were right is important, that you don't need a sort of degree in cryptographic caching to trust the results. they could both be true but to get to that virtue of trust, i don't see anything other than a paper artifact that can get you there. that's one priority. clearly there was not enough money invested to make sure the 20% of our country that relies on voting equipment that has no paper, the investment wasn't made that could move that he long. -- that along. the other priority though is certainly to get expertise into every local election officials office. we are a big office, 100 employees employees. we have partnered with the

chicago board of elections and hired a professional because when you get 20 emails a week from an information sharing center and you get great recommendations from the center for internet security and great recommendations from the brennan center, somebody's got to be able to ingest that. it is barely possible in our office but we are doing it, but most of the country's 800 local election officials come with a staff of three. the idea that they can absorb information, build the best defenses, institute the best audits, institute the best recovery plans without boots on the ground is a bit unfathomable. i would point out, in illinois, we've taken our half of the funds, $14 million and we sort of employed a team of, we call -- peopleal defenders who are going to partner up with local election officials and be their information security professional for the next couple of years. we think that is among the most states should employ, obviously we are all wrestling with

different challenges. i hope the illinois experiment works out and i look forward to some other states adopting it. your county is a lot smaller than cook county, but bigger than a good percentage of election jurisdictions in the united states. do you want to talk about anything that noah mentioned? has there been sharing in virginia between smaller localities and bigger localities? >> i think the advantage that we have in the city of fairfax is that we're close to d.c., so we are a little more aware of the different resources and partnerships that we can do. there are localities that are my size or smaller who do not have departments, so as noah was saying you are being asked to , ingest and respond to an awful lot of information that you really don't understand. i think i get a more e-mails a day about election security. i think the resources and these partnerships would be really

helpful, and i think it is important to note that while cook county is in the top five, there are a whole lot more state of fairfax small localities and the united states than they are and we need resources. >> we've had on the news recently a lot of attention to paperless touchscreen machines and try to touch on that a little bit. in fact, in georgia we have a in fact, in georgia we have a federal judge, although she denied a preliminary injunction motion to enjoin state from using the paperless touchscreen machines they have. she made pretty clear in her opinion that she was concerned about their security, and that the plaintiffs were likely to ultimately succeed based on the security concerns to force the

state to get rid of these machines on the argument that they were violating the voters constitutional rights. within the past couple of weeks the national academy of sciences joined many other experts in saying that these machines should be replaced as soon as possible. one of your more controversial moves, edgardo, as elections commissioner was decertifying the paperless machines been used in virginia. do you want to tell us about the process and why you did it? and again there's been a lot of concerns about the fact that as noah mentioned, about 20% of voters would use this fall, what that means. >> sure. last year virginia decertified its remaining paperless system, roughly two months before the november election. virginia has off year elections,

a gubernatorial last year. noah mentioned during the first question that there are a couple things about building resilience if no one is identifying and protecting against, and the other one is the ability to recover from them. so ultimately our decision in virginia boiled down to the fact that number one we couldn't identify in a paperless system whether or not anything had , happened to that equipment. there was no way to identify if inappropriate access, if anything had happened to the systems. even if we had been able to do that, there was no way to recover from that kind of system failure. unless you change those records, there is no way for you to recover in that instance, and so the ability to recover from it and not even know there is a threat there really makes it not

a resilient system. decision intough advance of the november election to force a number of local governments to deploy new voting equipment that has a people record, which does provide that independent record and ability to recover. scanners go down and become inoperable and you can have issues there. but if that happens, you have the actual votes that were cast on paper and the ability to recover from that. that was the deciding factor in virginia in pushing everyone to paper. we did an earlier round of decertification in 2015, that was around a specific system where we found security concerns, but ultimately, it may be voting process safer and more resilient and improve the

process for voters. add, i dowant to think that is the best policy adoption moving forward, but obviously, we are administering elections now with equipment we have now. those toward percent of the voters should feel strong. the secretaries of state and those areas take this seriously. andstate election directors the county clerks and city administrators are building a top-notch control, they are putting in all the defenses they can. these are low probability attack areors, just give and they low probability relative to the other systems we rely on. voters going into this election need to feel and should feel very confident that the people

charged with protecting the administration of elections are getting what they paid for. moving forward, as a policy i think we ought to be adopting one that eliminates the possibility of catastrophic election failure so i'm speaking years out for limitation, and implementation, and not for november. >> if i can add one more, i think one of the things we've been promoting , contingency planning is for those in terms of recovery and availability to address issues, they have emergency backups in case there are equipment failures that what you want as an election official for the process to keep going and to be able to continue voting regardless of what's going on with technology. >> brenda, you have actually

administered elections with both systems and i guess dealt with the consequences of it. can you tell us a little about the difference in running elections in those two systems? >> sure. i was in a really large locality for about 25 years and then i transitioned to where i now in 2015 when i got the job, they were on a particular system and by the time i took over the position, edgardo had decertified the equipment they were using. fortunately for me, i had been through a long procurement process previously so i was able to put that into effect in my locality. i think there's a lot of emphasis on the piece of equipment you get. while that's important, i think the things that take the most time to get used to the process changes that happen.

your election officers will have a different process, your voters will have a different process in the precinct. you also have accessibility questions about when you're going from a dre that is in itself acceptable ada compliant equipment, you may now have two different pieces of equipment in your precinct in order to vote in a paper system. also, ballots and paper. it's a lot of paper. it's a different way of getting the ballot even when we have absentee voting so now we are not relying on the machine to give the voter the proper ballot. ballotsto have those and have enough of them for everybody in our locality who votes on different ballots. i think the cost of those ballots is something that is lost on people and that's something that you have to consider.

it's an ongoing cost that you did not have before when you were working on the dre. i defended the dre when we had them because i wanted the voters to be confident in the systems that we had but i'm happy now to , have a paper-based system and i tell people this means we can essentially re-create the election if we have to if we have any indication that something has gone wrong. the other thing is challenging in the switch is the messaging. we really employed our election officers and gave them specific words of encouragement to the voters because they are really on the front line on election day when the voter comes in and says why are we going back to paper and doing this so messaging is important to people as well in the transition. >> brenda, one of the things you're now able to do with these systems, which you had a pilot just a couple of months ago , was something called a risk limiting audit of the system

afterwards and of the election. this is something in the election security world we keep hearing. i've even heard in hearing s senators asked about risk limiting audit, i'm not sure they understand what they are. maybe you can help everybody understand why there's been so much talk of risk limiting audits of our elections and what that means. i would be confident they don't know what it means. it's taken me some time to understand what it means. the first thing i had to understand was that a risk limiting audit is a very specific thing. it's not me sitting in my office trying to figure out how can i build confidence for my voters. how can i audit this election that we just had? it's a very specific set of steps that you take to do that. the gold standard of accounting for an election is really a full

hand count and while that might not be as scary in my locality which is small, even though understaffed, that's just an enormous task to undertake after every election in really large localities so a risk limiting audit allows you to mathematically create a sample that will give you a level of confidence that when looked at by hand will reflect the outcome of the election, or if it doesn't, it will show you that you have a problem and what that problem might be. >> i suppose in theory one of the benefits is that you're counting the minimum number , you are hand counting the minimum number of paper val ballots that you need to have high confidence that the

election -- that they run candidate didn't win basically. >> right, i refer to it as math-magic. that is another thing that happens when we talk about these audits is people get caught up in the math and how it works and there's more in a risk limiting audit then how you get to the sample size. we did three different kinds in the city of fairfax so we basically hand counted an entire precinct. we did a batch comparison where we have sample size of about 69 ballots and we compared those to what the machine said, how the machine said it counted that ballot and then we also did a polling audit which requires a larger sample size and different math and we did about 200 less -plus ballots for that one but that would be a lot less

than having to count the entire election and then that also introduces a certain amount of other human error with it as well. >> if i might add, in colorado and in rhode island if there is a discrepancy, the sample size grows. conceivably, you could get to the point where you need to hand count the entire election so this is done prior to certification so you are guaranteed you get to a point where you have proven a full hand count or you got to the confidence level that the software matches the hand count. that both picked out the same winner and that's the true and trusted virtue that we all try to establish as election officials. >> thanks for that correction, i said election result but this is supposed to happen before certification so there is no official results yet. >> that's the textbook, we are

all wrestling to try to get the value from it even if our state codes aren't up-to-date yet. >> in virginia it is not done >> that's the textbook, we are before certification. >> so i don't want you to think i forgot about you. you are running, helping to run the election protection hotline. i don't know if you said the number yet but it's 1-866-ourvote and this is the in toe where voters call talk about problems they are experiencing and to seek assistance. this doesn't just happen on -- i am forgetting what they election day is this year -- i think it is november 6. it is not so far away. it doesn't just happen on november 6. in fact, you guys were working over the primaries so how do voters process all of this when they are hearing all this talk about cyber attacks and russians and election attacks? what are you hearing from them?

certainly, and just to echo, we do run the 1-866-ourvote hotline for primary. we are launching tomorrow live through the election day to help voters with any types of problems or any assistance they need. on this particular issue, it does seem like voters are a bit more alert to the constraints of -- the concerns of the day, talking about cyber security issues. but as a practical matter when they go to the polls, we have voters call in when they face problems with technology and the polling place or you know, whether that's the voting machine or a problem with the voter registration system. you know, one thing i think it's worth reemphasizing is thinking through when there is a problem, what happens? we have voters who call that say

i filled out my ballot but when i went to put my ballot in the scanner, it wasn't working so they put my ballot in a secure location and said they're going to check it later and we are reporting to make sure this is known and through our network , we let election officials know they need to make sure the scanner sticks but the voters themselves, their ability to vote wasn't impacted by this problem. compared to voters that call when a dre machine is broken and that's the only option in their polling place. they say i've been waiting for , an hour, they say someone's coming but now i have to go to work and i can't come back because poll hours for a have to leave work. oll hours close before i have to leave work. these are problems that impact voters and disenfranchise people if there are contingency plans that are reasonable options for voters to take.

so, we do through our hotline get a number of reports like these and go back to advocate to election officials to think through the practical impact that it will have on voters .one -- on voters. one issue that we did cover, a number of calls about was in la county during the california primary this year, due to what we now know not to be a cybers ecurity issue, but it seems to have been a software issue, 120,000 voters, their names were missing from the polls so they went to vote, they were told they were not on the registry. thankfully in that case, polling officials immediately were telling voters to cast a provisional ballot. the county board of elections were quite transparent and the public about the issue that was happening. voters who had cast provisional ballot, their ballot was counted. there was no extra effort required on the part of the voter, so in that case despite a , significant error and a very serious issue that was later reviewed by external auditors

and proper follow-up being done with voters to inform them of what happened and remedy the issue, but those voters will -- were still able to vote, their vote was still ultimately counted and the voters themselves didn't have to take any extra step to vote. more recently in maricopa county in arizona where ballot printers at precincts were not set up on time, polling stations in many vocations didn't open until hours later. polling officials didn't really on the ground, didn't know what they were meant to do in that situation. their information wasn't consistently shared with voters, provisional ballots were not always provided. there were early vote centers available around the county, but other low income voters which had transportation limitations or voters with -- that had to go to work, were not able to

ultimately cast a ballot so these really can have some very significant impacts on voters ' ability to access the process. >> can i take a second to highlight the la county? those provisional ballots are part of federal law. that resiliency baked into the system, i think it's actually a good news story because while the type of thing that happened there was a software problem, locally, the impact is the same as what's a likelier attack vector, one of our voter registration databases going down. there is resiliency built into the process to help sort of overcome or withstand even a successful attack and i think it's a good news story. that's something that's available nationwide in all states, all counties to be -- are set to implement it. >> but training of officials and

kitchens a -- and contingency planning -- >> absolutely. >> great that los angeles had that. so i'm told that we should, it's time for questions from the audience. how are we doing this? are we having people come down? there are two microphones on either side and while we're waiting for people to come down i'm going to ask one last , question. i guess make this lightning round again but i want you to imagine all of you that we made it through 2018, no problems, everything went great. what are the big things you are thinking about fixing, tweaking personally? where you work or if not that, what are you hoping to see the state, your state or federal government do? >> we are trying to buy new election equipment. that is easier to defend and provides better capability of

detecting error and recovering and we've got to institute that over the next couple of years so we have all heard about how changes are hard to handle. that's going to be our top priority. >> i think for me, having conducted this audit for virginia, this is the first audit that virginia did. the laws came into effect in july of this year and we have a lot of work to do to kind of hammer out the details of that, implement audits in different kinds of localities, bigger localities. do a cross locality audit of a contest that involves different people with different machines and how we're going to do that. i think that will be kind of my focus in the next couple years. >> i think postelection heading into next year, i'll be focused on nationally pushing that 24 % of places that don't have

paper-based systems to transition, because they will have time before the presidential to do that and going along with that is this push for audits like getting people really more comfortable with them and looking at how to conduct them given whatever their unique setup is so that they are ready to do that heading into presidential. >> it is twofold from our point of view. advocating to election officials, to make these changes, moving to paper ballot again makes a much smoother voting process and its much more secure process in the long run. but also trying to shift to a more proactive outreach and reaching out to voters. so simple things or simple actions that voters can take like checking their name on a voter registration roll before the deadline comes, even if you're a long-term voter is just a good practice to take on to make sure if there are any issues with the voter

registration database, those are addressed and advanced and that -- addressed in advance and that folks are able to vote so hoping that not just through election protection but also hoping to see more proactive outreach from election officials as well to push that message to voters. >> great. >> my name is george ripley. i work with audit elections usa. and we are working on the aspects of the ballot image that are created in the new digital scanning equipment, as opposed to the older technology, the optics scan equipment which is still used in significant amounts here in the country but the digital scanners take an actual photograph of each ballot. and it's from that image that the cast vote record is obtained.

so you put your paper ballot in. the paper ballot gets a picture taken of it and then it gets shunted into the safe box. so then it is from that digital image -- >> i meant to say when we started, if you could make it a one part question that would be great. >> so i noticed that nobody addressed the concept of ballot image audits yet. so perhaps one of you could do that. >> i can address that. so i think from our perspective, the having people check the paper is really the most effective and best way to do this and when you're looking at digital images and kind of digital records, you have the same potential issues around malicious software or all sorts of other problems you may

encounter in a paperless system, so that's why we've been promoting and talking to folks about doing the paper ballots, the risk limiting paper-based audits, because that's where you're going to get that independent record check. >> the only thing i would add to that, i think you would agree there's nothing wrong with using digital images to audit. it's just that that should not be all that you're doing. there's plenty you could learn from that but at the end of the day, you need to make sure you are looking at the paper to. -- at the paper, too. >> that is a good there are a point. lot of things you can learn from digital images, not just on the audit side but from the voter side, you can look at issues around ballot design and problems people might have and there's a lot of good things you can do with those ballot images but when we are talking about it from a security standpoint our , focus has been on the paper-based audit.

>> you want to go first? go ahead. my name is stephanie, i'm with secure elections network, a group working on elections security. we are trying to let our activists know now what to do to make sure they are registered to know the hours of polling and to know if they need a ride. is there an online site that they can go to or is there a number? does your 866-ourvote help with that or is there something else you recommend? >> the 866-ourvote hotline which will be online from tomorrow through election day can help voters with any information they need any obstacle preventing , them from voting, if they don't know where their polling places, don't know how to track -- check their name on a voter registration roll, volunteers

are ready to help with that. if they need to find a resource to find a ride to the polls, we can direct them to local resources as well and of course, if they face bigger problems or bigger issues when they go to vote at the polls like long lines or broken machines, we are also ready to help so the 866-ourvote hotline is available and 866ourvote.org is also the resource where voters can find tools for those questions. >> that is fabulous, that's really important, thank you. and if i could put in a plug, please do your audits before you certify the elections. >> can i put in a quick plug? state election directors have websites where they provide information. everybout everybody, local election director has their own website so they know what county they live in, what state they live in, those are great avenues to go to. also google, facebook, they take

public information that each of us are giving them about registration, polling places, hours and they make that public in the run-up to every election so when you do a polling place search in google, that information is coming from us through api and being published out to you guys for finding -- so finding election information should not be a problem. but the best trusted source is the state election official and the local election official. >> i should say also for our volunteers, we are directing voters to state and county websites. which can be difficult to find but we help close that link for voters that are having trouble. >> optimization. >> hi, i'm beth, i was just registering high school student today and that's why i'm dressed like this. i'm also with the league of women voters and i'm a precinct chief at an election.

and so, we have, i was looking over the security checklist and i think where i am, we do a very good job with all of these. but, i was also reading page one and two about the connectivity for wireless networks. we use ipads to check-in voters. but they are linked to each other using bluetooth. i am at a high school, they are using the high school wi-fi so we're checking in with 5000 voters in this precinct and there are over 200 in our county, 200 precincts so you -- it said to have a good strong password that you will change after every election but with over 200 precincts, should we have a variety of passwords that we are doing and using and how secure is that with bluetooth

with the ipads checking in? they sync with the polls and such. >> i'll take that. it sounds like you are perhaps a northern virginia locality. >> i didn't say that. >> in virginia, we established electronic poll book standards a couple of years ago that require testing and certification of polls, not just to make sure if they work appropriately but they meet state security standards. so i think those pole pads in particular had specific requirements about how they get deployed. the use of networks to make them as secure as possible, certainly a preference is not to have those but those systems, kind of with the wireless functionality baked in, but i think there are some state requirements around security that they have to meet and also at the local level,

they are doing secure storage, they are doing updates prior to elections, doing kind of a password deployment so they are taking those additional steps at the local level before they feel -- field them to make them as secure as possible heading into the election. >> i guess another thing to add is having paper backups in the polling place and enough provisional ballots so that if something really does go wrong, it happens every election somewhere with electronic poll books that there's a problem and having enough so that you can get through a couple hours of voting on provisional ballots while they're trying to figure things out is really critical. >> i guess what scares me is that as a chief, if there were someone who somehow -- we have the paper poll back up and we have to double check that everything is correct before we start the election but during the day, we are using an ipad and checking every one in.

if someone comes and says this is my name, this is my id but we show they've already been checked in, if someone has gotten into the wireless network and started hacking that, we're kind of stuck. >> you're voting provisional and you've got a number of weeks after the election to sort of resolve it. >> right, but i can't tell if someone has -- i mean you've got to have the numbers matching all the time when they're coming in. >> this is a bit of a tough area. we cannot eliminate responsibly -- what weut is we do is we manage it, we recognize that the ability to be digitally connected to a precinct, to know what their bandwidth is, how many people are coming in, if

there are lines. value, i'm going to trade this bluetooth or this wireless access point could create another set of problems . so as administrators, we are constantly managing risk. to move back in time to a place where we eliminated them all, i think would make us pretty irresponsible so yes, it's an imperfect solution, but the provisional ballot covers the absolute baseline. that voter gets to vote. election administrators or tech staff gets the work through the program. you have information that says they already voted so you might be in a little more of a bind than a precinct, talking why you pushed them down that path but that's a reasonable risk reward balance decision and we've got a ton of those that we make and they're all highlighted now, especially as we think about connectivity and sort of cyber exposure. >> it hasn't happened. >> i hope it doesn't. >> it's a great question and i'm glad you asked it because a lot of us who study elections are

concerned that there's been a huge increase in the use of electronic poll books for good reason as noah pointed out but we certainly have to be focused on that we are keeping up in terms of what we are doing and backup contingency planning, security to make sure that we are keeping up with all that change. >> thank you. >> [inaudible] someone could not hack up from afar. that from afar. [indiscernible] >> i think the concern over the poll books in general, is once you start along the road of wireless connectivity and internet conductivity, that opens up a lot more avenues for remote access to systems. that is really get the ballots that noah was talking about and i think the push for securing

electronic poll books and taking a closer look at those and perhaps developing national standards around security is something that should be looked at. >> we have two more questions i want to get to. >> i'm mark levine, member of the virginia house of delegates. nice to see a couple of virginians on the panel. i am very concerned about cyber security, one of the first bills i introduced was a bill get rid of the paper ballots in virginia so i was glad to see your decertification. we had a number of contested house of delegates elections and i think a recount is where the rubber meets the road. i think people aren't as concerned when someone wins 55 to 45 but when every ballot counts, that's when every ballot counts and we had an election in virginia where they had to draw by lot and there was a real question about which ballot should have been counted or not so i'm going to be introducing legislation this year but my

question is, paper ballots are great. definitely the gold standard but in the recount procedure, what they do is they run it through the scanner first and then they only examine the ballots that are over or undercounted. and in that procedure, people are telling me the secret is you run ballots through a scanner one day and you run it through a different scanner another day and count is different, the over count, undercount, sometimes ballots fade. that's very scary to me. my question to all of you is what can we do to make sure our scanners are accurate? maybe we need better risk limiting audit procedures, or maybe we need frankly when the , scanners are different, that people go in there and hand count the votes which is what i would like to see, but if you could talk about scanners and how we verify them. so i'll step lightly into

that question. from a risk limiting audit standpoint, if you had a close margin that fell within the recount in virginia, it's .5% or 1%, you would be doing a full recount anyway. if it falls within the recount and if not get into law but if , the candidate actually asked for it to be done. and the recount standards are different from election day standards. so you end up hand counting ballots that you didn't hand count on election day and that's almost always going to create some challenges because you're looking now at voter intent and we had a ballot in the city during our audit where the voter clearly over voted the balance. -- the ballot. and the machine did exactly what the machine was supposed to do and counted that as an over voted ballot.

but it was clear looking at the ballot, when the voter crosses out two names and circles a name and says i want this one, it's clear that the voter intent is that they want that candidate. again, the machine responded to that ballot exactly as you would want them to do. so as long as you have a law that requires a different methodology for a recount than it does for election day, you are potentially going to come up with differences in your account your your account, -- in count, but you should be able to explain those counts. so in our recount, we could explain everything where the numbers differed. we could very clearly show the ballot and say this vote got counted as a vote for here whereas on election day it did not. and if you're coming up with a recount and you're not being able to essentially go back and say this is the difference

because we had this many ballots that were marked differently or adjudicated differently because of voter intent, then you know you have something that you need to look into with your machines and in our case, that was not the case. >> i completely respect what you're saying, maybe i didn't phrase my question. i have no problem with that. my question is the scanners themselves, you run the same set of ballots through one skinner and it might say 65 votes, no problem, four over votes, six under votes. you run it through a different scanner and it might have a different count. and so my question is what are we doing to make sure the scanners are accurate? >> we should replace most of them. all of our technology is free iphone. this is what the facts are on the ground. we don't invest in elections at the local, state or federal level.

we got a handful of vendors that work hard but on very slim margins. there are technology problems that can be solved if there's a functioning sort of market and there isn't, there's some really good technology. the new scanners from each of the vendors are really good but most of the country hasn't been able to employ them. the last real investment was in 2002, and we are all sort of together tonickels see if we can patch things up with old technology. >> with the scanners themselves you're looking at a couple , things. in terms of ensuring that scanners are counting appropriately, the first book thing we've been talking about and a lot of affection election -- a lot of election officials do although not all is to do logic and accuracy testing in advance of the election for 100

% of your voting equipment because that's going to identify potential issues in advance of the election with programming of the machines or with how the machines are taking certain ballots. so really having a rigorous and thorough 100% logic and accuracy testing i think on kind of we talkting the counts, about the risk limiting audits. the law that was passed as that coming after certification. moving that precertification gives you the assurance that those scanners were counting accurately and gives you that confidence in how they are counting and establishing like brenda said, voter intent but having those standards set up in advance of the election, in advance of any recount so there's an agreed-upon basis for tackling ballots that don't go through the scanner. things that over vote, under

votes, stray marks, all kinds of things but having those guidelines, those standards set in advance of the election so that everybody's kind of clear on them. all that leads into having a successful recount process where you have confidence, you built confidence along the way in both the programming, equipment itself that it's all counting accurately as you head into a recount and incorporating testing and you know, the verification prior to a recount as well. >> so we've got five minutes left and three questions so what i'm going to ask you to do is identify yourself, quick question and we will try to answer. >> my name is david from pittsburgh, pennsylvania where i'm involved with a number of groups and i have a case to be argued before the pennsylvania commonwealth court on whether we can have local like allegheny county appoint an expert

commission with cyber security people to review the machines because all these vendors have different equipment that has strengths and weaknesses to create an optimum system, but it came to my attention in the past week or so that one of the counties in the northeast of the state has an election director that has personal ties with one of the vendors who she's very much pushing for the replacement of their equipment, pennsylvania will have to replace all its equipment through the state so i am wondering what sort of protections and looking at the relationships the election directors who have influence in this may have with the choices. we have that same issue in allegheny county where it seems to be backroom dealing and not open in the review of the equipment. so that's my question. to questions, and then

we will -- >> i'm lisa with the league of women voters in arlington and my first question is actually a quick comment. the league has also a sort of hotline that's called vote411.org and it provides information about ballots and voting. and the second thing is, has any funding been anticipated down the road for voter education to prevent such things as over and undercounting so that voters know if they're filling out a paper ballot and they change their mind, they made their mark in the oval or square and they change their mind, they realize they did it wrong. the thing that they must do is to turn that ballot into the elections chief and get a new fresh ballot because that ballot has been spoiled? >> final question. >> my name is erica posey and

i'm also with the brennan center. building on voter education, i wonder if any of you can speak to efforts to educate voters about what are reliable sources of information as we've seen information campaigns, so letting people know about the availability of the websites and that that is their best option? >> great question especially , given that when we're talking about foreign interference in elections, and what happened in 2016, a big part of that is misinformation on social media in particular. so we've got, it seems like three, two questions, somewhat related. what kind of precautions can be taken or review of election director decisions and purchase of equipment? what kind of education is being done or can be done around things like over voting and

improperly marking ballots and then more generally about getting voters reliable sources of information? if we can go down this way, do you want to take any of those? >> local procurement codes drive all the decisions and i don't know how to advise you beyond that but i would assume there is one in a county and state that would drive decision-making. so with respect to voter information, it's tough because most of the country, we all make different choices on what ballots are given and how to use them so it's tough to put out national information. one of the interesting things as we move towards a lot more hand marked paper ballots, things like over votes show back up, one of the nice things about touch screen or ballot marking devices is they can prevent certain errors, over votes being one of them. you know, we are short on

resources and as we try to measure what we invest in, in light of the sort of admonitions of our intelligence community , which we've all come to expect, most of our efforts now are going to working internally to make sure that we are as resilient as possible and the sacrifices can often be some of these other outreach activities. so it's certainly a priority for a great many election officials but there's a constraint, and that's resources. >> brenda, did you want to take any of those questions? >> just voter education. after our first recount in my locality with paper ballots, we learned a lot about how voters interpret instruction. and so we changed our instructions and we post those instructions now right in front of the voter.

i will say that it's amazing how many times the voter doesn't see what's right in front of them. but, we are reactionary in that way. we look at what happens in this election and we will do our best to try and troubleshoot that for each election going forward. responding to where the sign is, what the sign says and all of that. i also have the advantage in the city of having a newsletter, full-color newsletter of many pages that goes to every voter in my city so finding those avenues where you can get information out to as many voters as possible, we live in the information age but i feel like it's harder and harder to know how to actually get information to the people where they're looking. >> i think on the procurement, noah mentioned that there are general procurement

requirements. on the election officials side. one thing i've been cautious and i know a lot of election officials are but should take notice of the appearance of what those relationships may be. it's not necessarily that there's inappropriate stuff going on but the appearance and there's a very small field when you're talking about election officials. kind of an extra caution for election officials to take note and avoid any appearance that there's something inappropriate going on but if they're using, i am following -- and following those procurement rules that should cover them from a safety standpoint, making sure they're making appropriate decisions. on the education side, unfortunately, noah is right, the resources, there's not enough resources to administer and do all the things that need to happen right now and on the spectrum of where the resources go, unfortunately voter education ends up being toward the bottom of the list because

the election offices have to keep the systems running. they have to employ the staff, they have to pay for paper ballots and they have to cover all these real and tangible costs so unfortunately the voter education piece gets short shrift and that's where there's folks like the voter protection hotline and folks down there promoting where you can get information, helping people find information, being those trusted sources to get it and filling in those gaps where the election officers don't have the resources to reach. >> do you want to close us out? >> just touching on the misinformation piece and how that can be managed, i think all of us probably in many different ways are doing what we can to counter misinformation. as brenda said, unfortunately the way in from people get information is so decentralized that it makes it very difficult

to find one solution that will work for all types of platforms. i know many of the big social media companies are looking into this issue and both we and our partners are giving recommendations on what might be done. in addition to that, i think unfortunately a lot of the onus falls on us, this room and people who care about voting and know how to find the correct information, know what the trusted sources are, might be able to find this state or county election board website to verify information. where you see misinformation or hear misinformation, reporting that or flagging that on twitter or facebook. doing what you can to disseminate accurate sources of information. this is unfortunately a multi-pronged approach that we are all going to need to collectively take as a country so just encouraging any avenues you have to counter misinformation and encourage you

all to take those and i know we're all doing that part to do that as well. >> thank you everybody. i'm going to stick around for a little while if there's questions we weren't able to answer. maybe some of our other panelists will be able to as well, but in the meantime, if we could give a run of applause to this fantastic panel. round of applause to this fantastic panel. >> [applause]