that happens. but if a trade is going on, then both parties were better off, no one forced me to buy, you know, that chinese noodles that i might have or no one forced me to buy the german car that i have. i must have said, yeah, i'm better off giving up my money and the german car manufacturer must have said, we are better off giving gary wolf from the car and unless we start thinking of trade as individuals then it will affect a great deal about how we do our public policy with regard to trade. ..

[inaudible conversations]

>> good afternoon everyone. thanks for coming and tonight we are in here to talk about the new book "weapons of math destruction" and we are excited to get into this discussion have the few housekeeping notes first if you have any noisemaking devices and also with the format you will have the

opportunity to ask questions after the presentation please raise your hand when you have a question we'll bring the microphone around to record that so everybody can hear we will be doing a book signing after this is over. if you like to purchase a book they are for sale up front at the bookstore also this is a politics and prose event if you pick up a

calendar you can find but to go beyond the simple challenge where we live in the age where it affects our lives to be paid for health insurance and by the mathematical models. and then the opposite is true. and even when they are wrong. cathy o'neil is the ideal person to write this book

those have been involved with occupy wall street would have the strongest voices and that algorithm will perpetuate that injustice. so many aspects of our lives. clementi switchover to the private sector with the software company --

[inaudible] [applause] >> so maybe we can jump then get some definitions because we were chatting in the back to say one project we have done in my lab if we use that before the personality traits. how many of you have done that? it turns out lots of businesses want you to take those.

and then to use that in lazy melee or may not. [laughter] >> so to talk about the of "weapons of math destruction" and then have an example with those characteristics and by that they do brcs about the rhythms that i care about and others that i don't put those that i do are destructive and nichol them "weapons of math destruction" so to start with example there is the guy named kyle who is a college student he wanted to get a part-time job at the restore -- crush restore if he had friends that worked at kroeger's they said phil

up the paperwork on-line. so 60% of java applications he was required to do the personality tests online will. but he didn't get to the interview process this is standard procedure so he failed. he was lucky because most people don't find out but he fifth found out because his friends said he was a red flag, and then also his father is a lawyer most people applying for a minimum-wage not have parents who are lawyers said they said what kind of questions were there? because i feel you are qualified to get the job to

bag groceries. strays a high-school 11 a problem the ones at all hospital now was being treated for bipolar. said that is a we could predict pdf and even some things like how he emotionally peace table or you? du did a angry easily? do nothing bothers you? it is easy to find that out from the test but those of social laugh and other data we leave behind as well. so he talked of his father but essentially the he said

that is illegal. as part of a process of hiring you cannot make them take a health exam including mental health. so that tries to prevent that creation of an underclass of people so his bother is suing the behalf of anyone who has ever taken the test but if he had not tried to get the job and six others in the land area it was the same test for all of them. so he was systematically prevented from getting a job so that is the first example. and. >> if you are talking about a test but with these types

of the insights it comes from these models built on algorithms and data but the fact that they can have a bias even though on the surface they don't is the nature of your "weapons of math destruction" concept. >> / adjust define that in an algorithm that is widespread and impact will. like those that i've built in my basement nobody cares about that. but they start carrying one would've fax lot of people so this is the perfect example. kroeger's to built the algorithm sold it to a bunch of companies to use instead of their age are people.

and it is secret. pile does not understand how a team was scored. and they just understood that they were. and finally it destroyed his chances to get a job but also creates that feedback like systematically with those with certain disorders. that is a local one to highlight the secretive black box nature and that is

a way to validate. live a neighbor getting teachers fired one. >> so the school chancellor visited the policy where some would get fired if they had bad teacher assessment some would get bonuses. it this complicated but they'll have a lot of spread so those who want to discriminate that what what is right in the scoring system so very few of those ways that they have any type

the spread so they instituted the assessment that is called the growth score or the value added score. i will go to technical but it is a very broad way to think about this and the teacher is on the hook what they got or what they should actually get. the estimates which each should get. you each got a score for your standardized testing in fourth grade. then you'd be expected to get 75 if you have 80 then

you'd get more then you were expected. it is not as clear and it is complicated because the test will vary every year. and find out if they had breakfast before the test. and the teacher is of the hook for the difference between the two. but from what each of you got that difference so sometimes it is also called monet's.

so it is on these terms. i don't want you to be confused it is normally called pretty meaningless. and they came down almost meaningless. not because i have that hard data the only when i got my hands on any real data i talk to one teacher 96700 did 2011. and then there is us secret system. so they go back to washington d.c. one teacher

got fired because it was a little too low. but the other what were 50% where the information was terrible value added score. so what more thing of lot of fourth graders had gotten really good scores at the end of fourth grade. but they could not read. they could not write. so she was suspicious of their goods course and in fact, has every reason to believe that the teachers cheated to get better value added model scores because

their kids were better than expected but they set her up to get worse than expected. so there is all sorts of erasures atom of your kravis scandal but it never systematically got investigated but in any case she was fired. so it is widespread and they're using more than half of the state's or the school districts she did not understand her scores were but when she appealed but then it is destructive so not only did did destroy her

life and she was rehire the following week in the affluent suburb where they do not use the scoring system. but there it is a nationwide war on teachers were and then we will fix education. but then in the case was to get rid of the teachers. and with that value-added model to get rid of the good teachers because and then

moving to the affluent suburbs. so right now there is a nationwide shortage of teachers. so this opaque model is very bad. and something bad is it is insidious about this so things that we don't interact with debbie algorithms that we talk about here have a time of similarities even like neff flex and we know how to handle those if you bought steven king then you say you are right. sometimes it is totally wrong but the all grit in

your talking about they don't talk about it 1.of the decisionmaking process mitt is even when they are poor leave validated and that makes them a lot more destructive because of the human intuition. >> i would argue it is worse than what you just said. >> i will give you two examples. facebook trending stories guess what? we noticed that that is not

a real story. that doesn't happen with the teacher value added model. they have a secondary assessment they were told this is your score. fell one guy that was ashamed of went from being a terrible teacher to a great teacher corrects there was no feedback mechanism to say i admire really good teacher to said don't want to see that because it is awful don't recommend anything like this to me ever again. >> they did not have access that way so they could not go in and override.

>> the other thing and want to agree with this the way i can about the algorithm is my friend is of principal and brooklyn and started to complain about the teachers getting bored and said tell me how they are being scored as a contact they said the dollar under steam and its mouth. like that isn't good enough? that is supposed to clarify not confuse. that is not okay so as they went through three different layers to see would not understand it is now off so

then they have the of white paper bois i've been doing this modeling seven years and i cannot understand so i filed for the freedom of information act request for the algorithm and it was denied. even the reason that i thought about that approach because it successfully had died all the scores as an act of publishing. so if they can get the scores then that makes sense. so i contacted someone at the institute they explained

that there was a contract with the city of new york to ever see how it is made. the secret from the officials of the department of education. >> it is a whole manifest of your argument because i thought you were building these for a while and then you became the enemy like joining occupy wall street. >> definitely. and through those many years the pro-life so i joined my hedge fund heading straight

into the crisis because very quickly those that i thought were experts didn't know much about what was going on not to say that i didn't understand it perfectly but by the way the person that introduced us make i have this idea of it being beautiful and to at the heart of the financial crisis with those triple a rating is on those mortgage-backed securities.

but they were promises could crunch the numbers to promise that they would not go bad so basically they were creating a lie. not only did they believe it with that scale of the market it was such a big deal. so i realized it was the aisle weaponization they trust the mathematicians that itself was not the problem but they were corrupt and shielding themselves to say don't look

here. you have to trust us. so that is what i realize. so then i left to become a a scientist. so is it a predicting the market i am predicting that people. with the historical data that did not seem so bad but the moment i decided to want quit my job and write the book he was thinking of investing in our company so

we will all said and listen to him talk about taylor advertising in the stakes of travel and that was relatively benign. you offer them a hotel room so you give opportunities to some people. so that is a mild way to segregate i did not think of as evil and then gave us this stadia of his future dream of the internet. i have a dream someday it did not sound like martin luther king. [laughter] let me take that back then i

am offered trips to aruba the men never have to see the university of phoenix ad again. so everybody around me will laugh. so what happened to that democratizing force of the internet? this is those fed is to segregate people by class from those that are scoring i we have that opportunity that we like to play with them people on the inside of the spectrum pest in mexico make money off of you for metrication but i make money off of them by exploiting so where do i make one money off of them if you are in a certain class?

>> right. think about the advertising because it offers those demographics so they are very good to show me the arts because i am very loyal to the alpaca. [laughter] sometimes they say that i want that but for some people the most profitable thing is to get that person to take out the federal loan and then saddled them with debt and then don't give them the education.

it is a relatively large industry i have never seen the university of phoenix advertisement i looked did up and i saw that the parent company was the same goal biggest ad buyer on twitter. so i will not see the failure of this kind of all the rhythm. everybody here saw that because it was so loud when it exploded but flu will see the failure of this kind of algorithm? because we have segregators cells to such an extent that if they fall prey to that trap we do not see that.

>> cowrie doing on time? so one of the things that you talk about is people who wore a building these algorithms because there is no ethics training for people like us. the reigning in said good part but the question that i came away with is if there is money to be made it seems like that will happen but we also thought that with enron in 2001 and just to make many on energy futures so it

seems like the market is set up to use anything to explain people to make money we will do what so how do we change that part of the system? >>, don't expect all that to end but i do want one particular forum to be challenged where because of the algorithm it has embedded values. it is just that those who create that algorithm can decide what those morals are.

so i have of very specific reason to define those algorithms because i perform triage. i told them where to focus the attention many of those are accidents because those who are greedy will always want to do that. so the all grin and the rises to that level potentially a weapon of mass destruction baird need to be laws about that there will be no free market solution and it makes money and is

profitable to be discriminatory you cannot say please stop being discriminatory that just will not happen that is why we have allies like fair credit reporting act those that were written in the '70s with those ficus scores and they are not perfect but it is better than what is going on with big data. so to push back and demand accountability so when you are assessed at your job i'll m i s as to exactly? they also have to have rules and regulations that is

widespread and impact falls civic you made it very clear that if you have an algorithm that seems like a shift work differently, and then that is because of a human. a fair it is discriminating because whoever program to didn't consider that could discriminate based on race. so the stars to its beauty holocaust denial mw because they did not think women are minorities are harassed on line all the time if you have an industry made up of people who are greedy or in and of the problems then they can be ignorant and

this accountability issue is to think about the problems not just to optimize the output. >> there is a lot of public examples so this is what we get to see. most of this is never seen by the public is the business dealings so those that are known to be under public scrutiny are that bad >> [applause]

>> we will start in the back. [inaudible] this is incredibly interesting. but it is valid. so reminds me of what we have to accomplish the same function whether constitutional law that nobody in this stance but they legitimize that structure to get to the bottom of this so they are the only ones that understand.

>> are you a renegade? >> and much to my shame those that are in that prefer it that way. is pretty flattering to be told your magical did is you are a mathematician not enough are worried about that but i do want to say that when i started the project i quit my job to write this book i was panicking because i didn't see anyone around me who was worried about this. but it now oh, four years later have the entire community including scientist and enter apologist but then with

those tools specifically to use into the black box. sinecure social logical experiment but then they would send two separate applications to see if the white names get more call back. there is the algorithm. it is crude ended is the first generation so long

story -- with have work to do but also for those that think about this sort of field that should be for the crows the did you feel that is coming but not right now. in of beginning midori to a place but but then they will send these people to jail even if they are a rapist and and then the reporting

team uses that data in the algorithm so there is work being done. >> this is wonderful. the one to go over to various that there is tremendous confidence the first did the committee with the electronic medical records i wonder if it isn't your field but another big

push of medicine and to take risks? to say if they did require so much care they would require more care than your anticipate. to make sure it is publicly available to figure out. >> great question. i am not an expert in your field but i do want to say i worry i would even think of

the other person if i were you. of these testing procedures. because i will explain by analogy the problem with teachers is there is no ground troops compared to what? had amino there any good? and with that accountability is made it to be trusted. then they said we have a way to fix that nbc i don't trust you. this has to be convincing in a real way because right now

they just say big data. but ultimately you need to be convinced and shown why that is good and wide that agrees with you and then you are not quite sure yourself. >> and example that i want to throw out there is the algorithm anatomy had really good prevention. even if we did we could predict the future, that is not necessarily a good thing. because in the hands of your doctor if that is done well

for one that can charge more for future alliances that is something that was not addressed by obamacare by a pre-existing condition not a future condition for those who can hire or not to hire. that would be really bad. and that would be used for good. in those that terrified me. so we are starting treatment with there first double pollex anonymous meeting.

and then to come out with a deere 90 percent accuracy. one that looked at women of those twitter accounts on the dais that they gave birth was 85% accuracy if they will develop depression or not. and then to monitor that and then that is scary. and also like diabetes or obesity or heart disease like depression or p.t. st and the sciences interesting. there is no regulation.

in the rich talking to the big companies especially to be down there to look at those algorithms. >> we are all terrified? right? said did not know you could do that with twitter. >> so this reminds me of in hippa in relation to kroeger's vocational testing why isn't as sensitive or worthy to be protected like the of hippa as the medical information or is it? >> thank you for sharing

that. >> it is in my book. >> to have that pervert down rhythm. >> with those education of those potential benefits so it seems the information of that analysis that helps to identify the instance of cheating.

that it is a really the algorithm so if it is the businesses in the second example. the did as a matter of efficiency. and those responses to that. >> >> the only way it was cheating in the first place because of the structure of the value added model itself basically there was not enough incentives in place. so that causes cheating.

i am not in the agitation expert so i don't have a solution for education and did this just a bad model. and i told to the "new york post" there was a high school math teacher who did something very clever. so i could not actually look at the model but if they had to schooler's because maybe they taught seventh grade and eighth grade math.

you unexpectedly the overall score as the teacher to be consistent. seventy-eight or 82 but he found uniform distribution he was as likely to get 90 or 40. it was almost a random number generator. there is no reason to think. i said almost because there was a 24% relation but it is very low but you cannot possibly take to say this is your fault. but it is not 0 percent.

and about uh test scores. that they are fundamentally that they have good teachers. so in terms of doc hiring practices of the personality test it is illegal and secret. essentially this against the law i don't have a problem

but it has to be legal. one and it should be transparent as well. with the misuse and algorithms. with the housing crisis is still done note any of the rhythm had anything to do with that at all. but yes i knew it wrote the

book so they're not even aware of the auger them were those who get their friends to see there post if you posted they may not see it but that teenage population doublets branding like coke then it will be pushed out which is funny.

>> the luck that the studies for of the great representative studies and to be concerned of the impact it is very clearly part of the economic status. those who are poor are less worried about those other rhythms which if you are talking about and i would not be exploited that much so in this insightful or nothing bad will happen so that makes it even more complicated but of course, it does not solve of

problem. because i like to go around and talk about it that is so impact fall with the literacy that we all need to develop. but it is not enough just to be aware but the reason i wrote the book is time and time again for those that the top are controlling those on the bottom the that is why many rules but that is not enough.

>> i na data scientists. i wirth can health care pavement -- payment and that goes back to where you were talking about and then to use those with hiring practices or to make a decision but this could be in your book but what about the algorithms' themselves? what is the correction for the inequity? . .

i think we can do that. part of it is the realization that we are not done. part of it is developing tools that audit to see whether it is currently fair and part part of it is developing tools to say how do we fix fix it? if it's not fair there how do we make it there? the good news is that -- let's compare company that has discriminatory practices for hiring with an algorithm of discriminatory practices for hiring to the public the company

as you interview people and ask them how they choose to the higher they will lie to you or maybe the action don't realize they are being discriminatory. whereas an algorithm will not lie so you can check that and once you actually have a successful algorithm that's good but you have to have sufficient trust in that algorithm to be able to be doing a good job. >> hello. can you hear me? my partner and i did work on bernie's campaign earlier this year and we experienced a lot of exasperation and frustration at the fact that bernie's campaign and i'm sure the other democratic campaigns used data from a company called van and what we were finding is that the data was sending in terms of

canvassing to a lot of neighborhoods that in our community did not match up with the demographics we need to be bernie supporter so on are and it felt frustrating that david didn't seem to be working for us and our higher-ups didn't see that just because there was data didn't mean that was the best demographic target. beyond that on a broader scale i'm a bit concerned about the fact that it seemed to be contacting a certain type of voter and not another type of voter. we seem to be going to a lot of wealthy neighborhoods and not a lot of less affluent neighborhoods so certain peoples are being reminded to vote where others aren't. first i wanted to ask if you've looked at that and if you haven't we ask you to look at van and in terms of if there's a place where i think there's an algorithm the existence of it is probably actually not the best thing for society. i don't think there should be a

private company that owns the data on voters and lent it out to campaigns. what can we do about that? where's the power struggle here behind back? >> a great question. i wrote a chapter on focus and and data and how data increases inequality and threatens democracy. this is the second part is democracy. i think just having be exacerbated quality, some people have such confusing lives that they don't have time to be civically engage but putting that aside the threats that i see from the data is the microtargeting staff and the way it is uneven. so i don't have time to go into it a lot but i think you are at sleep right that targeting only special people and the special

people typically are either people that can be counted on for donating a lot of money so they have ticker voices or people that live in swing states and are swing voters so a small sliver overall as you know. it also exacerbates these feedback loops where they will spend money on people that they think will definitely vote or who definitely they think will not vote and spend money on people they believe will vote but having said that if somebody who is supposed to vote doesn't vote that gets corrected. they will spend money on that person the next time around but somebody who doesn't vote, doesn't vote they are continually ignored. so that means the various socioeconomic leaving in terms of voting but the larger issue is this. the larger issue in politics as this.

what is efficient for campaigns is inefficient for democracy. what is it efficient for campaigns is to know everybody here, profile each perfectly and send you a the message of the campaign wants you to hear. that's not what is good for us as a group. what is good for us as a group is a broad public discussion of how the candidates stand on the various issues but that's not what's happening with political targeting. if rand paul targeted me he would find the things that i agree with him on which is breaking up the big banks and if i went to his webpage to see what he thinks about other stuff he would follow me with a cookie and say she is the one who wants to break up big banks, let's show her that. they are controlling the information i have about that. that's not democratic. we want information about candidates to be utterly open

and we want to to have more permission about that than they have about us and that is not what is happening. >> i'm going to play the let me freak you out more roll. understanding of people are going to vote for something we are pretty good at. in the last election we had like 97% accuracy but here's the scary part. if you remember in the last election they had a little thing that some of you probably saw that said hey here are all your friends who have voted or 20% and eight did a study on the impact that has. it's something like they could get half a% or people to vote if they showed that than if they didn't. so they want to get donald trump elected and they sign of the people they think will vote for donald trump but they don't show the people who will vote for

somebody else. and suddenly they are having a really significant impact on the election itself. i don't think they have any aspirations and that seems relatively apolitical in these sorts of things but i'm not calling facebook out specifically but the fact the companies that control what you see with those algorithms have the option to have real impact on elections by understanding who gets to see stuff that's going to make you go and vote in hiding that from people who don't. we haven't seen evidence of that happening yet that that's a real possibility with the technology that we have today. i think one of the most concerning things is that political space. >> the only reason i'm skeptical is because the population of people on facebook is already not bipartisan. there are more democratic voters and republican voters so they showed everyone that would be

fair. everyone would have more impact on the democratic selection. >> we have time for one more question. >> i looked for an on line advertiser that specializes on political web sites and something that's happened over the course of the last year is that a lot of major advertisers have gotten skittish about running ads on right-wing web sites. this is half noble intentions and half being able to say publicly things like don't worry we don't sell t-shirts to people who. right part for example but what is happening is whenever we pull out of a market we always look to see who comes and replaces them. in this particular case those

who're replacing us are on line advertisers who sell videos about how hillary clinton has a video that if you see, if any american seas will guarantee she is not elected and part of the really intense ideological segregation happening is because mainstream advertisers are going out on web sites to defeat that move. for them, even their ads are getting really biased. my opinion it was better when google was selling cars as opposed to selling really big knives. this is something that is done through algorithms. their hundreds of thousands that you don't.

i was wondering if you could comment on that? >> i mean i think it's larger than my book. and i don't know how to address it. it's a growing partisan divide in our country. it's not i don't think singularly due to algorithms and i agree that the way we live and silent spaces on facebook and talk to each other and that chambers is part of it but also not the only part. but i agree it's a problem. we will agree on that. >> thank you so much paper coming out and thank everyone else for coming out. i think it was a great talk and a great conversation. like i said we are going to have the book available for purchase in the bookstore and café will be signing afterwards. >> thank you guys very much.

[applause] [inaudible conversations] [inaudible conversations] there was a private visit. she would only go if no one knew about it.

she didn't want any press there and she went for the unveiling of the official portraits, her portrait and her husband's portrait and she brought along 13-year-old caroline and 10-year-old john kennedy junior and they had a very awkward dinner with the nixons were john kennedy junior spilled his milk. apparently he lighten the mood considerably because it was very difficult for her to go back. there were a lot of happy member member -- memories but it was also very sad for her. they're wonderful letters at of the kennedy library that are just incredible that john kennedy junior wrote to pat nixon and president nixon saying i can never thank you more for showing us the white house. i really liked everything about it and he talks about sitting on his lincoln bedroom bed where his father had slept in making a wish that he would be well in school. it was very sweet so it's kind of a touching thing that they went, that she went back, jackie

went back when pat nixon invited her. these women were rivals in 1960 during the 1960 campaign and pat nixon wanted a recount she was so upset at the results. so this was not an easy relationship. the most poignant letter though came from jackie herself written in her signature handwriting on her sky blue stationery. can you imagine the gift she gave us to return to the white house privately with my little ones while they were still young enough to rediscover their childhood she wrote. the day i always dreaded turned out to be one of the most precious ones i've spent with my children. may god bless you all. i think that really shows the humanity among these women, that they do go through so much and they do understand intimately what it's like to live in the white house and to go through this sort of security concerns and a painful loss that jackie went through and something that pat nixon could understand and

could sympathize with. during the 1976 presidential campaign, rosalynn carter was on her way to pay respects to labor johnson whose husband who had been the most recent democratic president or the day before the meeting the embarrassing "playboy" interview was published where he talked about adultery disharmony times and something you said in that very embarrassing interview was where he talks about nixon and johnson both lied and cheated and distorted the truth and really mentioning johnson and nixon in the same breath at that time so close to watergate was an anathema to democrats and very upsetting. here's rosalynn carter in this awkward position of paying her respects to lady bird johnson surely after the article ran. and roslyn turned to an aide who was very close to the johnson and said, what does mrs. johnson think about the interview in what should i say about it?

she said you don't say anything mrs. carter. you are a southern lady just like mrs. johnson. just be yourself, and it was. these women, there's a code. lady bird johnson knew better than anyone what it is like to deal with the husband who was offended and upset people at times. >> anita folsom what what do you do it hilltop college? >> i direct the "cam for college professors. we worked with economics and history and political -- close to what is your that? >> guest: our goal is to give faculty members and college professors at other campuses more information about free markets and current events and give them material for their classrooms. as as for how long have you been here? >> guest: 10 years. >> host: how