every component of burden may constitutionally be placed on the defendant. the burden with respect to number one is the burden of proving significantly sub average intellectual functioning of which the true score is a probable piece of evidence. >> why can't the state -- you told me the state can establish the cutoff if you told me the state can i find the burden to the defendant. in the case of someone that scores of 75 is it not the case that there is roughly no more than a 2.5% chance of that persons real iq is 70 so how does that square away any word enough proof that might be in the standard roof that might be assigned on that point, that's what i don't understand about the argument.

.. which is the constitutional inquiry is just a probabilistic inquiry into a person's true i.q. score, but true iq scores themselves are a statistical concept. it is the score the you would get on a hypothetical test that has no measurement error. then this is the point, true iq is not the same as intellectual

function. iq tests themselves, however perfect and maybe don't perfectly capture verses the selection of function. >> antigen that argument, but that does not seem to be consistent with your point that a state may establish a kharkov. the person does not qualify. >> this would not be a standard that would endorse. i believe that in light of the consensus tests that of professional organizations apply that was recognized a score that is above the standard error of measurement of two standard deviations above the mean would be okay. but the point, the converse point is not true which is we know for fact that many, many

people who obtain test scores of 71-75 in fact have mental retardation. if i just may point out that in this case there were six experts who fully examined mr. hall or supervised a full examination of mr. all. they were cognizant of the iq test scores that he had received. and each one of them opined without hesitation that he had a mental retardation, functional mental retardation. >> the finding that he did not show behavior. that was so, all of those experts that you referred to were speaking retrospectively. there was no evidence of what

the defendant current condition is. >> that is correct, justice ginsburg. the state trial board ruled that it would not accept evidence as to promise to end three, but it did allow mr. hall's lawyers to make a proper pursuant to the state's agreement that there could be a proffers in some expeditious manner. and that is a joint of one 5/8. one of the two grounds that we appeal to the florida supreme court on an addition to the heart kovrov and 70 was the fact that in fact an expeditious proffered did not in fact permit us to put on all of our evidence about prior to entering. and the florida supreme court -- and this is page 125 said we

don't need to consider that question because we upholder rule. >> the retrospective. the simple question, how does the defendant adapt and present. in fact saying at did not test to that. i don't know why i did not do it . >> you are correct. part of the expeditious proffer was limited to the testimony of one of the experts who examined him and did the adaptive testing function. and that expert did say that he did not test in prison. now, there is, again, a

universal professional consensus that adaptive functioning is tested by an adaptive functioning in the real world, not adaptive functioning that occurs after 35 years on death row. in fact, we also note to a clinical certainty that because mental retardation is a condition that is both developmental and not transients , that is there has to be and demonstrated on said during a developmental time but one does not emerge from the condition of mental retardation of like, for example, mental illness. >> talk about the condition of mental disability that is involved here. come back to something you say in response. the question was along the line, what does it mean to have disorder under the dsm? obviously one thing it means is that the scholars can talk about

it, they can all focus on the same subject. does it have any meeting other than that? it is an objective index, an objective characterization that certain people have certain mental conditions? is that what it means? >> that is exactly what it means, justice kennedy. as this court recognizes it is a clinical condition unlike, for example, insanity or competence. >> is there any evidence that society in general gives substantial deference to the psychiatric profession in this respect jack are there any studies on that? is there anyone to look to to see where that is true or not true? >> i actually not aware of anything that suggests that what the society is not like to professional evaluations to do this.

in fact, if one looks only at floor this system, florida uses mental retardation as a determinant for things other than the death penalty. they use is the existence of a condition for educational remediation, vocational rehabilitation and everything. and in those instances, as we point out in our brief, the florida does apply the standard bearer of measurement. >> later in the week we have arguments about economic theories. and it's a little different because in that case the court -- is the court's own jurisprudence, and we have not said that it is up to the states , do you think we defer to a psychiatric -- to psychologists, psychiatrists and more or less? >> i think it has to be much, much more. as this court has pointed out this is a clinical condition, is a condition that can only be

appropriately diagnosed by professionals. >> counsel, this is the same organization that once said that homosexuality was a mental disability and now says it is perfectly normal. they change their minds. they change their minds. 7475 as the new test. >> that is not true. the standard -- two things that are not in dispute in this case, we are only here talking about pa one which is significantly some beverage intellectual functioning and nothing else. and everyone agrees to all of the states agree, they all agreed at the time at his was decided that the clinical condition is defined by three elements and that the first element, significantly sub average intellectual functioning

is defined as a person whose intellectual function is to war more standard deviations below the mean intellectual functioning of contemporary society. >> can i take you back to a question that the chief justice ask? the chief justice said where is this as the incoming from? and it is the test makers determination that this is the margin of error that gives you an 95 percent confidence. i guess the question here or one question here is why do we have a 95 percent -- why do we need an 95 percent confidence level? and you can say it either way. you can say, gosh, we are putting somebody to death. we should have a 100 percent conference level. or you could say, as i ticket justice alito was making a point, the burden of proof is on the defendant here. so 95 percent confidence level

seems awfully high. we should ration it down to 80%. so why for this purpose to we have to go with the test makers determination that five is what gives you an 95 percent confidence level? >> so that the fact that to espn's gives you an 95 percent confidence level is just a statistical fact. take your question to be why does -- you know, why do clinicians and professional associations use that? >> that is not really my question. they might use it for a wide variety of purposes. the question is, what is their determination that it is useful for a wide variety of purposes have a 95 percent conference level? why is that stay stuck with that for this purpose? >> because the whole -- this goes to the reason that the use it. the reason that the use it is because of the enhanced in decision in testing in general, but in particular testing for the presence of something like

relative intellectual functioning. there are so many -- it is so common for people, for a variety of reasons obtain as 71 or 72, in fact they have mental retardation. because evidence of intellectual @booktv evaluation of intellectual function involves clinically much more than a tennis court. like what happened in this case. all of the iq tests that were administered to all of the wexler tests were company because they fell within the standard error of measurement. they were accompanied by the administration, a folder intelligence testing for confirmatory -- >> i go back to justice sotomayor is question. start mucking around with 95 percent. it is all over the law. 95 percent is a classical measure by scientists so when they have confidence that the fact that the regression analysis seems to establish.

it is in tort law, jerry chiles discriminating because they have black people on the jury. it is all over the law. i assume that we do not ask for this to muck around with that number is i don't know what the consequences would be. reduce it. the same tests six times. now we reduced it from 5%. above 70 all the time. one one-hundredth of 1% to read is that what you want to have happened? >> let me just ask you. >> you are not right in some of the things he said. the last thing you said is not right. >> the last thing. >> the last thing is important. >> a nice thing is that important. my thinking to the last thing is dependent on my being renovated before. mri before? >> you are right that 95 percent is just generally speaking a feature that is widely adopted

as a confidence level and particularly important here because the constitutional guarantee announced in adkins is against the execution of persons with mental retardation. >> to the last point before your time expires because i do think this is important. is there another way of proving liability? multiple tests, suppose someone is giving 25 tests and 24 times the person's coor's 76. one time the person score 72. what would you do? how would you deal with that in a state that as our car cut off? >> this is the last point that i wanted to get to. i think before my time runs out i just want to point you to page 10-foot of three of our reply brief which sites the ice for an book of clinical diagnosis. we have given you the pages. on those pages explains why we have a situation of somebody takes more than one test the

appropriate determinant is very much not the average but what is called the composite score. and the composite score is different and, in fact, the people below the mean, below the average because you have to take into account the fact that regression toward the mean and the fact that a person who takes two, three, or four tests, multiple test changes the bell curve on standard deviation. set the example given in the oxford handbook is very similar to this case. for tests. they average 72. the composite score -- and they're is a statistical explanation for how it is arrived at, the composite score is 69. the standard bearer of measurement is actually larger using a composite score. so that is why have -- as to justice briars last point simply averaging obtain scores does not, in fact, give you a better

handle. there are so few people who score significantly below the mean on multiple tests, what clinician's use and is a statistical analysis that takes into account the different calculation of what has standard deviation below the mean this. >> that is not consistent with my understanding. i don't claim i have a deep understanding of it. what will be your answer to my hypothetical? multiple stores above bark of but one that is below. i'll ask the state the opposite question. >> well, we know what florida does. >> what is the eighth amendment requiring your view? >> in our view the eighth amendment requires that the state chooses to use iq tests course as a proxy for intellectual functioning rather than a full inquiry into intellectual functioning it cannot refuse to employ this

standard error of measurement that is inherent in the test. >> seventy-six, you would not need to go on to the adoptive behavior. >> our view is that a state consistent with atkins could say that if you have no of chain store on of valid, properly administered up to date test that is 70 -- that is below 76 you may constitutionally be precluded. i think many clinicians would go ahead and do adaptive functioning and other intellectual testing, but our view is that states like mississippi in oklahoma that set 76 as the cut off to, in fact, complied. massive the balance of my time? >> thank you, counsel. >> mr. chief justice, and may it

please the court this course should affirm the decision of the florida supreme court because it represents a reasonable legislative judgment and one that is fully consistent with afghans in the eighth amendment. i would like to start by responding to your questions about what you do with multiples course. in fact in this case where are not talking about someone who has one or two iq scores. when you look at the wechsler test which is what the petitioners contend it is the gold standard into scores of 71, 72, 73, 74, and 80. as we understand what the petitioner had discord do was to take some of the lower scores and simply subtract five points from a. >> that is not consistent with the material that we cited in a footnote in this brief. if you look at the example there they did apply some statistical principles teresa's course. be not simply take the low score and subtract five points. the logic of that i would submit is obvious. you cannot have a situation, in this case you have a low iq of

one and a high iq of 80 and say at the same time that there is a 95 percent chance that the score is between 75 and 85 and also an 95 percent chance that is gore is between 66 and 76. >> this case, establish the principle, the very significant principal that would have a criminal defendant condemned to death for murder whose scores are 71, 72, 73, 74, and 80. that is okay. that is all you're trying to persuade us on? happy having to go through this in all future cases where you have somebody who has 69737475 in 81. i mean, don't you have some more general principles other than the particular scores in this case?

>> well, the particular stores in this case, we have a principle. when you're dealing with things like the medical field generally , a reason for this court to do as it has historically, to defer to reasonable legislative government. >> let me ask. suppose that the american psychiatric association and all of the professional associations to use as cn, suppose that. it seems to me with the state is saying here in declining to use that is that it declines to follow the standard for other set by the people to design and administer and interpret the tests. >> well, i have two responses to that. the constitutional -- if there were constitutional rule that the eighth amendment requires the adoption of all kinds of clinical criteria that the epa

-- >> this is not clinical but statistical criteria. >> there are two. >> the sc and is not a clinical judgment. it's a standard error of measurement. >> this the right way to look get there test. >> the test measures, the air measurement. they're suggesting how many deviations. >> they're not challenging the two standard deviations. if you're going to preclude functioning abilities and the other two factors of your test based on a score of a test that

says it has a sem of five then you have to use the sem. it is very different. we are not saying you have to take that number and declare that person mentally and intellectually challenged. you just have to apply and the other factors. >> well, it's a three-pronged test. you have to demonstrate with the assistance of all three prongs. with respect to the 95 percent antril. >> that's -- you don't have to go to the second and third. adaptive behavior doesn't come in the picture. in fact if the iq is above 70.

>> that's correct, your honor. it's a three part test. the medical community does not dispute that. to achieve a diagnosis of mental retardation you have to demonstrate that you need each of the -- >> what is wrong? there may be agreements on this. what atkins says is there are three parts, as you said. one part is significantly sub average intellectual functioning does the first part. and so what you say this if it's above 70 on an iq test, we don't go further. i want to tell the jury of the judge something. judge to have an expert here. thank you. expert. >> i want to tell you, your honor, that number 70 is subject to air. the state can do the same thing

if it 68. the number 68 is subject to air. you could bring in a witness and he was a 5% of the time it's within five points either way. now, there could be other ways of going about it. maybe you do the same tests six times with different questions. that may not eliminate but it might reduce the possibility of error, or maybe some other way to do it is recalling the psychiatrist. are an expert. seventy-two is still -- we have other ways. we have other ways. not just tests. i think you would do the same thing if you wanted to on the downside gas. that might mean people wouldn't be executed. and that is their position, though, i think. they get to do it on the upside. what is wrong with that? does not sound so terrible. anyway, the eighth amendment -- this is a way of enforcing the

eighth amendment. this is not need to be independent eighth amendment violation, but that is the kind of question i would love to have >> what is wrong with that substantially if you raise the limit to 75 you could. >> that didn't raise the limit. what it does is it says just what i said, and i don't want to repeated. once they are at 70 they call their expert to enforce the decision maker just what i said. that would take a little time. maybe 15 minutes, maybe longer. that is what they want to do. what is so terrible about doing it? >> you would end up increasing the proportion of people of a number of people who would be eligible for mental retardation. >> but only those who, in fact, are mentally retarded. >> they are not mentally -- there is no disagreement that 70 is the appropriate threshold.

this is almost an evidenciary matter. what does it take, standard of proof that they have as a matter of law and the standard of proof they do not challenge in this case. and they recognize the best measure of truth iq is your obtained an iq test score. >> the ultimate determination here is whether somebody is mentally retarded. the iq test is just a part of that. as a part of one prong of that ultimate determination. what you're cut off does is essentially says the inquiry has to stop there. the question is how with that at all consistent with anything we ever say when it comes to the death penalty? we have this line of cases this has been the comes to meeting at the death penalty we actually do individualize consideration and allow people to make their best case about why they are not eligible.

the essentially what you're cut of does is stopped dead in its tracks. a person who mayor may not have a true iq of over 70. >> well, with respect to mediation this is completely separate. there is individualize decision making with respect to whether. the still as an opportunity. >> but he does. and with respect to this critical question. you cannot execute somebody who is mentally retarded. he said now you are preventing me from showing you you are mentally retarded because you have an iq test, a part of one prong of a three-pronged test, an iq test says i am honestly retarded.

that iq test may be wrong. >> with respect to the i.q. test being one part of the intellectual function, that is a very recent development. one of the problems we have is that it's changing. if you look at the form which was in existence, they said the intellectual function is defined by iq. >> we don't allow all factors to be considered, do we? with this state have been able to receive future his assertion of mental retardation by pointing to the fact that he is the one who pushed her into a car, drove a car with his accomplice following in another, and to kill there and killed a

policeman, too, later. somebody who is mentally retarded has not to be responsible for subject to the death penalty certainly could not have pulled all of this off. significantly mentally retarded. >> the adaptive functioning portion of the test, a three-pronged test, the intellectual function until very recently, adaptive auction and tells us about how people react in the ordinary world to difficult situations. some of what you talked about may or may not be relevant. but responding to the earlier question, it is not that florida is not allowing evidence of florida is making a finding they cannot. that is why you don't. >> it seems to me to follow from

justice pagan's question and it is a very important question, we have been talking here about the inaccuracy, to some extent, of iq scores. in your rule prevents us from getting a better understanding of weather that i.q. score is accurate and not because we cannot even reach the adaptive functioning problem ido prevent it if it is under 70. >> you would have to satisfy all three. >> the reading of the iq, we submit that is not the case. that is why these are discreet inquiries. via multiple test scores -- >> very close cases, doesn't

eliminate whether or not the i.q. test is exactly as reported or to some -- increase to a decrease depending on what the evidence of adaptive function shows. >> that will be the position of the modern dsm. ague's define the intellectual function. >> i will read it again. it refers to the adaptive function. >> you made a mistake. there is an adaptive function inquiry. that is one of the three-pronged spear greedy have to prove adaptive function. >> that is dsm for. >> that has been a part for decades. what is changing is the way the medical community looks at the iq or what to do with i keel. >> at the very least you give somebody an iq tests and is close to 71. he might actually have an iq of

71. he might have an iq of 69. you would let him go and show that, you know, he cannot function in society in the ways that actions seemed to care about. this accused phone-number might be accurate and might not be. >> it's a critical component. no matter what your deficits are in adaptive function you do not qualify for mental retardation diagnoses without also showing substantial deficits and intellectual functioning. >> i know that this has emphasis on the i.q. test. but when the i.q. test was used that they always use it as a fixed number or include the sem as informing the clinical

judgment? >> the sem has been part of the equation. >> it has not changed. >> but in all of medical diagnosis. >> i think that the application has been a component for some time. we do not dispute that. we do know that the emphasis on iq is decreasing and that the medical community is now suggesting that you should rely less and less on iq. >> you are not arguing for that. they're just arguing that we should say where it is always give which is using the sem. >> what they are arguing is that you should do this, apply the sem that same way that clinicians' do because that is the way conditions do it. if you go down that road is difficult to understand and the principal way where there was stop. >> a case that those who use iq tests always require an 95 percent, for as little? and always must require an 95 percent confidence level. let's suppose a school on the

other end of the scale wants to identify gifted children. they said child is gifted of the show as 9q wire repair your above. so they say if you have been obtained score of 130 your and. even though there is the same percentage that would be the case with respect to someone with an iq of 70. the person is below 130. would there be something wrong if they're doing that? either places that do that? >> certainly. that is up to the decision makers who is relying on the iq for whatever purpose. published, part of the test. the decision maker, they can set that as high-rise low as they want to. that is where we are dealing with here. it's a fundamental difference. we have an adversarial process

with respect to contested cases, a burden of proof, is not shared in clinical settings. there are a lot of reasons why it is difficult to make the diagnosis and a clinical setting, particularly now when the medical community is providing services and making services available to the people where you don't have the same. >> could this state change its statute to say we are now using a threshold of 60? >> well, this day certainly has. it would be more difficult. what you want to do is go back and looked at what is a part of that. acting before making a decision he would want to let the whole picture question about that the 70 was longstanding.

everyone has agreed it is 70 for many decades. maybe forever. if -- why would this day say no to that? >> again, the special interest. and the fact that the state may need to be more descriptive. the incentive that inmates would have to score lower than they would. you would not have been a clinical setting. >> that is the two prongs. >> i'm sorry. >> that's why you have the other two prongs. when you have a fixed cut out you have the ability to use the other two prongs. you're stopping them on a test based on tests but that has a margin of error.

>> we are not stopping them. all we are stopping is the consideration of the other prongs. if someone came in and it were undisputed that he could not satisfy the adaptive functioning brought in you would not necessarily have to. >> did i misunderstand the case? in effect my words that the iq or the threshold in order to make defense, you have over 70. please correct me if i'm wrong. >> that's correct. what happened in this case, there was a motion by the state recognizing that the iq scores or all above 70. and so the evidenciary motion, if you had a case where you had a proven ratio.

>> if you do not satisfy prime-1 you do not get to price two or three. >> by the same token if you don't satisfy prong to be gutted to prong three and so one. simply a recognition. you have to satisfy all three prongs which is a factor. >> right now today under the law of florida, a similar case with an ice cube @booktv i.q. score 71. and the prosecutor points out to the judge. the defense lawyer says, your honor, i would like to bring in my text desperate year who will explain to you this test data shows 71. there is some fairly small but significant probability of error and it could in fact be as high as 76. you would like to explain that is the situation. therefore can i have him testify. does the judge have to let him testify not?

>> it's fine and is an hypothetical, the test or 71. without an obtained to score 70 or below he would not. >> this is a dispute in the case. it would like to present that expert and you would say no. >> that's right. >> i just like to be sure. my first question which i won't repeat. is that man has been on death row for over 35 years. >> 1978. mental retardation until ten years after his first conviction >> that's right, your honor. he went back and at some of the same evidence. >> 1978 is when the killed this woman. >> there have been a number of appeals in this case. yes. >> the last ten people, florida has executed, an average of 24 years on death row.

do you think that that is consistent with the purposes of the death penalty? is it consistent. >> i think it is consistent with the constitution. >> that wasn't my question. >> i'm sorry. i apologize. >> is it consistent with the purposes of the death penalty? is it consistent with the orderly administration of justice? >> it is consistent -- >> go ahead. >> it is consistent with the purposes of the death penalty. >> maybe you should ask us. >> most of the delay has been because of rules. >> of course most of the late. in this case it was five years before there was a hearing. as the attorney general florida suggested to your legislature

new measures are provisions our statutes. >> there was a statute enacted, number of issues presently being challenged. >> let me ask why you have this policy. >> i'm sorry? >> why you have the policy, administrative convenience? >> the people of florida have decided that the death penalty is an appropriate punishment for the most terrific crimes. >> why you have this 70 threshold. >> that is what i was getting at insuring that the people who evade execution because of mental retardation are people who are, in fact, mentally retarded it would double the number of people who are eligible. that is inconsistent with

florida's purpose. >> that is just to say that it would double the number of people eligible. some of them may have been mentally retarded. decisionmaking with rias -- >> not mentally retarded if they don't have an iq of 70 or below. >> not mentally retarded if they don't have an i.q. score of 700 below? you don't believe that yourself. this is a tool to decide whether someone is mentally retarded, a tool that functions and one prong of a three-pronged test. >> the iq threshold is the first prong. no matter what your adaptive references are you might demonstrate and again he must demonstrate peppermint together the have an iq of 70 or below. we believe @booktv is a 95 percent chance that my iq is somewhere between 68 and 78 you have not satisfied the first

prong. what like to talk about that. it is not the case which you have a 95 percent chance. it's a very small chance. you have a 95 percent chance that your true iq is within five points of the measurement. but it is not that you would have equal chance of having 67, 68. and so over and over again use gore near the bill curve most of the time. at the end of that 95 percent threshold is a very small likelihood. and with each additional testy take the odds would go down. so it is simply not the case. as 72. satisfy the first prong because it's a statistical matter. a factual matter.

admitted into evidence. seventy-one, 72, 73, 74. in so you would have to say what are the odds that that group of testing, the dry q. is it possible? possible it's over a hundred. it could exceed beyond the 95 percent conference level. nobody disputes that that's true iq is incapable of being measured. the act -- iq test is what the community has and is the most objective of the three prongs which is why is particularly important to focus on because it is the most objective test that we have. >> how many states retain that practice with the measure of 70? >> your honor, by our count there are a states that have both the kharkov and a 70 or

standard deviation which approximates to the same debt has been recognized by the state . there are a number of other states that have statutes in similar quotas that have not been interpreted one where the other. >> how many actually have that fixed cut off and how many have a sem? i thought it was only four that did the test. >> your honor, all have a fixed cut off our 70. >> by judicial decision they considered -- well -- >> in most instances they have done what florida has done which has a statute is interpreted by the courts. >> that is what i am saying. only four without the sem. >> i apologize. >> only four have interpreted without using the sem. >> interpreted their statutes. >> only for by florida. >> we have alabama, florida,

kansas, kentucky, north carolina , virginia, maryland woods as repeal the death penalty. that was their standards. we would ask respectfully of the court affirmed them. >> thank you. you have a minute remaining. >> in state versus jerry which is the florida supreme court decision that established this rule that if you're low score or your only score was 71 you're out. that applies to the take one or multiple. from the supreme court decision, it is a universally accepted given that the sem is a universally accepted given and as such should logically be considered in determining whether a defendant has mental retardation. what the court said was we have to read the plain meaning of the florida statute. the florida statute says two standard deviations. the notion that the florida

legislature -- may i finish my sentence? the florida legislature, the people of florida have made a concerted decision not to account for the sem, baseless and beeline by the legislative report that accompanies the statute which said 70 is 75. >> earlier this morning president obama made a statement about the ukraine. here is what he had to say to reporters in the white house briefing room. >> it morning, everybody. in recent months the citizens of the ukraine have made there voices heard. we have been guided by a fundamental principle, the future of the ukraine and must be decided by the people of ukraine. that means ukraine's sovereignty

and territorial integrity must be respected in international law must be upheld. and so russia's decision to send troops into crimea has drawn global condemnation. from the start it has mobilize the international community in support of the ukraine. reassuring our allies and partners. we saw this international unity again over the weekend when russia stood alone in the un security council of defending its actions in crimea. as i told the president yesterday the referendum was a clear violation of ukrainian constitution's. i recognized by the international community. they are announcing a series of measures that will continue to increase the cost and russia and as is possible for what is happening in the ukraine.

first as authorized by the a executive order signed two weeks ago we are imposing sanctions on specific individuals responsible for undermining the sovereignty territorial sovereignty in government of ukraine making it clear their consequences for their actions. second, i have signed a new executive order that expands the scope of our sanctions. as an initial step bump authorizing sanctions on russian officials, entities operating in the arm sector and individuals who provide material support to senior officials for russian government's. and if russia continues to interfere we stand ready to impose further sanctions. third we will continue our consultations. to them of the head with their own sanctions against russia. tonight there will meet with the leaders of our nato allies,

poland, estonia, not the kind lithuania. i'll be traveling to europe next week. our message will be clear. as nato allies we have a solid commitment to our collective defense and we will uphold this command. fourth, we will continue to make clear to russia that further provocations will achieve nothing but to further isolate russia, diminish in this place in the world. in nassau community they oppose sovereignty and continued russian military intervention will only deepen russia's diplomatic isolation. going forward we can calibrate our response based on whether russia chooses to escalator diaz that the situation. addressing the interest of both russia and ukraine. that includes russia pulling its

forces in crimea back to their bases, supporting the deployment of additional international monetary in ukraine and engaging in dialogue with the ukrainian government which has indicated its openness to pursuing constitutional reform as they move forward toward elections this spring. but throughout this process we will stand firm in our unwavering support for the ukraine. as i told the prime minister last week, the united states to stand with the people of ukraine and they're right to determine their own destiny. we will keep working with congress and international partners to offer ukraine that economics is what this needs to weather this crisis and improve the daily lives of the ukrainian people. we continue to look at the range and whether we can help our ukrainian friends achieve their rights and prosperity and dignity that they deserve. thank you very much.

>> are you surprised that this is taking its course? >> next, valuing privacy and security and concerns about data collection by the government and private sector. an official with california's justice department and two attorneys that represent target are among those speaking. stanford law school hosted this discussion. it's about two hours. >> before we began we want to give you all one reminder. this is being taped for c-span. that explains the light. also as you might be asking questions or if you want to interject we will try to get a microphone to you. we might ask you to repeat. also keep in mind because of that you may want to think twice about what you say. so we are delighted this evening have a great panel.

i practice here in california, the san francisco office. a certified privacy professional . that is enough about myself. joined here on the screen, a beautiful while the new jersey, he is actually a director of the center for information technology policy and among other things has a pretty interesting blood on his website. i will let the professor introduce himself a little bit more properly. to my immediate left is john with the department of justice here in california. helping training. more last. and she knows more about privacy and policy issues than i can possibly ever get into myself. to her left is michele, the

chief privacy officer of mcafee. certified privacy professionals. but michele more of me is the author, one of the authors of this brand new book that came out this week. the privacy engineer manifesto, policy keewatin to value. it's available. and to her left is my partner and colleague. a litigator with our firm. he has been doing an amazing amount of privacy litigation. he is done everything from dating breaches to. [inaudible] to. so i actually learned how to do some of the things i have done [inaudible]

for those of you want to compare our gray hair. anyway, each of you will start out with about five minutes with the discussion to set the stage and to explain their perspective a little bit more introduce themselves. combined ask professor take us away. >> great. thank you. it is a pleasure to be addressing you from here in the cloud. sorry i could not be with you in person. let's give of their brief introduction to myself to give you some context on one level we talking about. i'm a professor at princeton and computer science and end the woodrow wilson school which is our public policies cool. i'm a computer scientist by training. i have done work on privacy of from the technology side and policy side. i served at the federal trade commission in 200011 and 12.

and obviously recent events have drawn a lot more attention to the issues of privacy and a data . we hear about leaks of data from maps and other kinds of technologies. we are hearing a lot about collection and use the data by governments. we are realizing increasingly that we need a break and brought in that burst real data behind us. as we move to the physical world as well. it is no longer the case that the kind of private -- pervasive trailed is only in the digital world. it's now the digital world and the real world. we leave these records behind this.

as the dated trail that we leave consists partly of data that we provide willingly the people, data that you enter in a website or information about a particular affirmation that we provide to someone, and the data that we provide is sometimes available for collection and capture on a device that we on, say dated a you enter into your phone or your computer, information that you sent via e-mail. that information is sometimes available for collection are capture and our own phone or computer. the information then typically traverses across the internet to some kind of server affiliated with the servers to that we are using or that is somehow connected to our activity in the data is then again typically at rest on that server. and potentially its available. and then, of course, and transit between the local devices and

the servers and data centers where data ends up the date is also often available to be captured there because we rarely use it just for testing. so in addition to data that we provide explicitly there is lot of data that is collected about our behavior and activities based on what to do. for a sample almost any time you read something online information is collected about the fact you read that particular piece of content, whenever was. similarly information about where we drive and what we do in the physical world is often available for capture even if we don't explicitly provided. so what does this have to do with the data? why does it matter that the status said and this collection of data is big? well, what is really important about big data for privacy is that data allows inference to be made so that when a piece of data is provided it reveals not only was written on the face of

that data but it also reveals implicitly everything that can be inferred from. and so if we're going to understand the privacy implications of data that we reveal what that is collected many did think carefully about this issue of and france. a problem is that inference is notoriously difficult to model or understand predict. even experts have a lot of trouble understanding what the implications are of releasing that marginal piece of permission might be. generally more data means more inference. and each in france that is made about you is a new fact that somebody knows. that fact can enable more inferences in a chain reaction. one of the reasons why it's tough -- difficult to understand in model is is difficult to think about those sort of chains of inference that we can set off by reviewing one more seemingly not very sensitive fact.

so we have often in the past talked about government collection of information versus corporate collection of permission. one of the things we have learned in recent months by thinking about and learning more about government collections is that government and corporate collection are really connected to more tightly than we thought. ..

to go beyond what i think many of us expect good was going on. and that sharpens the issue for us as well. so with that very brief introduction, let me pass it off to my fellow panelists. thanks. >> good evening. very happy to be here. first of all, i give you a little bit about me. my training is as a medieval -- medievalists. i always expect a connection for medieval studies as privacy turned out to be very byzantine world. i was working in privacy since 2001 i headed up the former office of privacy protection in california, which is a small

state office treated by the legislature originally part of the department of the tumor affairs that was basically a consumer education and advocacy organization, didn't have any regulatory authority. from its first operation, do was part of the beginning of a glorious privacy passing california, which continues to this day even as we speak. california has been claimed in some circles as the leading state and consumer privacy and some circles to describe it differently. we have a lot of privacy statutes. we also have a constitutional right to privacy, which if you haven't looked lately is not just the right to pursue, but also to obtain privacy security and happiness. i hope we are all getting our share. the office of privacy protection

cease to exist, eliminated at the end of 2012 and not exactly coincidental. pamela harris at the time created a new privacy unit to which i moved along with some of my farmers have. it represented a real advancement in privacy protection by government in california and it added to the same kind of work that was done of education and policy advocacy and added accountability and enforcement. so in addition to educational activities, we also enforce privacy laws, both state and federal laws that allow attorney general. it is significant that this level of resources is dedicated

to privacy protection by the general office because most of our privacy statutes in california are pretty much left to the attorney general and district's attorney to enforce. most of them do not have a private right of action. thanks to you and initiative a few years ago allowed individuals only if they demonstrate monetary or property damages, which is often difficult to do in describing privacy harm. so that leaves it to enforcement our current ag is very interested in using this tool and enforcing any of our new and older privacy laws. and looking at bringing about compliance for information

practice principles with good privacy practices, respectful policies, we use more tools than just enforcement. i usually call it three east. so there is encouraging businesses to practice to a higher standard than the go compliance by educating them in working with them on developing fast track status to educate consumers and empower them with not only information about their right, but also strategies that can haunt them protect their privacy even when they don't have rights, which is often the case. enforcement can be very educational. we are looking among other things at practices we wish to raise to people's attention and

point out the wrong way to do things and suggest better way to do things. i am very interested in this big data topic from the regulator and big data -- i guess i'll probably talk about defining it. a lot of data. enough data -- enough different data points about an individual that even if no one of those points is personally identified, the accumulation of them makes the whole stream potentially identifiable. our basic legal structure for providing with control over their information depends on notice and consent and dealing with personal information in restricted ways. when you can't tell -- limited

at the time of collection and aggregation isn't necessarily personally identified, that whole structure doesn't work very well. both from a to those big data and perspective of privacy protection in general, i am interested in regulatory innovation. so many things are changing so fast that affect our personal information, not just things that big data, but new technologies and business practices that the log really cannot keep a. there are things we can do to sort of keep refreshing on henry understanding them. but it's hard to keep out than i am interested in a number of different proposals that are being discussed now for alternatives to command and control laws in rags. before i pass it over to the shameless promoter of her book,

i was very disappointed at a technological and marketing failure on the part of amazon who i understood was going to anticipate what i would want to order an advance and they should have gotten it to be now. i had to order it. it wasn't hovering in a blimp over my house or anything. when i get home i think it might be there. [inaudible] >> a drone, that was it. >> privacy manifesto -- that would be a really good thing. >> good evening, everyone. so many friends in the audience. and the chief privacy officer. anything i say can be a cheerleader to rosie greg. i started out as a young lawyer, which he allowed her and then i started out as an intellectual property lawyer, lucky not big

data. one of my very first cases, the ones i will talk about ways. the rom of cocktail drugs and pharmaceutical ownership over molecules and specifically the molecule network tonight it stands a dozen years ago. it was because of certain molecule at a version of chirality, fingers, times, but they're not identical. it turned out that one was affected and one was not. that was the case. i think that's really interesting when it comes to datasets is instead everybody knows sometimes can result in analytics that create absolute a fundamentally new advances and in that case began a path where we could go after disease and malady was no therapy available at that time.

so here we are again back with data all these years. i wasn't going to shame, but now i will. i wrote this post with microsystem visiting hours are their local director and tom finneran, who spent a data architecture security field and that is enforced and because i think big data is the ultimate in getting together artistry, chemistry, law, policy, consumerization of data in the truest sense and i am getting to a better place and not necessarily armageddon. i will shamelessly reach a couple paragraphs that could change my again in this area from the introduction. the road is certainly flat. everyone said so. the government said so. the church said so. the richest guy in town said so.

everyone except a few explorers, dreamers, scientists, artists and plainspoken folks who look out of sky that looked more like a pole and noticed the ground in the sky always met paris for the observer wonder closer in the media became elusive once more in shadows and tides and other indications suggest there might be something more than the edge of the world. so as it turned out the world was not in fact flat. there is a seemingly endless set of new possibilities to discover. privacy assert money that. everyone said so. rich people with the cia in the 1970s, you know who you are, founders of important hardware companies, he knows who he is. someone whose blog said so. the government can't make up its mind which person should say so, that it might say so. if somebody treated event really

all technologists invented the whole thing said so. everyone said privacy was dead. except a few explorers, inventors and philosophers, children, parents and government regulators who looked at in defeating the endless sea of data and still could see how a person can distinguished from a pile of metadata. this is true for people who wish to decide for themselves the story they wish to tell about themselves and to whom these people see a different horizon. the privacy engineer sees this were privacy security combine to create value as a similarly challenging and exciting time for expiration. this is a book for you. so that's a restart of the book and i will shamelessly plug it tonight, but it really is the culmination of 15 years of thinking a partnership with security architect, written by an artificial intelligence

analytics ceo. we really do think that yes there are more and more identifiable moments happening out there, but i also think we can think organically. we have all the elements for armageddon. we also have all the oman are doing some rain and human surveying. so we are at the world policy statement for stanford. that's all i've got. >> hello, everybody. thanks, jonah michelle. i am mark szpak. i've been in a letter since 1985. i am intrigued we are going to even work on finding -- we call it large data.

the intellectual property connection. part of it for me, years of my country and ago we had a case where we were working on a trade secret case that has to do with who invented the pump sneak for those of you who remember it. but there is an aspect of trade secret law that carries through the context for me. it's a very fundamental rule that in order to deal to assert rights with respect to trade secret, you got to keep it secret. if you don't keep a secret, it's gone and your rates go with it. just like data. once your data is out there, it is gone. trying to reel back rights is i think part of what everybody here is talking about, whether that's even possible. i've been working on data breach in data security cases, like jim

said, ever since 2007, seven years ago. at least for me and a lot of people that sort of started a lot of attention in this area. it is time for the largest cyberattack of its kind and stayed that way until the heartland payment systems came along at 2009 with a good portion to work on that case at the council as well. we advise and represent privacy matters, including the criminal intrusion, at nationwide, where a motion to dismiss the class-action litigation was followed by lack of injury, which i'll talk about later on probably as well. we work for it -- we assisted for my colleague doug miele is

challenging the authority claimed by these again from the pact and by the ftc and the staffers to claim the ability to enforce reasonable stability standards in the hotel franchise context. we now work on issues that target as well. i'm not able to talk about any of the particular client matters in detail, but it is a very interesting time to be here and be able to talk in general about the thing better in the news and the last few years. so my kids, my general perspective is that i do look at big data issues from the perspective of someone in the trenches as a litigator dealing with the cyberattacks against company data, bodies of company data, aggregation company data. sometimes even as the attack is

still potentially underway, we get called in. right at the mall that accompanies are faced with sudden challenge of dealing with the reality of finding out what is happening, finding out what exact data and issues working with law enforcement and dealing with all of the range of things that have been immediately, regulatory and korea, class-action complaint, counterparty claims and obligations fall around you all at once as a company that's been attacked like a ton of bricks with extreme pressure to act quickly and also to do it right by all of those various other actors, very, very challenging that we've gone through a number of times. so my economist on this is in

the litigation context, where i live, the analysis all start and end with injury. if any information about an individual is taken or exposed to law, what is the injury? and the recent decision involving a challenge to the size of provision, involved in surveillance, but the supreme court issue in much of this year. supreme court makes clear the federal court we are not talking about a risk of injury. we are not talking about the possibility of misuse. not even an objectively reasonable expectation. instead, to get into federal court under article iii of the constitution coming at actual injury but just to satisfy that

standard, you still need to have enough of an impact, a nascar mass of the injury to make the element which are trying to assert whether it's negligent or the level of injury that might be necessary under your statutes. otherwise, if you don't have those two levels of injury, they don't prevent where the courts irving was one socially engineer the situation for you by porting a remedy. that is a structure that companies find familiar. that's what litigation is. companies know how to deal with that regime.

the facts are revolved around whether it's real injury. companies can work back here wanted that may not reason this space are regulated and you can see with the attorneys general and they are usually operating under statute for those deceptive act acts or practices. -- [inaudible] they are starting to appear and talk about privacy data security issues with different language. that is something we can do as well. and that contacts, the injury often seems in a deception case

you talk about materiality that is injured. he talk about whether it is a material one. and unfairness case, many of the cases you talk about doing t. evolve on atomic last. you sometimes hear concerned not about what has happened, but about what might happen. you know, as professor feldman said when you're dealing with data inferences from people buy things that are difficult to model and predict. but that is what you year people expressing concern about. you sure do of words like inconvenient stress.

creepiness, worry. all of those being discussed for inquiry impossible action. there are almost no judicial decisions in the regulatory context validating or addressing the difference in approach as it's applied by regulators. at teen under statute. she don't have a body of case law where courts are weighed in and validated for adjudicated adjudication. not in the same way you do and court obviously. at least in the data security context. i think that lack of cash that lack of judicial authority, sort

of lucky not to difference in approach pc motivated, sometimes implemented in the radio tories fear, part of the difficulty big data faces with liability and predictability and those values analytics software that are growing hand over fist, very quickly. what can we do with this data and how responsible will i be if a breach occurs or other kind of privacy challenges arise. it's less clear the regulatory area than it is i think in the traditional -- [inaudible] so that's my discussion. >> thank you all. perhaps maybe we should pick up from not feed. you touched on the damage piece

of good, thinking about it. you might not be starting from the beginning. one of the things that seems to drive a lot of our thinking is what is the harm of data collection, data analysis and putting it to use in some way. you did mention the inferences is almost a sure thing that occurred that is the maybe or maybe not as 20% or 30% of phone traffic. if you would comment a little bit on your blog recently. the society at this point think there is no harm in this branding for joan, your perspective and a break in later perspective, what are the potential harms you are concerned about is we are regulated not? and leave it to you to take that from the first perspective.

>> sure. let me talk some about this from a technology standpoint. as i said before, it can be difficult to understand, to figure out how data might be used and it's difficult to tell if data being used in ways that are out verse two ranchers this information but a lot of particular persons life is published in a way they don't want published or the somehow. they may not ever know about the job offers to calm, about the invitation to something that does that, or whatever it is. so the result is people tend to think about this stuff in terms of risk. risk is really the rational way to think about a situation in which causality is believed to exist but it's difficult to tell

the exact mechanism was the exact way in which a negative consequence can manifest itself. so that's the rate people tend to think about this. to think in terms of a body of data being collected, being maintained, being used as bringing on risk. it's almost like an environmental risk. in some ways he think of data being how does the mike oil in an underground oil tank. there's a lot of things you can do to inspect the tank to make sure it's safe, to replace periodically and so on so you minimize the risk and oil doesn't end up seeping into the ground. if you want to minimize the risk of oil leaking into the ground, the best ways to not have an oil tank in the first place. now that doesn't mean you shouldn't have oil tanks either. it just means you need to think in terms of risk and exposure

and that is the way i think many people think about these issues and i think it makes a lot of sense. >> i actually would step back from the regulatory position to policy philosophy is. so when we talk about protecting from harm, what are we trying to protect? i would suggest that it's not merely a matter of protect individuals from harm to them by the use or use they didn't want to their information, but there's also a societal dimension to privacy that respects for individual privacy or lack of it has an impact on society as a whole that for a healthy, functioning society we need to be autonomous individuals with a sort amount

of respect for her dignity and our atomic said that we can operate freely. we can delivery. we can be innovative and speculative. it's essential to scientific method does the society as a whole from the massive of information on individuals to pervasive surveillance images that effect how we act and how we live and how we develop as individuals and how we operate as a society so i think we could sort of fun ourselves down the wrong path away if we define what we are protect dean is simply protect even individual from harm.

so that is one thing to bear in mind that their societal values and benefits to respecting privacy. look at what the harm to individuals? and that's speaking as a regulator because i don't think i was as we've been hearing necessarily have come to receive the unrecognized privacy harms that meant that many of us would perceive. is it or interesting study by george washington university in which he has categories unharmed that includes things like surveillance and collection, like disclosure -- the distinction between information that is public about me such as stuff published in phone books are public records that is left

in a record or book someplace compared to that same observation, which was 30 public been posted on the world wide web for now it is exposed to more people. it is surgical, to a degree of exposure and i can't tell you how many times we get calls, constantly calls, letters. regarded in the old office and people who are gracious cover information on the website or information -- not like medical information, but their name, address, other staff scraped from not just public records, but social networking sites. they are sure there is a law that says that can't be there without their permission put there by somebody they have no relationship with exposed to all the world. so i don't have the legal answers to how we can start to expand our definition of harm, but i think we should both with

the impact of individuals and on society. >> so just to kind of follow-up on that, do we have once already better prepared and goes directly to some of these data collection issues that pop up when he think about these entries and data collection for various purposes. >> identity theft. we do have laws that speak to some very specific types of harm, but not for the others that have a real impact on our lives and our society. >> i think that is important point. a couple points. one is we are talking right now. this is only the u.s. view. if you are a practitioner, if you've got.com you are walking onto the global stage and it is very, very different.

so harm outside of the u.s. borders is based on your freedom and it's much, much broader and tried to to find an individual case for standing. so that must just always be kept in mind particularly in the context of the data because we aren't yet tagging it with a profit onto geography. if we did, we'd be accused of overly monitoring. so that was one point i wanted to make. the other is thinking about public databases. to give a personal example, there's all these real estate sites, public filings, good reasons to be on public filing. i also do something i call the identity project that john has been a part of for a long time and really helping kids and parents understand about the dangers of identity theft for children and how that ties into

human trafficking, which has escalated in a shocking and society should not allow its going on out there. whenever i publicly talk about identity theft for children and particularly exporting from usually mafia sources, i at least get some sort of some thing. a male, he tweaked that is talking about i am firm so and so when i get you some person out there. eventually it is going to be my published home and that bothers me a great deal and it is a chilling effect not just one individual who happens to have a cause she believes in, but sometimes collectively been able to stand up against things and clearly speak your mind without being worried someone is going to be near your children are attacking your children when you are traveling around, trying to prevent the very crime that's happening.

the other thing i think with how the firms collectively happen if the person example of health care. and you're available. the good side is we will teach people and is doing so you have people coming out of your room. and having a child here at stanford and it happened to have a submission. it's like a constant flow. what you'll find when you open your bill is that you've got all these billable for people if you have a law degree. track them down and say what services were provided in this colossal 80,000 big bunch of them are just observing is the

baby. teaching hoss at all and a student or group of student is something i would say to be an insurance problem and there could be hundreds of thousands of dollars. it is hard to articulate my anonymous gale bleeding and context of fort law. still very early. >> my only thought is there are specific areas where legislatures have implemented the video privacy, stay privacy. clint leach bliley have rules that apply to financial institution that the consenting

to deal with the medical situation. that so far has been a productive way to proceed because tools and regulations are implemented in the kinds of rulemaking process. that's generating the predictability. at least in certain areas that i think commerce is looking for. so i think that is good and i'm sure that will continue. there's a lot more specific that will be coming out. those statutes say, let's now overhaul everything and create one statue. we couldn't have a single grand leach bliley soap rocks. so the tailoring of the rules

and treatment really has to be context specific. >> you start it from the other side. it's actually a parochial discussion here because it's very u.s.-centric. the european approach here in the united states. so it's not just the nsa. data centers in the united state. it's a difference in rules and protection that companies in the e.u. have to worry about than whether data is located at those differences to make a difference. but, you know, that's why --

[inaudible] [laughter] it doesn't necessarily mean -- an interesting point about to find a way to reconcile ever together. here the tradition united states has been must exploit the opportunities available for big data. you may have to make a choice between collect data in the first place so that it can be analyzed for its maximum potential ornamenting in an internal us to quite a so in deference to privacy concerns and span and eliminating it from its availability for analysis i think we've gone more in the other direction. there's a great illustration following the floods indonesia. we were a utah phone data

collected to analyze where people are going and helping people react and in relation to the flood was a tremendous boom to these studies as a result of being able to get that data in the regime we say we're not going to allow the data be collected. you wouldn't have that information. and so my sense is they think the united states, my sense is we will be more reset it in the united states. we're going to focus on issues with data in the context specific way and not the collection. >> maybe that's the point because i think the discussion that the piece of that with risk mitigation and tried to get that risk is close to zero as possible. most of us would agree that there's not going to be

accompanied that would take it to the zero-point and say let's not put the oil tank in the ground. there's this opportunity. but there is behind the surveillance but the potential financial or jaime data is lost in some way that we have someone. the president is asking the various folks from washington to look at big data and think about how to rebalance these of these potential harms any opportunity of how we earn innovation. michelle, going back to your experience as a patent litigator, looking at some of those data analytics we saw in the genetic space or the medical records space, talking about geographic trends, tracking of course incredible value nation we know comes from data analytics. if people come up to you and ask

you, how would you advise the president to balance that risk and opportunity, how would you approach that? we haven't heard that you, professor felten. we'll start with you. >> it's too much to ask the president to solve this problem. certainly there are some important public policy things that can be done. they start a sink some pretty fundamental concepts to, for example, to try to strengthen the idea of consent from consumers and citizens, especially in a certain context. we often have this phenomenon of privacy theater in which someone pretends to disclose to me how my data is going to be used and then i pretend i've understood

and consent. consent it doesn't help anybody. getting closer to something like real consent is how paul. consent of allowing people to pretend there's been a meeting at the minds about what the expectations are. i also think outside the policy process we are starting to see and we are going to see more technical self-help by end-users. that means people act in a way that denies data to collectors or that adds noise to the data by either at securing information or even behaving in a way that causes incorrect data to get collected. we're going to see all kinds attacked ex carried out by consumers and we're already seen a rise in extreme upper privacy protect it products for use online. we are going to see more sophisticated ones.

with respect to big data, there's not only sophisticated theories of how to analyze that data, but there's also a sophisticated data on how to modify and manage big data in ways that further protect privacy. use the consumers and end-users take matters into their own hands. we see sometimes people avoiding medical care because they don't want something in the record. they are orchestrating for simulating certain online behaviors to give the impression that certain things are true. >> one thing to the point of asking the president to solve everything, first of all, the countries where we do business. i'm not a current commissioner.

would love to come as a technical community and just how the closed-door for ip address actually is good for example it is the internet as it turns out. don't you think a little more technical on everything? he said no -- hats off to you. you are the person and the fact you can name a guy is problematic. i don't think ed can do it all, but hats off to you dr. szpak. you're the technical awareness engine for the government and i applaud you for that. obviously he knows a lot of wouldn't have given this job. the other think really excited in this area is as it turns out, people are not that dean. it seems really fun.

it turns out that we've got about 3000 years of a data, behavioral data. we understand what people do and we understand to entertain, et cetera. the big movement i'm really excited about and artificial intelligence in particular is to sandbox and use dummy data to test applications before they went by. remember when that was the thing i never once said we can't possibly add a new i.t. name. as it turns out you can. you can have a simulated data set and test it. in terms of big data and looking at how people behave in malls, how they shot. man walking and turn right. we don't really know why, but that's what they do and you don't need to pay what amount to $5000 per shopper to have a bluetooth, have a phone, have

permissions out of her than out then turn right. sporting goods stores to the right of the entrants in europe to the races. it sounds silly, but what you can do is simulate various patterns and models of data sets you want to combine so you don't have to actually get live cdc data. we found this when we were creating federated identity solutions in the early 2000. we said okay cdc common set of august data and putting sensors in airports and things, we can simulate first but would it take if there was a massive sars outbreak, how to fly? a student silicone, anonymous practice run. as it turns out a lot of information was learned for the information wasn't being shared, where there was too much sharing. with a smaller case

practitioner, that will continue to come in companies that can so telecom monitoring is a large static data sets are the huge winners in the space. a couple companies are looking right now. i've tried a couple more dollars a month down in that industry because it is such a hot topic and takes a lot of risk off the table to do things. so that is the number one. the other thing is that we also need to have been building standards. we need our engineering and completely biased in my latest project it is absolutely responsible that in the engineer graduates without fair information practice principles. they do not have to learn about security. it should be absolutely no player gets to practice in the u.s. without a license. not so for developers and other people creating critical infrastructure, mass quantities of data.

it's not the big silver bullet, but every engineer must be conversant in uses of data collection strategies, transparency strategies, all of the known information principles we've had around us as the 1960s. they may not need our technology, but the aspects of the framework we haven't even tried to exploit yet. i refuse to believe that something that's been that static over time and tested is one. the thing in the back of my head which is as a practitioner, a real-life example without the company attached to it. if a crisis for one of our very important researchers was cut off by a raging wildfire that we wanted to sign up to real-time kids needed to be picked up in schools. families and it's another loved

ones for safe. we had to evacuate and figure out who is traveling that day or not. we had to crack open databases and in some cases personal databases where we have not parents who had access to care. the point of that was during that context and time because of the planet wasn't necessarily a technology thing, we knew and there's a beginning, middle and end. i knew when we shut down access to databases just as soon as i knew when we opened access. we are in a crisis, people say in what was the crisis but you have to does for the end of the crisis when everyone is bored and is going home to make sure the data, which is critical data has been closed again. it's the boring details that really amount to safety. it's kind of the big way data is saying it's really great and

terrific here but handwashing turns out to be the number one thing that lowers diseases in large facilities. so there's a low-tech, high tech and understanding education component that i think his renovation needs to happen. >> john, what about yourself? >> this is sort of an extension may be of the sandbox and in a way. there's a lot you can do with big data and get a lot of benefits when it is not identified. we can learn a lot about climate change. no pii involved. there are different rockets are different types of data and context in which data is collected and used to make a difference. they could have different rules. the kind of massive quantities of data used for medical research is probably going to be

pretty well identified in most cases, much more than for other purposes like marketing must say. you actually need to know more about cases to do a world in which there is not only a governance model and a multi-thousand year history culture of confidentiality regarding information. the rules do not world are stronger already, whereas in a marketing context, there were many rules and you even want to be looking at what is the ultimate purpose here and how does that stack up against using data for medical research, for example and might you establish

some rules that would significantly reduce the risk of using big data in a context that has less public benefit at the end of the line camera that can be done without the data being personally identified. [inaudible] >> however you do it. there could be any number of ways. in the medical context, and very many sources of the rules, some of which are cultural, but there are also laws and governance procedures. there's lots of ways to establish rules and we can learn from some of those pieces. before he went to washington state. in which he proposes companies create, like irp's, ethical

review boards for big data projects which is an internal thing, not a government paid. a board that is appropriately representational within the company that has a basis within a policy that has some values underlying it and big data projects i reviewed an arena of research in human subjects are reviewed when you look at this benefit analysis and the risk not to the company, but the data subject. [inaudible] weathers the chief privacy officer. are you saying that market is a little towards companies more and more about that. >> i was thinking that's one of the primary roles that i play. make him a look at the privacy

impact assessment, which is now including design guidelines analysis rules for buildout and when you put those data sets together or using the vendor said because retailing analytics and big data. then we'll put things together and i will be the line arbiter and if we have to escalate about that, we do. more and more people are doing that and i think the hard part of some of this stuff is happening for smaller organizations after procurement level. so it is the young navy attorney, usually a contract manager who is facing some sort of question about liability for a vendor who's going to be providing data. they may or may not know anything about these risks or rewards her with the controls are. they may have a sense that there should be security liability training, but this is for the

trans-nation piece is alarming. >> what does that look like on the back and when something goes wrong? >> you know, my governance system being that you look at when you win in that situation, people are trying to evaluate whether this incident has occurred at a company that is diligently taking care of information are not. my reaction dealing with companies that struggle with these situations that a lot of the steps that are being discussed are very expensive. very time-consuming and and the the way of the business you're trying to serve customers or engage in research activity. i think we all need to think carefully about sort of pushing that model into areas where the value of the data you are

protecting isn't really worth it. on the other hand, there is -- people used to always say there's two types of companies and already companies that will inevitably be reached because everybody will be reached. i cite terrific variation of that the other day in "the wall street journal" that they cut like interview with ted schlein who is a venture capitalist partner at kleiner perkins. he says look, i'm a firm believer that there's only two kinds of companies. those that have been breached and know what i'm those who have been breached and don't know it. most of what we do intervention

and prevention. great. just knowable work. know that they're going to get in. that really is kind of the world we live in and that's what we see dealing with people who come to us for advice. so if you are a big data company, you want to sort of recognize that, you have to have a good feel and part of the incident response plan in the overall approach to security in the first place you want to have a sense of how you explain the situation to regulators, two people in the puppet. what we see us whenever there is a breach, there's always by definition a broken link. somebody got in and away that maybe he would've been been able to prevent by design or by luck.

somebody has gotten in. what is interesting in this area is that is where the plaintiffs in lawsuits and regulators are going to focus. they will focus on the weakest link. our job as counsel is to persuade this decision takers to the public, make sure that they understand that imperfect security doesn't mean unreasonable security. look, you have to look at the totality of the circumstance. if you have a data center that is a fort knox type facility with guards and biometric pasco's, if you've got firewalls and segmentation trekker knotwork on the strong access and authentication goals, security policies and privacy policies and training throughout the come any, they advise you on

aspects of your design. former ability skinny. you follow up on the vulnerability scanning results. you have performance monitoring right diaz or dlp. you spend an enormous amount of security. you are still going to be at risk of being hocked and you will be. so, yet whenever these things happen, the focus is always that one thing the way of us. think when we have to gain a mature approach to these situations that there is a world were imperfect security may be the best. we all agree no matter bring it down to zero in that approach is got carry through the way people

react to incidents when it occurs because the overreaction and intense amount of traction or placed on companies by the brand damage, but the cost of litigation really makes -- there's a cost to that. it's great for lawyers. the companies themselves striving towards reasonable security you have to put so much more into it because you want to be because of the concern that half of the brain damage is done and goes wrong, but damage will be substantial. >> just to extrapolate that to the marketing department to the analytics to progress up, taking the governance model and that they not out in the innovation innovation -- [inaudible] you know, i want to