reducing and protecting secrecy wasovernment, and his ideal to open up lots of the documents that had been sealed, and he was trying to figure out a way to get jesse helms, who was on the board, so then he came up with a concept, secrecy and form of regulation, it senator helms was not a fan, so he came up with this concept ,hich would appeal to somebody who one might think would be perhaps opposed to releasing historical documents, however long they might have been under

so i think it was that quality about him. and he had the ability to bring out the best in people by trying to figure out what do you want, what are your ideas, and i think it was that capacity to reach and he servedle, presidents, regardless of party. >> he served president nixon. >> he served president nixon and president ford at the u.n., and in the months before he passed away, he had worked on a commission that george w. bush could together on social believe that it was that capacity to reach

across the aisle. >> i went to get some background on you for growing up in new york, but before we do that, i want to show you a video of someone who looks awful lot like you. he also went to columbia university, and he is currently on the massachusetts circuit court, and tell us who this man is when we see him. >> i have had the privilege of knowing him since 1980 when i served as his first law clerk on the u.s. court of appeals for the first circuit, and i also appeared before him on numerous occasions in my capacity as chief attorney, and supervisor of appellate litigation, and one judgeaway from knowing breyer with the sense that here is a man whose goal is excellence, but excellence in and oneuit of justice,

can feel good about the future of america when people such as judge breyer are considered for the supreme court. >> who is that man? >> that man is my identical twin brother gary, and he is a judge on the massachusetts appeals court. i think he is an extraordinary judge and also a very kind human being, and he has had an amazing career. court has been on the doing basically what i do but in the state court system for some and he has been a prosecutor in the department of and he is an author and scholar. his book has been translated into russian.

his work on securing justice for , resulting in a book with the workings administration. he has been actively involved in boston in civic engagement, me,ic engagement and like he is a c-span addict. >> well, go back to the beginning. where are your parents? are they still with us? >> my parents are still with us. my parents are my heroes. 88, and he is a refugee from nazi germany, and incame to the united states 1941 with his mother. andfather had died, 1938, my mother is from brooklyn. she is 85, and her parents were and my parents are

really my heroes. they have really given all of us, the four children, a sense that anything and everything in life is possible, and that whatever you do, you should do it with modesty, with concern of others, and with the idea making the world a better place. worked for 40 years and went to night school. >> what was his job? >> he was an engineer. he would come home from night school classes and never and iined about anything, think that i am often asked, and i often work with immigrants,

trying to provide counsel for immigrants, and i think part of the inspiration is that i can still remember the voices and accents, and those coming to the country to make it a better thought itthat they would make the country a better place, but they thought this was a great nation, that they wanted to make it even greater. siblings, what are they? girls or boys, and what do they do? >> i have a boy into pewter is, and he is in connecticut, a very smart guy, and a sister who has and has raised a great family. on worked several years ago and we're allcer,

very close. >> what about mom? what is the background there? >> my mother was -- raise the and what a great job she did in terms of attending to each of our needs. single placember a where my mother ever said to me, and i think this is pretty imarkable, and i do not think am idealizing it, because i was talking about it to my siblings, anything she said that was unkind. we have had the blessing of having two very supportive parents. the difference between, and who is older, gary or you? >> i am eight minutes older. i used to think i was five minutes older, but about 10 years ago, i saw the birth certificate, he jokes that he anyway, andt, but,

we basically do now the same thing but in different systems, one in the federal court, the other a state court system. >> what would you tell a lawyer who is going to come before your court not to do? >> i would say that is a great question. i would say do not caricature the arguments on the other side. it's a lawyer is to have credibility in court, certainly with me, i want to have the sense that that lawyer is fully respectful while advocating for his point of view, the point of view of the other side, but at is not -- and at

the same time is not exaggerating the other side's arguments. also, another thing not to do is not to know the record. if i ask a question about something that happened or that was in the record, i would want the lawyer to be fully knowledgeable and prepared to , and then that record the third thing for me is answer the questions, which is to say don't think -- if i ask a question, don't answer it with an answer you want to give to another question. please answer my question. is a clip of here a man i know you disagree with. watch a little bit of this. this is from a 2012 interview, and this will give us a chance to get into this book you have

written. let's watch. [video clip] said in some supreme court opinions that sometimes the letter of the law is its experience and that experience must prevail. at is nonsense. the letter of the law is the letter of the law. that is what we are governed by. there is some determination of ,pirit, which could be anything but the statement comes up often . it is an in powermad of judges, and judges can simply say, oh, yes, the text says that, but that is contrary to the spirit of the law, and we're going to go ahead and do whatever we like. a democratic self-government of people, that they cannot have which is.t, >> here it is, the new book,

and could you be farther apart on this, and what is the issue? >> well, let me first begin with what we agreed. do youissue is how interpret a statute if you are a judge? do you look only at the words of the statute, or can you also materialse body of that congress produces in the process that accompany the statute? the court. to come to an agreement with what it is they are doing, and often there is a vote of confidence, or the committee report -- they often give

background about the backgrounds. >> can you define what a statute is? >> a statute is a law of congress, so it could be the civil rights act of 1964. it could be the hobby protection act of whatever year that deals with protecting memorabilia from being copied or coins from being copied. it could be the clean air act. when congress passes a law, that is a statute. so where justice scalia and i i that theld agree is language of the statute, the law, where it is crystal clear, then, of course, fidelity to written meanst is

you do not have to go beyond the words of a statute because it is clear, so if i said to you, there is a law that makes it distribute in a 30-day period three grams of a and not you personally, because you would never be involved in any of this, but let's say a person had been his grams,rams, having 30 well, it is very clear that the statute says three grams. the interpretation is very, very clear, but what if i were to say to you that there is a statute it will be unlawful

for anyone who has been convicted in any court of a crime punishable by more than ?ne year to possess a gun now, what does "convicted in any court" mean? court in theany united states? does it mean any court in the world? that congress could go either way. >> you have a case you talk about like this. >> yes, i do. i have a case like this. >> or three judges that sit on these cases. >> yes, yes. >> were you there? , and you can there say to be convicted of the court, perhaps in the world, or maybe the united states, but

then you think about it, and you think, well, they're all crimes in parts of the world for freedom,g religious for trying to practice freedom of the press that are punishable by prison of more than one year. do we really want to say that if you are convicted anywhere in the country, with a court that does not have the same values that we do, that that should count? it, why you think about could we not then look at the accompanying materials? >> would justice scalia look at that? look at that.t here is the way i look at it. congress is part of the law. that is the first article.

charged by the constitution with in enacting laws, and so what congress thinks of as important, in terms of our understanding as judges, as administrators, as citizens of its work product, it should simply not be discarded, but it should be paid attention to. obviously, some of the materials are more important than other, but we should not simply discard it. scalia notesce that legislative history increases, legislative materials the judges, and they can pick and choose, and the real problem is if the statute, if the law is unclear, and the only thing i am going with on is the words of the

statute, and it is ambiguous, and i do not look at anything why use my own discretion? because i am not constrained by trying to understand what the legislators meant and how they thought the statute was meant to work, often with language in their committee reports. you understand your book say that justice scalia and clarence thomas are the only two who are textualists? >> they are the only ones who do the text.ond i mean, justice scalia, for a career sitein his , but he no history longer does so, and -- you know why? >> i think yes just come to an issue.

you would know better. but i think that is his view. case some years ago where i actually cited his reference to legislative beenry, and that may have one of the last cases that he cited, where he cited legislative history. you are saying about this case of the second circuit, how beingo you think about reversed by the supreme court? >> i do not really think about being reversed by the supreme court. i think about trying to follow the law where the president -- aecedents directly to go in direction, and there are times when i will reach a where, if i could rule the world, i might have written

the law differently, but it is my job to follow that president, so i believe that we would have a lawless judicial system if we were not attentive to what the supreme court directed us to do, but where questions are left open, we have more discretion to try to answer the question, and the, of course, can lead to supreme court. >> do you have any idea how many cases you have actually 1999?cated since >> that is a great question. i would have to think about that. it is in the thousands, in the thousands of cases, because i think there are probably about, argued, and0 cases

then we have the immigration docket. it is probably -- about 500 cases a year. >> from the court? is that what you sit with? >> i sit with a three-judge panel, so every judge with me. i sit with all 22 judges, but we sit at any one time in a three-judge panel, except in those rare instances where we hear it as a full court. sat -- all ofer the judges on an appeal with in the court? >> very rarely. our courts go in bank rarely. other courts go in bank more infrequently. >> why is there a difference? >> i think it'd difference is if the case is that important, it should go up to the supreme

court right away. apecially if there is three-judge panel, and there is a strong dissent, and let's say is,s 2-1, then the issue the issue is the differences are firmly drawn, and they should be -- the case should just go up to the supreme court for resolution. in other courts, the view is rest -- thee the court does not like what the is injudges did, it consideration. it is unwarranted. i think one of the reasons our court is so collegial is that we do not go in bank that much because there is often a lot of proceedings, and i for one think the tradition that we have had which goes back to

leonard hamm of not going like a wise tradition. >> what is the politics of your appointment? by billointment clinton, but you also give a nod to gene sperling, with barack obama for helping in the fall thing -- whole thing, and there is also patrick moynihan. did you want to be a judge? primarilyhe position because of senator moynihan. and he had -- i had worked with him on so many projects and there was a vacancy.

the persondentified who works for senator moynihan, now a judge. senator moynihan was the force, and gene sperling, who had been a research assistant to steve i firstbrookings, when got to brookings -- nine >> who used to work for eisenhower. >> who used to work for eisenhower. and so he was helpful on the white house and, and then the other thing that was really quite fortunate for i was known to people in the hill because of work that i had hatch was veryor supportive of my potential and his team let it be known, and senator hatch was

chair of the committee, and he that i would get through, and senator leahy was and so i supportive, was nominated in a few months. 1999 confirmed in july of by a voice vote. the first court of appeals in 19 99.t confirmed >> as you look at the court, there is 700, 7 hundred 50 federal judges in the united states. how many of those are determined to be on the court by the members of congress, do you think? >> i think the district judges are almost certainly largely senators, whenhe the senator is the same party as the president.

the court of appeals appointments are traditionally viewed as white house and that is why, i think, in my case that senator to getn worked overtime and theation through, white house and john podesta, they were very helpful. >> on another issue, justice scalia a again on this, and i will explain why we do it in a moment, and watch what he says about this is not a legal issue, but it is an important issue, as you can see, before the supreme court. [video clip] really thought it would educate the american people, i would be all for it. if the american people got together and watch our proceedings gavel-to-gavel, they would never again ask, because i

am sometimes asked,, "justice scalia, why'd you have to be a lawyer to be before the supreme court because the constitution says you do not have to be," that is right, but we are normally dealing with the internal revenue code, with all sorts of stuff that only an lawyer can understand and perhaps get interested in. if the american people saw all of that, they would be educated, but they would not see all of that. what most of the american people 15-second takeouts from our argument, and those takeouts would not be characteristic of what we do. they would be uncharacteristic of what we do. >> it almost sounds like a setup, and i do not know what you think of this, but your

court is allowing us to cover monday the aclu versus clapper -- tuesday, in the court in new york, and we have covered several things in the second circuit court. why can we cover the second circuit court, and what do you think of his attitude about television of the supreme court? what in terms of the court of appeals, the reason that we can cover it, you can cover what we do is that the judicial conference, which does not include the supreme court as part of its jurisdiction, essentially in 1996 allowed for a local option, so that each circuit could determine for itself whether to allow cameras in the courtroom in noncriminal and the ninth circuit are the two circuits, i believe, that are participating in this, cameras in the

courtroom, and the way that it works in our circuit is that a request goes to the panel, and if the panel agrees for the coverage, then there is coverage for you if a member of the panel has any concerns, then the to decline the request for cameras in the courtroom, but by and large, i isnk that the experience that there have been a number of cases, as you have said, brian, where there were cameras in the courtroom, and i think that in terms of the court of appeals, i a positivethis is development. >> has it ever appeared, in your opinion? >> no, there is no record that it has ever interfered. that often,happen

because there are not that many cases as exciting where people want coverage, but i think the value of c-span is that it covers everything. it covers the whole argument. i think that that is important for the educating process. there is also now an experiment that was authorized in 2010 and began in 2011. i think the project will end in of5, the pilot project, various civil proceedings in the district courts across the country. i know there are many district courts you are participating, and there is a website where you can go and watch the proceedings. the district court, doesn't always have a jury? >> it does not always have a

jury. there are many cases which are so we will see how that experiment works out. >> does the circuit court ever have a jury? >> a circuit court never has a jury. >> and does the supreme or ever have a jury? >> know, the supreme court never has a jury. judges, andthree the lawyers come before us, and we do not have the excitement of the district court, with the defendants and witnesses, and you see. the court of appeals does not have that. >> we have some video that was taken earlier, just to show what your court looks like. there are more judges sitting up there than the normal three, which we will show, run just a little bit of this, and then ask you to explain what we are watching. [video clip] the united states court of

appeal of the second circuit. shall be heard. >> when they are working with the syrian's, providing the syrian's a dossier of questions, getting them back. >> yes, that is an allegation of complaint. dossier of questions was the fbi, identical to and the syrian said they were provided this information to the united states. the very sames at officials who told his attorney that he was on his way to serious, or was it the case of one hand not knowing what the other was doing? more than three

judges. you are not in this video. we saw more than three judges come out. why was that? >> i believe that was a new bank >> that was a 2008 case. and what you see is all the judges on the panel reviewing a lower -- they are reviewing a question that was , asidered by a prior panel three-judge panel of the court. they are not so much reviewing what that three-judge panel did but looking at the issue of fresh, and you see the lawyers in the courtroom. one of the reasons i asked you this is that justice scalia said the public would not understand what is going on, and it, though he did not say

they are worried about the jon stewart program or the john colbert program -- stephen colbert program. does that concern you. >> i think there is a concern about taking things out of context, and there is an argument for the whole thing to be shown. if you are going to watch c-span, you're going to be a very intelligent person who is wanting to learn about the process. you asked me about the supreme court. i know that they are at least at this point certainly releasing audiotapes. i think that is a very good development. on the issue of cameras in the courtroom, i would want to know more about why some of the

justices, who at one point seemed favorably disposed to cameras, have concerns, because there may be something that we do not know about that has to do with those concerns, maybe security concerns, and so while at the court of appeals, nobody knows who we are, and we could be covered, and there is no sense of security concerns, and maybe different at the supreme court that may suggest why some have changedes their views, and it would be interesting. the next time c-span does that, you could ask them. >> we did not ask them during that round. this book, what is on the cover? >> what is on the cover is a text of the statute, which i had and the question

in the case was this. bar on suits against involving the miscarriage or loss or negligence of transmission of materials in connection with postal delivery, and so the question was, what does andigent transmission mean, the reason i think it is such a great cover is and gives you a sense of here is a small phrase in a large statute, and that can become the subject of a whole court case. >> what to think of the experience of -- oxford press publish this, and it costs

$24.95. what did you think about doing a book like this? >> it was a great experience. nyu on theture at subject of statutes, and there was a representative with the new york times you said you should really think about andnding this into a book, so, i did, and the oxford press , and i ame been great excited to be part of their stable of authors. >> we did not talk about the and thest papers governance institute. are you still on the governance institute? >> yes, i am, which is run by russell wheeler, and it is an

institute that tries to deal with problems of governance. they have done a lot of work in judicial administration. they have been very helpful in the project i directed on securing, trying to secure counsel for the immigrant ppor. -- poor. >> who are they? where are they located? >> they are in washington, d.c., and they are a nonprofit, a and it is a great organization. >> now, we don't have time for this, but i am going to ask you anyway, because i have never seen this word before. versus --ualism efulsim.s

that we understand what they meant, and justice breyer i think has written eloquently about the importance of not diversity -- divorcing law from life, and that means understanding what congress had in mind. >> and at this stage in your how long do someone like you think about continuing to be a judge? >> i would love to stay on as long as my brain is working. privilege togreat try to serve the public, and one of the great things about the job is that there is not a retirement age. upyou feel that you are not

to it, or if others feel you are not up to it, they will tell you. twin it the same for your brother gary on the massachusetts court? a mandatory retirement age. i think that is unfortunate. >> our guest has been achieved judge of the second circuit court of appeals in new york, judging book is called " statutes." thank you very much. >> thank you. [captions copyright national cable satellite corp. 2014] [captioning performed by national captioning institute] ♪ >> for free transcripts, or to give us your comments about this program, visit us at q&a.org. " programs are also available as c-span podcasts.

>> here is a look at our prime time schedule. the second circuit court of appeals here is an oral argument clapper case,rsus which challenges the constitutionality of the nsa collection of american phone records. on c-span2, book tv, with authors who have written about american presidents, and on c-span3, it is american history tv, with programs on the 1812 burning of washington. the pentagon, a rear admiral briefed reporters on several issues, including the ongoing threat of isis and violence in syria. here is part of that. forcese there are u.s. on the ground? >> no u.s. forces on the ground. >> before after? >> no u.s. forces on the ground

before or after. >> with the forces? >> all i can tell you is we continue to work with partners in somalia and in the region, but i will not get any more specific than that. before ithe mission took place? >> i do not have that on the notification process, but this is very much in keeping with the kinds of operations that we conduct throughout the region and in partnership, you know, with the leadership there. >> this the mali government announced a new operation, about combating al-shabaab, particularly targeting their toess to parts -- sports take off their courses of revenue. >> not to my knowledge. if the reporter was killed, and i know you will not speak to

that, but if you were to be killed, what would that say about the group -- what would it mean for the group going forward? how important would that be? >> whether he was, in fact, he was theot, recognized appointed leader of the al-shabaab in syria, and if he was killed, and this is a very significant blow to their network, to their organization, and we believe to their ability to continue to conduct terrorist attacks. now, mind you, it is a network, and we understand that, and we are mindful that there remain other leaders of the organization at large, but he is the recognized leader, and it is a significant blow to their organization and to their abilities. >> thank you, admiral. more broadly, can you answer the critics that are saying that the administration does not have a

strategy, does not have a counterterror strategy, a middle east strategy, one that is good enough? if there is a strategy, can you articulate that? >> absolutely, there is a strategy of our approach to the middle east. now, i can only speak from a military perspective and for the pentagon, but we have been consistently going after the terrorist threat in that part of the world, and not just that part of the world, as i just read to you, and inside iraq, the mission is very clear. iraqi there to support and kurdish forces as they take the fight to isil, and we are there to provide humanitarian weistance where and when can, and we just talked about that over the weekend, and we are there certainly to defend and protect american personnel, so the mission inside iraq is very, very clear. military strategy with respect

to the middle east also has been very clear, and it is not just something that we started doing. we have been going after terrorist networks in that part of the world for more than a decade with very good success. it does not mean it has been eliminated, but we certainly have been very active and very energetic, and the objectives have been very clear. if you had gone after isis as soon as you possibly could, the question is how good and how early was the intelligence that was being briefed to the white house about the isis threats, and could more have been done sooner? >> well, i am not going to speak about intelligence matters, and i am certainly not going to speak for intelligence issues raised at the white house. that is a question better directed to the intelligence minute, tobut wait a your larger issue here, we have talked about isil for many months now, and as i said

before, we were very close in tracking their progress, their development, well before they rolled into mosul, so this is not an organization we have not been watching. the speed with which they took control of northern iraq definitely got a lot of people's attention, and i have said that publicly, two. no one expected four divisions of the iraqi army were going to fall like they did, so there was a speed there that certainly did not go unnoticed, but this is an organization we have been long watching, and i think it is helpful to go back and just look at the last couple of months. onean, we are all fixated targeted airstrikes, which would are conducting with very good tactical effect, but long before that started, there was our presence in the persian gulf. personnelore security in and around baghdad.

we still have two joint operations centers, which are active and helping right now as we speak in terms of advising and assisting and sharing information with iraqi and kurdish forces. and then we have done numerous airdrops in two different operations to alleviate suffering, so the military has been very active here. the other thing we said, just in, and this is not a small point, is that it will not be just a military solution. ultimately, it has to be inclusive and responsive, with good government in iraq to help alleviate and take away those conditions that folks like isil can exploit for their own purposes. does that answer your question? as you may know, the majority of the foreign fighters who are joining with isil are going to syria through turkey.

my question is, how do you eliminate turkey's role in fighting isis? >> well, it is not for me to evaluate turkey's role. a strong relationship -- i know that. i am not going to answer a question that should be asked of the turkish government. what i atement. what i am telling ment. what i am telling you is that turkey has a stake here. we understand that. it is an important element in the region, an important ally. the turkish government has concerns about foreign fighters, and right, they should, and we are going there next week, and i think that no doubt this will be a topic of discussion between secretary hegel and his counterpart. what? >> do you think they are doing a helpful role? >> a helpful role. we believe that turkey, because

they have concerns, just like other partners in the region, are expanding their efforts and their energy in trying to address this as best they can. i am not going to go into more detail than that. >> can you give us a clear situation on the ground at the mosul? why does the u.s. keep launching airstrikes at that location? >> because isil keeps trying to take it back. said last week, as long as they continue to pose a threat to the facility, we are going to continue to hit them, and we will. >> secretary hegel will be asking nato allies and other nations for help in this potential campaign against isis in syria. how much more can you tell me about it? which ones, and what they will ask for specifically? >> i think we are still putting together an agenda here, phil, for the nato summit, but you are right, secretary hegel and

secretary kerry want to get together with some partner nations. allies, ofm are nato course, and i don't have a list right now. i think this is going to be more of an informal relationship. again, on the sidelines of the already very full agenda on -- will try to get together and talk about the contributions that have been made by these other nations, and certainly to encourage others who have not been to be did yet to look at intuiting, as well, but i do not have a date certain on the calendar or time that they will do that, but we are looking for those opportunities, and it could be more than one. it could be that they have these discussions in more than one setting with smaller numbers at a time. >> do we have a sense of what they will ask for? do they want partners for airstrikes? what will they ask for? >> we want partner nations to contribute what they are able and willing to contribute, in

whatever fashion they are willing to contribute, understanding, of course, that they have domestic concerns, as well, and their own legislative bodies to work through on this and populations with different views on assisting against the isil threat. we respect that, so it is not about going there with demand or a laundry list. it is about going there and thanking them for what they are doing and encouraging them to continue to assist in whatever way they deem fit. >> that was part of a briefing held today at the pentagon. you can see the entire event any time at c-span.org. u.s. college professors released their findings on minority students and whether current policies are helping or hurting their chances of getting into college. they spoke earlier today at an event hosted by the ucla civil rights project. it is about one hour, 10 minutes.

>> ok. well, i hope you are all refreshed and ready for an intense panel here, so we're going to have three presenters on this panel. sylvia hurtado, nicholas nunez., and anne-marie why don't we start at this end of the panel and have anne-marie go first? >> my paper is co-authored by someone who could not be here today. we focused on how the shortcomings of a post-secondary system could lead to hispanic-serving institutions.

research indicates there are well documented offenses in student institutional hsi'steristics between non-hsi's, so our analysis at the beginning of this paper, in the previous session, we talked about the possibility of adjusting for some of the student and institutional characteristics, and so, in fact, at the beginning of our paper, we find that when we account for student characteristics, financial resources of institutions, policies and institutions that might influence completion, whether or not it institution has an open admissions policy, that the gaps in graduation rates, and so the most commonly used measure being discussed right now to measure institutional performance, the six-your graduation rates, that the gap in graduation rates between hispanic-serving

institutions and non-hispanic-serving institutions disappears. and this is what sylvia was talking about, and if we were to calculate a rating system with the data we have available in data sets, like the post ,econdary education data system that is really the only data system now that is national, that has data on all the postsecondary institutions in the u.s. we will talk in a minute about how there is problems with missing data, but it has the most comprehensive data, so what some have talked about is using a method of adjustment to level the playing field in terms of assessing institutional performance. minority serving institutions and non-minority serving institutions. >> this is something like what

deborah santiago was talking about. >> exactly what deborah santiago was talking about, and the idea is to reject a graduation rate based on what we would expect, given the student and institutional characteristic, perhaps policies to help, and compare it. than the actual graduation rate of an institution, and that institution might be seen as underperforming, but it's a graduation rate that was expected was below what we sought to be the actual graduation rate of that institution, it might be interpreted as over performing, and so i think one of the takeaways that sort of has been coming up is that they are doing more with less, and so a lot of them, in fact, may not be underperforming as they appear when we only look at outcomes like graduation rates, but when

we take into consideration these other factors, they may actually be doing better than we think. next slide. ok. so we went ahead, and once we sort of established that the cap disappeared, we then went ahead and examined different ways of looking at regression adjustment , and one of the problems we initially came across with this data set is that we can only adjust for characteristics on which there are available data, so that regression adjustment may be especially useful for looking at groups, large samples of institutions, and comparing them with one another, like others did in their analysis, but they maybe a little less useful in terms of looking at the performance of individual institutions when we break it data set has a lot of missing data, especially for hsi'a, and this is also the case

for minority serving institutions, in general. we talk about it in the paper, and we have two sticks on it, but it is significant missing data. perhaps one of the most important things we heard in the last session is academic preparation, the course work that students take, how they perform in that coursework in high school, and using an , that that is the most important predictor of college completion for students, and that data are not available -those data are not available in and so as was being talked about how completion is not a college but we cannot, with the current data available, we cannot necessarily take into account as completely as we might like to, yet. go ahead?

so one of the things that we did in this paper was we ran two adjustments, and in one of them, we looked at student characteristics, institutional characteristics, financial context characteristics, and we took what we could best get at or not itf whether institution had an open ended admission policy, and so that may not sound like precollege, but that was the closest we could get you and the most available data for hsi'a, and then there is also a common measure for looking at student of incomingics admissions test scores, so incoming test scores, sat, so we ran separate adjustments, one including an open admissions we found iswhat

that whether a college seems under or over performing depends on the variables that are included in the model, so in our analysis, nearly 30% of the institutions changed directions between when we included the admissions policy variable and when we included the test scores thatble, and so the fact this calculation can change this much should raise a red flag for the idea of high financial aid for these results. we know that in 2015, the performance institutional rating system is proposed, and so we are trying to look at data issues, but if we go into it further and think about applying for funding decisions to these kinds of ratings, that should really raise a red flag. next slide. and so this brings us to our recommendation.

the first one is to collect more data known to be predictive of institutional performance, and so what came up in our last session was the importance of academic preparation characteristics, so that is one example. another example that was raised is being able to track students from institution to institution, and so right now, the graduation rate measures used in the data set is only students who begin at that institution and finish at that institution and are full-time students. and students who finish in six years, and so when we consider a lot of what we have in talking about, they may take longer to finish. they may not be full-time students, and latino students in particular, are more likely to transfer. those students are being left conversations, so if it is possible to collect think one of the

things that sylvia is going to talk about also, not only is it collecting data about the inputs , and also maribeth, but rethinking what it is that these institutions contribute, what other measures are indicators of performance, might there be, at these minority serving institutions and that hsi' the second recommendation is we don't want this to be an unfunded mandate. a lot of institutions don't have the resources. thatding capacity to do would be a really good idea. potter -- part of this could be partnering so -- it would be a good opportunity to track transfer students.

resisting the temptation to implement a high-stakes policy based on inaccurate data and assessment is really key as well an imptant take away froour analysis. finally, there are several higher education associations who are making efforts to find accountability metrics and find reasonable ways of recording them. some of them involve bringing in measures like transfer students. to build on what those institutions are doing and involve them as partners and not -- hiringhe wheel leaders who are trying to account better for institutional performance. thank you. >> thank you very much, professor nunez. the deputy -- i want to make

space for her at the table. please move downward and add another chair. >> thank you for the opportunity . it is a work in progress for sure. i would love to have crustaceans about where this could go. the central argument for this paper were called differential impacts of college ratings and hazards, the central arguments here is a very straightforward one. it is a simple one they could overlook, but one that is critically important for thinking about implications of the federal-aid system and tying funds to a colleges rating. place matters here it is often a research and college access, we

focus on the process of opportunity here to russians like, does the student take -- opportunity. questions like, does the student take ap courses? procedural opportunity is obviously important. geographico opportunity. the geography of opportunity is something that matters here and it will be explored in this paper. it draws on what we might be familiar with. there's a family of research on community built environment. we have research that shows low income and racial segregated and high poverty communities across the country happened to have built environments. have limited access to public arcs. they have high density to industries that pollute. they have got all of these characteristics that define the

community, including access to healthy and affordable, nutritious food. communitythink of colleges as part of a built environment of our community. on top of this, moving away from the process of opportunity will help us thinking about how people make choices about where to go to college simply based on the built environment. not everyone has a lecture to shop around for college or search across the country or even across the state for a place to go to school. there are places around the country that we will explore in a second on this paper. i think we are only scratching the surface. these opportunities are mostly constrained or highly constrained. is usually along the lines of race and class. but the slide, we can hopefully see a map of the united states.

sorry to anyone from alaska or hawaii that is not included in this map. this gives us a mental image of how spatial factors that could matter in choosing a college. what i did in the study is i took all of the counties and cluster them around what our commuting zones. this is not something i made up. this is something people have been using for years trying to identify the commuting patterns of counties. tennessee foris, example, people cross state lines. that would be captured in the commuting zone. they share a common commuting pattern. it makes sense to me that people would also he more or less ifling to to me to school they live in a common commuting zone. what this paper does it should -- it shines these counties to the commuting zones -- it shrin

ks is counties to the commuting zones. how many public colleges are even available? are there key reading zones that only have one public college? -- commuting zones that only have one public college? do they have the capacity to serve their communities? pickednext slide, we this cluster. i would like to hear your thoughts on this. i know dr. nuñez has some comments made us all. here is an example. is represented by a largest planet population. it is a community with high unemployment rate -- large hispanic population. it is a community with high implement rate. has about 100,000 people that

live there. you could seeide, a student who wanted to shop around for college. if they pull the logic of -- to go to this website out but in the website and do the search for a credit colleges are nearby . this is what they would find. you enter your zip road in the left. you could say within 100 miles of your zip code. how many schools are there? four schools. the first two or for-profit -- are for-profit schools. a student who lives in this community is probably going to be place bound. if you're looking for colleges, they have one of two options -- to go to a community college or to a for-profit school.

you have to think about how people make choices in these communities. listed a quick snippet of some of details -- let's take a quick snippet of some of the details. group ise the middle the community college in which the largest program is general studies and has about 5000 students. a lower net price than other schools for the four profit schools that serve a tiny handful of students. if you are not interested in cosmetology and live there, you don't have any choice. here is the tricky part -- and there are several. this opens up the slippery slope . one of the criteria that will probably be included in a rating theem i would imagine is percentage of students who borrow in a school default on

their loans. you bethat would make put on notice. that is a policy. this community colleges at 24 per astra 4% already -- 24%. we could play this logic out and pretend that aid is eventually tied to a colleges rating. that means this college would lose access to aid in one way or another. one of the students attending the institution, they're trying to find a better place to go. where are they going to go? this is not just a problem in texas. it is across the country. there are commuting zones that have a similar all in environment -- built in environment. i use this dataset that i would be happy to share.

a simple count of institutions in each commuting zone and recordings of economic profile and unemployment rate. statistics that might matter. it would be adjusting to see patterns and what these communities are like. ideally, there would be no patterns. as would be completely random. it is not at all. these commuting zones draw line along the lines of race and aass, and just like drive line along race and class and other ways -- drawing a line along race and class in other ways. colleges -- five public colleges, i'm sorry -- are located in these communities . these communities tend to have growing hispanic populations that tend to have what i'm

assuming will be within a becauseonal inequality they are communities that have low education levels already. the we think about implications, we think about implications here. it is an unattended consequence -- i'm not sure if i like to use that term. i think we have got to call it 'smething and realize it environments differ depending on where you live. if the policies going to accidentally affect colleges that are serving these communities, it is by virtue disproportionally affecting minority communities and that are working class and have low levels of education. of place.t the role

the current discussion around college rating is so focused on consumer information and consumer choices. is dominated by people who maybe don't have a lot of experience living in these communities or having institutions represented in their framework. what we have to think about is the geography. waivers or atege least identify which ones they are. >> missing -- thank you. our last presenter of the morning. >> thanks for having me here today to talk about ongoing research. or some of you who have been talking and have looked at graduation completion, we have been doing that for people who participate in our national survey for 20 years.

we have the working with campuses and providing them equations and calculators so they could understand how they are doing relative to the kinds they are recruiting. they have been using more of the national database to really look at what we call efficiency scores. we will talk about work related to these areas. let me start with the work we have been doing on metrics and understand graduation rates, particularly in stem fields. both of these issues are vital to the u.s. economy. a broad access is vital to the economic revitalization of diverse communities. where they are located is poignant in terms of the communities they serve. to students and

frick institutions and serve many low income and i'm represented -- and unresp groups. those targets and first generation are underrepresented. they're most likely to graduate colleges or taking courses they know they will succeed. students are still leslie to regulate. let me -- still less likely to graduate. -- me get to the it doesn't identify the challenges of educating large numbers of these targeted populations that stand to gain the most from college.

it is not a way to rank institutions, but help them understand where they are and relative to peers and how they might move forward in terms of improving. to havethat we need metrics that take into account the type of students and resources that are available. one of our metrics -- and we'll start with the student input adjusted measures using a freshman survey. about 700 institutions that use this data. combined with the national data. we are able to produce predictors of graduation and forever institution, we could give them a score in terms of actual versus predicted. recentlye projects we completed is looking at those institutions at the highest

performers in terms of the student adjusted measure. we found several institutions in which we have the data.108 were doing better -- data. >> 108 were doing better than expected. not takenmetric has into account resources. actual versus expected, i have on the first slide basically three institutions that are 10-20 points higher than it acted in terms of -- than expected. atput up 50% line regulation the center -- graduation at the center. 10-11% points higher in terms of completion. there were some institutions that were expected to be below that, but are above 50%.

some institutions started above 50% and the increase that rate. we're looking at the talent development happening. let's go to the next slide. institutions and tried to predict what was a characteristic of those institutions that were doing better than expected? the key predict who's at the ones you see on the slide. they have lower selectivity, them out of financial aid among first-time students, the instruction expenditures per capita. they should be supported if you have large numbers of low-income students and instructional spent teachers if you have underrepresented group, you need

to direct your resources toward those types of students. it. is one way to look at another way is we use the time online,ou have gone time has a rating -- ranking system based on graduation and affordability and dissect ability. they have a holistic rank that combines the three secrecy which colleges come up the highest. three.ines the then you have the list of which colleges come up with the highest. read and abide colleges that were -- we look at colleges that had an actual date that was 10-20 points higher for low income and underrepresented students. a lot of them do rank higher on the time graduation rank. those with a high proportion of minority students would rank lower.

even though they are doing better than expected, it is higher than what they expected given the student body. they will be ranked lower on any kind of similar type of rating system. usingxt slide is really the time holistic range and using access for graduation and upward mobility. -- there are few institutions that are very high on both. to not to ber one high on their holistic rank. -- turned out to be high on their holistic rank. find of theing you top 16 is you notice they ranked lower on the graduation because they probably are less affordable and less accessible

for the top 16 better do about. to my house small numbers of low income first generation students -- they might have small numbers of low income first generation student. at thet one really looks stem efficiency scores that have a different technique. because there are some locations and the fact that it is focused on first-time all-time freshman, we ended up using certain data and looking at transfers and five years of data to understand the completion. comparesis modeling institutions based on resources. faculty labor and student enrollment and expenditure per student. this is another way of identifying top performers. one of the things that we found ,s that for the most part

private institutions look better on those kinds of rankings. what we find that using these efficiency scores that they are more efficient. example that minority institutions are more efficient at producing stem degrees for blacks and latinos. the other thing is because these particular scores have not controlled for input, a different kind of measure, more selective institutions do better than expect it in terms of efficiency. what we have done is broken them down so you could take into account access to find the most .fficient transit institutions what we're doing is try to visit these institutions. what is the bottom line? any metric you should take into account the students, focusing

on the degree completion from first-generation and underrepresented minorities and resources. i think old indicators help to identify those top performers and the social mobility. i'm hoping the rating system nationally will be used for that and not necessarily to basically limit financial aid. these institutions -- i will reiterate what some of the panel even though their rates may not look that hot, we see when they can do with the student is amazing. one of the questions that came up earlier is how do we handle the input? how do we handle some of these things? four yearonly institutions. two years editions are doing the bulk of this work. those are additional areas to look at in the future.

>> thank you. we have a number of respondents. >> let's play musical chairs for a little bit. >> i do want to thank the authors and the respondents and those in attendance today for contributing to this important dialogue. myning to the papers for colleagues, i applaud you for taking the lead on attempting to do what many have said it would be a good approach to the president's ratings plan, which is to rate institutions based on estimates on how they should be performing compared with how they are performing. as you'll see when you read the paper and i hope that everyone

can come away perfectly reasonable approach and. flawed. it sheds light on how other analyses that might attempt to do something similar will run into the same danger. that is at the very data collected by the department of --cation, particularly what it is inadequate for the purposes of rating institutions. leon --e missing data beyond the missing data, look at , there isoints another underlying data reality. it is not new. it is something that is worth addressing here. again, if that data is used to rate hispanic serving institutions in particular and

other minority serving as to since coming it is not really reflective of the students that attend those institutions. we have done some looking at the national was secondary study that is another data source that is very robust and has a national representative. point, 49% ofthe latinos are enrolled part-time. that is half of latino students. the students will not be representative in the outcome measures. track artistic. many delay secondary enrollment. this further means that other institutions that serve the students well will go unrecognized. building on the first and second recommendations in the paper, i

want to caution against the need to collect more data. what we need and i'm certain that the authors would agree is to collect accurate data. yes it is better accounting of so, the arguments in the paper that raises another issue which is essential for consideration and that is how the department and other agents -- agencies could empower and incentivize institutions to not only report out that, to utilize their own data to improve student outcomes. this means in addition to them rethinking data, the department and other agencies have within their power with these institutions and doing a better job of utilizing data to identify performance and

research gaps and to meaningfully act on the information through institutional policy and. this means allowing institutions the capacity to have my institution of research efforts within their very own walls. i wanted to say a couple of what angst. i really applaud them for addressing environment. we have some more coming up to look at how state really influences institutions and regional contacts as well. i want to say quickly that we institutions can reveal vastly different characteristics depending on the strengths of system that surrounds those institutions and the makeup of the community and the students that they irving and undocumented student access to higher education. statek it was -- look at

data and the federal government to partner to and sent states to use the data to allow for accountability measures. a final point and i know i'm almost out of time i want to let people know that we have a paper that came out earlier this year to look at the college choice behavior of low income students. there's a paper that shows the way that students choose colleges. if we believe the rating system is a good fact ranking which i think it will become, you hispanic students don't -- we know hispanic students don't use rankings. rated rankings as an important after. congratulations again to the authors on their papers. thank you for having me. >> maribeth.

beth.y >> ok. i'm responding to sylvia and her colleagues. i will be short and sweet. does i am short and sweet. -- because i am short and sweet. [laughter] i thought i would get that in there. i really enjoyed her paper. is important that you read this paper for a number of reasons. one of the things i like the most is in the beginning of the paper, she talks about how he wanted to use metrics that were fair to both students and institutions. i really like the use of the word fair. i think too often where we see a lot of national reports, you don't see the fair use of metrics. another thing i thought was

interesting about the paper is she does a really good job of reminding us how for so long researchers have been calling for measures that considered input as well as institutional input. so long, right? and yet we still have all of these efforts to don't consider student input. now all of these ways of looking at institutions that don't take them put of who the students are that attend the institution, yet we have been asking for this to not happen. whenetting a wondering policymakers will listen to that -- i'm wondering when policymakers will listen to that. the other thing i thought was really in wharton's is the work that sylvia and her colleagues are doing to help people look and did deeper and try to figure

out why they are achieving a atater efficiency and access any these institutions, i was glad you said you would start visiting them. there is success taken place in a variety of -- -- it isdig deeper and not as big of a deal, but it is a large problem that many -- at many places. it is difficult to collect institutional data. i think that is important. two things i thought of when i read the paper -- i hope people will -- will start using it as a litmus test. i hope you will caution people against that. i could see people taking it and

saying it is not a action. what i would like to see happen to the institutions that don't come out as being efficient is that perhaps they are given a mentor institution that is of a similar type or size or maybe within the same sector that could work with them to become more efficient. i hope that will be one of the recommendations. i could just imagine someone coming along and saying -- that would be the one thing i would really want you to think about. >> i will make comments across the papers. i want to recommend that you look at this in full. you can get a taste of it just now. is of the important findings

the record permit and that instructional spending. the inputs matter. they show this in the paper. it is a marketable that we have lost sight of it. in this discussion about accountability for higher education, the question -- what is this policy going to do to help ensure that more students get access to those resources? if those inputs matter, what is accountability going to do to increase the likelihood that it is a students who need those things to graduate and will get them? will it bring new resources to the institutions that don't currently have them? will he create better matches to train students need to be educated and high resource settings?

indication that it will occur? i don't think so. it is extraordinarily doubtful. will happen is the institutions that don't have those things, that don't have much instructional spending for the students were that enroll large numbers of disadvantaged students would be poorly rated and cut out. where is that going to leave us? does it mean that the savanna thatnts are going to know institution was badly rated and they should avoid it? whether or not they know it is they are unlikely to do anything about it. are you going to close the poorly rated institutions and leave them with no options? is that a better scenario? these things like fundamental questions to ask and there are no answers. they haven't been addressed. we know that the close some

family schools. pyrenees wanted to send their kids there. where are they supposed to go? just because someone declares the schoolbag because it has a lot of black and brown kids in it does not mean it is a bad school. is it takes a starting presumption with the idea that schools don't care about the outcomes of their students. i don't think i have ever been to a school that actually doesn't care. is this is dumb prims any efforts to establish equity and adequacy in the resources among schools. it does nothing to grapple with the facts of the system that expose the disadvantaged students. that is the worst part of the system right now, how risky it is tucson son who is

disadvantaged enter our system and how likely they'll end up with debt and no degree. i think if we want to experiment with this accountability, we should start with schools that have high resources and using substantial federal funds. there are more than 500 private institutions with an average endowment of more than $100,000 per student for using federal tax dollars to cover their cost of attendance which are over $43,000 a year. students attending these schools are using federal financial aid to cover 20% of their attendance. spearmintt our grand in those settings where those students are at no risk of noncompletion, but we have significant concerns about the resources that we are spending. >> thank you.

>> i'm going to comment really on nicholas paper. most -- that's what the importance of geography and terms of student college choice. about seven in 10 students in the u.s. choose to stay close to home and how important a local secondary -- close second reaction is. percentage of them lives in an education desert. as we heard, such institutions are more likely to receive lower scores or they would be in a lot of the metrics we have been talking about in any accountability system. gooding them less funding

constrain the capacity toward less students. they could even lead them to close. it might be the -- one of the few post secondary options. one of the things i would encourage us all to consider is to emphasize that are in the limitation of post secondary options in certain geographic areas is influenced by the historical context of those areas. this includes historical segregation and discrimination in areas such as texas and other areas of the south. i have often had few post secondary options. theye federal level,

sought to bring an post secondary options to less subtle areas of the u.s.. time.the frontier at the -- theates continue actblishment of the second was required to encourage the states to build separate situations for african americans and funding from the higher 1965 was also awarded in order to continue their important work. one of the things that is important to consider is the legal context. the institution where i work that has been directly impacted by this.

in the late 1980's, texas had in equilibrium funded this institution, particularly not funding the system is into shins -- those institutions that have -- they have also written about this. casee that it is still the that they talked about. they teach students at the institution that they identified. we bring a program out to them so that they can have access to the educational leadership and policy studies. even though i live in a metropolitan area, even though we are located in a metropolitan area, we have to reach out to

them. texas is one of the 25 or so states. we are still held in outcomes. we serve over twice as many latino students. that istant point about there might be of ripple effect even further away from those regions. it does not necessarily in education desert. we're trying to make up for what is going on in some of the counties around us. we also get penalized. that has even greater ramifications. i think that is important to consider in future policies. >> thank you. this session is bringing

researchers to talk to people about policy issues. we have a policy maker here who is in the middle of this discussion and was generously willing to come and meet with us today and talk about her reaction on what she heard and what is going on in the government. we have an honored guest at this session. we are pleased to have her with us today. .> thank you it is good to be doing something with you in the center again. i'm not the only one here working on this project. some of my colleagues are here as well. 's challenge if it has done nothing since august is along with a lot of other work i many of you along the way is moving the focus to better restaurants.

is what question of all institutions are successfully contributing to college completion with a meaningful education for students with the least income and the least a stork opportunity? that is where we are coming at it. i sensed this year that we're asking smarter questions with that as the objective. to do that, we have to ask what practices and investments lead to this positive outcomes. what educational choices and what kind of recruiting and selection process and what financial aid practices and student support and we can just with jobs in the workplace will actually help us accomplish those things. one alternative in trying to squeeze months of thinking and yes to is to simply say

the questions. every question i have read and very in the papers is a appropriate one. you would be pleased to know about the conversations we're having our eerily similar to the ones that we have heard this morning and are engaged with. turning think about how to design methods that would capture the important inferences and advance the policy goal. my own personal test we have whathing to test would be are the places that you think are doing a good job in terms of of quality completions that are well graded in the process? that will be my own version of whether what we do is working. , thenes to be wary about

ones that you say that place is struggling, i would hesitate to let my nephew go there. very practical and logical. is we willrealities and continued conversations what we think is the best possible nomination of the issues that will be published for serious conversation. we will want the expertise of all of you to help us. as we think of the design, when we pick a few items, i want you levelw at a more granule of how we're thinking about some of these. we are thinking that the student academic preparation. given that not all students do the same kinds of things on their way to post secondary

experience, we are thinking of what exists and what markers there might be. is there way to evaluate high school? coats -- zip codes. do they tell us something about the academic preparation in a way that is usable enough to be helpful. i haven't spoken to a single person who isn't well aware that the graduation rate is a very committed indicator. folks like you know it will be better and account better for our time and transfer students with a couple of cycles. what can we do with that? how will it up at things over time? will that solve the problem to get at the population? , thereeneration status

is another element we are thinking about. you could look at postgraduate outcomes. or could look at repayment default rate. looking at whether people work at or above the fair minimum and whether they have been off in terms of their capacity after school, there is an important set of questions and it is one that matters. it is complicated and freighted by existing discrimination in the market lace. -- the further out people are, the more the

earnings might be affected by local and geographical income realities. i don't want it all of you that this is hard work. i want you to understand we are working as hard as you are to understand what those effects would be. we could find something that does have meaning and achieve and incorporate the kinds of things that you are looking at strongly enough, seriously enough, that we get answers to what schools are contributing to ,or outcomes for low income previously excluded people. how do you do that to the extent that there are consumer student thaty counselor desires

people have to make based on their own options and what might happen for student at a particular school and that there are policies and potentially federal investment decisions to be made here it would you do the same thing or different things for them? what would those differences be? when you are trying to do two things, you might need variations for those purposes. there are challenges in using different data. it is likely that we would have some experience with rating and the ability to try them out and improve them before proposals came forward to congress which would have to approve effects on

the way student federal aid or other resources are allocated based on a rating system. our hope is that we would achieve those olds and possibly revise it to satisfy the questions that you could ask and be able to use ratings for several different or business. this is a rare place where someone invests $150 billion and doesn't have any accountability with how it is spent. we have soft subjective informal measures. to sustain national confidence in the student aid system and to be able to continue to justify that we are getting what we want as a nation from it, which is education opportunity and results for all, we need a

political structure of support that is strong enough to keep is putting precious and highly competed funds into student aid. one clear purpose is to help push the state investment in student aid. theral money cannot change decline we have been experiencing for so long. that reorientation of the traditional bargain about the feds and the states in institution and family and philanthropy together is something that we need to improve. maybe we are moving in the positive direction.

something that is overlooked if we think about the dangerous side of an allocation that that is effective in educating first-generation -- first-generation, , werrepresented students look at the negative side of who might lose. i would encourage us all to think about the positive side of having the information that reinforce the to resources and grow the resources to those institution to have the track record that many of you were talking about we put the resources into their hands to be able to do the things that they are doing or to do it for more students or do it with less struggle. being the models were looking for for that kind of success.

this is very complicated work. many of you have been very generous with us and helping us think about these issues. i hope we can count on you to do it as we get closer to versions that we would like to share with you and share with the entire public so that we can achieve the kinds of goals that you are talking about and do it according to the test -- it's sarah, right? that she was talking about. those are my questions as well. those are the benchmarks for us in the system that succeeds in understanding better who is succeeding at these in portrait goals and where students are disadvantaged and where institution should not be able to dissipate -- participate. >> thank you very much. two or threefor

short questions. come down to the microphone. this is your chance. >> hi. i'm from the national women's law center. my question is short. the slide that you showed comparing public institutions and for-profit institutions and the cohort default rate is at zero. could you explain that? >> i forgot about that are. my guess isthat they don't need to report that data out to the feds. if you have an institution that has less than 30 hours, you would have -- that is my hunch. wet speaks volumes about how as you things that we think are pretty straightforward, but are

not. >> it could be zero-100. -- 0-100. >> hi. thank you for your research and for this panel. i'm a reporter with the black dress. my question is more about the proposed standards for accountability. it seems a lot of thought has gone into these factors. any thoughts about helping poorly ranked schools actually turned themselves around and improve? happense have seen what at the k-12 level, but is that something we're trying to duplicate on a higher education level? >> go ahead. >> we have lots of programs in place that do try to strengthen institutions to be able to carry out these purposes. including notion of

improvements of institutions wherever they are if they are on a positive trajectory toward greater effectiveness on whatever measures we end up shooting would be recognized and be protected. it is hard to talk about something that doesn't exist. everything i say -- people have suggested that it is possible that we might. the goal is to have lots of opportunities for people to go to school that will serve them well. our get to -- our objective is not to eliminate, but improve the overall system availability. the more that we can identify the next to die the people's awareness and identify innovations in any part of education that has value for educating the population, we are talking about the happier we

will all be. we ares no reason that not looking to eliminate unless there are places that truly not using federal funding to get educational value for people. departments would be very important -- improvement would be very important. >> that was the last question. i want to take a couple of minutes. we know you have given us your morning and we appreciate that tremendously. very odd full effort to understand and think about what could be done about the accountability that others have made in higher education. worries and the plans that exist in a number of our states and in elementary and

secondary step have produce counterproductive impacts. these different analyses have many in many ways -- the ways of looking at the issues. the geography, precollege preparation, a variety of factorsional resource in many respects. we have a very complicated system of higher education. at this stage of the primary actors in the public sector your. it is a country where the federal government has been the primary after. -- actor. toast amount of money goes the hopes of generations of americans that the children will have the chance to be in the middle class. the stakes are extremely high here.

the reason we did this conference is we want to make sure that they are not being -- there is no unintended harm being done. especially when it is so close to the american people -- do no harm, do no additional harm. think about the ways that you could think positively and contribute. i don't think anyone who has presented today thinks it would be better to have more accountable institutions. it is not a simple issue. some panelists have pointed out , ithe data you want to have doesn't exist. theoretical data that doesn't yet exist, you great albums that are not perceivable. i think what we will be doing as we go forward in this area is to be taking these tapers.

the authors of had a chance to speak to each other and respond and hear your questions and give awful responses, think deeply and revise. we will publish these as it goes through review. certainly appear in a professional volume. it is a thoughtful way to try and contribute to a thoughtful discussion. we greatly appreciate it. we welcome these contributions. i would like to thank the authors and commentators. i want to tell