40 minutes. he is an assistant editor at politico. thank you for your time. appreciate all your participation in the show today. we will be back at 7:00 eastern time tomorrow with another edition of the washington journal. now it's off to the new america foundation, where they are going to take a look at student learning in this country. they will talk about standardized test, accountability, university standards. this is live coverage on c-span. enjoy your day. >> good morning everybody. is kevin kerry. i direct the education policy program here at new america. we all appreciate you coming here on a rainy spring morning in washington, d.c.

the subject of our event today is assessment in higher education. and may be technical seemingly kind of boring topic that i actually think is becoming central to a lot of the discussions we had about american higher education. i say that because i have found that assessment of college student learning is often the place that you end up after trying to deal with a bunch of other naughty questions in american higher education. it's not the thing that people talk about first when they talk about college. our national conversation around college is very much focused on price and debt and cost. when you have a debate about the price of college and you go to colleges and say, why are you so expensive? why do people have to borrow so much money to attend your institution?

when you get back is a conversation about value. often colleges will frame the value of higher education in economic terms, particularly if the question is framed in economic terms. we have become very used to using terms like return on investment. u.s. department of education's publishing information about how much graduates of individual colleges make. from my perspective that makes a lot of sense, but at the same time i don't think anyone would assert that the value of a higher education can entirely or even substantially be encompassed only and a dollar figure. theou are going to measure value of college in some way other than how much a diploma is worth in the labor market, it leads you to wonder how to do that and how you can gather information about how much students learn. if you are dissatisfied with the accreditation system as many people here in washington doc seem to be -- there have been hearings in congress criticizing

or denouncing the accreditors in of thent ways because satisfaction with the way they handle karolyi control -- quality control. if not accreditation, then what? if you want to think about innovation in higher education and new ways to use the federal to givel aid system entrepreneurs or people to ability to create new systems are new ways of approaching higher education but at the same time you are mindful of the potential for fraud and abuse, how does the question you find out whether some new higher education operator is really doing a good job? , just theroadly central importance of colleges and universities. the students moving through them , the very high-stakes nature of that process. and the traditional black of eggs -- lack of a really solid

research base. there is surprisingly little given how many people go to college, given the value proposition and the promise is made for higher education. we don't know all that much i students about how learn, particularly at the individual or department or institution level. invited frederick deboer who is with us today to write a white paper that was released last week about the state of colleges today. it is, where it is going, what it means. what i think you find from reading the paper which is both synthesis of where things are today and a provocative look at where things can be going, you will find there is more out there than people realize. there are very smart people and we have some of them in the room talk about this web

been doing great work over the last decade and more. developing new ways of assessing student learning. people have become very interested in these assessments. the broader idea of using standardized assessments and learning is one that is more familiar to us from the k-12 where it has remained very controversial yet very much part of the fabric of our schools. there is a lot of trepidation. some of it warranted, some of it not in higher education about whether it is possible or appropriate to assist people in a more consistent or standardized means. complicatedlot of technical questions about how to do that and there are a lot of broader philosophical questions about the meaning of assessment and how it should relate to higher education. that's what we are going to talk about this morning. we are grateful for you in the audience and everyone watching on c-span and we're going to

start with a presentation from frederick deboer who's going to talk about his research and the paper he has written. frederick. thank you. [applause] good morning. thank you for coming. i would like to thank kevin and new america for commissioning the paper. it's a great opportunity. i try never to read too much when i present but i do like to hear myself talk. if i'm going to little bit long you can throw something at me. the thing i want to talk to write a why i would paper like this when i consider myself someone who is still within the liberal arts or humanities and someone who opposes the corporate turn in higher education. why would i come to write a paper like this? it is the case that now most of my research is quantitative in nature, but i grew up in the humanities and i still consider

myself fundamentally a humanist. i want to tell you about where this research came from. my interest in standardized testing in college came from my own local context and my life. when i was getting my doctorate at purdue which i completed last may, a controversy erupted over proposed implementation of the collegiate learning assessment plus which is one of the major ofks -- standardized tests college learning today. administrationls -- he is a former republican governor of indiana and is now the president of purdue -- wanted to implement this test at wide gale in the university. they wanted a very large portion of the incoming freshmen and outgoing seniors to take this test in order to monitor undergraduate learning. the daniels administration has made value their sort of keystone word. for example, in indiana it is now dotted with billboards that say education at the highest

proven value. this was bound to be controversial for a few reasons. mitch daniels administration has been controversial at the school. this is for a variety of reasons. maybe the biggest one being that he does not have an academic background, which made many of the faculty unhappy when he was hired. it's also the case that -- the way in which he was selected was controversial given that he appointed all the trustees who in turn appointed him. when he was governor he appointed the trustees that appointed him as the president. the assessment in particular became a linchpin of a lot of other issues that have been bubbling along on the college -- because assessment in a very deep and real way asks what we value in the university. and it is inevitable that assessment will to some degree control learning. i think you can have minimally invasive assessment, that's part

of why i wrote the right paper -- white paper. doubt that no assessment will always have some impact over what happens on college campuses and so the fortion is, was this away the daniels administration to sort of take control of undergraduate learning away from the faculty senate where it has always been invested? that was the proxy issue. the fight was about this test. there was also about faculty control of learning and the gradual the professionalization of the professoriate that is happening nationwide. . when i was doing my dissertation, a theme that came out again and again from the faculty was, we want to assess. we want to make sure things are doing a good job. who controls it? one of the biggest issues with standardized testing is the standardized. what faculty members do not like

is that it removes the local control and local definition of what success means. yet i still think standardized tests can be useful even though i myself believe in and understand the desire for faculty control and local context. -- i spent the past five or six years becoming versed in educational measurement, statistics, metrics with an assessment. -- within assessment. i have done a lot of work. i came up with a very conventional liberal arts background. i got a ba in english and philosophy. 10 or 12 years ago i was reading thomas hardy. these days for some reason i'm staring at a spreadsheet all the time. i wanted to acquire this basic literacy and quantitative skills because it seems clear to me don'tome people -- i

think everybody should be this way, it's not even a large proportion of us -- some of us within the humanities have to be able to speak the language of numbers and the social sciences because that is the language of power. it is abundantly clear that the policy world speaks in a certain kind of language and that language is statistics, validity, reliability. i became concerned that that was not a skill set that was in the hands of most humanists. one of the things i did up finding in my dissertation research was humanities professors were complaining that they were cut out of major policy initiatives and think tanks and commissions -- but the question becomes, what would you talk about if everyone else is talking about numbers? that's a bigger fight. but that was why i'm here. myself aaintain to core belief that i'm pursuing what the humanities are all about. anyway. the story of purdue had kind of an anti-climactic resolution. it became a very big local

controversy. the local paper ran a front-page story that said, daniels and faculty in battle of wills. which of course made that true. if it was not a battle of wills before as soon as the paper said it was, both sides dug in the cause they had to save face. theyended up happening is delayed -- they did what universities always do. they had committees and subcommittees and those subcommittees had subcommittees. it was a delaying action. the students would not sign up to take the test. you had this big battle going on. one of the major impacts was that -- you could not get an adequate sample size of the size that the administration wanted. they are still going forward with it at the school. but in much smaller numbers than they had originally proposed. 18 and 22-year-olds are not eager to take a standardized test that they did not have to take in the past. that's one of the issues.

the controversy was really interesting and important and we are going to see these debates play out in many schools across the u.s. because assessment is not going away. of presidential administrations, both george w. bush and barack obama, have had educational officials that made a strong call for more standardized assessment of college. this is an issue on which the -- who will does what and the dynamics of how it's going to happen -- there is great agreement between them on this issue. people within higher education cannot just close their ears to this. was particular plan eventually scuttled, but the obama administration is going to performance on standardized assessment and college rankings to availability of federal aid. that is a very big stick in duty. that is the kind of thing that no college, even really elite colleges, can afford to ignore.

going to move forward as a system of higher education that can define what's going to happen to itself rather than have assessment happen to it -- in my field in composition there's a guy named edward white. whitelaw is assess or be assessed. perform yourself and you are not eager to get involved in assessment, you will find assessment being done to you. the reason i'm so invested in this is for that reason. i think faculty can take an active role in assessment. we can get out ahead of these problems and we can become a major force in shaping how assessment happens. in faculty simply says, we are not going to do this, it's going to happen anyway and it's going to happen in a way that does not reflect faculty interest. i think that's reality. some people see that as fatalism. i want to make a few things people -- clear to people within the humanities who disagree with me. assessing in many

ways. there is already all kinds of assessment that happens on campus. many of these are ad hoc, idiosyncratic, lacking in validity and reliability. they can't talk to each other from one campus to the other. the entire process of the accreditation has supposed to have an assessment function. the idea of assessing colleges is not some new neoliberal enterprise. it has always been central to the process of accreditation. fact that the accreditation process is seen as toothless is not really a good thing. if we are going to have a process it should mean something. the fact that so many schools have become used to a context in which the accreditation process does not threaten them just means we need to reform the system. rankings like the u.s. news & world report rankings -- in a very real sense, that is an assessment. i think it's a very wrongheaded

kind of assessment and i think it's casually distraught to what we really value -- destructive to what we really value on college campuses. they are saying, let's assess the quality of these schools. they pull out -- put out rankings that many students and parents pay attention to. elite.ly perpetuate the but there are many things elite colleges do well. harvard, yale, stanford -- they do a lot of things very well. but the notion we know that they teach undergraduates better has no evidentiary basis. the sense that harvard is the best university in the world has almost nothing to do with how well it teaches undergrads. that data does not exist. because they skim off the top and take only the truly elite high schools dunes, you could probably put those students into any university and see them excel. so when they report help other students do after graduation, you can fairly ask, does that come from your undergraduate education?

i went to public high school. the local private school always bragged about standardized test scores that its graduates got. they would ignore the fact that you had to do well on the standardized test to get in. it's like having a height minimum and then bragging about how tall your student body is. that is the system that is perpetuated right now. a lack of assessment data allows elite institutions to maintain the fiction that they teach better without having to provide any evidentiary basis to do so. this is one of the reasons why assessment is a social justice issue. if we want to make colleges true vehicles for equality we have to be able to generate data that says, what is perceived as being an elite university has no solid basis for saying that it is. every year there's a controversy about one school or the other slipping or rising or falling a little bit in the rankings of the u.s. news & world report, but it's not like harvard is going to show tomorrow and be at 50. that doesn't happen.

just shuffling around these very elite institutions at the top. it's also the case of higher education is under a really profound threat right now. we have had massive amounts of defunding at the state level. we have a lot of people in the policy world who think that physical colleges and the traditional liberal arts should be replaced altogether with online only programs and certifications. i think it is profoundly naive to think that online education is going to sweep in and within a generation we will see a reduction of colleges of 90%. that is something i read a lot about. thing i think that underestimates the persistence of institutions and the inertia of how hard it is to change institutions. i also think we do a much better job of educating than online only education. i think what limited evidence we have now suggest that that is true.

that, we going to say have to generate data to say it. the same people who tell me that we should not be assessing hate the idea of online only education. they hate the idea of the demise of the liberal university. if that's the case and you need to be able to say to the rest of the world, we do something very well. to assess more in college and i think this can have a lot of benevolent institutions. the message should be to everyone involved, we take a lot of resources from society. college is very expensive. we are draining hundreds of thousands of dollars on the backs of young people who graduate into an uncertain economic climate. colleges receive hundreds of millions of dollars from the federal government and we have invested college with this unfairly,- somewhat but this function of being a key linchpin of having a healthy economy. there is no enterprise in the world where we pay hundreds of millions of dollars and no one bothers to ask how the we are doing. besides the defense industry.

so i think we can do more assessment. but i also agree with skeptics that this kind of assessment is very hard. one of the things that i wanted to say -- it is useful in a context like this in the policy world. these problems are not just political or theoretical in nature. of beinge frustrations a humanist and talking about thessment is when i'm in other world, the policy world or the educational testing world, the assumption is that resistance to these instruments is always about political resistance or the self-interested resistance of the faculty. that when we say these are hard to do its really about, we just don't want anyone to check our work. from a simple matter of empiricism and social science, it is actually quite hard to run these kind of large-scale assessments. there is a lot of data cap that we can collect --

we know for a fact that colleges have profoundly different populations coming in every year. the reasons elite colleges invest so many resources in their admissions process to make it truly exclusive is because they know that it works. theou pull the cream off top who are to go on and excel. true of these is standardized test in colleges that the best predictor of how well both your freshman and senior populations will do is their sat scores coming in. we know very well with a great how mostertainty colleges are going to stock up in rankings simply based on how they stack up on their average sat score. andeed to use value models things like that to correct for differences in income and populations. is a fantastic public university with a great student body. we still except about 60% of our applicants. which by the way, that is

actually quite a leap in context. i don't think those people understand this. going to an exclusive colleges extremely rare. this is one of those things that people in journalism and the policy world underestimate. there are over 3000 accredited four-year institutions in the united states. maybe 125 of them and that is a generous estimate reject more students than they accept. the vast majority of colleges are taking a majority of applicants and most colleges take essentially any student that applies. economic to for pure reasons. if we compare colleges, we need to bear in mind that they have different populations and we have to correct for that. there's also all kinds of issues with scaling. we know how to address those things pretty well. but let's be clear that they are empirical problems, not just political. they are not just self-interested professor saying we do not want to be tested.

it's a false choice, is my major point i want to make between a invasive testing, testing everyone all the time, constantly testing students, ,aving them teach the test dramatically making college -- or no assessment. my great frustration is the conversation so often boils down to, either we enact something like no child left behind for college or we continue to do almost nothing that is replicable. that is a false choice. i believe we can have minimally invasive college standardized assessment that can still provide a lot of the data. we don't need to test all the students all the time. one of the frustrations with k-12 debate is it's based on the premise that we need to do a sense of style group testing. -- census style group testing. almost all the students are tested all the time. this is part of what parents

hate about it. of inferentialer statistics. one of the things we know how to do very well is form stratified samples that are representative of a student body and extrapolate from that sample. we know how to do that. we can take a sample of students from the average college and make sure it is adequately and we can develop a sample, have students take the tests and understand very well how the student body is doing. that's an ability that we have and it is frustrating that that ability from people who push more testing and who resist it is often ignored. we don't have to test everybody all the time. we have the beauty of stratified sampling. it's essential to involve faculty at every stage of the process and that is true whether you like faculty or not. there is still some power invested in faculty coming even at institutions where the higher administration has clawed back a lot of power.

at purdue for example, president daniels has a really strong mandate. although i'mthat critical of him, i have agreed with a lot of his initiatives. he has a strong mandate, he has the backing of the board of trustees. some people can see that he is still the most powerful politician in indiana even though he is not the governor anymore. he still was unable to just hen't want -- implement what wanted by fiat. my recommendations for the faculty control disciplinary assessment. -- one of the things that bothers me about it is they didn't even really try to assess disciplinary knowledge. tose tests that were going talk about today are tests of general ability. often defined as critical thinking or academic aptitude. they don't attempt to assess what you learned in your major classes. a are not even attempting to

say, how well did a nursing major learn nursing? how well did a computer scientist learn to code? you can't have a standardized test that works all the way across the institution testing major knowledge for people that are not in that major. when people say there is limited learning and college often they are talking about not looking at what many people consider the most important thing you learn in college, your major. that is an excellent way for us to involve faculty. will always control disciplinary assessment. the computer science program gets to decide success for computer science. we need that to be more standardized and interoperable. we need to be able to talk with other kinds of assessment. that's a great way to involve faculty. let me close by saying this while i have the attention of the testing industry. havenk that we have to greater access to data and information and mechanisms of standardized testing and instruments if we are going to implement these.

we need to open up the books on these tests a little bit. to their credit, major tests developers do a lot of internal research and they have proven themselves willing to publish research that is critical of their own instruments. dts -- they are very progressive and having independent researchers who investigated or internet and say, this is a problem. they are very willing to do that. internal research can never replace truly external review. even the most principled researchers can audit themselves. information is made available by testing companies. it's my opinion but not nearly enough is made available and what is available frequently has two onerous -- too onerous requirements for access. i was taking a seminar in length which testing and i needed a data set. my professor said, you can use this that i have from a testing

company that will remain nameless. thought it in and she it was very good. i said, i would like to publish this paper. she said, that data set is a proprietary data set of this educational testing company. here's what you have to do. you have to write the paper, submit the draft to the testing company. the testing company will decide if they want to review it or not. if they decide to go through with the review, that process can take six months. they will send it back and say, this has to change. back forit and send it review again. if they thought my reviews were good enough they would send it back to me. then i can submit it to an academic journal. the academic journal again would take her typical three months or to referee t paper. they send it back to me with the recommended revisions. if i made those because the draft had change, i then need to submit it back to the testing company. the testing company can sign off on give me revisions to the revisions.

once the testing company is satisfied, then i can submit it back to the journal but i would have revised it again in ways that the journal would. have advised me to use of they could come back and sell me again. it could easily take three years or more. time isreer academics' of the essence. if you are a grad student you need to get published work on your cv to have a job. is a powerful disincentive for people to publish. there has to be a better way to do that. think that it's possible -- the concerns of the testing industry are number one, test security. the fear that giving people data will make it easier to cheat on the test. and industry trade secrets. think we can provide data and still address those things. test like the sat people have been trying to game it for years. is still remarkably durable to cheating.

whatever princeton review are and their test prep company tells me, they have not been able to demonstrate they have really named those tests. -- gained those tests. the core of assessment is saying, trust us is not enough. universities are saying, trust us, enough. is true for the faculty it has to be true for the testing company. in other words, they cannot expect to trust us either. i think there is all kinds of data you can put out there. multiple-choice, external resources should be able to do a external item analysis. stuff that real test developers can tell you about better than i can. i think it is very appropriate to provide research collection of real assays that have been generated and scored internal to

the organizations. prompts used. make it readable. ultimately what is appropriate and fair for test companies to use and put out there is a negotiation. they will air on the signs of getting less for more. this can help the test companies, to. faculty says we cannot look a hind the curtain. i am an expert in educational testing or expert in developmental psychology oars statistics. a professor site may -- a professor may say why cannot i amat your mechanism? going to stop talking now. close by saying the

notion we have to either have no assessment or high-stakes professional regimes that dramatically changes the university is just wrong. there is plenty of opportunities for us to gather data and make it publicly available to better understand how our students are doing in college. we can gather information and there has to be external .ressure on teaching it is the institutional response that institutions never change. we are in crisis and need to change. thank you. [applause] >> that was terrific. mentionedmarks you the instrument there.

controversyll-known a trip. the man who is responsible for the cla plus. i ask you to note we invited alberto i'll schrader to speak today. he was on his way down from new jersey and there was a problem with amtrak and will not be able to make it. he did send the great remarks. we can be a little tougher on amtrak, not that we would not have been anyway. i would like to start by hearing your thoughts. >> i enjoyed it. that i am thesay , a politicalae

economist, and also, i was an academic for a long time, and still think the title professor is the one i like the best. was the dean of arts and sciences. i view myself as fundamentally a product of global -- liberal arts. i am married to an distinguished arts historian. her world was illuminated french manuscript. she would not bother to attend this subject as she totally world. and -- on that let me just say on behalf of dts , one of the problems we have is measurement scientists, nutritionists, there are only a handful being turned out from

iowa, iowa state, illinois, minnesota -- places like that. yet, as kevin noted in the intro, increasingly assessments are being recognized as a very important subject. numbers,o not have the let alone the polity of people in that field that we will need going forward. the importance of ets cannot be overstated. i think for example the director kline, who, steve persontired as a young worked under sam messick under them metric sampling approach. pretty much the gold sampling for the kind of test that makes

a lot of sense. on was alsoles important. he has ties to stanford. ets is doing excellent work. lydia lou a distinguished researcher working with assessment and higher education. , who used to be a colleague of mine when we were at rand in the late 1990's now has a chair. now he knows a great deal about the issues that frederick was talking about. i want to say a little bit. paperttom line, frederick is a serious critique, and poses

important. what i want to do is make a couple of comments stimulated by his argument, and then i want to all you a little bit about new way we may think about framing research on higher education. leading perhaps with educational assessment but going beyond that a little bit. i think accountability did have a lot to do with the ramp-up of in higher education. and talking about the spellings commission and so on. group,case of our

concern was the principle of motivator. steve and i and other colleagues began to think about introducing introducing higher education because we felt it was a good idea. steve had introduced performance assessment to the bar association via the clinical part of the bar association, now all the major states in the late 1970's and light -- early 1980's. we thought in the knowledge economy it is very important for the next generation to be able the critical thinking skills and use information , and youou can google really need to be able to be stronger, critical thinkers.

the board led by richard acted adkinson, michael archimedes tucker, michael kroll and others, really believe that this was a worthwhile undertaking, and we still do. we focused on avalue added approach, because we thought the improvement question was a good way to start or post secondary education. we were starting to do research development. so we decided to really take the value added approach to scale.

it is a worthwhile endeavor. it turns out there is a point for four standard -- deviation testowth across 1400 , and that is a very important growth and social science. controlling and social sciences now an ect, and propensity matching model we have created. the amount of growth in similars -- similarly situated .nstitutions there are about 20% of the

colleges that really demonstrate best practices and important ways. obviously those in the middle and those at the bottom are somewhat problematic. about four years ago we began to develop a version that was reliable and valid at the student level. sticks -- no stakes approach did not cause motivational problems that affected the institutional level results, but it did affect motivation. so we have created badges. we have career connections and employers beginning to actually

ask students whether or not or indicate they would be thesested in getting results. so we are trying to solve motivation issue that way. we could havetrue much more reliability and validity data. we truth of the matter is have a critical thinking assessment performance, and about 100 external publications. ca of this published by papers. there are many dozens of them. there are several independent reviewers that take a look at them. that is on our website.

now, the idea has a label of going forward. giving you something to think about. name of a paper i have just drafted. if you are interested, i will leave my card somewhere and will be glad to send you the draft. donald stokes wrote an intriguing book called testing quadrant, basic research and technological change. the time reverses honored assumption that basic research drives applied research . that is the way it goes. he has numerous cases in which practical problems drive research.

he was passionate about doing something about lactated milk that was killing millions of .abies along the way in his professional career he invented the building block for microbiology. folks himself talks about an important democracy. i came up with three or four historical cases in which this approach focused interdisciplinary research, reusing the value system of science, peer-reviewed and the ability to replicate results as principles. we in the middle of the civil war congress founded the land grant university, and the persis -- purpose was to make agriculture more of a science.

we have done a very good job of that. the green revolution, etc.. there are complaints about -- modifiedetics genetic crops today, but i think it is a success story. after the success story 1910, there was commitment to make much more of a science. that has been a steady climb. the recent decades with molecular biology, i think basically, the tide has turned. there are tensions between the clinicians and practitioners and , but again, it has been a good story. finally, rand, where i was for a decade, there at the end of the war, where we got worried about

the soviet, congress founded and, and the goal was to task group of researchers to come up with a better group of decision-making for national bettery to make decisions. in the first decade, they invented game theory, cross benefit analysis, and the prototype for the internet. not bad. argument in say my the paper is higher education is the next obvious candidate for this kind of approach. why? first of all, i think there is increasing agreement human capital is the principal resource. this is the former venue for

preserving and enhancing that for preserving and enhancing that capital. there is therefore reason that policymakers at every level will be more and more focused on how to make sure the system and proves. the hired sect is very important standard forts the teachers to move toward. that is why it is important. what evidence of problems are that make it warranted specifically? i really was not going to them but i will say it is the usual sus ex. curves in the the consumer price index in the growth over the past 40 years compared to cpi, it shows almost a cost disease problem almost

abundantly in evidence. therefore, any ended -- any increase in funds, those inflation and the primary symptoms we read about, student loan debt is a huge problem, and one that i will mention that i think is very important, and that is almost unnoticed, the demise of the tenure-track and running the sector. or less of have 25% the whole sect that are primarily inand the top institutions. department government is the way we said about organizing the sect there's, decisions about what to teach, recommend to teach it and who to assess. a problem.

addition, you have the equal opportunity. that is something i am very very -- i am very interested in. you do not have an eight plainfield with students to have a crack at the good jobs as they leave college. studentse 30% of the who graduate from the selected colleges score in the top five or 10% and only 10%. when you do the math, you are talking about 1,000,500 students in the less selective colleges. students that should be given

opportunities. so there is an inequality problem. that is a huge issue. it is a major public policy issue. , a big debate about whether higher education is public or private good. and it is frozen. both sides not moving, it is becoming what we call a common core problem. are not aggressive. that is my argument. -- when you talk about and bring your research to bear, higher education leaders and faculty are worried about certainty and causing negative publicity and other budget issues and so on, and there is something i call -- the department-based government and something i call a double-edged

sword. this is one of the principal reasons we have the greatest research universities on the planet. anythingt want to do to disturb that. , the samee time principles mean the faculty in the department view their role as determining what subjects to teach, how to assess it, etc.. third-party interventions from are not looked at favorably. think not only to really consider upping the game to review this research inspired game to consider the field of inquiry that share the value of science,

just not just have one but data analytics, and , reallynal technology withe very helpful joined practitioners who are very important, a point that frederick makes, to move forward. the principles for engagement that i will simply assert are the incident -- principles of peer-reviewed and transparency you need this to be able to replicate results. to joinly or impossible what we've done in our altar or security?

i way of and it wrote, a couple of years ago and was talking to a man named carl weinman, i think the director of the white house council on science advisors. was that your title? something like that. so he is won the nobel prize. he decided to do something really hard, how to do a better undergraduateg

students. like a lot of people that come to washington from the outside at some point decided i would like to get something done before i go. the thing you decided to get was an idea of requiring any university that took federal research money to report back how they were assessing the quality of instruction and undergraduate students. i say to them, as long as we're ideas, why not have exam to thee same student,uate physics on my idea that maybe it is one of the less committed realms and report how they are doing. he just laughed at me and said you are a fool for even bringing

that up and i said site that, it to peter ou in this effort, which was torpedoed anyway. the resistance of -- among political.not just i do think it is sometimes as if you take learning seriously and want to do a good job, you have to recognize how complicated it is. are there opportunities in some realms where we could move ?elatively quickly where are the opportunities? colleague.your this is the national physics

knowledge test. this is the kind of local control i want to maintain. to make this a case for local control, some fields people will acknowledge off the bat as a political component that influences the decisions. maybe one of the first controversy was over howard's and, a controversial figure of history -- ho physics knowledge test. this is the kind of local zin, a i want to maintainward controversial figure of history. you impose this from the outside to impress upon this people. but look at a field like computer science.

of computer science is one of coaching. it is a classical layer of computer science when in fact it is filled with theories. these differ enormously in terms of their approach to theory and practice in the kinds of languages they think students should be learning. within individual departments you will find professors who have very passionate disagreements about how to teach computer science and the best way to prepare students for the work race of the future. that is the top-down thing i would like to avoid. the american historical association or something like that, the professional that, the professional organization of histo-e % a%n tm guideline interior to a discipline. so say that this is the document we believe in. within this means an outcome statement. opportunities to specialize in different ways framework, and when we are assessing we are assessing based on this

framework. appropriate and possible to determine on this scale. is the most powerful force in higher education. this is not happened in the past and will continue not to happen unless someone provides incentive. you mentioned the name. a lot of times people ask me what i want. i said we have all of these problems with standardized test, manages to nape avoid almost all of them. no kid is crying because he had to go through yet another round of the nape. teachers are not complaining that they had to go through another round. the pearson company does not exert too much control over k-12 curriculum because of the and thee in the nape testing materials they put together. again, that is relatively very

numbers of students taking it. the same students are not taking it over and over again. aboute very powerful data what is going on because of that test. this: have we backed into in college? >> no, because that focuses on non-18-20-year-olds. there is of course -- oecd has still an and there is renewed to is being develop higher education version of that, which is another

subject, but we are involved in that. we suggested years ago when we got started something like the college nape. they said something like in your dreams, you have lost your mind. they were not happy to go down that road then. probably not now. these problems will continue to grow. foring about the need standardized assessments with anything that has fixed -- stakes attached to it, because if you cannot see the results of the test as being reliable and valid, how can decisions about it? that is the problem. and yet, frederick has correctly identified the importance of

staying true to academic or relative account -- economy and local control for assessment. about that. mitch daniels in some ways almost an extreme test case of a politically empowered repressor. possible to envision a small, notwell-known college having much power but of powerful institution like purdue you cannot come in with more backing like you had. it ends up being a negotiation with the faculty.

if you are serious about the teaching this will help you be better about the job. >> one of the frustrations about one who willno project the assessment will write things down. u.s. world news report? no, i hate it. the reason it existed is because students do not feel they have information about what is a good college. as productive as they are and as many parts of the instruments i disagree with, if you are an 18 or 19-year-old you are looking at this one is three or seven and i know which one i'm going to. u.s. world news report? no, i hate it. about who and whyted is becausk

you go to college, it is important to remember the conception of college is very much bound up in elite school where you go to school in the northeast and all your peers come from all over the world. for a public university the median distance from home to school is 18 miles. college students go to school within 30 miles of where they grew up. geography is number one. if they are not attending since college, i am someone who is complained about the dorms. part of the reason tuitions so incredibly high is because we're building things like at purdue $95 million gym, which

construction began in the heart of employment depression. gym, we hadmillion a pool where you can run against the current. a 65 foot climbing wall. many colleges have dining calls with full-time sushi chefs. lazy river rides. many dorms are based upon the idea you can have as many singles as possible. they did not go to art school, they went to competitive institution. it is not because of the faculty student ratio, a made-up number

that means something. visit, the dorm looks nice, the dining hall looks nice. and 19 years old, you do not have the most long-term thinking in mind and would be cool to have the vertical certificate. 23 may be 73 as far as a long-term plan goes.

college fixes some of this album. one of the lies that we used to be good at educational testing but as long as there has been international testing, the u.s. has done poorly. motivation is the question. they did a fantastic paper on what the two groups control and experimental. one group of students was told that the test score would follow them so it would go on to transcripts and employers who graduate schools