school reserved -- received a prize of $750 for a documentary on the national debt. and a student won a prize for his documentary on marijuana. at thomas edison middle school, a number of students won honorable mention and $250 for a group. given an awardas for the national debt, and students one for their documentary on heroism. also received honorable mentions for their documentary on global warming. thank you to all the students who took part in our 2017 studentcam documentary competition. to watch the videos, go to studentcam.org. and studentcam 2018 starts in september with the theme the constitution and you. choose anyents to provision of the constitution and create a video illustrating

why it is important. former judges from the colorado and indiana supreme court's joint and indiana university law school per hazard to discuss the future of law schools. this panel was part of the seventh circuit bar association annual meeting, about an hour and 20 minutes. for makingand thanks time on the program for this conversation about the future of our profession. as brian welch said, i am randy shepherd. i ended up spending is my life as chief justice of the indiana supreme court. i now sit doing senior service in our intermediate court, and i have an appointment over at the indiana university school of law. corliss,ht is rebecca who after a career as a trial judge an entire practice, spent more than a decade on the colorado supreme court. she left there perhaps 10 or 11

years ago to be the founding of the institute for the advancement of the american legal system. she does all sorts of research and programmatic work on the improvement of courts and the improvement of legal education and of the legal profession. and then professor william henderson, certainly a man of the circuit, if there ever was one. after his legal education at the university of chicago, came a clerk at the seventh circuit, but happily has spent his recent career at the indiana university school of law. nobody whohere is has done more intriguing work about the future of our and becky than bill corliss. i am going to begin with words of setting the stage. there are many things about

modern legal education that we ,s lawyers know a little about but not necessarily know about in detail, and they provide a considerable foundation for evaluating current state of law schools both now and in the future. now, you could actually describe recent developments by using just eight words. the eight words would be "rising tuition, massive debt, fewer jobs, and fewer applicants." i will say a few words about laying those to begin the foundation for our conversation today. no mystery to anybody that tuition in higher education has been rising. indeed, it has been rising for the lifetime of virtually all of

us here at a rate that exceeds twice the consumer price index. in fact fact -- it has been rising at a rate so rapid that the finance officers of the american university systems have created a new measure. it is called the higher designed price index, to get a more accurate reading ought nowh what really be called the sticker price as opposed to the nominal price, but how price the sticker price has been going up. that, say,decades just ended, late in the last century and the first decade of this century, the general rate of inflation for higher ed was , actual adjusted for inflation, cpi. 71%. at the rate in law schools was not sent a 1% over that period,

but 317%. it has been rising at an average rate of 8% a year in recent decades. led to course has stories we all read about the largest amount of debt that students, both undergraduates, and, of course, law students, experience, rising to the point where americans now owe more in education debt than they owe on their homes -- really quite a stunning change any american debt picture. the average law student that on the way out the door is about $85,000 in the public schools that,out 50% higher than 100 $22,000, $125,000 for students who have gone to private schools.

this continues of course that in general to be easy to read about. last week in "the wall street journal," a new week in debt. there is a program called parents plus, the front page of the second second chama about the opportunity that parents have been given to empower their children to borrow more money if the parents would cosign, and then the fairly rural government would enter a guarantee so more money could be borrowed. it is now clear that parents plus is having an effect on the disposable incomes of, if you will, working middle-class parents, just on their children, that on the collection of parents. so it has become the pervasive part of what it means to be not only a university graduate, but a law graduate. jobs. fewer

one of the things we as lawyers did not notice, unless we had a child or a very close friend who was looking for work over the last two decades, is that we --e been turning out j mor j.d.ing out more graduates than we had lawyers in a long time. the reason we cannot see that is the way in which employment for new lawyers was reported did not do a very good job of telling us where they were employed. it simply told us by and large whether they were employed and for what kind of organization they were employed. so it was quite ordinary as recently as, say, 2000 two read announcements once a year from the national association of all places or the american bar association that would say this was a great year, 90% of our

graduates have found work within nine months, now 10 months is the measure. and that sounded like a very happy result. well, as you will recall, there were a series of scandals about the way in which this was reported. one of which of course occurred next door in illinois. our friends in illinois will particularly recall. the result is the change in reporting system asked tougher questions. in particular, how many law findates, license holders, a full-time ongoing job for which you have to have a law that is the hardest way of asking the question -- and in the last half decade or so, the answer to that has been last year it was 59%. that is to say, of all the people who turned out of the school, how many people found ongoing work full-time for which you had have a law license, and

the the answer to that is 59%. .ow, that number has stabilized it has been as low as 57%. stabilizedber has over the last couple of years. peoplel means a lot of do not find work for attorney general's offices or prosecutor offices or private law firms. the other reason it has stabilized is the number keeps declining. the way you calculate the percentage gets altered. the last fiveer or six years, the number of jobs , full-time, license required jobs, has continued to decline as recently as five years ago, something like 20,000. 40 245,000 --ting

40,002 45,000. that number this year has fallen to 23,700. decline perhaps that earlier in the decade but still a small number. seniors and second-generation people and advisers are no dummies. been a dramatic change in how many people apply to law school. decade, the normal number, and it was the highest theer seen in the county, high was 100,600 applicants. applications is a funnier number so it seems to me more important to focus on applicants.

likenumber was something 100 thousand, 90 8000, for quite a long time. dramatically and in 2016, the number was 2006 -- 206 -- 200-6000. 206,00. 0. down from last year about 1%. i think that number today has probably stabilized. but there are things happening inside that number that are attractive. for instance, for people who actually get into school, those who come into school with stats are half of what they were five years ago.

a number of people who come in under 150 is up 80% from where it was. there is a noticeable shift in .he metric, the easiest word others might say talent or suitability for law school. the other thing that has happened inside the number is a change in student legal aid. as the competition for law increases,, changing the numbers, as would like to ine the smartest students law school. the amount of merit-based aid over the last decade has gone up 10%. there is a third category called both merit and need-based.

i have not yet spoken to an -- al law and -- officer most of that goes to the people with the best master -- best metrics. the post at the back end of the line and up with a far greater share of debt. i have been particularly worried about the effects of this with minority applicants because their profile and gpa is not as good as asians and caucasians and hispanic applicants and african-american applicants have a lower metric. it is harder for law schools to hand them scholarships or financial assistance, if you will. better thought these days as discounting.

we have schools in the circuit who don't charge any tuition at all for a substantial number of students. few schools who sends the students money. they not only don't charge tuition, they write checks to twitch and. it partyou do that but of a greater trend. week, no great surprise they have likewise been falling. february, not as good a measure as july. the february rate was the lowest we've ever experienced. an important on note. the number of people in the academy who say this will pass, is a shrinking number.

done by various entities with which professor henderson is associated, about how we change legal education to what it looks like professionally is very encouraging and substantial. the easiest to see is the change in what is now referred to as experiential learning. how many actually hands-on experiences do we give to students is dramatically changing in the last two decades because schools are determined to do as good a job as they possibly can. we will hear some intriguing news about those trends. we will start with justice corliss. professor henderson and i have to stand because the

monitor is only visible here. i apologize raking up that trio and we prefer to be seated. as you know, i have a background as an appellate court judge but i am running a research interest -- institute of the university of denver that we refer to as a do tank -- it is the worst acronym in the world and i apologize. by a federal judge and he would not let me put justice in the name because he thought justice was a term that was too susceptible to the interpretation of the day. in four different areas. i have spoken to this assembly

in prior occasions. we do work in legal education, civil justice reform, the family justice side, and then we work on judicial selection and performance evaluation. our process as we gather research and put together wonderful groups of stakeholders . we then facilitated and monitor the implementation of the models and we measure outcomes. the problem here we are talking about this morning, chief justice shepard has already identified, which is that in the context of which -- of education, we are struggling with how to prepare lawyers for

the profession as it evolves so we can revitalize the numbers in the role of lawyers in our society. these are the drill down numbers to it she has just referred. this is the percentage of 2015 law graduates who did not land full-time employment regarding bar passage. we just chatted and we think the 2016 numbers are due out in the next couple of days. i apologize that these are yield -- a year old. these are the best set of numbers we've had in a moment. percentage of 2015 law grads who did not land full-time employment recognize things that are law degree. 25% did not land full-time employment including

professional and nonprofessional. these are the priest in numbers. loss -- law students who believed they had sufficient skills to practice. this set of numbers is really interesting. lawercent of 30 year students who think they are locked and loaded and ready to however,aw practice, the law professors who teach and,say that that is 45% wait for it, the percentage of practitioners who believe new lawyers have sufficient skills to practice is 23%. there is a problem there. we developed a project intended to shed light on ways to address this. we created our foundations for

practice project. we have support from the hewlett foundation. this is a three-phase project. we put together an advisory committee consisting of practitioners, judges, justices, representatives, representatives from the national association that's the national conference of bar examiners, representatives from the public and even thefice, attorney general of colorado. we developed a plan. the first piece of the plan is to identify foundations that entry level lawyers need to practice. the second phase would be to develop measurable models of education that support those foundations and the third would into align market needs

order to incentivize positive improvements. surveyne, we developed a based on existing research for with of you, such research the assistance of our advisory council we distributed that survey across 37 states in the country. the bar association is in 37 states and distributed the survey. little bit of crossover of people who practiced into the states. we got 24,001 hundred 37 valid responses to the survey.

if there are any of you in the audience who answer that survey, thank you for your time. it was an undertaking because itself mostly five minutes long but i got over 40,000 actual responses and 24,000 valid responses. base mirrorst legal professions. some of the numbers are not wonderful in terms of demographics, but in fact, the response base is representative both geographically in terms of the size of firms in which lawyers across the country practice, the amount of money and then, demographics. we asked about 147 different foundations in this survey. andategorize the foundation

they broadly fall into three different categories. some of the examples are identified relevant facts and legal issues, critically evaluate arguments, conduct and listenedpositions, respectfully, make decisions, .ork as part of a team, these three categories in which we divided the foundations are as follows. characteristics, and competency. 100 47 foundations and we are dividing them into three categories. 27% of the foundations were skilled based. 28% were characteristic based

and 45% competency. graduated the questions we asked of respondents. we asked what is necessary in what ist term, we asked advantageous but not necessary we asked whaterm, is necessary over time and we asked what is not relevant at all. it does not pick up grace so it looks like i'm missing a cylinder in the middle. fortunately, not on your screen. here are the results. were 77 foundations identified as necessary in the short term by 50% or more of respondents.

foundations, 77 were identified as necessary in the short term by 50% or more of the respondents. what emerged surprised us. we call it the character quotient. here it is. 20 of those top stone identified as necessary in a short-term, nine are 10 oureristics, competencies, and one is the legal skill. there it is, research and the lot number 14. the 10 competencies keep confidentiality, arrive on time. this is a melodic -- millennial issue. arrive on time, treat others

with courtesy and respect, listened and death attentively and respectively, take individual responsibility. speak professionally, right professionally, exhibit tax and diplomacy. characteristics, the other nine, honor, commitment, integrity, trustworthiness, diligence, a strong work ethic, attention to common sense, intelligence, and a strong moral compass. this as the whole lawyer cumulatively. about legal skills? this is extensively why we go to law school. legal skills show up in the must be acquired over time set of responses.

these 20 characteristics are the top 20 or the foundations are the top 20 that must be acquired over time according to our respondents based. here they are. riskpriate risk minute -- strategies. depositions.efend and participate in mediations. all of the skills i suspect all of us what identify either on a transactional or litigation side of practice as being necessary. the takeaway is we expect people to acquire them over time and we do not necessarily expect them to show up at the time that they sign up for work ready to do these things or being qualified to do these things. where it becomes a

little more relevant to all of you. in the second part of the survey, we asked respondents to ansider the helpfulness of set of hiring criteria in determining whether a candidate has the foundations they identified as important. given the fact that they already answered the survey and identified what they thought was important. then we set how would you hire in order to snuff out the individuals who had those foundations? we did not ask them how they currently hire. oliver went to yale and harvard and was editor of the log review and get a summer clerkship.

is this personn would get an interview pre-much anyplace in the country but does this person really have on his resume something that would suggest that he has those foundations that people said they were looking for? what would people focus on if they wanted to hire the lawyer with a character quotient. 17 people and we asked respondents to rank them from very help to very unhelpful. criteriaok at the 17 and i will show you how the results worked out. here they are.

top of the list, legal fromyment, recommendations practitioners, or judges, bottom of the list, log review, and journal experience. respondents. note the first eight have to do with experience. what the respondents told us is that the character quotient or the whole lawyer metric emerges through experience. we broke out the data by government employer, legal plusces, with firms of 100 ofnd law school, with firms to-100, a little bit of a shift

in the top eight, particularly the second floor. it is still the same top eight. in smaller firms, solar practitioner settings, specialized loss school classes move up because there is a for actual demand ability to jump in on a particular topic. government employment is still the same, a shift in priority. , you see aces particular lawsuit in a criminal area. ast does all of this tell us we are thinking about the future of legal education and the future of employment?

first, it tells us practice be the best term to define what we expect from new lawyers entering the profession. we expect new lawyers to learn on the job. emerge fromem to law school as whole lawyers with legal skills, professional competencies, and the character quotient. when it comes to evaluating whether a new lawyer has the character motioned, or the blend , experience really matters. date, the foundation for practice project has put out two reports available on our website

. first focuses on the evaluation terms of what respondents are looking for a new lawyers. that is where the character quotient emerges. the second report focuses on hiring the whole lawyer and experiential component. we are moving on to phase two of the project, developing measurable models, leach -- legal educations that support the foundations, working with wherehools to ship ways authorize incan thatother form of hiring these individuals have the criteria they have identified that is important. note, i want to give credit where credit is due. of the foundation

for practice project of our legal education section, is a woman named allie who some of you may know or have heard speak. she was supposed to speak today. wanted to note she is the one who developed the project and her information now appears before you. i look forward to the discussion about this and what it means for the evolution of law school and of the profession. [applause] quite so that was a very good pinchhitting job by justice kourlis. i am very grateful to be back giving a talk.

i will go through today or the central thesis i want to advance is we need to work at the future of law practice and law practice is changing pretty dramatically. it is like the boiling frog metaphor at the -- as those of you have heard it. put a frog into the water and heat it up and he will die there. water, he will jump out immediately. to a certain extent, that model is going out to death with millennial's today. we lack that fresh perspective. the bar is changing dramatically. i will give you a little bit of an overview. a couple of descriptive remarks. idea that judge shepherd

talked about what is going on in law school. here is the data i was able to calculate. class-size entering for four decades, for those of you who entered in 1971, quite a while ago. 246 andage size was hovered in that facility for over 40 years. of 262 andark dropped to 182 in the last four years, it set a record. what has gone on in the background is we have added more law school since 1971. of 204. high water mark to have them mergers so we are down to 201. those are a tremendous amount of fixed costs spread over a larger number of loss.

imagine a hotel or airline or restaurant chain that has lost 31% of paying customers, between 282 and 182. you see massive consolidation taking place in order to regain pricing power. at universities, we don't think that way. we were trying to figure out what the next move is going to be. i want to tie this into the labor market somewhat. more piece of labor market before get into that. tos the number from 1973 2017. extrapolate by the size of the entering >>. showing we the bar's will go to levels of loss -- law

school graduates we have not seen since 1970's. it could be structural eventually, it will rejuvenate. it is actually the nature of law practice is changing in such a way that we cannot just bank on number.r base it was a little little bit in the report given by the chief administrator of the second surf -- second circuit. look at this from a holistic perspective. market for total legal service in the united states, $290 billion. these are enterprises or businesses purveying legal services. it wonderful classification by the sentence -- census bureau,

clash of customer data. how much of you -- how much you are selling, how much you are selling to organizations you can see there is a big split. lion's share going to organizations. there is a famous study call the chicago study done in the early 1980's from a sample drawn in 1975 of the chicago bar. the finding that came out of this was if you want to understand the structure of the bar, find out who your clients are. from your clients you can tell a lot about a lawyer. this was called the two hemisphere theory because half the bar was individuals and half was serving organizations. they replicated the study in 1995.

they found out the lawyers serving organizations enjoyed enormous spoils. solos servingd individuals were much more economically distressed. this is going back 20 years. certainly that trend or pattern has continued forward. this has really big implications for law school because my students show up on day one with an idea of what love practices that comes from pop culture. --s is very different statistics are overwhelming about the number of vote -- of individuals who want to vindicate and get full justice. they just don't have the higher competent legal representation. something that disc into my attention a few weeks withecause i was working judge corliss in denver.

here's a couple of shocking statistics. i can't believe the numbers i am hearing. the median judgment is $2400 in state courts. that's from a civil matter. -- your typical lawyer quits say it is hard to make a living from this. 75% of cases had one self represented party. from theee the quote report. picture of civil litigation that emerges from the landscape data set confirms the long-standing criticism that the subtle justice system takes too long cost too much. it is just continuing to get more exacerbated to a point where it is pretty close to a breaking point. this is not the kind of legal system by which lawyers can earn a living.

it looks pretty close to a breakdown. you can get a lot of lawyers for the bigger cases but for the typical case it really does not work for the lawyer. theme ofued up this the segment of the bar that serves individuals and the segment that serves individuals -- organizations. visit concept of latent legal demand. solve theirant to legal problem if they are aware to have one but cannot afford a lawyer. so they are technology products coming online by which they can engage in self-help. there is a famous one in silicon valley. this news report talks about 11 legal tech companies being being financed by the y combinatior. there's a quote that basically

says it is really the individual market -- the legal tech startups are reticent to commit dollars to the large law firm segment. for a variety of different reasons. on the one hand we have these tech startups that are beginning to fill the void, the justice gap, the inability of the civil court system to accommodate or to be able to marry up practicing lawyers with cases in a way that can make the economics work. this is a seachange. i would call the legal industry. we go back to this chart, now i am breaking it down by total amount of spend. on the one end of the spectrum, 220,numbers at the bottom, $ average legal budget for an

individual in the united states. if one of us gets divorced we white that out for several hundred people probably. it's an imbalanced system. it's not a lot of money. over half of the dollars spent are spent from the basis is of organizations that calculate with her average budget is. over 50% of the market has $2 million or more to spend on legal services. this is a list of companies you can get on a book. it would be shorter than the l pages. these are the -- then the yellow pages. are now in a folks position where they are complaining about the cost of legal services and they are beginning to look for alternatives to traditional law firm practice. here's an article that comes from the wall street journal, talks about law firms facing new competition. their clients are in sourcing legal services. here's another clip that comes

from bloomberg on jpmorgan. basically hired a bunch of developers to develop software to automate jobs that would autumn -- ordinarily be done by lawyers. not just because lawyers are too expensive, but they could not get through the amount of work that needs to be done. ai-enabled hiring developers to develop tools for the services. this is a trend that is starting to take off the ground. a picture of what is happening in law practice today. you can see this is three trend lines going back to 1997 and it shows against a 1997 benchmark, this is the first year the census bureau had a north american industrial classification system up and running. you can go forward 20 years. you can see the change of

employment in employed lawyers in the law firm sector, government sector, and in-house sector. in-house has been joined grammatically. it grew from 34,000 in 1997 to over hundred 5000. -- to over 105,000. it's pretty staggering. it is growing. timesring rate is 7.5 more lawyers being hired in-house than are being hired in the law firm sector. even there, it should surprise folks. ignore the top line and look at the bottom two. government hired almost twice as many lawyers in the private sector at least since 2006. i don't think the government has been i hiring boom at all. something big and structural is taking place in law practice.

basically big corporations between building your buying, they are choosing to build. thisaving thought about topic for quite a period of time and doing more field research as opposed to just reading, i spent a lot of time with in-house legal departments trying to understand the changing buying patterns. i came up with this. the real take away is what i call the type six client to the far right. sophisticated law firms embedded inside major corporations that have the specialization and budgets that rival almost any law firm in the country. to a certain extent they have made the decision to build the capacity in-house because they cannot buy with a need externally in the market. there is a variety of reasons firms transition to fill this gap at the biggest one is

the metric of the billable hour. it is how we sell our services, compensate our lawyers. that is antithetical to efficiency, the more for less mantra that we hear. these folks are taking matters into their own hands. the most significant segment is probably the legal operations folks, the gold boxes to the right. these are sophisticated folks tried to get data process and technology, people systems to drive of quality and drive down costs. they're also being aided and abetted by the green bar, the procurement folks who are using all sorts of systems to rapidly during portions of the legal services market into commodities. so this is a big seachange and it definitely affects -- ultimately who we hire from law school in the skills they need to succeed.

can you raise your hand here if you have ever heard of clock? really? nobody has heard of clock? ok, we have one hand. corporate legal operations consortium. --s is the fastest growing it is big and growing very fast. they have their convention next week in las vegas. this is the fortune 500, the profession or's -- the professionals in gold and together to compare notes, to build faster for the internal clients. this convention in las vegas will probably have 1500 people in attendance. every possible vendor in the legal ecosystem will be there. basically there rapidly figuring out ways to drive down their costs in a way to keep them pretty far a field from civil disputes.

this is rapidly changing, driving down the demand for legal services and to a certain extent, a change in the nature of practice and very, very fundamental ways. so, i hope that a few more people have heard of richard, raise your hands. ok, quite a few. richard wrote a book called "the end of lawyers" in 2008, and it had a question mark on the end. the purpose of lawyers is to serve clients in the state and ultimately derive a living after we have served those two duties. he thought challenges to rethink the nature of legal services. he wrote another book in 2013, which recommend to anyone. $20, you can read it on a plane flight between here and the east or the west coast.

it's a marvelous, accessible book. achard has been writing for long time at a fairly high level of abstraction. but now in my fieldwork i see more of what he's talking about is coming into fruition. so, a model that richard uses -- to the am going to speak future of legal education because it is the future of what the market is telling us it wants. or the start of ecosystems are tapping into latent demand for individual legal services. when we think about legal arvices, richard is making claim everyone can travel along this five stage progression. bespoke is tailor-made and what we learned to do in law school. tailor-made an argument to win in a court. a contract to consummate a desired transaction. we can all get would bespoke is. tos talking about moving

commoditized. wouldwyer including me agree that commoditized is a frightening place because it means it may be free over the internet and it is very hard to make a decent living if our work product is going to be free over the internet. so a lot of the lawyers retreat back to this idea to the most technically sophisticated stuff, the brain surgery that we learn and claim we know how to do and learned in law school over decades in practice. richard is making a claim that the systematized and packaged is where the future of law is going to preside. when you think about the individual hemisphere are talked about before, what is it struggling from? lack of lawyer productivity. substantially a function of how the legal system is set up to its procedures and

its trial practice in the presumptions that are built into it. makes it uneconomical to access legal services through the standard format, the court system. then from the corporate perspective, there is such a massive amount of need they cannot afford to just pay large firms to continue to do that. they need efficiency, so they are doing it themselves. the systematized packaged solves the lawyer productivity problems on both side of the hemisphere equation. information systems, systems engineering, finance, marketing, product management, and law. these are other disciplines not traditionally taught at law school. i think it would be beneficial to have an eye dropper of those disciplines within the legal education. -- the future is the ability to communicate across those disciplines.

engage in teams of multidisciplinary practice and come up with solutions that really are outside the domain of legal expertise -- that require deal expertise that can ultimately provide the best service of client. that's a pretty tall order, but i see every time i log into the cloc listserv and i see the things being talked about, i see the future across from what those professionals are talking about is definitely moving in this direction. here with thisp idea of people and process and technology. it is going to be some variance of that that we are going to need to incorporate into legal education. it's going to happen very slowly, much too slow for my satisfaction, but we have to work backwards from demand in the marketplace. folks can put pressure on the law schools to change what we

teach and how we teach. so i'm going to do one quick exercise then we will be done. by ais a famous study done commission by the law school admissions council, the people that make the lsat. it was published in 2008 but all the data was gathered in the early 2000's. basically they did a study that was basically a can to what george -- to what justice kourlis's organization did. it was based upon the university of california berkeley alumni base and the university of california at hastings alumni based. about 1200 lawyers in the sample. after they delineated these effectiveness factors using psychology methods, then they came up with behaviorally anchored rating scales to actually judge the quality of , how they performed

in these 26 different dimensions. i would in the sample, have a score for my supervisor, my negotiation skills, creativity, innovation, practical judgment. then they took the data and it put it in these redactors -- predictors. so i will put this question to you. let me see if you can get some of the takeaways from this study. which attributes below the better predictors of lawyer effectiveness? grades, orgraduate personality assessments, emotional stability, self-control, conscientiousness, and ambition. was it a or b? b? b? everyone says b. you got it right. hire? your chambers do you rely on b or a?

b? i suspected that my experience has been with we rely quite a bit on a, even now we know b -- even though we know b is how people flourish in practice. i am happy to help on this, we can dig deeper on how the practice of law is changing any he kind of professional skills that we need. into a dialogue and agree at what is prized and that is how we're going to hire. i think we could break out of some bad habits and into higher some great people in change the curriculum in ways that serve our society. i thank you very much for your time. [applause] >> we have persuaded brian welch to preside over q&a and

comments. all of us are grilled -- are glad to respond to anything you have heard. hands.n't see any raised do we have any questions? we will try to walk a microphone around so that people can hear the question. >> good morning. is the son? -- is this on? going back to your last slide, b, is thathe reflective of the criteria that law schools use in choosing law students? [laughter] prof. henderson: guilty as charged. i knew you were going to throw a fastball. law schools, we feel like we are locked into kind of a positional competition. the u.s. news & world report.

we do look for lsat and undergraduate credentials to admit on, and that is how we reward scholarship money. writ large, this is how most of the 200 schools do that. maybe some of our self regulated organizations could figure out ways to break that logjam because it clearly has all sorts of deleterious consequences on society and i think we will come to regret where we are at. having been on the inside of law schools, not only in my school but talking to a lot of other law schools, it is a very serious issue to flout the conventions of u.s. news & world report. all can beants easily thrown into a tailspin if you drop 10 points in the rankings. i am a believer unfortunately in

rankings management, but there are ways i think that the law schools themselves overweight lsat and gpa. i know one school that actually focuses on employment outcomes and actually admits people would lower lsat's and lower undergrad they will have more back and and they have run the math. but i would love to break out of that dilemma. >> almost impossible for an individual school to do that because you killed in the marketplace. changing the discounting for r faple, would be darn nea tal. from the ohio state university school of law, they said important work trying to tease out possible options for collaborative work. she has gained attention rightly

because of that. >> another question in the middle. gpa, but isat and don't know that b category. do you know of any entity that used b-based hiring? >> i was part of a research covering for six years and we held law firms utilize that. let me stay centered on the study. they used a variety of different instruments -- the industrial organizational sky theology -- psychology and gold standard. call thean assessment personality inventory and a developmental survey instrument. they were looking for the positive attributes and negative attributes for on-the-job performance. they have been validated over a , in manyod of time

different professional settings. is someone in the law firm space, a psychologist with a big practice built around the hogan products because they are excellent. he probably has tens of thousands of law firm partners, especially law firm leaders, that participated. there are folks out there that are using them. there's also other tests, situational judgment, the biographical inventories, which is like a money ball-type instrument. all these things predicted way better than lsat and gpa. some predictors were negatively correlated with the specter. the higher the lsta below are --

so, there is a huge amount of effort that can be put in for my law firm to begin to think thoughtfully about who is successful in the process and why. you can't build self reflection unfortunately. processthe legal genetically under invests in thinking through lawyer development. tremendous research is out there and i'm happy to go off-line and equate someone with where to begin on that inquiry. >> talking little bit about the role of what the bar exam plays in this process. the correlation between bar passage and lsats and the extent to which we had built-in sort of our own piece of this puzzle that we may need to unravel. i'm not sure i: know exactly where you want me to go with this but i will go with what i know.

the lsat is a pretty good predictor of ultimate bar passage. something about six times more effective is your law school grades. your law school grades should be a function of your lsat will you put it into a big statistical model you effect -- you measure the effect of one versus the other and your law school grades are about six times more important in ultimate bar passage. lsat, whatdent of would predict your law school grades? motivation, all the character factors that justice kourlis was referring to. showing up. i always took attendance in my class and every year i got the same results. you come to fewer glasses, you get a lower grade. am -- some of those people have high lsat scores. guess, well i should ask

justice kourlis, is that the point you were trying to directly toward? justice kourlis: -- trying to direct me toward? justice kourlis: it is easy to say this is all u.s. news & world report's fault and there's nothing we can do about it. but there are things we can do. collectiveked about action by law schools. talk about changes in hiring practices that therefore incentivize law school to produce different products. alternatively, to create different measures for those students who are graduating. , overhere is the bar exam which we do have a significant amount of control and questions about whether we should be thinking about changing the bar ,xam, asking different things directing different kinds of skill set development through the bar exam is he issued that i

was trying to put on the floor. in newenderson: hampshire they actually have a pilot program, i will get the name right -- i won't get the name right, the franklin pierce report? ok. it basically allowed law school graduates in new hampshire to completely opt out of the new hampshire bar if they actually took a series of courses that were simulations. they basically built a portfolio. you take away for the employers was the young people that went to their program were just demonstrably better on so many different dimensions, which i think to justice kourlis's point, the initial demand on who gets admitted in the criteria, if we change that we end up with people who work and are prepared and more effective. justice shepard: the early days of that option, a relatively

small number of students chose to do because it thought it would be harder than taking the test. i think there would be some interest in altering the final entry. something valid and reliable. i remember being in a drug court setting one day and a lawyer came in on behalf of his client and said my client doesn't want to drug court, he would rather do present. -- do prison. when he really meant was drug court is pretty darn tough. it would be easy to spend two years in jail. the key would be to you create a measure that actually holds people accountable and separates out people that we should not turn loose on clients from the ones that are ok to turn over important things to. other questions? we have a microphone right