i think the past commission failed in this respect when they viewed things like radio, to beating with radio and some instances it was fm radio computer with fm radio whereas i think the market, everyone is fighting over the same eyeballs, the same attention, the same advertising dollars and it's figured out how best to reflect our current rules while doing so in a thoughtful way. >> watch tonight at eight eastern on c-span2. >> now former judges from their colorado and indiana supreme court's discuss the future of law schools. from the seventh circuit bar association annual meeting, it's an hour and 20 minutes. [applause] >> thanks, and thanks for making time on the program for this conversation about the future of our profession. as brian welch said i'm randy shepard.

i ended up spending most of my adult life as chief justice of the infamous supreme court and i now doing see scenes of it in or intermediate court and i have an appointment over at the indiana university mckinney school of law. to my right is rebecca kourlis who spent more than a decade on the colorado supreme court. she left there perhaps ten or 11 years ago to be the founding director of the institute for the advancement of the american legal system which does all sorts of research and programmatic work on the improvement of courts and the improvements of legal education and of the legal profession. and then professor william henderson certainly and that of the circuit if there ever was one. after his legal education at the university of chicago became a clerk at the seventh circuit but happily has spent his recent

career at the indiana university maurer school of law, and i say there's nobody he was done more intriguing work about the future of our profession than bill and becky. i'm going to begin with the words of setting the stage. there are many things about modern legal education that we as lawyers know a little about but not necessarily know about in detail, and they provide considerable foundation for evaluating the current state of law schools and, both now and in the future. now, you could actually describe recent developments by using just eight words. the eight words would be rising

tuition, massive debt, fewer jobs, and fewer applicants. i will say a few words about each of those to begin laying the foundation for our conversation today. no mystery to anybody that tuition in higher education has been rising. indeed, it has been rising for a lifetime of virtually all of us here at a rate that exceeds twice the consumer price index. it is, in fact, been rising at a rate so rapid that the finance officers of the american university systems have created a new measure. it is called the higher education price index, designed to give a more accurate reading of how much, what ought now be called really the sticker price as opposed to the nominal price. but how fast the sticker price has been going up.

in the two decades that, say just ended, late in the last century and the first decade of the century, the general rate of inflation for higher ed was 71% real actual adjusted for inflation, cbi, 71%. but the rate in law schools is not 71% over that time, but 317%. it is actually been rising at an average rate of 8% a year in recent decades. that, of course, has led to stories we all read about. the largest amount of debt that students, both undergraduates and, of course, law students, experience, rising to the point where americans now owe more in education debt than they owe on

their homes. really quite a stunning change in the american debt picture. the average law student debt on the way out the door is about $85,000 in the public schools, and about 50 50% higher than th, 122-$125,000 for students who have gone to private schools. this continues of course that in general to be easy to read about. just last week and the "wall street journal" a new wrinkle on debt. there's a federal program that's been in place for a while but i think it's called parents plus. it's on the front page of the second section about the opportunities that parents have been given to empower their children to borrow more money if the parents would cosign. ..

very close friends over the last two decades is that we have been turning out more jd graduates and turning out more law license holders then there were jobs for lawyers. for a long time. the reason we didn't notice that unless we have somebody directly involved with the is the way in which employment for new lawyers

was reported didn't do a very good job of telling us where they were employed, simply told us by and large whether they were employed or what kind of organization they were employed. it was quite ordinary as recently as thingy 2000 three announcements once a year from the national announced nation with the american bar association that would say this is a great year. 90% of our graduates have found work within nine months, now 10 months has been measured. that sounded like a very happy results. well, as you'll recall, there were a series of scandals about the way in which this was reported and which of course occurred next-door in illinois. our friends will particularly recall. the changes according to ask tougher questions in particular how many law graduates, license

holders find an ongoing job for which you have to have a law license. that is the hardest way of asking the question. in the last half decade or so, the answer to that has been last year was 59%. that is to say about the people who turned out at the school, how many people found ongoing work full-time of which you have which you had to have a law license and the answer to that is 59%. that number has stabilized. it had been as low as 57%. but that number has stabilized over the last couple of years. a lot of people don't buy alert for attorney general's office as a prosecutor offices or private law firms. the other reason it's stabilized of course is the number of new law grads keeps declining. the way in which you calculate

the percentage gets altered. the fact is over the last five or six years, the number of jobs full-time license required jobs has continued to decline. as recently as five years ago, it tended to be something like 28,000. mind you we were graduating 40 to 45,000 or maybe a little bit more. over five years ago about 20,000 jobs which most people think they are looking for when they go to law school. that number this year has fallen to 23,700. slower decline perhaps thin earlier in the decade, but still a pretty small number. college seniors and second generation in college law advisers can read those numbers.

so the result has been a dramatic change in how many people apply to go to law school. within the last decade, the normal number, and the highest numbers ever seen in the country, the all-time high was 100,600 applicants. applications is a funnier number so seems to be more important to focus on applicants. that number has been something like 100,000 paid 100,600, 98,000 for quite a long time. it dropped dramatically. in 2016 the number was 56,000. so it's on the order of a 50% decline in the number of people who come down and put in their applications. as of last week, so far this year there were 50,200. that is down from last year just about 8%. i think that is the number that can say has probably stabilized,

but there's something is happening inside that number that are particularly attractive. for instance, of the metric units commend the people who actually can in school, in order to come into school with lsats over 160 are half of what they were five years ago. the number who had 150 to 160 stabilized at the number of people who come in over 150 is up 80% from where it was. so there is a very noticeable shift in the metric to use the easiest word. others might say talent or suitability for law school. the other thing happening inside the numbers the change in student legal aid as the competition for applicants increases, schools have used more and more of legally chasing the numbers because it helps

ratings and of course because you begged to have the smartest possibly in your law school. they resulted that is the amount of merit taste aid over the last decade has gone up 10%. the amount of merit aid has doubled and there's a third category called with merit can be based. i haven't spoken yet to an actual law school officer who doesn't tell me both men merit. that goes to the people of the best metrics. so it is the folks at the back end of the line who end up with a far greater share of that bad when they walked out the door with their degree. i've been particularly worried about the effects of this of minority applicants because they are profiled in gpa and lsat

isn't as good as asians. the hispanic applicants an african-american applicants have lower metrics and it's harder for law schools to hand them scholarships or financial assistance if you will. as i said earlier, better thought of these days. we had some schools in the circuit by the way who don't charge any tuition at all for a substantial number of students. indeed, we have a few schools to send students money. they not only don't charge any tuition, they write checks to support living expenses. there are only a few that do that, but it's part of this greater trend. and of course, again last week, news that the bar passage rates in the country come in no surprise have likewise been falling. february is not as good a

measure as july, but the february rate in this state is the lowest that we've ever experienced. i want to end on an important note, which is that this is not gone unnoticed by our friends at the academy. the number of people in the academy who say this too will pass, would've heard this before but the shrinking number. research done both by various entities with which professor henderson is associated about how we change a little education to adjust to what it looks like the profession will be is really very encouraging and very substantial and the easiest one to see is the change in what is now referred to as experiential learning. how many actual hands-on experiences has been dramatically changing over the last few decades because schools are determined to do as good a

job as they possibly can. we are going to hear some news about all of those trends we will start with just this kourlis. >> professor henderson and i have to stand because the monitor is for my peer, not from our seats. i apologize for breaking up our little trio and would refer to be seated. as you know, i am becky kourlis. i have a background -- thank you, as the trial court and appellate court judge, but now am running a research institute at the university of denver that we refer to as the sink to tank piles, which is the worst acronym in the world and i

apologize, but we were named by a federal judge and he wouldn't let me put just this in the name because he thought justice was a term that was too susceptible to the interpretation of the day. we work in four different areas and i have spoken to this assembly on prior occasions, but usually in the context of our civil justice reform. we do work in legal education, civil justice reform. we work on the family just decide that things in state courts or not we work on judicial selection and performance evaluation. our process as we identify a problem. we gather research, put together wonderful groups of stakeholders. we create models.

we've been monitor the implementation and measure outcomes. hence they do tank. to do problem i talk about your current chief justice shepard has identified, which is in the context of legal education, we are struggling with how to prepare with lawyers for the procession as it is evolving so that we begin to revitalize the numbers and more importantly the role of lawyers in our society. these are the drilled down numbers to which he has also just referred. this is the percentage of 2015 law graduates who did not land full-time employment regarding bar passage. bill henderson and i just chatted.

we think the 2016 numbers are due out in the next couple of days, so i apologize for the fact these are year-old, but this is the best set of numbers we have at the moment. 30% is the percentage of 2015 law grads who didn't land full-time employment recognizing their law degree. 25% who didn't land full-time employment including professional and nonprofessional. this is the buries the number. 71% of third-year law students who believe they have sufficient skills to practice. this set of numbers is really interesting. 71% of the third-year law students who think they are locked and loaded, ready to go into law practice. however, the law professors who teach them say that is 45% and wait for it, the percentage of

practitioners who believe new lawyers have significant skills to assist 23%. there is a problem there and we at iaals develop a project intended to shed some light on ways of addressing this. we created our foundations for pratt this projects. we had support from the foundation from the three-phase project. we together an advisory committee consisting of practitioners, judges, justices, representatives from the national association of bar execs from the national conference of bar president, the national conference of bar examiners. representatives from the public defender's office, the aba and

even the attorney general of colorado and we developed a plan. the first piece of that plan is to identify the foundations that entry-level lawyers need to practice. the second phase will be to develop measurable models of legal education that support those foundation and the third will be to align market needs with hiring practices in order to incentivize positive improvements. phase one we developed a survey based on existing research for those of you who are mavens in this area, such research as schultz and vatican with the assistance of our advisory council and distributed that survey through bar associations across 37 states in the country.

actually, they were individuals who answered, but the bar association said 37 states distributed the survey and we had a little bit of crossover of people who practice in two states. we got 24,137 valid responses to the survey. if they are any of you in this who answered the survey, thank you for your time. it was a little bit of a prodigious undertaking because the survey itself was 25 minutes long. we got over 40,000 actual responses and 24,000 valid responses. the responded base mirrors legal professions. some of the numbers are not wonderful in terms of demographics, but in fact the respondent is representative, both geographic and terms of the

size of firms in which lawyers across the country practice, the amount of money they make in the demographics themselves. we asked about 147 different foundations in the survey. we categorize those foundation and and broadly fall into three different categories. some of the examples are identify relevant fact in legal issues, critically evaluate arguments, conduct and defend depositions, listen attentively and respectfully, make decisions, work as part of a team. the three categories into which

we decided those foundations are as follows. skills, characteristics and competencies. so remember, we have 147 foundations and we are divinely mad into these three categories. 27% of the foundations were skill base. 28% characteristic-based and 45% competencies. we also graduated the questions that we asked of the respondents. we asked what is necessary in the short term. we asked what is advantageous, but not necessary in the short term. we asked what is necessary over time can we asked what's not

relevant at all. the screen right here doesn't pick up grace that looks up grace public up grace that looks like i'm missing a cylinder in the middle. fortunately not your screen. so here are the results appeared there were 77 foundation identified as necessary in the short term by 50% or more of the respondents. 147 foundations, 77 identified as necessary in the short term by 50% or more of the respondents. what emerged surprised us. we call it the character quotient and here it is. out of the top 20 of those foundations identified as necessary in the short term, nine are characteristics, 10 are

competencies and one is a legal skill. there it is. research of the law at number 14. the patent competencies, and keep confidentiality, arrive on time. this is a millennial issue, right? arrive on time. treat others with courtesy and respect. listen attentively and was actually, respond promptly. take individual responsibilities. emotional regulation, self-control, speak professionally, right professionally, exhibit tact and diplomacy. the characteristics, and the other nine, honor, commitment, integrity, trustworthiness, diligence, strong work ethic, attention to detail, conscientious common sense,

intelligence and a strong moral compass. cumulatively, we refer to this as the whole lawyer. what about legal skills? this is substantively why we go to law school, right? the legal skills show up in them must be acquired over time set of responses. these 20 characteristics are the top 20 or these foundations are the top 20 that must be acquired over time according to our respondent base. here they are. develop appropriate risk mitigation strategies. provide quality in court trial advocacy. cannot didn't offend depositions, case on appeal, prepare for participate in mediation. all of those skills that i suspect all of us would identified either on the transactional or litigation side

as being necessary. but the take away here as we expect people to acquire these over time. we don't necessarily expect them to show up at the time that they sign up for work, ready to do these things are qualified to do these things. so now, here is the second part of our survey and here's where it becomes a little more relevant to all of you. in the second part of this survey, we asked respondents to consider the hopefulness of a set of hiring criteria in determining whether a candidate has the foundation they identified as important. so, you have to understand, we took as a given the fact that they had heard the answered the survey and identified what they thought was important. and then they said okay, how

would you hire in order to suss out the individuals who have the foundations. we did not ask them how they currently higher. and here is poor oliver barrett the fourth who went to jail and harvard and with no senator of the law for viewing a summer clerkship of wilmer cutler. so, the avowed notion is that this person would get an interview pretty much anyplace in the country, but does this person really have on his or her and this is his resume initiative that would suggest that he has those foundations that people said they were looking for. those foundations that people said they were looking for. what would people focus on if they wanted to hire the whole

lawyer with a character quotient? we identified 17 criteria and we asked the respondents to rank them from very helpful to very unhelpful. take a look at the 17 criteria adjust quickly and then i will show you how the results worked out. here they are. top of the list, legal employment. recommendations from part addition there is where judges bottom of the list, log review journal experience. now, this gradation is all respondents. you will note that the first days has to do with experience. with the respondents told us is that the character quotient for

the whole lawyer matcher eric emerges through experience. we broke up the data by size of firm, decrement employers, legal services rpd with firms of 100 plus class rank and law school move up with firms of two to 100. there's a little bit of a shift in the top eight, particularly the second floor, but it still the same top eight. in smaller firms or solo mission are setting, specialized classes which may extend because there's a little bit more demand for actual ability to jump in on a particular topic. government employment, the top eight still the same.

a little bit of a shift in priorities. legal services or the public defenders office. you see particular law school courses specializing presumably in the criminal area of moves that. what does all of this tell us as we are thinking about the future of legal education and the future of employment lawyers. first it tells us practice writing may not be the best term to define what we expect to new players entering the profession. we all expect new lawyers to learn on the job. we do expect them to emerge from law school has told lawyers that they plan to political skills, professional competencies and character quotient. and when it comes to evaluating

whether a new lawyer has that character quotient for that linda foundations do we have label the whole lawyer experience really matters. the foundations has put out two reports, both of which are available on our website. the first really focuses on the evaluation of the data in terms of what respondents were looking for a new lawyers here that is where the whole lawyer in the character quotient emerge. the second report focuses on hiring the whole lawyer in the experiential component. we at iaals are now moving on to phase two of the project, develop a measurable models that support these foundations, working with the law school is

in employers to shape ways whereby employers can recognize on a law school resume or some other form of hiring that these individuals have the criteria they have identified as important. as a final note, i merely want to give credit where credit is due. the director of the foundation for this project at iaals and our legal education section is allie kirkwood, whom some of you may know or have heard speed as she was supposed to be here today and a medical circumstance protector from you. i just want to note that she is the one who really developed this project enter information now appears before you. i look forward to the discussion about this and what it means for the evolution of law school.

[applause] >> well, that was a very good pinchhitting job by judge kourlis. i've learned a lot from this session and i'm grateful to be back here at the seventh circuit, giving a talk to a court that i used a clerk on. the point i will go through today are the central thesis that we need to drive from the future of opera as a mob practice is changing pretty dramatically in that it evolves over a long period of time in this kind of like the oil and frog metaphor. put a frog in the water and heated up and it will just die there. if you toss it into hot water, it will jump out immediately.

to a certain extent, that's a lot of what's going on with the millennialist today. to a certain extent of those aging, we lack that fresh air, but the price is changing very, very dramatically. i'll give you an overview and then maybe a couple remarks. going on this idea judge sheppard talked about on what's going on in law schools, here's some data i was able to calculate on the website. so for those of you that entered canada in 1871 quite a while ago, the class size is 246 and harvard for over 40 years, had a high water mark in 2010 and then dropped precipitously to 182 and set a record for a small

entering class size. what's been going on in the the background as we've been adding more law schools since 1971. about 147 law schools in 1971. we had a high water mark. there's been some closures that were down to 201. that is a tremendous amount of fixed costs spread over a larger number of law schools. if you imagine a hotel or airline or restaurant chain that it lost 31% of his pay cut as much as the delta between 282 and 182, you see massive consolidation taking place when it regains pricing power. universities don't think that way. and so we are kind of moping along, trying to figure out what the next move is going to be. i want to tie this into the labor market somewhat.

one more piece of labor market data before you get into that. this is the number of aba graduates from 1973 to 2015. we can extrapolate what they will be by the size of the entering classes. you can see the bright green showing we are going to go two levels of law school graduate that we haven't seen since the 1970s. this seems to be far beyond cyclical. it certainly could be structural and a lot of people look at the data without digging too much deeper to say eventually this is going to rejuvenate because we will drop down to what we need and everything will be fine. it isn't actually the nature of law practice is changing in such a way we can't just bank on a smaller base number and kind of build back out. some of it was alluded to a

little bit in the report given by the chief administrator of the seventh circuit. so, to look at this from an holistic perspective i get data from the u.s. census bureau and just look at the total market for illegal service in the united states, so about $290 million. desire and the prizes -- enterprises. there is this called class of customers. you can drive how much you are selling, how much or provision of legal services to individuals and how much organizations. you can see a exploit. about $70 billion to individuals in the lion share of it going to organizations. there's a famous study of these famous in a circle called the chicago study published in the early 80s but the sample in 1875 of the chicago bar.

what finally came on the base, which was if you want to understand the structure of the bar, find out who your client fire. for their clients you can tell a lot about what the lawyer was. this is called the two hemispheres. because half of the individuals and roughly half of our serving organization. they replicated the study through a sample drawn in 1995 and they found out the lawyer serving organizations enjoyed enormous in-house lawyers and that the small firm and solo portion of serving individuals was much more economically over 20 years now and almost certainly that trend or pattern has continued forward. this has really big implications for law school because my students show up on day one with an idea football practices comes from pop culture. this is very different than the

statistics are overwhelming about the number of individuals would like to vindicate their madness and get them full justice. they just don't have the capability to hire competent legal representation. so this is something that just came to my attention a few weeks ago because i was working with judge kourlis in denver and her organizations and the state courts to put together this report. here's a couple of really shocking statistics. i care members sitting there and secondly the #-number-sign hearing. the medium judgment is 20 per $100. it's going to be really hard for me didn't make a living from that. 75% of the cases have at least one representative party and you can see the quote that is from the report.

the picture of civil justice that confirms the long-standing criticism of the civil justice system takes too long and cost too much. the critique has been in place for quite a long time, but just continuing to get more exacerbated to a point where it's pretty close to a breaking point. this is not the type of legal system by which lawyers can learn a living. it looks pretty close to a break down. you've got a lot of lawyers on the bigger cases, but for the typical case, it really doesn't work for the lawyer. so, i queued up this segment to bar the service individuals in the segment that serves organizations. there is this concept before that date so of the 10th legal demand. individuals out there would like to be able to solve political problems they are aware they have one but they can't afford a lawyer so there's these

technology projects coming on line buy ways they can engage themselves how. why a famous silicon valley news report talks about being fairly excited by being financed in pretty bullish on the legal sector. and it basically says the individual market that the startups are pretty to commit dollars to the large law firm for a variety of different reasons is difficult to sell efficiency and not let sam altman was getting not. on the one hand we have this tech startups beginning to feel the void justice gap by the inability of the civil court system to accommodate what to be

able to marry a practicing lawyers in cases in a way that make the economics work. i wouldn't even call law is. i would call it the legal industry. now i'm breaking it down. one hand of the spectrum the numbers at the bottom is $220 your average political budget for individual in the united states. appointments can divorce we wiped that out for several hundred people probably. so it is an imbalance system of $220 is not a lot of money. if you look to see over half of the dollars spent is spent for the basis of organizations to calculate what the average budget would be. over 50% of the market as 2 million. a list of companies shorter than the yellow pages and the legal

services. this group of folks are now complaining beginning to look at the law firm practice. and so, here's an article that comes from "the wall street journal" talks about law firms and their clients are in sourcing the legal services. here's another clip that comes from jpmorgan, ray software to automate jobs that would ordinarily be done not so much because lawyers are too expensive. there's no way they get through the amount of work that needs to be done. so they are hiring a.i. enabled developers to develop tools that give him legal services and this is a trend starting to take off the ground.

so, here's a picture of what is happening in law practice today. you can see the three trendlines in the show's 1997 benchmark the first year with the north american industrial classification. i can go forward or not they find in the the employed lawyers. in law firms sector, government sector in the in-house sector. the big-league firmness grew from about 34,000 in 1997 to over 157. there's as many lawyers in-house that are employed domestically. it is pretty staggering and hiring rate is about 7.5 times more lawyers being hired by

being hired into the law firms are. even there, that should surprise folks if we ignore the top line and look at the bottom two lines. governments are at almost twice as many lawyers in the air. at least in 2006. i don't think the government feels it's been on a hiring boom and all. something big and structural is taking place in law practice and basically big corporations between building or buying they are choosing to build. so having thought about this topic for quite a period of time by doing more field research as opposed to reading, i spent a lot of time trying to understand the buying patterns. i came up with this apology in the real takeaways a real pickle is what i call the type six planned to the far right. these are sophisticated law

firms embedded inside major corporations that have the specialization that rival almost any law firm in the country and to a certain extent they made a decision to build this capacity in-house because they can't just buy what they need externally in the market then there's the right of reasons offered this transition or failing to fill this gap, but the biggest one is the metric of the billable hour, how we sell our services and compensate our lawyers and that is antithetical to efficient the intent of the more for less mantra we hear from in-house legal departments. these folks take matters into their own hands. the most significant are folks to the right. they use data processing technology to drive up quality

and drive down costs. there's also aided and abetted by the green bar, which is the procurement folks using all sorts of systems to rapidly turn a portion of legal services market into commodities. so this is a big change and it definitely affects -- ultimately who we hire in law school as the skills we need to succeed. so can you raise your hand here if you've ever heard of clock? really? who's got one hand here. corporate legal operations consortium. this is the fastest growing. they've got their convention next week in las vegas. this is the fortune 500. this is the professionals in gold that are getting together to compare notes to provision

legal services to internal clients. they will have 1500 people in attendance at every possible vendor in the legal system will be there. they are rapidly figuring out ways to find out their cost in a way that keeps them pretty far afield from civil dispute. this is rapidly changing. again, driving down the demand for the go services and to a certain extent, changing manager practice in very fundamental ways. so, i hope a few more people have heard. raise your hands. quite a few people have heard. richard wrote a book called the end of lawyers in 2008 and a question on the end which was meant to signal the purpose of lawyers.

he wasn't making a declarative statement. in the states and ultimately after we've served those two duties than he thought challenging to rethink the services. he wrote another book in 2013 called to mars lawyers which i recommend to anybody. it's only about $20, 160 pages. we did on a plane flight between here and the east or the west coast. for richard has been writing for a long period of time it is fairly high level of abstraction. but now i'm a fieldwork i see even more of what richard is talking about is coming into fruition. soy model that richard uses and erin go into speed to legal education because it's the future to either clock with the started ecosystem to track latent demand for individual

legal services. when we think about legal services and richard is making a claim that everything can travel on the five stage progression, it is tailor made and what we learn to do in law school. taylor made an argument, tolerant and a contract with a desired transaction. we could all get with the focus, but he is talking about the standardized products to monetize. any lawyer including me would agree that commoditize is a really frightening place because that means it may be free of the internet and it's hard to make a decent living if our work products is going to be free over the internet. a lot of lawyers treat back to the idea of doing the most tactically sophisticated staff and that we know how to do that we learned in law school over a period of decades of practice.

richard is making a claim that the system ties and packages for the future of us going to reside. if you think about the individual corporate -- that i talked about before, what's is struggling from? lack of product committee. the inability partially or substantially perhaps a function of how the legal system is supposed to do its procedures and trial practice and of assumptions built into it you do makes it just uneconomical to access legal services through the standard format and the court system. from the corporate, such a massive amount of need they can afford to pay large firms to continue to do that. they need efficiencies that they are doing it themselves. i think as far as the problem.

information systems engineering, finance marketing, project management and law. these are other disciplines that are traditionally taught at law school. i think it would be beneficial to have an eyedropper of those disciplines within the legal education, but the key -- the future is the ability to communicate across the disciplines, to effectively engage in teams of multidisciplinary is to come up with a solution with legal expertise that require illegal expertise to serve in the best interest of clients. that's a pretty tall order, but i see every time i login and see the stuff that is getting talked about, i see the future comes across in what is moving in this direction.

it's going to end up here with this idea of people in processing technology. it will be some variant of that they're willing to incorporate the legal education. it will happen various lovely. much too slow for my satisfaction. we worked backwards from the demands of marketplace. you guys are the folks ultimately putting pressure on the law schools to change what we teach and how we teach. i'm going to do one quick exercise done here. this is a famous study done by a commission by the law school admissions council, the people that make the lsat published in 2008 but all the data was gathered in the early 2000. they did a study that was somewhat akin to what judge kourlis' organization did. they eliminate the factors that need to be successful and is based upon the university of

california berkeley alumni base and also the university of california's alumni base about 1200 lawyers in this sample. after they delineated these different areas using gold standard psychology not ask, then they came up with the anchored reading skills to actually judge the quality of the lawyers and how they perform on this 26 different dimensions. i would have a score from a supervisor in my negotiation skills or animation are practical judgment. they take the data, correlated it back to the lsat in the undergraduate gpa that are prized on admissions. so i put this question to you and let me see if you can get the takeaways from the study. which attributes are the better predictors?

the lsat undergraduate grades in law school grades measure emotional stability, self-control, conscientious and ambitious. was that day was that there was a beat? everyone says b. you've got it right here. let's go back. how does your law firm or have your chambers higher? i suspect my experience has been we were like quite a bit and b. is the right way of people flourish actually in this. i am happy to help on this. it's how the law is changing it and what we need the new dialog and not how we are going to

hire. we can break out of some bad habit in the great people and changed the curricula in ways that are of our society. thank you very much for your time. [applause] >> we've persuaded bryan walsh to preside over q&a and comments that we are glad to respond to anything you've heard the three of us say. >> i don't see any way it stands. we are going to try to walk a microphone around so people can hear the question. judge parker, you've got the floor. >> good morning. going back to your last fight, bill henderson, amd, is that reflective of the criteria the law school's use them all

students? >> guilty as charged. i think to myself judge parker get to. the law schools do we feel like we're locked into a position on competition the u.s. and world reports and so we do look for graduates credentials and how we reward scholarship money. what large this is how most of the accredited schools do that. maybe some of our self regulated organizations can get together. we can figure out ways to break that logjam with all sorts of deleterious consequences on society. i think we will come to regret

where we are at. having been inside a law school, not only my school but targeting law schools. it's a serious issue to flout the convention and your alumni is students and applicants can be easily drawn into a tailspin if you dropped 10 points in the rankings. i am a believer in there are ways to vent the law schools in gpa in various ways -- i know one school with folks of the employment outcomes admits people with lower lsat is because they know their more employable, but that is very early where we are added and i would love to break out of that.

>> almost impossible for an individual school to do that because you get killed in the marketplace changing the discounting, for example it's darn near fatal. john mayer from the ohio state university school country and state law school important to tease out a few of the options that might exist for collaborative work. she's gained some attention rightly because of that. >> we have another question here in the middle. >> nine by noah al-assad is any gpa. are you talking about it breaks my first test and if so, does the law firm or any entity use the feed? >> i actually was part of a leg research company for six years and we help law firms utilize that. they used a variety of instruments that the industrial

organizational gold standard of these hogan product. and so they use an assessment called the hogan personality inventory and developmental survey instrument and they look for the positive attributes in a negative attribute on the job performance. they have been validated over a long period of time in many different professional settings. larry richards is the law firm space that has a big crack is built on ground. -- the practice built on ground. tens of thousands of law firm partners and leaders that participate. there are folks out there that are using them. there is also other tests in the situational judgment the

biographical inventory like a money ball type instrument. all of these things predicted way better. as a matter of fact, some of the academic rigor or his word negatively correlated. so the higher the lsat, blow your ability to develop business or the higher your undergraduate gpa, the lower your development. there is a huge amount of effort that can be put in for my law firm that begins to think who successful in the practice and why. they can't build self reflection unfortunately. the legal part is dramatically under a bus kind of thinking through lawyer development and there's tremendous research and i'm happy to go offline and equate somebody were to begin on the inquiry. >> they talk a little bit about

the role of our exam plays in this process. the correlation between bar exam passage and al-assad and the extent to which we have built-in sort of our own piece of the puzzle that we may need to unravel. >> yeah, i'm not sure i exactly know where you want me to go with this, but i will go with what i know. the lsat is a pretty good predictor of our past. it is six times more effect to as your law school grades. your law school grades should be a function of your lsat, but you measure the effect of one verse together in your law school grades are about six times more important then. so independent, what would predictor law school grades? motivation, all the factors that

judge kourlis was referring to showing now. i at least take attendance in my class and i have always struck every eric at the same results here. you can if your, get a lower grade. some of those people have high scores and so i guess i should ask judge kourlis, is that the point you're trying to contract me towards? ..

whether we should be thinking about changing the bar exam, asking different things, directing different kinds of skill set development through the bar exam is the issue that i was trying to put on the floor. >> in new hampshire they actually ran a pilot program, won't get the name right. put out by the franklin pierce report that basically allowed law school graduates in new hampshire to opt out of the new hampshire bar if they actually took a series of courses that were a simulation, and basically built a portfolio, and the

takeaway for the employers was the young people who went through the program were demonstrably better on so many different dimensions which gets to judge kourlis's point itself the admission demand from the bar on who gets admit, we could have people who are better prepared and more efix tv for the service of -- effective for the service of clines. >> the early day of that option a small number chose to do it and thought it would be harder. think there would be some interest in altering the final entrance, if it were vat automobile and reliable i. was in a drug court setting where a lawyer couple in on behalf of client and said my client doesn't want to do drug court. heed rather go to prison. what he meant was drug court is pretty darn tough, and it would be easier to spend two years in a jail m -- somewhere to go to

prison the key is, what do you want to measure, and can you create a measurement that actually holds people accountable and separate out the people that we shouldn't turn loose on clients from the ones that it's okay to turn over important things to. >> other questions, judge. >> all right. i'll just -- >> we have a microphone right there. >> i want to go back to the slide about chase and replacing equals with technology and the like, and there's a lot of that going on, whether it's discovery or -- [inaudible] -- i want to know why we're not talking about shrinkage of capacity in law schools, fewer law schools, fewer lawyers, over the long run, because these jobs are not going to woman back. the document reviewer jobs are not going to come back and they

probably shouldn't. >> so, should we repeat the question? >> the question is, why isn't really what you're talking about just shrinkage in the size of the legal profession, which at some point is going to mean we don't need 200 law schools. maybe we need 150 law schools, maybe we should watch that curve go down further. >> i agree with you, it's going a be a long, slow, painful process, and which law schools close -- the academy will have a strong opinion on that. i completely agree with you. i think that we are clearly headed down that road. we have a lot less people making cars in the united states than we do 20 years ago. >> i just want to say, this is not a problem unique to law school. >> it's not. >> the federal trade commission has done studies about how

industries should -- the word they uses is "rationalize" in the face of declining demand if you're the horse and buggy industry, demand is just declining so there's a big debate, at least in antitrust circles whether you should let the market do this because it will be a better result ultimately even if painful along the way, rather than from the top down. >> i would add just one other consideration, and bill alluded to rebecca sandifer's work on the just gap, the unmet legal need in the united states, is burgeoning and you can see that from the self-represented litigant numbers, pro se litigant numbers. so, yes, think part of it is downsizing of the profession, but i think there's another component here about the profession's sending the people who need legal assistance and

doing it in innovative ways that are affordable. unbundled legal services is an example. maybe we develop things like limited license legal technician options such as the state of washington and now utah. maybe we become more involved in the kinds of online services that are being provided, and playing a role, and then julien henderson -- >> hatfield. >> yes. >> that would take the position that there is a unique role for lawyers in the kinds of systems that are being built through ai, autonomous cars, for example, thinking through how to build out the rules for what autonomous cars do or don't do. i think there's a demand for legal services that is huge, it's a question of how we're going to meet that demand and whether we're actually going to

come up to the mark in changing the way that we think about the product of what a lawyer does and how that lawyer is trained. >> one thing you can say is that the law schools are shrinking and was very surprised last week, spent time with the fall of 2016 matriculated laws and mat ma trickett rated as few as 60 or 70. the justice department has taken a position that any check tv, a aimed at shrinking the number of competitors is frowned upon to say the least. one thing hat happened on most grads the law schools sent money into the central administration budget of their universities and now it's quite common to have -- money has to flow in the option direction. -- the opposite direction, a very difficult time to be a law

dean in an american law school, and the number of people who find employment as legal teachers has been shrinking. and the number -- the amount of money that people like whittier say they choose no longer to subsidize the organization and the subsidies ran back into central administration for departments for which there was not much market demand. >> who paying for this? it's all fedly financed and the u.s. taxpayer this guaranteor. that should fight the trust analysis somewhat. >> other questions. yes, sir. >> i was wondering if you could talk more about -- over here -- >> there we good. >> talk more about the studies trying to correlate essentially young lawyers' at tribuschs --

attributes, how do those studies measure career success? i assume if, as you say, the highest paying jobs are using the wrong metrics, using lsat and if you just measure, say, lifetime career earnings you'll find strong correlations between, say, -- that kind of metric. how are you defining effectiveness or success in those kinds of studies where you're finding stronger correlations with things like conscientiousness and timeliness and those kinds of traits. >> you have to back out from that analysis the kind of initial effect of lsat on four first job. your first job affects your employment, your lifetime earnings, but specifically going back to the berkeley study, the schultz study, they took the 26 effectiveness factors and

through focus group developed descriptions on the positive and negative side for what is a -- seeing the world through the eyes of others look like, or the ability to develop a business or integrity. what it looked like positively and negatively. and then through a series of iterations, were able to -- where they asked practitioners to place those along a continuum to give them a specific score. this led to the effectiveness was that the rate scales were very reliable and valid for predicting performance, and so that is how they -- how the berkeley study was conducted. just took those behaviorally anchored rating scales to supervisors and employers and got the scores and correlate with lsat and it was de minimis.

so in my own study, when you look to see how elite school does in terms of persistence to partnership, they good at oversames and then the persistence of elite law school graduates to kind or persist until partnership is lower than regional law school graduates. probably because of fact that te awe firms so preference top ten or top 14 law schools and so they kind of look askance at evidence that maybe the person really doesn't have the motivation or the characteristics to succeed their long-term , and think that is rooted in a historical bias in the bar that privileges -- academic credentials extrapolate too far. to be an effective lawyer is a bigger tool set than just academic ability. the hiring bar in the judiciary

overweights academics. you play it over a loaf crime and academices matter but not as much as we think. >> there's time for one more. >> i want to talk about law school curricula for a moment. just kourlis' survey said something i thought for a long time as far as hiring, things like law review. you have the top of the top members of each law school classing themselves running for law review and learning a skill that is frankly useless. doesn't correlate with good writers. they learn how to write for law rescrews. they don't learn how to write briefs or how to write as lawyers. wonder if the law schools have looked at focusing on what helps them learn how to be a good lawyer and a good writer. >> sounds like they should. >> take her answer. >> go ahead, you can answer. >> yeah.

so, the answer is there's some pretty innovative things going on in law schools around the country that are directs toward trying to produce better writers and better thinkers. the problem is it tend -- the innovation is concentrated in the bottom two quarters of law schools in the country. the top ranked law schools doesn't seed the need to do those sorts of things. they have higher employment rates and are resting on their laurels. so from our perspective there are really innovative wonderful law schools but most of them are the ones struggle with their market share, which is not the right message either. >> all right. think we will leave it there. thank you to our panel for this morning's presentation. [applause]

we have a 15-minute break on the schedule. please come back. nexted to say to you there's wi-fi available if you want it. the password is 7 circuit. should be on the slips on your table. see you in 15 minutes. >> today a look at the influence of isis in south asia. that discussion owessed by the atlantic council's south asia center is live at 1:00 p.m. eastern here on c-span2. online at c-span.org and the free c-span radio app. this week on c-span: >> the eclipse is important because these bodies comp into align independent a cosmic

moment that we're all being part of. >> tuesday at 10:00 p.m., live coverage of president trump's rally in phoenix. wednesday, at 8:00 p.m., former presidents george w. bush and bill clinton on leadership. >> a bert life if i help somebody else have a better life, too, and i liked it. got lucky. all these people tell you they were been in a log cabining are full of bull. >> we'll look at pending proposals for the federal budget. and friday, a profile interview with agriculture sect sonny purdue. >> my political heat was i tell people when i was born in 1946 in perry, georgia, they stamped "democrat" on your birth certificate. i made a political discussion, call it truth in advertising, in 1998 to change parties and became a republican at that point in time. >> followed at 8:30 p.m. by a

conversation with jeff moss. >> there were no jobs in information security for any of us. only people who were doing security were maybe people in the military or banks. so this is really a hobby. at the internet group -- there were jobs and people were putting things online and there was money at risk. all of a sudden, hackerred started getting jobs doing security. >> which on c-span and c-span.org and listen using the free c-span radio app. >> c-span's voices from the road. at the national conference of state legislatures summit in boston, asking attendees what it the post important issue to your state. >> what is really important to our state is that washington makes sure we maintain health care for the poor, the elderly and the infirm. we have to make sure if we replace obamacare, we replace it