going to be like what we did. i do think our actions have had a trickle down -- >> i think charlie referenced this, but it bears repeating. the commercial payers by and large are looking at parameters. the doctors aren't necessarily looking at fee-for-service, but they're insisting there's quality reporting. but the so-called quality reporting does not take consideration into cost. it's not high value care. it's measures that don't consider costs. and i think the next generation of reports that physicians are expected to turn into the payers will be high value care. good outcomes but high value.

i think this will move it forward. >> if you're a patient, you care about your metric. it's wonderful we're monitoring if doctors give quit smoking advice to patients. i'm not a smoker so it doesn't matter to me. what you want to know is -- i know somebody who's about to have heart valve surgery. they want to know which doctor in their city performs the most heart valve cities and how many patients are alive a year after that. the fact we cannot answer that for that patient today or tomorrow or even next year, like, that's the question we want to answer. it's not like, do your diabetics have their a1c under control? i'm not a diabetic, but if i'm diabetic and it's under control for me, i don't care if your other patients are, i care about me. we're a "me" culture but we have to make sure the data releases can be customizable so the person interested in their own health and their own situation

can get some answers from it. >> charlie, at the same time, patients are complaining all the time about high co-pays, deductibles. we spend billions of dollars a year in unnecessary care or marginally useful care. and this conversation and this data release gets us closer -- or gets us away from spending all of those extra dollars that we don't need to spend. >> no question. >> so i think it does come back to haunt the individual patient. if your co-pay is going up, your deductible, the cost of your insurance is going up. the reason is, we're spending too much money. how can we provide high-quality care. we have to factor in koos long with outcomes. >> we have a question here. tell us your name, where you're from and your question briefly. >> yeah. my name is bea young, i live in

maryland. first, i congratulate you, transparency of data system is great, but i wonder how far can you go, how do you deal with -- you didn't get data from internal system because the nurse did not give it to you or you did not even maintain the system -- the information you receive from patients. it's very critical, very important. currently today there's a letter about who are the physician in which hospital in which area, the checkbook or reference you or government agency, there's nowhere you can find it. a lot of -- they use false name [ inaudible ] so how do you catch the system, the real information that you

should need, like from computer, consumers or patients. a lot of nurses don't even give patient treatment if they ask for it, the record. they don't give the record. if they do, they give it fraudulent record, fraudulent charges. that type of thing. things are very critical, very essential. for you to compare a patients, physicians and real professional and fake. because a lot of emergency staff, they ask why are patients -- give the liquid to the patient so they can perform the procedure. and if you ask any physician for

proof, they say, no, they don't have it. >> let me try to wrap this up so we can get -- go ahead. so, misdiagnosis as well as patients -- all the various reporting -- >> system and other procedure that i -- >> i got up. >> i got it. this touches -- thank you. this touches on a couple of things our other panelists might get into it, especially our last panelist. a lot of our work started with the physician data set. but as we're hearing, there are so many streams, and you touched on this, wanting to know about the referral network as the net set of data that could be important to triangulate. we're talking about painting a better picture on the complexity of health care services. do any of you have some comments on -- we mentioned hospital, we talked physicians, drugs or

devices, other pieces of this puzzle which are much more complicated than what we've touched on. >> so her end point, when you're talking about physician services and medicare part b, that's a payment going out the door, sflit when you're talking about part d, prescriptions filled, at a pharmacy, they log a prescription being dispensed, maybe it's not bes dispensed. they're being paid for it being dispensed. when you talk about a referral, there are two different claims that are made. when you talk about quality metri metrics, chuck, i'm sure can you get into this. the quality of electronic records, the quality of paper records leave so much to be desired, when you're trying to adjust for things like that, we're a long, long way away. >> and thinking about different settings. hospital, hospital, physician offices and then thinking about er settings, intermediate care settings. the complexity of all this can be so overwhelming it could cause you to say, why bother

releasing anything because we can't do all of it? it seems like the first was the hospital data, which you talked about. and thinking about physician charges because of the florida case. know we've also got -- new to me was the information around dialysis and some of these other services we do have data released on. are there other transitional care or care settings we think we really need to have better transparency around that would be important for patients to have? >> well, not to -- well, let me plug the commonwealth publication on this. for those who haven't -- >> unsolicited. >> there are two things that need to be stressed. first, the physician data has to be integrated with the hospital data, with the pharmacy data, to look at any of this data in an

isolated way, in a way it addresses some of what you're bringing up. to do it isolated doesn't give you the right picture. have you to put it all together. and comparisons are really difficult, but as the commonwealth publication pointed out, until we get into real detail about outcomes and health status and demographics, we'll never really be able to make good comparisons or the perfect comparison. so, hopefully one day we can get there. and we're moving in that direction right now. >> i'm going to do a lightning round to ask very quickly, ten seconds, 15 seconds. what has -- what has been released to date? what have you learned from it? something just to give our audience in the room as well as on the website something that may have surprised you about what has been released to date.

i'll just go ahead and kick it off. i will say the thing that did surprise me is exactly what charlie touched on. this was actually mo to -- not my patients necessarily but hi more physician colleagues and health insurance plan colleagues kind of say, this is really exciting and interesting and we'd really like to get into that. something that surprised you or something you learned from this release process. >> so, i'm a geek. i like looking at patterns. in the physician data, just looking at how different services were concentrated or not concentrated in certain specialties. i think the other thing -- the thing that surprised me the most, actually, was how few cpt codes physicians built. if you had woken me up in the middle of the night before i started the project and said,

how many cpt codes are physicians billing, the average primary care physician, i would have said easy, 50, 60 a year. maybe more. for many -- for a significant chunk of physicians, they are billing 10, 15, sometimes even less. but they're -- some of them are really cranking through the cpt codes but they are specialists. so, that was the biggest surprise to me. >> perfect. >> i had grown up as a health reporter lerping about differences between hospital referral regions and how important hospital referral regions are. i think what i learned through here is just how much variation there is within those hospital referral regions and you can literally go one street over and the quality of doctors as measured by services they use, drugs they prescribe is so vastly different and it makes a difference to look up an individual doctor. >> chuck? >> i looked up my data. i thought you were paying me more.

>> i have to take a cut off the top. >> that's niall's new job, gets a cut off the top. >> i have to compliment cms because i think you got it right in my case. but i've been doing this for more than 30 years. and it it was really hard for me, despite all of that experience to make any sense of it and i began to imagine, well, what are patients thinking? how do they figure out what's going on here? >> that's a perfect segue for some of our future conversations. join me in thanking our panel. and we've got -- let me do this. if i can ask our second panel to come to the stage.

i'm going to have paul ginsberg from usc schaeffer policy, who also has an extensive background and bichlt o, i'm not going to read, as well as a number of publicationings related to research in this area. he's going to be leading our second panel. let me make sure i don't get in everybody's way here also take off -- i'll have paul -- oh, good. paul didn't need to use the stairs. paul, sit down and then we'll remike. we'll see if we can swap out some glasses for you guys. paul will be leading the discussion on implications and perspectives from researchers, policymakers and payers. we teed up a few of these in our first discussion, but i think we'll have a robust one with the second panel. thank you.

>> thank you. i think we can start now. really pleased to be moderating this panel. i think stu at the beginning of the conference set the tone about this is transparency. this has been a long time interest of mine. i've always believed some of the confusion about transparency comes from the fact that we talk about transparency because of its aspect of sunshine. we believe public institutions but other institutions, their operations ought to be more

transparent. we also talk about transparency in terms of it being useful to different types of entities. we think of patients, consumers. when we think a little further, we think about physicians, hospitals, health plans, researchers and policy makers. and this panel is really about what the three different audiences can do with this information. and we will given with jay white, a very distinguished researcher, talking about what this means for researchers. then i'll hear from lew sandy, an executive with the united health group, talking about what this release can mean for health plans and lawyers who are their clients and some of them do it themselves. then we'll hear from bruce with

a long policy background to speak from the perspective of policymakers, what can they learn from this information that will help them in their work of making policy down the road. there are two contacts i'd like to ask the speakers to think about. one is that as we were going to talk concretely about the april data release that niall brennan was describing to us, but we're not going to dwell so much on what was in the april release but also to talk about what could be in further re -- future releases and whereas niall couldn't talk about what will be in the next release, they can talk about what they want to see in the next releases to make it more useful. the other context is this is not -- these releases are not the only game in town. you know, with chapin as a researcher, he can go through

the process of paying a lot of money and taking a lot of time to negotiate a data use agreement with cms and get some of the data that's really raw behind this release. li lou with united health care can either work with a qualified entity or apply to become one and also has another channel. this is also part of the context. i'm going to stop my talking. >> thank you, paul. it's really terrific to be speaking with you all today. and i think the first question i want to get into it whether this data release of the physician charge and payment data is a big deal. again, i'm coming at this from the perspective of a research. we had a couple conversations at rand, salivating over this new data source. what can we use it for? what questions can we ask? we figured out, well, there

isn't much new we could do with research. any research questions we would want to get into the claims data that we do all the time. does that mean this data release is not a big deal? not at all. i think this data release is really important and is a step forward. why is it important? number one, i think the release of this data has laid down a marker. for decades as niall mentioned this, there was an injunction in place that prevented cms from reporting what individual physicians were being paid by medicare. by releasing this data, cms has

said that balance between physician's right to privacy and the public's right to know what's going on in the medicare program, it's now officially tipped in the direction of the public's right to know what's going on in the medicare program. the other thing that's important about cms's release of this data, they've laid down a marker and said, we can release data for a specific physicians and cpt codes without running afoul of hip a. if you do research, hipa is a fast barrier reef that you have to learn to navr gate. and cms has charted a course others can follow it now. that's the first important thing about this data release. the second thing is these data by themselves, aren't informative or telling but data becomes useful when it's put in combination with other data. as cms releases more data over time, we'll start to see trends.

now that we have the physician level data, we can confimbine i with the hospital level data for geographic regions and get a fuller picture of what's going on. as other data sources become available identifying physicians by npi or national provider identifier, we can start combining this cms data release with those other npi level data. and conceivably at some point private sector health plans will begin releasing data if it's released in a format compatible with the data cms is releasing, then you have a three-dimensional picture of what's going on in the health care sector. the other last point i would make is that this data release is a small step but it's a step in the direction of building our data infrastructure to understand where we are and

what's going on in the world and to help us make better policy choices as a society. and i think one thing we forget is that our data infrastructure has been built through conscious, sustained effort. and concepts like gross domestic product, we take that for granted. we track how it's going up and down over time. it's grown by such and such percentage. the concept of gross domestic product was defined and the tools formering it have been honed over decades and we devote significant resources to tracking that. that's a tool for the use of businesses, government, individuals and so on. it's a collective data resource that has been built through effort. and i think that is an incredibly valuable effort. i think the cms data release is one piece in that valuable effort. in terms of where we're going next, what i would be interested to see, the private sector is

kind of the black box. medicare is starting to lay all of its cards on the table. but in terms of prices, practice, patterns, geographic variation in the private sector, is more challenging to get a hoojdz that. and to the extent we can get private sector data releases that are structured in the same format as the cms medicare data releases, then we can start comparing prices, start comparing practice patterns, identifying geographic identification patterns in both of those sectors. that to me would be the most fruitful and probably also the most difficult avenue to go. i'll leave it at that. . >> one question fof you. probably better serves many of researchers' needs. any thoughts about how that process could be streamlined, could be made more efficient as

ab an aspect of the transparen initiative? >> well, the data could be made cheaper. we pay taxes for a reason. we pay taxes so the federal government can provide services that shouldn't have a high marginal price attached to them. data is one of the thijs that should be made freely available even though it takes a lot of resources to produce that data. once you make one copy of the medicare data claims file, so, the price could be made lower. the turn-around time could be reduced. that's a significant problem. hipa, that's just a fact of nature at this point. have you to satisfy all the privacy constraints. but the turn around and the price, i think, are barriers to research that could be lowered.

>> thanks. lou? >> well, thanks. thanks, paul. appreciate the chance to be here. i appreciate stu and the commonwealth fund for sponsoring this. what i wanted to do is talk about what is useful in this release, what are some limitations and some of the suggestions, to your point, paul, about where do we go from here. first thing i do want to say, as i really want to commend cms for their efforts to promote transparency and the use of these data is refreshing. and it's incredibly important. it will evolve this idea of evolution over time is an important theme of this event. you know, even in this current release, there is some utility in this information. we've heard a little about this today already. i think it's useful in raising awareness of variation. the pervasiveness of variation that we see out of the data.

useful in understanding the variations in that data. useful -- frankly, it's so obvious, doctor cutler mentioned this, useful in bringing transparency around what things cost. just the basic information there. and i think claims data useful for some thing, not so useful for other things but it has useful dimensions around volume, around the service mix. i'm not in practice anymore. i'm a general internist but i looked up some of my friends to see what their practice pat ites looked like. their practice would be heavily concentrated in a senior population. it's important, and it will provide useful insights in its

current, limited form. now, what are the issues and limitations? i think the most fundamental one is health plans have learned over the years that looking at raw claims data is really more an enter size in high hypothesis generation. you can't look at a raw data table and say, a-ha, there's an outlier, let's do something it's really more a hypothesis you have to triangulate around, look at multiple data sources, look at trends over times. and then have a further exploration, particularly it may involve, this is something health plans do regularly, have a conversation with that physician to see -- you might find an outlier that statistically is an outlier, but they actually are referral practice and have a unique set of services they only -- they're

the only ones who provide that in a region. i think hypothesis generation is the way we should think of raw claims data. the second thing, as i said, a lot can be done with claims data, but it requires analysis. all of us will say, it's not just putting the raw data out there. starting with some simple analysis, some simple descriptive statistics can be very useful. percentile rankings as we her from pro publica, standard deviations. then there are even more sophisticated uses of claims data through the use of technology episode groupers and the ability to do case mix and risk adjustment. so, i think there's quite a bit that can be done with claims data, even with some of its limitations. and i think it's an interesting policy question about how much the sponsor does, in this case,

cms, versus having a raw data file come out and have third parties do that kind of analysis. that's an interesting question. and i think in terms of physicians or any stakeholder, i think probably the most important at bud of this kind of data is the ability to make comparisons. and to do that, you have to have credible data. dr. cutler mentioned physicians want to compare themselves naturally against other physicians in the same specialty. that would be logical. and i think that's another thing that health plans have learned over the years. and that relates to my third suggestion or limitation, is that what we have learned over the years is particularly when releasing data around physicians, it's a good idea to run what i call service bureau function. the ability to serve the stakeholders and in the case of physicians, to answer their confess about the data, for them to be able to look at it and

correct it, if there are some errors in it. physicians are very data hungry, interested in data. if they view it as credible, but they want the ability to do what they call drill down. to drill into the data in a way that makes sense to them. all of this is what i would call a service bureau function and the things united health care has done, as in our premium designation program, we've done since 2005, we have a physician portal that has hetus data on it open to all physicians in our network. this service bureau function is an important suggestion how to enhance things going forward. lastly, let me make a few suggestions for where i think this could go from here. i think i've said some of these already. first is to just clean up the data and offer some simple descriptive statistics for all stakeholders. i think that's a reasonable thing for the sponsor to do,

even as others can do other kinds of analysis. second, i agree with your point, the ability to combine data, data sources to get an important view, in this particular case, it's really quite a limitation that you can't get the full picture of what the physician is doing in medicare. we have found in the commercial private sector that we might find a physician expending a great deal of resources in the office but if you look at total cost of care or episode basis, they're actually quite efficient. the reason is, that's their practice style. you won't see that without the ability to integrate and combine desperate data sources. you know and it was already mentioned, the health care cost institute, that's why we thought it was so important to contribute data to the health care cost institute as a multi payer incident. that's an interesting for example at a couple of levels. it originally started as an effort really to just be a

tracker of private sector costs trend data and utilization data. as was mentioned, once parties got together and they started putting the data in, the hcii said, we could use this for transparency purposes. we could start to look at quality data in addition to just cost data. as some of you may know, that's an effort that's been announced and will be deployed in 2015. to some up, there's a lot of opportunity. this is the beginning of a new era. transparency is the realm. the current release has been useful in some respects and limited in others, but i think

it will evolve and strengthen over time, particularly if it moves along some of the directions that have been suggested already. >> thank you, lou. one point you made that got me thinking. you were talking about one of the steps forward is to go from really raw, unanalyzed data to at various levels analyzed data. as this is evolving, we're on track to have physician value modifiers in medicare, which will be presumably fairly analyzed data. i don't know what the transparency plans are, but assuming there are transparency plans, any thoughts about these two different tracts things are moving on? >> again, whenever there are multiple initiatives going on, it's an interesting challenge for any organization to sync them up. but i think -- and i think that's why it's a great opportunity for someone like niall to have a broader view of

all the broader resources, to use them internally and externally. i think in general the idea of moving -- i think probably the most important thing is moving beyond fairly rapidly beyond just a raw claims release into analyzed data that is consistent in its thematic direction. the measures might be slightly different but as long as, this is the domain, i think most people view a best practice, for example, is not just to release cost data without quality data, to combine those. and i think having so that any user can have a picture that integrates both quality information, to the extent one can have it, with cost data would be useful. that's a partial answer, i guess. >> thanks, lou. bruce? >> my job -- thank you, paul -- is to take a broader policy perspective and the u.s.

congress and how they might benefit from this data release and subsequent data releases. and i'd like to start as well by keeg off stu guterman. it's easy for us to value transparency because we're the lookers. we to want see more. but it's the lookees, the residents of that black box who need to accept and even embrace the concept of transparency, if we're ever going to really achieve it physicians and other providers need to get used to the idea of having their performance evaluated. as dr. cutler has suggested to us, their willingness to do that is partially dependent on how accurate and useful the data are that are being released and how they'll be utilized over time. you asked, paul, for some suggestions of what we would

like to see. i actually have some predictions about what we will see. now, niall has not been able to say they're going to release more data. i'm going to predict they will. in fact, i'm going to predict cms and its contractors will correct a number of errors that the media organizations have pointed out. missing data, poor specialty designations, other things of that nature. i think providers will contribute to that data. they have an insen stif knowing their information is going to be available to a wide range of lookers, they have an incentive now to make sure their specialty is accurately recorded. they have an incentive to match up the national physician identifiers with the billing codes. i hope we'll have more kinds of different providers available for analysis in the future.

most importantly, as soon as we get that second data release, we have the beginning of what, chappen? a time series. every economist wants to have a time series. i'll give you an example of how that could be useful. the government accountability office took in 2005 and 2006 data, the exact same database that cms released in april, and attempted to identify beneficiaries who are high utilizers of service given their health status. the correction for health status, as everyone knows, is not perfect but we were able to identify a group of beneficiaries that were high identifiers. then we examined those same beneficiaries in a subsequent year. did they remain being high ut lighters. they had a tendency to be high utilizers. a tendency.

we identified the doctors as high utilizers given their mix of patients, health status and how many resources they consumed. then we looked at the doctors the following year. they were twice as likely to remain high utilizers as their patients were. and i'll be glad to give you a reference to that work and -- i can't go into all the details. certain specialties and certain geographic locations. it's the kind of thing one can do, time series data one can't do with a cross-section. i worked for congressional support agencies for about 12 years. at gao i supervised the preparation of roughly 50 testimonies and reports to the congress, different congressional committees of jurisdiction over medicare and other health care spending. in case you were wondering, what do members of congress members reading, gao reports or

newspapers and news summaries? anyone care to guess? by the way, if you ask the same question about congressional staff, you get the same answer. so i look at these news organization's reports and analysises of the data cms released as another potentially important source of information. maybe it's not in depth. maybe there are mistakes being made. but i think it's especially useful when you consider that congress objects much of its information, many of whom are constituents that have a vested interest in health care spending. i remind you, of the five top d.c. lobbies in spending, four of them are health care and the fifth is warp. to have another information source with incentive for producing information is useful. in fact, i would say that these

news organizations' publications are complements to, not substitutes, to the research congressional organizations do like gao, med pac, and rand under contract with cms, these are complements, not substitutes. i also would expect because these news organizations will produce the information to congress and give them ideas on what could be investigated in greater depth, there might be some benefit to the congressional support agencies to be asked questions that were raised by these news organizations to be investigated in greater depth by the organizations that have the resources and capabilities of doing it. my bottom line is this is not only a good thing. it's a necessary thing. it will produce more meaningful information as time goes on. we all believe transparency an

integral part of health care reform. i look toward to the day where our database is not just cms production on medicare -- because after all, medicare, the entire story is utilization. because prices are fairly constant except for geographic cost of business differences. you get into the private sector, you see a huge amount of variation so not only would you have the advantage to combine across different payers to get a fuller picture of what individual providers are doing, to get the private sector data into that mix and you'll have a lot more opportunity to investigate very meaningful differences and how services are performed around the country. >> thanks, bruce. you said something very interesting about how some of the media that, you know, was based on this release, you know, made its way to members of congress and staffs. and likely shaped what they were

asking congressional support agencies to look at or even talking about policywise, legislativewooiz. do you have a sense about interesting directions this has spawned? >> was that the only interesting thing i said? >> the most interesting. >> the most. i don't really know. this comes under the heading of hope for. i did look up and saw that the hill, which is a publication for members of congress did pick up a number of these stories and reproduce them in their own media. but i can't cite a specific instance of where an issue was suggested to a member who then requested work of gao or any other support agency. >> maybe in the course of conversation others will have ideas about that. probably a good time to turn to the audience. we've got time.

why don't you wait for a microphone. >>. [ inaudible ] >> doctors, i don't think, are required to post online what they charge. and i was wondering what you would think of that? it's one of the few services that are important, that do not have the market visibility that you have in other areas. do you think we should change that? >> this presumably is not a medicare issue because medicare physician -- i guess they're very limited constrainted about what they can charge in medicare. of course, those prices coming. i guess that's more of a

question about private insurance? i don't know if you want to take that, lou. >> yeah. i think it's just -- the question reflects, again, the desire to have greater and greater transparency on all aspects of health care, including what providers charge. two things. even though there's no national requirement, i know at least in massachusetts, if there's anyone here from massachusetts, there is a new law that requires exactly what you suggest, which is the posting of crisis so that will be an interesting experiment to see what impact that has in that state. there may be other states, i'm just not aware of it. i think the other thing is, from the consumer point of view, and we'll hear from the consumer panel after us, i think what consumers want to know, our experience has been what consumers want to know is not what the provider charges so much as what am i -- what do i have to pay, which then relates to what is -- what is the charge in relationship to the benefit package i have, the network that

i have and what does it mean to me. >> and perhaps even a whole episode of care. >> exactly. sort of if i'm choosing option a over option b. so, it will be interesting to see. that information is increasingly becomi becoming available, again, in the same transparency theme we're in. >> i think the gentleman there are-h a question. >> thank you. i was wondering about the difference between the raw data you have to pay for and the data that's been released, is it primarily a question of consolidation so you download less data to answer the same questions? is it primarily hipa? is it some combination? >> if you go through the process of entering into a data use agreement with cms, you and you guarantee you're going to satisfy all the data security

safeguards they require, which are serious, they will send you claims data that has a beneficiary i.d., that has all of the age, sex, zip code of residents of the beneficiary that has the microlevel services. the cms physician public release only reports combinations of physicians and cpt codes with 11 or more services provided. obviously, if you buy the microlevel data, you can see all the services. but it's a major hurried toll set up data security systems that are impenetrable and hipa compliant. >> capability to pull together all the different services that the beneficiary has had. >> and you have to promise not to release anything that violates hipa. >> question back there.

>> david costa. if the genie is now out of the bottle, from your perspective, what would be best next relative to cms data release? >> i can -- well, paul ginsberg actually mentioned the physician value-based payment modifier. i'd be curious, how many people, if i say value-based payment modifier, know what i'm talking about? some? a lot, great. if i had a wish list, it would be that the raw data and the final output of the physician value-based payment modifier calculations be made available at the physician level. i don't know if that's realistic, but that would be taking this raw data dump that cms has just made and doing the analysis that a lot of the other presenters have been talking about. the idea hyped the payment based

modifier is can be measuring quality and measuring efficien y efficiency, including all of the services physicians order, not just the one they provide themselves. so, i think that would be a significant step forward. i doubt it's going to happen. >> bruce? >> well, i think a second cross-section, as i mentioned, would be very useful but it would be especially useful if cms could correct their errors in the first release. so when they issued the second release, the first release is more accurate and, therefore, you could make better comparisons over time. >> i think a theme a number of us have talked about is more comprehensiveness and more linkages of the data. for example, part "a "and other data sources so that you can have a more comprehensive view, either of an attributed population or of a more 360 view of what a physician is doing for the seniors that he or she is treating. >> if i could add one thing,

this would be even more advanced, would be, grouping data into episodes of care to look at how a physician handles, say, a joint replacement or some other, you know, well-defined episode of care. to really be able to compare them to others. yes, sir? >> i'm frank with the brookings institution. my question deals with what this might mean for the health exchanges, particularly for each of your constituency groups, you know, policy makers, researchers and payers. you know, even though this data is only medicare, what might be some interesting future developments we might see with the health exchanges with more cms data releases? >> that's a tough one. i think only in a general way, if you can learn things about, let's say, drug -- prescribing patterns as a general utility, then the exchanges might be able

to use information of that nature. but since they're not generally treating medicare patients, i think that there's a limitation in what -- >> yeah, i would echo that. we've learned, and we have -- i didn't mention this in my comments, but we have since we provide services to medicare beneficiaricy, united health care does, we actually have data ourselves on a medicare population. we have found -- we originally thought we could combine all those data with the data we have from the commercial marketplace so we would have larger sample size but we actually have found that these are different population. it's pretty obvious. the commercial population, the working age population, is different from the senior population. so, when we do quality measurement and reporting, for example, we use different benchmark levels of performance between the medicare and commercial populations. i think that's -- my guess is the same phenomenon would happen

in the exchange population because that will be a different population. particularly, it would be evolving over time. >> thank you. any questions? oh, there's one back there. >> rachel hornsteen, i work in data policy at hhs. last year the office of management released the memo and one concern was the mosaic effect, how when you release different data sets and link them, there's the possibility of releasing identifiable information on beneficiaries or clients. can you please touch on that? >> i don't have anything. >> so, this brings up the elvis' doctor problem, which is if you report physician-level data and

don't have a minimum number of procedures or patients treated, and if you report on the practice patterns of elvis' doctor and everybody knows who thatis, then you can find out everything else about elvis. so that's the motivation for the 11-plus minimum claim requirement. you know, there's obviously a risk that really sophisticated data analysts are going to be able to take this data release and consume it with other resources and figure things about individuals. my guess is that probably a bigger risk from hackers just getting into the cms mainframe, but i don't know. i think that we're definitely in an area of risks and tradeoffs

and there is a benefit to these data being made available. there are increased risks and the more data sources are made available in greater granularity, the more risk there is for people to combine things in novel ways and kind of undo the privacy protections. i guess, just as a researcher, i would say let's keep in mind the benefit of these data being made availability and not just go with always the safest approach. >> i think the principle of the greater good does apply here. and, in fact, even knowing the data releases, there whether always be some errors and some misinterpretations of the data. and there will always be a physician who is unfairly singled out as behaving poorly when the person hasn't. but you have to look at those are the costs, but you have to weigh the benefits against those

costs. i think we're of the mind that they are exceeded. certainly privacy is one of the risks as well. >> use our credit cards all the time even though our credit cards are for sale to, you know, through other world hacker websites. so our financial system is constantly balancing the benefits of electronic data exchanges against the risks of malicious hacker activity and i think we're -- you know, we're gradually groping our way towards a similar kind of balance with health care data. >> well, i think it's time to thank the panel for their really good job they did. [ applause ] i guess we'll execute the same thing. we'll transfer this to senior fellow at brookings institution who will be moderating the final session.

>> okay. are we live now? i think we are. now we're going to shift the focus. we've talked about this data release and several other data releases and talked from the point of view of researchers and health plans and the policy community. but what it's all about is patients and consumers. and so now we're going to shift the focus to the patient, the consumer, and talk about what they need to know. and i hope we can be a bit more visionary and expansive in this

panel because we are where we are, but where do we want to be? i hope that these three representatives who work very hard on behalf of consumers over quite a long time can help us think about what do consumers really need, what do they need it for and how can that data be generated. to help us with this question, we have first robert krugof who is the president of the center for the study of services/consumer checkbook. and we have robert restucia who is the executive director of community catalyst, a consumer advocacy organization. and we have doris peter of health rating center part of the

organization consumer reports. and i think doris' organization and robert krugoff rate lots of things. not just health care. that's where you go if you want to know about cars or toasters or plumbers or whatever. but now we're talking about health care and a really important aspect of so many people's lives. so we'll start with robert krugoff. tell us a bit about what you think consumers need and what they might need in the future. >> okay. well, i think the -- first of all, i appreciate being here. and i believe our priorities should be to have good, raw data readily accessible. if the government can do that,

that's an amazing, wonderful achievement. and fortunately i think the government has recognized this is important and has made significant progress. i'm very happy about niall's recent appointment and loved the president announcing his appointment. siting the commitment to frame works promoting appropriate external access and use of day to whether in agate or granular form. that's what i think should be the government's role here. unfortunately there's still a lot to be -- there are good things that have happened and there's a lot to be done. i'll give you examples of the good and the bad that i've seen in the government making data available over the years. just sort of always gives me some anxiety that things might not go so well the next step around. in 2006 there was hhs's denial of consumer checkbooks, requests

we get claims so we could give consumers some information on the amount of experience each physician had with various high-risk procedures. and the government siding with the ama in an appeal after we won in the lower court. that was disappointing. the release of claims data that was much more restrictive than it had to be. the cms rules included anyone who wanted the medicare data in order to be performance measures nationally to have matching non-medicare data in every region. that's an enormous practical obstacle and it wouldn't have had to be interpreted that way. on the other hand, there was the great news this past april the government's release destroyed

the privacy against release. so that was an enormous accomplishment, i think. and then that was followed by cms's quick action to get physician identifiers in the large data files that we and others have been using for years. getting those identifiers right in there quickly was just exactly what needed to be done. and cms recently created the virtual data program to make those more accessible to allow them to be used remotely. still more expensive than i wish for a single person to use. although cheaper than trying to get those files straight up which would be in the hundred thousand or more range and quite unfortunate to cost that much. but there was some bad news when

we were told this virtual research data center system could not be used to produce quality measures on doctors. couldn't quite figure that one out. and so the door sometimes swings open or closes. but cms has to continue to work for openness in spite of opposition. the focus should be on making raw data available. i'm not optimistic that the government should be -- should itself be a producer of performance measures. but it should be a supplier of diverse -- supporter, i should say -- of researchers to develop performance measurement methods and put them where they can be critiqued by purchasers, consumers. we want the data, we want the

methods. but it's fine if the government wants to do its own performance even in the face of provider resistance. but we worry that such reporting would be too cautious to give consumers simple information that will make it easy to choose providers that would best meet their personal preferences. difficulty of compare website, the real lack of information -- development and information in the physician compare website. give one quite a bit of reason for pessimism on the government doing the reporting. and contrast that to the creativity of propublica which finds a way to put this data in a form that consumers can actually use and to be responsive, difference response to you than to you and to be able to find the data. all that stuff i think really private entities. and what data is needed to

support them for perspective let me share with you what we have seen from consumers. of all the many times of service providers we evaluate, the consumers look at most often are ratings of doctors. of course people say well, they don't look at quality ratings of doctors. well, maybe they don't look at them, but they look at them twice as much as ratings of plumbers and autorepair shops and about 13 times more than they look at ratings of hospitals. so that's an interesting perspective. we've also done tests of consumer interest in different measures of doctor quality. by far the greatest interest was in consumer survey ratings of doctors. and in results of surveys of doctors asking for their recommendations of other doctors for care of a loved one. both of these types of measures were chosen at least three times by at least three times as many

website users as ratings based on board certification and training or ratings whether nationally find evidence based medical guidelines for quality and efficiency of care. so where consumers are starting from may be different from what we might have expected. so now here are a few thoughts on possible measures for future expansion. and i'm not going to limit myself on -- you know, to talking about these large data sets. but just sort of try to think about anything that consumers might possibly find useful. so i think one measure is whether doctors effectively use electronic medical records. such information can be collected via the meaningful use program. this doctor use of these things. and the importance needs to be forcefully explained to

consumers that this really will matter to you or could matter to you. another might be to explore with specialty boards so they can let doctors identify themselves as performing let's say in the top quarter of other doctors on the certification process. that's something i've never seen done, but it seems to me it would be interested particularly because it might reveal something about doctors' diagnostic skills and one of the problems i think is that almost all of our measures out there, process measures, outcome measures even don't get to the question of diagnostic skills which are so critical to having a good physician. another opportunity would be more measures of clinical bad outcomes. in analysis we've done, we've seen at the physician level

statistically significant levels in death rates, complication rates, readmission rates. this kind of information needs to be public. if we had better data, measures might be possible to assess whether a doctor is prescribing too many of some kinds of procedures identified in the choosing wisely initiative. a very important challenge is to report on patient-reported health status and outcomes using similar types of measures. calculated some of these type of measures requires assembling better data bases. there's a continuing need to push for all-payer databases where it could be understood that it would be used for provider quality measurement. so talking about, you know, large national databases like that, that's a great thing. as an all-payer data base, i should say.

but participation and registries is important if the registry rules allow the data to be used on public performance. registries can collect information that goes well beyond what you're going to find in claims data. it makes sense to find out, i think, whether public reporting which physicians participate in appropriate registries and explaining why participation matters could put pressure on doctors and providers in general to choose to participate. but the big opportunity is in health information exchange. for performance measurement and various other reasons. that's -- it could be the heart of a lot of stuff. so that means we need to push for continuing expansion and standardization of information and electronic health records including patient-reported information and push providers to share this information in large regional and national

databases. a public measure telling which providers are participating in these exchanges might actually create incentives. certainly the government needs to invest heavily in this path. think of it as the interstate highway system for health care. it's a major investment and it really should be a focus. finally i'll focus -- finish on a relatively easy but strange path. getting a database of survey results on individual doctors. a database which could be to the group level and site. but starting with the individual level doctor. why the government does not push forward with this is baffling to me. we have a good instrument in our cg caps survey.

this survey of variants give good information on how well doctors listen, explain things, keep track of a patient's history, engage the patient in decision making. this is important stuff. a doctor doesn't listen, for example, the doctor can't do a good diagnosis. or come up with treatment plans that fit the patient. if they don't explain well, they won't be motivated to do their care. as one article in the journey of american medical association put it, effective communication has been linked with better treatment plans, more appropriate medical decisions, and better outcomes. and measurements should be at the physician level. dana and others in the journal of internal medicine showed that the individual physician, not the practice, group, or plan counts for variance in the communication aspect of patient

experience. so why don't we have survey results at the physician level. we hear it's too expensive. to prove this wrong, consumers checkbook did a demonstration in four metropolitan areas in collaboration with united -- thank you, lou -- cigna and local blue plans. showing a large number of significant part for physicians. the demonstration showed this could all be done at a cost of about $120 per physician and still show meaningful results. that all have demonstrated across the type data. and that $120 could be cut to about $60 or $40 per year since it's not necessary to do the survey once every two or three years. although physicians can be enabled to do the survey more

often if they choose. this is a big opportunity and one the federal government could make happen with a little support and encouragement. thanks very much. >> thanks, robert. that's a wish list of very interesting things. >> so, commune catalyst is an organization focused on gives voice to consumers particularly low income consumers. we have historically been fighting to open up the black box in support of robert and consumers union. i think it's important to look at consumers from two perspectives. one is the role of a user and payer. and other in terms of their public policy role. you know, with the trend towards cost sharing and health plans,

and it's become more important for consumers to be good purchasers. with the implementation of the aca, millions of people are looking at the exchanges in the market place to choose health plans. so i think we've had a lot of experience with that. and frankly i think they're still either not looking at that black box or they're looking at it through the lucite lens. because there's a lot of problems. consumers and unions -- consumer and payers even find the most basic information illusive. the urban institute recently came out with a report that is difficult to get information from many market places about websites, the market place websites about whether a health plan is tiered, what providers are in the network, and what are the cost sharing differences associated with those tiers. for the low income consumers who

most of the people going are low income consumers, quality data is complex and more -- very difficult to understand. it's clear that one size doesn't fit all. information needs of a 60-year-old is different from someone coming to checkbook. in terms of what we're talking about today, we need to start with getting the basics right. we need better information about who and what's in a network and what are the costs to consumers. charlie mentioned the importance of accuracy. the data needs to be made more understandable for consumers, taking into account the wide ranges of differences. community catalyst has been working to take up a single quality measure that could be broken down to component parts. focusing on the particular needs of consumers.

we agree with robert in terms of patient-reported information. we see collecting patient activation as a quality measure and a focus on outcome. and finally in terms of consumers, they need help doing this. you know, the average person is not going to be able to make the kind of cost quality equations we need in terms of coming up with values, without support. medicare beneficiaries get this sort of assistance through the state assistance programs. we have a consumer assistance program that's part of people focusing on enrollment. we need to sort of think about expanding that program. so in terms of consumers in terms of public policy, there's important pieces to really think about. community catalyst was one of

the first organizations to push for transparency around the conflicts of interests of pharmaceutical companies and physicians. it was actually the courts that opened up. charlie in propublica took that database and made it into an effective tool for looking at what the is relationship of a pharmaceutical industry to the physicians. that perhaps hasn't helped in terms of allowing the individual consumers to make choices about either their drugs or their providers. but it's had a significant impact on medical schools and hospitals. from a public policy standpoint it's made an important change. the other area we have been working on is around hospitals and community benefits.

hospitals want to keep what their community benefits are off in a mystery in the value of those community benefits. a mystery to consumers. community organizations want and need to that have information. we've been working hard to get a database of the information around schedule "h" which provides some of that benefit data with george washington university. so we're looking to move forward in improving that kind of data for consumers. consumers need to understand on the basics of health care, the idea of some of the broader trends in health care are a black box to consumers. and the government and consumer organizations need to push hard for opening up that black box. >> i'm from consumer reports and

i direct the consumer reports health rating center. our role is really into trying to translate that data for consumers and to try to put it in context and raise awareness of the problems of that data, bring it to light. that's what we focus on. and so it's really about having people use the data, not just having the data available. by using it, we are able to identify some of the strengths and weaknesses of that data. so at the ratings center where we work, a group of scientists and doctors and consumers and writers who take data from various sources and much of it is the government and translate that for consumers. and we look at the areas of drugs, preventative services, physician quality, and also health plan quality. so we have been doing this for about ten years. we reach about 20 million consumers per month. even more through our partners. so not just through our own

magazines. you're probably more familiar with our work in the area of cars and electronics. we often complete with those areas and make fun of each other. i'd rather be testing cars than hospitals. and we can't bring hospitals into the labs and all that. so it's a healthy rivalry, but i was just looking up some data the other day and i was really excited to see that in our own measurement of how our readers use data and understand these stories, that our stories on health care costs and quality whether it's about physicians or about hospitals or about drugs, sometimes supersede or rank higher than stories about cars and electronics and washers. so it's really a top of mind for consumers, these areas of cost and quality. consumers can understand it and use the information. so that's another common question i get asked. well, you know, consumers can't

really use it. but consumers can use it and they do use it. and we hear from them about it. and we hear from them about changes they've made either in the hospital that they've chosen or the drugs they take. more importantly the discussions they have with their providers. a lot of our content drives people to have discussions with their provider about the drugs or hospital. so it's important to reinforce that relationship. so the thing is we really appreciate what cms has done in releasing data. we advocate to continue that release of data. the important thing, though, that i see that's sort of been skipped over is the issue of safety and errors. that's one area that's hard to measure. and we seem to have jumped from sort of processed the measures all the way to patient reported measures like whether you can climb stairs. but skip measuring patient safety and errors. we've asked consumers in our own surveys about this and our last

summary about 30% of people self-reported they experienced one or more errors in the hospital. and that's probably a huge underestimate. because that's just what the patient noticed themselves. maybe not what went unnoticed to them. so i really feel like that not only in the area of data collection but also in the area of how we measure safety and errors, that this is an important thing for us to focus on. another way to get at this issue is through registry data. registry data is probably the most powerful out there. you've heard a bit about it in this panel. and i think that's an area where we'd like to push more to get that data available to the public. we've been able to be successful in a small area of heart surgery. we've worked with heart surgeons that have voluntarily released their data to the hospitals. we have doctors and hospitals that are high performing and some that are low performing that release their data. consumers can look at a

procedure they may be undergoing like heart valve replacement or bypass surgery and find those hospitals and physicians that perform the highest in these ratings of quality. and those quality measures look both at outcomes and also at complications in a reliability way. that's what we would ask that we sort of make more public and i think cms with the power that they have to require these sorts of quality reporting systems that they start making that more of a requirement in terms of being reimbursed by cms. and then some sort of more requests that i have is to not try to reinvent the wheel. i think cms has sort of required to do public reporting as well. it kind of makes a lot of confusion at the level of the consumer because those websites that are out there now that government puts out, hospital compare and things like that are very difficult for consumers to understand. and so i would suggest they

focus more on the data and the quality of the data and requirements for reporting the data rather than the consumer side of things. then the final thing which is kind of a little geeky is i would ask cms not to forget about infrastructure. those of us who use the data have to deal with the structure of the data. the missing elements in the data that make it harder to release accurate and timely reports to the public. so thank you. >> we've heard a lot about choosing doctors. we heard some about choosing health plans. i'd like to ask each of you to say a word about how you would hope the feedback to the providers themselves and to the health plans would improve the situation over time. because isn't that ultimately what we hope consumers having better information will accomplish? any thoughts on that?

>> well, we've watches this with patient experience data most carefully and tried to work with some of the -- especially societies to figure -- to tell doctors how they can improve and where they can find resources to improve. i must say, i don't think we've been particularly successful in that, but i think that's probably because we're not very good at it. we tend to be a measurement organization, not a quality improvement organization. but there ought to be potential there. however, i don't think -- and yes, that is one of the objectives. but i also think that one of the objectives is helping consumers find the good ones and that also gets you good care. there are two paths one can go down here. >> yeah. i think that when you look, for example, the pharmaceutical data, i think that you have the professional societies stepping up and understanding that there are implications to the conflicts of interest and trying

to address those with changing policies within the societies, within medical schools. and within academic medical centers. i think it is necessary to have a partnership here. i don't think the market's going to move us away from all providers who are quote, unquote, poor quality of care. it's not going to happen. we need to rely on a partnership between hospitals, doctors, consumers, to get it right. >> i have to say in our reporting of data, we really do see a big response by hospitals. first they get angry because they don't like to be rated. but then they tell us that they're focusing on the things we're reporting on. the more we report on things to change, the more they focus. that came in a complaint. you focus on what you want them to do. i thought thank you i did my job.

so that's one area we see changes in. and we also see areas where hospitals will call us and say thank you, thank you we didn't know "x," "y," "z." those are the calls i really like to get so i know that people care about it. and then we also hear from hospitals about patients canceling procedures and surgeries and then losing revenue. so there is some market change going on. it may not be as much as we'd like right now, but there's something happening. >> both doris and robert krugoff talked about getting the raw data out there. you were quite vociferous about how the government should get the raw data out there. but the risk of putting out raw data that hasn't really -- risk of misinterpretation as we saw in this they -- data release

that triggered this event. >> or will be misinterpreted. you know, i've been troubled by that response. we've had a lot of experience. we have a health plan comparison tool that we've tried to put out. we put out in the illinois exchange, for instance. we asked the federal government can we have the benefit, the description of benefits, the copayments and coinsurance for these so we can do an estimate of out of pocket costs for each plan. the best estimate for somebody with your health status, et cetera. they even made it in a coherent way most people don't have a chance of doing. and the government unlike the part of cms we're talking about here, the government said, no, we can't -- we're not going to give you those data. and then they finally said,

we'll give you the data the day before open enrollment starts. these estimates take many weeks and complicated calculations to do. we get it the day before open enrollment. so the reason -- two reasons. one is the plans wouldn't like it. and the other is that we might put out something that's confusing. or that is wrong. i believe in the market place of ideas that you put out something that's confusing and wrong and nobody listens to you again, then somebody sues you. i like that way of having the world work. and we take some chances on that front opposed to not giving the information out. >> lest we get too focused on the government not releasing, big companies don't like it very much either. >> right. >> i think it's time for audience questions. and we have possible microphones.

there's one right -- question back here? >> hi. i'm lauren hershey. i was here about 18 years ago. so hi, alice. i follow medicine a little more closely given my aging process. and i'm curious to learn over the last couple of years, mayo clinic book, cleveland clinic book, there is something called wellness medicine that is happening. i don't know how to define it. there's alopathic medicine. i never thought in those terms until a few years ago. that's health care delivery. so my curiosity is to ask each of you three panelists, how do you evaluate what's happening? how do you measure it? and how do you describe it? has this wellness movement been around for two years or six

years or nine years? i don't want to connect it to some personalities on tv. i want to connect it to university of michigan where i've read about it or yale university. but the question is data. so if i move 10,000 steps a day, big deal. but it's supposed to make me healthier. how do you measure that? get my point? >> i think i understand what you're trying to say. i mean, we've -- that movement's been around for a long time. if you just look at the products we have. diet and exercise so you have that aspect of, quote, unquote, wellness. we take in the form of preventative services. how that can keep them healthier. i don't know if that's along the lines you're thinking of, but that's our focus. that's the aim we take. >> i guess i see there's a trend towards payment reform that would move you in the direction, move hospitals and payers in the direction of treating a person

more holistically. it's been around for a long time. i think it's getting much more traction now. >> some of it's not medical. >> exactly. >> so much of it is not. it's nutrition, it's exercise. >> exactly. >> and are there evolving systems for evaluating that from the point of view of the consumer? >> well, if you think about the dual demonstration projects, i think there are evaluations going on now of that and that's really looking to take the medicare and medicaid payment, combining it and keeping people out of hospital and out of nursing homes and theoretically healthier. you know, that's something i think the research triangle is doing right now. >> i think there's a -- i think the visibility of patient self-help and non-alopathic

medicine has increased over time with tv and every place else. a lot of news letters and things like that. i do think there's progress. and i do think many of the health plans are looking for alternative ways to help their members hoping that these will solve problems and reduce the health plans. and some of these providers, i mean, for instance, checkbook does evaluations of acupuncturists. that's a more straight up sort of thing instead of some things you don't know whether the person is hitting the bottom of your foot right. some are much harder than others to do. at least some of those you can get some meaningful feedback at least on the ability of the providers to explain things and -- or to give miracle cures. i've had a miracle cure by an acupuncturist. one time i went in and they said you have to listen for all this

type of stuff. then he put on music and let me sleep for a half hour. i wasn't supposed to be sleeping but i was. he said how did that work for you. i said it was terrific. but when he was talking me i was just thinking, warm lamp and sleep, how can you beat that? i was thinking did i lock my bike out front. so people have different views on how this works. [ inaudible question ] yeah. well, i think we haven't done much in general that has not been as -- it's a bigger and bigger part of the system in the economy. i don't see it as having the same level of visibility and measurability that we're trying to see in the regular medical care system. >> other questions? >> i have a question that comes from twitter.

is there any information about increasing consumer engagement through transparent cost and quality data? and how can we increase engagement? sfl i can speak anecdotally. we see an increase from consumers in both the data that we're providing out there and also with the consumers telling thinker stories to us. i think what can make it more useful would be making them more personalized and making it more about their day-to-day decisions or the decisions they're making about procedures they have enough time to make those decisions about such as maternity, childbirth, knee replacement, things like that. >> i think that's -- there's been a lot of literature about why consumers get involved or not involved in cost and quality data. i think from low income consumers it's -- it is somewhat different. and you need to think about how to engage folks who have historically been insured.

it's where we think consumer assistance is important. the medicare program has assistance in people choosing plans. we think that would really help. >> yes? >> i know there are some providers and organizations that are in the business specifically to help people. and there are others that are in the business to make money. and i'm just wondering from your analyses, can you differentiate one from the other? >> i don't think we can get to the heart of their motivation, but i think we can look at their performance. and we can see are they, in fact, making people better and are they keeping costs

reasonable? are they overprescribing? so i do think it's possible. even some of those things -- i've talked about this choosing wisely type, you know, assessment. which is obviously very important. and we don't know how to do that. i mean, and we're going to need better data to really decide -- you know, start identifying providers who are really abusing as opposed to just getting a bunch of referrals for somebody who needs that treatment. >> before we wind up, let me ask each of you very quickly if you look ahead ten years, what would you like to see in this world of consumer information about health care and consumer choice? what -- what are the most important things that you think the world ought to look like ten years from now in this

dimension? start with doris. >> i'll go back to where i was before in safety and medical errors. i feel we've been trying to crack that nut for ten years if not more. so if ten years we're able to have a way for consumers to understand their risk of medical errors in any particular hospital or any particular surgeon, that would be where i'd want to be. >> and there are fewer risky providers, one would assume. >> i think we need to consolidate information, make it easier for the consumer to use. i think we need to focus more on outcomes and the patient experience. >> i would agree that we have to push toward measuring outcomes. different people are in very different situations. somebody who has no chronic or acute condition is not much interested in outcomes except they want to avoid something going wrong. somebody who has a serious chronic condition really is looking at measures that are very clinically oriented, et

cetera. so those things have to be tailored as you say to -- the measures have to be tailored to the patients or to the consumers. >> given the demographics, more and more people will have that. thank you very much, all of you. this has been a very good panel and i hope that some of these things come to pass quite soon. let me turn it back to -- well, yes. a round of applause for the panel. [ applause ] and then let the turn it back to kavita for some final remarks. >> thank you, alice. go ahead and stay because this won't take very long. i just wanted to thank all our panelists again and thank or excellent staff at brookings. a number of people who helped make sure that everybody's -- every was as comfortable as possible and also for the folks watching on the web. i just want to highlight a

couple of things. i mentioned some of the work we're already doing with this data set and data releases. we're going to have several more briefs that talk about topics covered today. and i wanted to be a little provocative in what might be coming after today's conversation. we've talked about kind of uses that are not just consumer centric but what happens if some of this data is used to create high value networks that we're seeing in the private market place as well as on these health insurance exchanges. and what if we could actually get researchers together and do almost kind of a crowdsourcing of valuable research cases or findings, things we all find interesting that we don't necessarily have one entity taking lead responsibility for but that we could work with nonprofits, research organizations, as well as consumers on what they find interesting and try to put that together in a way that's not

contingent on what the government would do. and then finally i think doris kind of talked about a conversation that's been missing around safety and errors. it wasn't that long ago we had crossing the quality chasm and other colleagues who highlighted these problems unless we be remiss to think we've solved all of them, how can we be pointing towards places to improve and think about concentrating efforts from transparency in data to show people not just what's bad but what we can do to improve the health care in our country. i did want to point the conversation won't end today. we are going to be doing future publications as well as hopefully extending this in other ways. so thank you for your time and attention. and have a great rest of the afternoon. take care. [ applause ]

and live coverage today on c-span3 when the domestic violence committee talks about professional sports. we'll hear from troy vincent and others. that's live at 2:30 eastern. and in the hill reporting on boehner plotting two-step course on immigration to gop rank and file that would keep the government funded past the current december 11th deadline and also push back against president obama's executive actions on immigration. he plans to bring a bill to the floor this week and the house would then vote next week on a

health officials have --ñr$ eb treatment centers with more expected in the coming weeks.xdi researchers who are trying to develop an ebola vaccine and the president's scheduled to deliver remarks just after 5:00 eastern. you can watch that live on c-span.org.

next we'll hear from the head of the u.s. national intelligence council about how the intelligence community is adapting to new national security challenges. this was hosted by the atlantic council. it's just under an hour. >> good evening, everybody, and welco welcome. our online audience, welcome as well. we're so pleased you can join us to look at an issue of critical importance and i think we're understand i understanding this is becoming real world and an even more compelling fashion than we thought maybe even a couple years ago. and that is how the intelligence community responds to escalating

threats. the relationship between intelligence and strategy is an area the atlantic council has focused on on international security which are sustained analysis of global trends and our foresight initiative where the director and norm formerly e intelligence council. we pride ourselves on building a network of experts dedicated to understanding what the future holds. and then to think creatively about solutions and creatively about strategy for the future. it's not easy work. often argues that strategy during the cold war was easy because you had containment of the soviet union, containment of communism, and then the tactics were difficult. and now he complains that we're all tactics and not enough

strategy. of course intelligence is a lot about informing what our strategy ought to be. one of the best experts i talked to over the years on any number of issues but as you know i'm a recovering journalist of "the wall street journal" and you always try to turn to the smartest people to inform their ideas to help your own. i always liked stealing mr. treverton's ideas. we met in europe i think more often than the united states. recent events in the middle east and eastern europe have changed the policy conversation in washington and reinforced the importance of timely, accurate, and appropriately intelligence analysis. very often it may seem the

intelligence world is different from the policy realm. looking from the case of wmds in iraq, the hunt for osama bin laden, just lists and lists. you can list positive cases as well. the next landmark global trends 2030 report forecast that we are living in a world of rising non-state actors, revisionist powers and exponential change, i think we're all seeing that accelerating faster than the thought it would. i think that's probably one of the most interesting findings of these global trends reports is if they've underestimated anything, it's been the pace of change. so, with that as prelude, we have the pleasure of welcoming a thought leader tackling this. dr. greg treverton. he's previously served as director of center for global

risk and security as well as a professor at the graduate school where he serves as associate dean for research. prior to joining rand he was vice chairman where he oversaw production of the government's premiere assessments of international problems, the national intelligence estimates. what years were you vice chairman? >> '93, '96. >> okay. so i think i may start there. the director of national intelligence james clapper called you to do this job for your combination of analytical skills, broad substance of expertise, passion for the intelligence profession, and a deep understanding of the unique role and mission. i think we'll talk in this conversation, i'll ask you a few initial questions and then we'll get into q & a. and so clearly he call you the right man at the right time and i think that's absolutely right. so thank you for joining us. >> thank you. >> one last housekeeping note.

we have a hash tag for this event. that's probably something new in terms of your appearances. so we encourage you to tweet away using the hash tag -- and i apologize in advance for this. #acdisrupt. why don't we get started. let's start with your time you were vice chair. you're back now and there i think you were under joe nye who was the chair. so coming back into the building, coming back into this capacity, how's it different? both in terms of how the place operates and then also substantively. >> pleasure to be here. appreciate the opportunity to talk about the intelligence community and intelligence task in general. the biggest change coming back as you can see, i'm a slow to rise. it took me 15 years to go from vice chairman to chairman. but it's great being back.

i still have the same phone number i had all those years ago. so some things change more slowly. the big change, really, is the operating environment. before we basically only did more strategic work, not just national intelligence estimates but more strategic in putting pieces together and into context and taking a longer look at particular issues. we didn't have much role at all in current intelligence. that's a dramatic change. now the nic is responsible for doing the intelligence preparation of the principles committees and the deputies committee. >> when did that begin to shift? >> it all really began with clapper, i think. when jim came in. and working out the arrangements between the cia and the director of national intelligence. the cia still does most of the work on the president's daily brief, but now the dni delivers

it. ditto on the pcs and dcs. we turn to all the agencies for help, but we do the quality control, put them together when an intelligence community wide view of a particular issue is wanted. so that's the big change. it makes us on the good side makes us relevant. means we're in the thick of things. it means critically we know what's going on which is always a great problem in intelligence knowing what policy makers -- have somebody see what they want to accomplish over the next two years. but they have no idea about next tuesday. trying to keep up with policy and those circumstances is a real challenge. the cost is that it is a hugely consumering effort. and so for the busy accounts like the middle east, like russia national intelligence officers feel like they spend most of their time either preparing for meetings, going to meetings, or writing short response memos after those meetings.

last year we did like and arrange 15 national intelligence estimates. we did about 900 pieces of paper. more than half of those 900 were memos, specific memos to susan rice about particular issues that came out of the dcs or more rarely the pcs. dcs needed them more often than the pcs. that's the big change. the thing that hasn't changed other than my phone number is the people. a terrific collection of officers and their deputies particularly. just a world class set of people. i feel like i get to go every day to a world class intellectual salad bar and it really is a treat. >> well, before we get to the substantive issues, where do you think you want to adjust that

needle between realtime and long-term? >> that is the big challenge i face. issue number one on my agenda is trying to think about and to the extent necessary, recalibrate that balance. think about, and to the extent necessary, recalibrate that balance. it means finding ways to let people have time, energy to do somewhat longer or somewhat broader thinking. i had a really good deputy of national intelligence officer in russia who said, i like getting to do this straty piece, longer term strategy piece. he said unfortunately i only had six hours to do it. so deep thought in six hours is

probably not a great idea. so finding ways to let that more strategic, not always longer term, but putting things in context, finding ways to do that is really the big challenge. now, happily not -- many of the questions that come out of the dcs are pretty straightforward. but many of them are quite interesting. we'll get a what-if. what if we do this, how will these -- >> dcs, deputy committees. >> the main policy committee in the u.s. government in the policy making side. as a practical matter, it's the dcs that do most of the work, and sort of tee up decisions for the principals. sometimes they'll ask us interesting questions, just the sort that i'd like to engage in with policy. if we do this, how will putin respond. what's your assessment of how putin will respond. those are ideal as a first task that i need to begin to develop a better count. because it's not the case that all those memos are only information and only the nies are more strategic. there's a middle ground, trying to figure out exactly how our work breaks down. that's the first task. >> before going present tense into the future on substance, since you've raised russia, let's go back a little bit to when you were there before. everyone argues, should we have

known where things were going, could we have altered things, what is our role in the outcome we're experiencing today. with my knowledge, and your knowledge in these areas, what is your take on when you were last at the nic, in terms of how we were looking at that? one reason i'm asking this is this interesting mixture of long-term trends and short-term action. if you're trying to look at russia out ten, even five, certainly 20 years, it's hard. but five, ten years, you may make smarter decisions about today. so take us back a little bit right now, and then how do you play that out right now? what sort of intellectual process did we go through then, and what sort of intellectual process ought we be going through right now? >> one thing that strikes me is this seems to me an interesting

inflection point in global politics. maybe the third in the last generation or so. one was obviously the fall of the soviet union, and communism. the second was 9/11. and 9/11 was easier because it seemed to come with instructions attached. so here's what you do. you go after these bad people. so this feels to me a lot more like the fall of the soviet union, the end of communism, where one geopolitical framework for thinking about the world is gone, but it hasn't quite yet replaced with another. it seems to me the first time when the soviet union fell, we very quickly said, that's over. and while i think that as a policy person at the time, i thought that the expanding of nato and all those things was a good thing to do. but we probably were in retrospect pretty dismissive.

and part of putin's attitude plainly is, it feels like he and russia were dissed by the west for a couple decades. and that obviously does have some effect. it doesn't explain him. but i think it happened because we sort of quickly went to, well, now that's over. the cold war's over. and now russia's no longer a threat, no longer a major power, and we sort of jumped quickly to a different attitude toward russia that is in some sense part of the sweep of what we're confronting now. >> it's interesting to think about. >> sometimes we think that change takes a long time. if you look backward, the distance between the evil empire, and the fall of the soviet union, which is slightly over a decade, so it does mean that things can go very quickly from time to time. >> my staff here, i'm so glad you raised the point of inflection point. we're kind of living by that

argument right now at the atlantic council in a way, drawing it out of the global trends 2030 report, where the inflection points listed there were 1919, 1945, 1989. and the argument being that at these points in history, decisions of leaders had outsized importance, because you were at a plastic moment in history where things could be molded and shaped, et cetera. you used different inflection points here. and this gets to my point of how is your job different than it was then. how big is this inflection point? is it of this historical dimension, end of world war i, end of world war ii and the cold war? >> i think obviously, i don't know whether it's as big as those other ones. it seems a very significant one in the sense that we are sort of scrambling around for a view, a

lens with which to apprehend this world, to mix a metaphor, and that i think means a real challenge for intelligence, trying to help people build that lens, or that story. i've come to think that intelligence is about helping people create and adjust stories in their head, and we know when the story gets too firm, we call that mind-set and often results in what gets called in intelligence failure. but if there's not a story, then new information just kind of bounces around. it's a factoid. it's hard in policy to have a conversation. so one of the true churchill lines he was supposed to have said after a particularly undistinguished meal, asked what he thought of it, he said the pudding, that's dessert for the

english, lacked a theme. well, i think our world lacks a theme. so trying to provide a story or a lens, that seems to be a job -- a formidable job for us to do working with policy people. as i said, whether this ranks with 1989 or 1992 in importance, don't know. but it does seem like we're again in a pretty shapeless world, where it's easy to get dominated by tactics. >> how about regionally, as you're coming in, and how the

world is different than it was then. what were your regional priorities -- you know, in this world of challenges, the middle east, ukraine, you know, the far east, south china sea, ebola, global financial stability, what are your -- how do you set your priorities in the job you're doing right now? and where would you set them both in regional sense and in a subject sense? >> we set our priorities by what's in front of us. the big crises that are going on that need to be handled, the challenge with those is trying to help people step a little bit up and say, here are the tactics, what are we trying to get out of this? what's a realistic end game, offer at least end point to this particular set of crises? not very easy. hard to do. so those we'll define. i think for the remainder of the administration, of course, the middle east and russia are going to be dominant themes. i think we need to keep trying to raise people's sights a little bit to beyond the immediate tactical to, what are we trying to do here, what's our ultimate point here given realistic possibilities. i don't get to have too many priorities in my current job, though as i said, somebody the other day, i spent my whole career avoiding the middle east. that's over. as the kids would say, that's so over. right? so i've learned a lot.

>> #acdisrupt. >> otherwise, on my personal list, i think china and east asia has to be at the top of the list of so much going on there, so much chance of multiple issues at play, connecting in ways, people making miscalculations. we know there are going to be bumps ahead in the road for china. we don't know how bad they're going to be, but there are going to be some bumps. and there may be good ones as well. i think that whole combination of issues there, in a region that doesn't have strong security agreements, mostly bilateral with us and allies, so that's going to be at the top of my list. my own personal list is certainly cyber. from where i sit, we still haven't calibrated the threat. it's very hard to calibrate the threat.

it's very dynamic. we're divided in ways that make it hard to have a complete view. we have, you know, the military part of the government, the private sector that has all the infrastructure and does all the stuff. we have the government. and we have offense and defense. and working across all those divides, i think it's no surprise that we haven't really calibrated the threat. so that's high on my list. and then -- >> and globally offense is stronger than defense at the moment. >> i think we don't know. it's part of calibrating the threat. and then i -- as you said earlier, fred, i was really happy when the cold war ended, because it had become boring. it had become a sort of a managerial problem. and we knew that if we kept our alliance together and kept ourselves strong, we would win in the end. we didn't expect to win in 1992, or in our lifetimes, but we did win. so in that sense it was kind of