better than that, if your actual return is better than the policy return, it simply means you either picked managers who have done better than the index. that's contribution of active management or positioned the portfolio to be overweighted to the things that did well and underweighted to the things that did less well. the policy is what you would have earned -- >> my question is, base on the hypothetical, that you said, is that a trend? are we missing every year? >> no, no. it's not a trend. i'm going to show you in a minute, the under-performance versus policy is really the mechanics of one of your benchmarks being aspirational benchmark that has been different. >> for private equity? >> yeah, i'm going to show you

that in a minute. what i will say because i generate the same reports for many other public funds, is it's unusual for the total fund to do materially better than the policy over long periods of time, because it's hard to pick managers that outperform and tactically position the portfolio. so your results here in doing that are quite strong versus others, even though you don't see a big positive number. but i don't see any trend in you getting better or worse in the 1-5 year period. the 10-year period you've definitely gotten better. your 10 year result versus policy, if we went back to the first page, you were under the policy considerably. for the 3 and 5 you matched and the one year, you did better than policy. >> thank you. >> the statistics on page 31, again if you go to the far right, total fund return, policy

indexes, 10-12, virtually the same. the alpha which is what we didn't talk about earlier, is the amount of return you earned above the risk of an equivalent fund of the same risk. that is 1.17% which would put your fund in the top 12% of the universe of public funds. so in other words, the board has set a policy for this staff of asset allocation. you determined how much you ought to have in each asset class. the markets move over and above those indices and the staff can accept what they've outperformed in the one year because you were overweight to u.s. equities and

they did well. and then the staff put managers in areas they think they can outperform and that's the other component, that manager selection piece is the most applicable through time and the numbers are strong. >> first one analyzed return, that is saying basically 10%, but we're doing with less risk? >> that's right, the annual return is 10%. if you look at the standard deviation of 5.5, you have a top 7% performance on the return aspect, but you're only in the 40th approximate percentile due to risk. >> that's doing it with the private equity benchmark which is tough, because it's 5%. and it really should be more like 3. >> that's right, and we're going to see that in a minute. >> ok. >> so that is the presentation there.

now if we go to attribution, on page 38. there are detailed tables. in the one year we outperformed the index by 13 basis points. where did that come from? 52 basis points came from allocation effect, meaning we were overweight things that did good, like international equity and u.s. equity. and we underweighted things that didn't do as strongly in the period. if you added that up, the positions of the portfolio added 52 basis points. manager selection which is the net of how your managers did against their benchmarks, that detracted 39 basis points. on the surface you say, gee, our managers didn't do as well, but if you look down the categories,

it was private equity, one of your stellar performers, underperforming the index that was the source of most of that under-performance. remember, your private equity portfolio for the year generated 14.89%. if you got 14.89% out of your private equity portfolio every year over a long period of time, you would be delighted. but remember the u.s. equity market in 2017 was up 21% and your benchmark was to do 5% better than that. so the benchmark was up 26%. your private equity return 15. and when you multiply that by the percentage in private equity it appears to be under-performance. that is not a manager selection issue, that's a benchmark mismark. you would expect that over time to work itself out, but remember

we're in one of the largest equity uptrends we've seen. so when we look at the same number for five years, you should almost take that number out of the under-performance because it was generated by a benchmark specification, not performance of the manager. >> allan, couldn't it be if you happened to put a lot of money in private equity and it's drawing down? could that also be part of that? >> clearly, if your private equity program is very young, and you're just investing and paying a lot of fees, that will drag you down. >> ours offset? >> yours offset. yours is a mature program. >> i'm still trying to understand, you're saying 217, we were up 14.9 and what was the benchmark up? >> for private equity? remember the private equity

benchmark is public market plus five. >> ok, that's why because the public markets were up so high. got it. >> so you were chasing a fast rabbit. your private equity program when we look at the actual results in terms of versus peers, you're in the top 10% of your peer group. >> so 14.9 is top 10%. >> that's right, every plan that has a benchmark of public markets plus the spread is facing the same as you are. if we take that same analysis and flip it over another page where we now have the five-year results, the five year, you can see was under perform, the total benchmark by 24 basis points. the allocation effect was all of that and there you were, you'll see the components. but the biggest component is you were underweight asset class

that performed as well. you're under weight in private equity wasn't because you did a bad job with private equity it's because you were building the program over five years and you were below the benchmark on something that did well. if you look at the manager selection effect which again is what the staff has probably the most control over, is every single asset class in which you invested outperformed with two exceptions. one, again, private equity under-performance because even in the five-year period public markets did well and u.s. equity, an area that bill would indicate hasn't done as well competitively as the other asset classes have done. the bottom line in the asset class level is one, the plan has taken a high are than median equity risk in a period where that was rewarded. so the plan did very well in

terms of its aggressive asset allocation. with respect to the managers that you hired to run those assets, other than in equities, the managers you hired by and large added value. and this is net of fees. so this is return after you paid the manager fee. and it ranged from 57 basis points for real assets, fixed income 28 basis points, international equity 23 basis points so that's pretty significant on the size of portfolio that you run. the asset class returns are in the tables that follow. you can see all the detail. in the interest of time what i was going to do is flip you to page 49 which puts -- 48, which is the five year returns for every one of your asset classes. so public equity which is 45% of your total.

17.79% would have ranked in the middle of the peer group. that ranking of your public equity when you then deflate the return by the amount of risk taken, you get a number called the information ratio. that is the return per unit of risk taken at the portfolio level and you can say your information ratio was in the top 12% of the peer group. u.s. equities did not do well. they actually underperformed the benchmark. you don't even see information ratio or you shouldn't see one, because when it's negative you can't rank it anyway. u.s. equity was exception. international was in the of the group. you ranked in the top 40% in terms of return, but when you adjust that for the fact that you ran a low volatility

portfolio, the top qarlinngua tile, if you go down the you'll see you outperformed the benchmark. private equity which is 15% of the total earned 15.28%. you see that for five years. annualized, there is the benchmark again, public markets 21.5. so you underperformed your bench mark, but versus peers that was top 13% and it was low volatile. page 49. >> for context here. this mismatch between private equity returns and our benchmark returns for private equity, this is going to be a systemic issue

because we're matching infrequently priced assets with daily priced assets. so this is always going to be the case. i think a better relevance would be what the cambridge private equity lagged benchmark did. and for the five year ended private equity as asset class 13.3 which is on the page that alan referred to earlier. in that light, we outperformed by two. you look at this page, you see outperformed by six. but that's because of just how equity has done so well in the last five years plus a hurdle of 5%. >> i don't want there to be confusion about that point. the benchmark was selected to outperform public markets by a spread. that is a reasonable expectation over long periods of time. but we've now been through one

of the biggest bull markets we've had in public equities and private equities are a lagging indicator. so from a performance aspect as bill said, versus your peer group, you've done well. and certainly 15% a year for five years, if we could do that going forward, we would be quite happy. it's just public markets as an anomaly in this period where the fed has reduced interest rates to zero, that effect that driven equity market high. investors faced with the choice of earning zero on their savings have moved into the stock market and that has driven the price of the earnings high. that is not sustainable in all of the forecasts. all of the forecasts you'll hear from other professionals, going forward, we're not going to see 21.8% a year in the u.s. equity markets. we'll be lucky to see 5-6. >> you see three rate hikes?

>> i've taken a bet with another member, i bet two or less. he bet four. if we're wrong, that's where we'll be wrong. the thing that will upset the market is the fed raising rates faster than expected and that will hurt the markets. the other wild card is the driver of global prosperity and earnings growth has been global trade. if the u.s. undertakes competitive tariffs and things like that and there is a reaction abroad that could upset world trade and have a very negative effect on the growth of gold medal. >> what do you think will cause rapt rate hike? >> if inflation ticks up faster than people think. it would be on the wage side probably. the tax cuts could be more stimulative than people think and you could see a spike in

buying and bottlenecks in production that would cause prices to go up faster and people would be concerned about inflation. the rest of it goes into the details by manager. if you flipped the pages, every place there is a red, it's where a manager has underperformed the benchmark. you won't find very much red. in fact, every manager that has significant under-performance is indeed on your watch list which is the next topic. so i wasn't going to go into managers much. the two things i was going to point out at the manager level, if you go to large cap developed markets -- >> what page? >> page 56. before we go on, i have a question that is relevant to pages 36-40. i wanted to know if you could speak about the manager selection effect.

you said that is because we have aspirational benchmark. is there any other information that explains this, or -- >> remember, at each asset class level, you have a benchmark for the asset class. u.s. equities, it's the s&p 500. when you compare how the managers did against the index, it's very relevant and if you outperformed it, that means you selected good managers. when you get into private markets, the benchmarks are more difficult to set and in private markets the rationale you've adopted which the bulk of the peers have done is a view that why would we be in private markets if we don't do better than the public market equivalent. many funds have a bunch mark that is public market plus the spread. that's how you measure the performance of your private equity. that index in the last five

years because of a massive government intervention into our capital markets reducing interest rates to zero, has propelled the growth in equity markets as an extraordinary high level. much of that has been multiple expansion, so u.s. equities have generated even over five years higher than 10% a year, and that makes the benchmark you're measuring private equity against a very difficult benchmark to beat. so i'm saying that is not that you did poorly. it is usual bench mark. >> so that should be standard. we're talking ourselves out -- >> all i'm saying, victor, there are benchmarks indicative when you perform well against them of a performance issue and benchmarks that are aspirational that may not work for large periods of time and this period of time has been a difficult period for any private manager to outperform. >> i don't see it as

aspirational. we set a benchmark as a true benchmark, signed off by everybody in this room, is exactly what we think we expect to perform or better, so why should etalk ourselves out of it now? >> i'm not trying to talk you out of it. i'm just trying to say it is what it is and this case, under-performance is not reflective on the private equity results. when i said aspirational in public markets when the benchmark is the s&p 500 you can go out and buy that. it's called index fund. it exists, you can invest in it. in private markets you can't guarantee that you can buy s&p 500 plus five. it doesn't exist. we hope to get there because of the returns we earn in private markets earning a spread, but you can't ever guarantee that benchmark and it makes it slightly different. we could do a session on setting benchmarks but when you move out of public markets where you

can't trade the benchmark, it becomes more difficult and that's why i'm trying to stress the interpretation here. >> commissioner makras: i'm hearing you but it's still like we're excusing it. we're not hoping to get there, we're working to get there. i never heard we're going to come under that number or it's perfectly acceptable to be under the number. >> if you go back in history, private markets, private equity, the reason people invest in private equity, they've earned a premium to public market. that range is 3-5%. we're saying in normal times we would expect to be there. all i'm saying is the period we've been through is not normal times. it was coming out of a difficult economic meltdown. and it was a period where the federal reserve intervened in the capital market and that affected prices which drives equity down. >> commissioner makras: you take that as all true, our stats have

not come back to change the benchmark. >> yes, they have. >> we have for going forward. >> commissioner makras: but it is our current benchmark and it's still underperforming. >> it's the benchmark, going forward, the benchmarks have been modified. >> commissioner makras: but we're still underperforming the modified. >> that's correct, i'm not trying to hide that -- >> commissioner makras: i'm just calling it out, because i think the benchmarks are serious and -- >> you underperformed the benchmark, there is no question about that. what i'm saying is the private equity program did very, very well relative to any other private -- there was no private equity program i'm aware of that earned public markets plus three or five in this period. so it wasn't you did a bad job, it's everyone who used this benchmark fell short. you actually fell short by less than most others.

i'm just saying don't go fire the private equity staff because they did a bad job in building the portfolio. >> commissioner makras: i didn't suggest that. >> i know you didn't, i'm trying to -- >> commissioner makras: benchmark. >> i think the interesting thing is, now we removed it to s&p plus 3 when it should go back up because markets are fully valued and you don't have a big down 38% time where end of feds you see the market. i see it in the industry. five years ago, people were changing it more rapidly back to s&p plus 3 because they saw that. shame on us we didn't do it earlier because it was too hard to reach. now we're going to s&p where it's probably going to be s&p

plus five. >> when setting private equity benchmarks -- >> if we're going to compare public returns to private returns, you're always going to have a ton of tracking. >> it's a bad kind of thing to do. >> the way you manage, you compare the returns in the private equity portfolio to page 5 in the cambridge. >> to others like us. >> and there we've earned 15.2 or 3 and the peers have earned -- >> page 5. i would like to comment briefly on commissioner cohen's question. on page 36-40. while the private equity numbers are all negative for the reasons talked about, most everything else in terms of manager selection is showing as positive. most are. >> supervisor cohen: i know that. focus on the negative to get an explanation as to why they're negative.

>> i'm trying to explain, not trying to justify. >> supervisor cohen: i know, bill is saying don't overlook the positive. i get it. thank you for taking time to answer my question. we can move on. >> on the managers again, you can look at the individuals. the two i thought i would mention on page -- you have cap guardian. and cap guardian in the last little while has indeed done better than the benchmark. they've outperformed the benchmark relatively consistently although they're still about in the 50th percentile of all managers. again over the last six months or a year, cap guardian has improved relative to its benchmark. and gotten a little better relative to its peers and then the other one on page -- opposes sorry.

oops sorry. >> supervisor cohen: when you're staying -- it's about us trying to find the best managers that we can find. >> that's right, i'm just saying a year ago they were very much trailing the benchmark and i was just pointing out in the last little while they've gotten better relative to benchmark. >> supervisor cohen: because the benchmark is much easier to beat. >> that's right. the other one i wanted to mention was page 59. and you see himalaya there at the bottom. we see a year's return and quite strong. other than that, the managers that have the red are on the watch list. we can take that up as the next question unless you had manager detail i was not going to go through. by and large your managers outperformed the benchmarks. >> supervisor cohen: i have to say, taking time, we've had new staff come in. mr. coaker's built a new team.

but these managers that he's brought in, the teams brought in, as we moved this portfolio like himalaya. i mean they've been phenomenal. so i know we couldn't come in and make a wholesale change, but one by one of adding these managers, the return of 59% a year? unbelievable. so if that starts adding in the manager selection, that is going to be really, really exciting. >> page 60, can i also note outperformed by 12.6% in its first full quarter. >> i saw aero street. these additions. it seems like the board was complaining that we had these mundane managers that were kind of our long equity managers that we replaced with some really

high active share managers that are very different than the benchmark. and we've been rewarded for that. it's a good start. i just also want to just acknowledge -- make sure we give equal voice to our challenges concerning parts of our portfolio. but also recognize himalaya in particular as the great investments that are performing exceptional so my compliments to the investment team that did the work, convinced us to support it. and this is like your victory lap, i told you so. >> thank you, thank you. and thank you to the board for, you're pioneering. and you've been rewarded as well.

>> supervisor cohen: as you're going down the list, one of the things that blared out -- >> there is still work to do here. >> i'm looking at the aflcio. >> what page is that on? >> all of managers that are on the watch list which is next topic if you want -- >> keep going, ok. >> there is still a lot of work to do. >> i'm done. >> president stansbury: commissioner. >> review of the next section. starting with page 43, suggestion putting rhfi when the other line is the measurement if that program is working.

>> we put just as in private equity, we put both here. one is the benchmark, one is the competitive standard of how other programs have done. >> but the over, the under, that is going to cause more confusion, i think. on page 45, then to show even though it's program only affected for one year, if you look at the annualized return, and the annualized standard deviation and compare them to the total fixed income numbers on the top row, the issue is has the asset return portfolio improved the performance of the fund or not? >> yeah. >> looks like risk went up, but return as well. >> again, joe, we're looking at returns and of course the absolute return portfolio is designed not just to provide a

decent return above cash, but also to have better characteristics in a down market which we haven't had tested yet. february will be interesting test. but remember, the absolute return was not put in as substitute for fixed income per se, it was to earn a good return but have characteristics on the downside that held up. we haven't seen a downside. >> commissioner driscoll: one year number is a start. can't draw conclusions. so far it looks like it's performing what we expected to? >> exactly. >> commissioner driscoll: that's part one. the tradeoff between risk and return, this one worked. ok, i'll come back to the next issues -- is there a total fund standard deviation number? that's what i was looking for.

>> it's the volatility, it was in the charts -- >> it's looking like on the grass that we are moving in the right direction again. >> that's correct as well. >> it's ok to move right as long as you move up. >> just trying to pass judgment on when you review. now that we're getting ready to invest a lot more money in the area, it has to come from someplace else. this is total. we go back to the issue of benchmarks and that came up a few minutes ago. in the private equity, that benchmark is a premium over the s&p. what is the correlation between public equity and private equity? >> very high. >> .7,.8. >> but numbers like that indicate they're going to be out of alignment if we use that benchmark. >> that's right. >> because there have been periods where the portfolio has beat the benchmark. >> yep. >> especially when the market is going down. >> i mean, as i was going to

say, there is a whole science of bench marks. your benchmarks are consistent with others, but private market benchmarks have the phenomena they're difficult to -- they don't always match what you've invested in. >> commissioner driscoll: the next questions i'll bring up on the next subject. thank you. >> but, joe, on the risk return charts we looked at, every one of those is the risk piece is the standard deviation of the portfolio and the standard deviation of the san francisco portfolio on a relative basis has been getting smaller through time. >> commissioner driscoll: that leads me to the question -- tactically mr. coaker and staff have been wise to put the uncalled money in public equity and the program to equityize our cash? that has paid off, that's a tactical decision away from the belgium mark. the question is -- benchmark.

the question is, should we take on more risk? that's the balancing act, we never talked about it, but we're patting our ourselves on the back, about you should we take on more risk? it gets into uncertainty, yes or no? loaded question, but these reports have a lot of information where we never figured out how to use it without patting ourselves on the back. >> when the board went through the asset allocation system, i know brian led the discussion, of being more concerned going forward about the potential fort equity market correction and the desire to have lower risk, lower volatility in the portfolio to better protect against adverse times, while at the same time trying to add asset classes to keep your returns as high as you can get them in a market that is going to be difficult to earn 7.5% going forward. but the whole discussion how much risk we should take is one

that was part of the asset allocation discussion historically. you've been comfortable taking a fair amount of equity risk and you did it in a period where it was very rewarded. was that luck or skill? it was augmented by the actions of staff in terms of manager selection and positioning of the portfolio around that. >> any other questions from the board? comments? >> president stansbury: the linkage between the information and the decision is important and mr. coaker, that's something that you guys are working on with the public market portfolio, working through funds one by one, and figuring out where we're going to go from here is this >> yes. president, the strategic plan we're going to be working

through with the board on next week, a week from today, will include public equity. our plans for public equity. and between that meeting on march 21 and may 16 is the complete portfolio. so, i hope everybody will be able to attend, participate. there is a lot of data and a lot of planning and courses of action and there is going to be quite a bit of change, going from 35 to 41% in public equity. that's a lot of change. and fixed income to take that down to 9%, that's a lot of change. there is a lot of action coming forth and we put a lot of action into the time line, and the like, so there is a whole strategy to reducing our systematic risk, our beta, in improving our alpha experience, meaning our excess returns.

>> last month i asked about absolutely return and performance during the last mini correction in february. will we have -- >> i do have that data. i can tell you now. so public equity was down -- u.s. equity down 3.3. and international equity was down more than 4.5 bonds down 60 basis points. the hedge fund index was down about 2.5. our portfolio was down 0.6. so significantly outperformed. >> commissioner makras: those are great numbers. correlation, do we have that? can we maybe do a little more of a detailed analysis upon how it was that we performed -- the high level numbers are really good, but i think it would be helpful for the board to have something for us to look at on paper that breaks it down with

detail. >> i'll ask david to put that together. there he is. >> the market was down 9 or -- >> you know, i don't know we get data pricing enough. >> ok. >> it's hard to do that on aggregate basis. we do get some of that data for certain managers. but to add a little bit what bill was saying, i think it's important for the commission to recap and set context with you about what our objectives are with the absolute return program. give some examples of why the market environment in february is exactly why we have it. so, to be specific, the

objectives that we have are to dampen the planned volatility, to provide exposure to sources of return that are uncorrelated to equity markets and traditional fixed income markets. as bill mentioned, the aggregate performance of our return portfolio in february was down about 60-70 basis points. so on a relative basis to the traditional fixed income market to the global markets it's a very good measure. and also if we think about what the beta of our portfolio is, we have a limited number of data points because we've only had this portfolio funded for about 18 months. but we're tracking right there where we would expect to be with a blade of .2 to the global equity market, we're right in range where we expect to be and

outperforming a beta adjusted measure of what our beta measure would be. the other point i want to mention is that we've had very strong absolute performance from specific managers. case in point of the managers that we recognized for investment for the direct part of our absolute return program, all but one of those managers was positive during the month of february. so it's very strong manager selection. and more specifically, one of the managers that we funded recently outperformed its relevant benchmark by 7% in the month of february. so if we calculate a measure of how much we had invested and that out-performance, it's $5 million in one month. so that's significant alpha creation. now, it's one data point.

i would love to say that we could annualize that at the plan level. and i don't want to set an expectation like that, but i think what these data points show is that this is exactly why we have absolute return program. it's why we have a program that has a structure that it has. that gives staff the ability to bring recommendations forward. to identify some of the managers that we have high conviction in, that we think are going to be alpha contributors. we've seen that in february and we expect to see that again in the future. [inaudible] >> about the same. about.2. yeah, great news, thank you very much.

[inaudible] how much was that leverage on average for the month of february? >> we don't have the data points yet for february month end. but it's within our guidelines and our guideline is 3.5 times. so we're close to that guideline. probably in the range of 3.2 to 3.4 times. and just one last data point if i may -- the new guideline has been revised, correct, but we're still operating, kind of ramping up so we're still largely adhering to the prior guideline, with some of the additional investments we make, we expect to get closer to the new guideline. [inaudible]

>> our february month end nav is about 1.85. and the last data point that i wanted to share is that february, as i mentioned, it's one month, one data point, but the performance that we received in february was not at the complete expense of giving up performance during the month of january when the markets were significantly positive. so our portfolio was positive in january. and of those managers that i referred to in the direct part of the program, all but one who were positive during the month of february. those managers were positive in january and thus have produced a meaningful positive year-to-date number through the end of last month. >> great, thank you. in case some of that got lost in translation for some of the members of the public, what people are commonly calling

hedge funds made money in february, while the markets lost money. which is exactly why we're doing this. thank you very much. are there any other questions or comments from the board? seeing none, why don't we open to public comment? >> i would like to say, it's simpler than building cable cars. [inaudible] most pension funds would be very happy to get a 7-8% return on their investments. and that is not difficult, not very difficult at all. one of the reasons we can't get 7-8% is because we spend

hundreds of millions of dollars listening to investment consultants that give them bad advice, giving the kind of advice to invest in every fad that comes along. investing in hedge funds. i think hedge -- investment consultants got started in the 1600s when you were advising people to spend thousands of dollars. i think that's when the investments started. let me give you a portfolio returns, the past 90 years. ok. starting in 1926, 2016, 92 years, in those years you had the great depression, world war ii, iraqi and afghanistan war. 1926 to 2016, if you had 50-50

investment in stocks and bonds, you would have annual return of 8.3%. portfolio 60-40. [inaudible] 10.2% annual return. so that's -- convention of 19 years. if you're all on the board in 19 years if now, i bet the exchange would be the same. so my advice is -- well, don't invest in anything -- >> thank you. are there any other members of the public that would like to address the commission? seeing none, we close the public comment. nexttime. >> there is a -- item.

>> item 8, report on managers under review. >> the second and third sentences are new. and they're repeated in each document. this is because i'm going from 45 45-31% in public equity. >> the new guy has to -- >> yeah, alan gets to share the good news and you get to share the bad news. >> as you know, this memo provides retirement board with details regarding the public market investment managers have been placed under review in accordance with the guidelines, manager of monitoring and retention. managers can be placed on the list for a variety of reasons. most often it's because of performance. we're mindful of the nuance of

their benchmarks. they can be placed on this list due to organizational changes, philosophy, style. guideline violations or other matters deemed relevant. managers placed under review are not eligible for additional funding and may be subject to asset reduction as we talk about the reduction of public equity and the reduction in liquid credit. at the end of the fourth quarter, two managers were added to the list. advent capital. and fidelity which runs international small cap strategy. removed from the litt, we know the -- six other managers remain on the list. four equity and two fixed income.

we've provided comments as bill noted for each of the eight managers. i don't know if you have comments, otherwise we'll turn it over to you guys for questions. >> president stansbury: any questions? >> yes, thank you. advent capital, the performance issues, trailed benchmark at 3 and 5 years. how can you both trail the benchmark and be above peers? >> it's not hard. the benchmark again is set not as a median of all peers, but s&p 500 for example, let's take equities, the benchmark for s&p equities is s&p, that is the performance of the cap-weighted set of equities.

as was mentioned earlier, over long periods of time, 75% of active managers underperform that benchmark. so again, the benchmark is sort of the theoretical what could have been achieved on a passive basis and how managers do after fees can be above or below that. in this case, advent is below its benchmark, but above its peer group. the whole peer group has done poorly. >> when we consider managers that underperformed, you have to consider both, compared to a benchmark and but all of the others. >> in this peer group there were 30 managers. >> can i give an illustration. consider three active managers and index. so universe of four. the best performing of the four was index. the second was the active manager and the next two are the

peers. right there, they underperformed the index, but outperformed their peers. >> i'd only ask for convertible bonds specifically. it's a very -- the performance of convertible bonds can be diverse and different managers focus in different areas. some bonds trade with not as high of volatility as equities, but similar to equities. while other convertible bonds are bond-like and they don't move much in response to what the equity is doing. this was a period when there was a lot of equity performance and over the last several years. and i think as a group, these 30 odd managers focused on areas different than what the underlying benchmark did. that's why as a group they

underperformed. that's why this manager underperformed the benchmark, but outperformed the peers over that one time frame. >> supervisor cohen: additional request of the staff, if it's possible for us to add the year of the original investment, approval in the memo, just as a little bit full of context and perspective. just the year of investment approval. when do we approve the investment? how long has it been sitting on the books, investment where managers have been under review for a long time. >> yep, very good. >> supervisor cohen: thank you.

>> commissioner driscoll: the under review are way too kind. they're nice people, their performance was poor. you can go to page 54 and see it. there is a reason it's in red. it's a long-term number. that's not your decision, kurt, but i'm bringing the observation, we don't want to embarrass the managers, but we have to talk about the performance and we do it publicly. similar numbers on page 56, the capital guardian number. it just happens to be positive. but that was another group when staff did recommend and i can remind you what you said, but i remember staff recommended terminating capital guardian. i cannot make the case if we had or had not done that, maybe that would be a higher return. but there is the result. so it's not about simply manager selection, it's about manager monitoring.

it's buy, sell. if you want to change the process, we better talk about what has been going on. >> supervisor cohen: i agree. >> for a long time, either we pick up the area underperforming, but it's a good discussion item. is it coming to us? good. ok. >> may. why isn't this one coming faster? >> we've brought it forward over the years, since i've been director. there has been different approaches to it, but this time we believe that we want to find out what best practice for a

plan our size. >> ok. >> and make sure that we document why we would consider making these kinds of changes. i think commissioner driscoll remembers a time when there was a delegation of authority that was removed and now historically there hasn't been a delegation, but we're going to bring the whole process and recommendation most likely to the committee. >> i believe commissioner cohen requested this in the january board meeting. it just takes time for p.c. leading the research effort to gather. they've made a lot of progress. and they're in very good position to bring it in may. >> i would caught you about alluding to historical perspective unless you want to get it right. i brought this very issue up in the board retreat. it is a huge governance issue, i was asked to wait and be patient

while we sort things out. >> that's why we're bringing it forward. we're not going to point to anything historical, we're going to point to what we believe is best practice among plans our size. >> and complexity of the investment. >> ok, it's about the decision-making process which i know i and other staff members have been look at how to do, when to move it. it's a change at the top. >> certainly. the delegation to staff is only in one piece of the private equity portfolio. that's where it was. >> in terms of -- we've been talking about revamping write-ups and trying to get the information to the board in a format that is easier to glean the things we need to, to make a decision.

on the memos, for managers under review, when you guys have time, we can just very simply put in the performance numbers, 1, 3, 5, you have them, but if we put it in a sample table like in the preceding section, it would be easier. >> very good. >> is there any way to get more detail about it? you know, qualitative aspect of the review. we understand performance, but there is a lot behind it. oak tree. we kept with them and i understand why from what i understand howard marks does and has done. but i don't see it in here. if i was a board member and read it, i would say, why have we kept this person for this many years? there is nothing down there i can understand that. if you could bullet point the

main factors. he's made a call to quality and we believe that will be a long-term call energy the school, whatever that is, just like with convert. i think it's helpful. i also want to add, i thought mr. martin's presentation was excellent and mainly because he really went to the pages that mattered and polled us with this large document exactly what we needed and got us to the main points of that. i thought that was good because that was a big document, so thank you for doing that for us. and not walking us through every page. >> along that, you may recall information received when a manager is being recommended. we see their calendar year numbers. we don't see that anymore. that's the problem with using annualizing 1, 3, 5, 10 numbers.

one bad years affects the numbers. usually on recommendations, they're recommended to. think about presenting them, if it's a lot of red years, versus one or two bad years. let alone issues of concentration. >> i think that's good point. i love to see individual year numbers. i find them more insightful than one, three, five. >> they're useful, too. >> i like to take them into context of what is in the market. on review and managers. many times managers will be think being the m.i.t. letter that was sent out, managers in our program that says they have to have five-year, so it's really kicked out teams that have left and proved to be great after three years. are we going to revisit all the

criteria we have with the managers just to get in the search and what allows that to happen? >> yes. >> good, because i know we've done that let's say with the hedge fund, with keystone. where they lifted out within a year, we can put them in and use them. that would technically violate the criteria, so i think we need to revisit that. >> the answer is yes. >> ok. >> any other thoughts, comments, questions from the board? >> seeing none, we open it to public -- >> can i make one comment. there has been a lot of discussion on indexes. a point i did not emphasize, we actually show what you would have earned had you put 60% of your money into a global equity index fund and 40% of your money in a global bond fund. for ten years instead of earning 6% a year, which you did, you would have earned 4.4% a year out of indexing.

on the $13 billion you started with, that would have made $4 billion in difference for choosing not to index. >> so if we had gone with the 60-40, we would have lost billions of dollars? >> yes. >> and if we indexed rather than have our own manager, that flies in the face of just having index portfolio? ok. >> thank you. >> we'll open up to public comment. any members of the public that would like to address us regarding managers under review. next item, please. >> i have item 9, proxy report for 2017. >> let me take it as is.

would you like to call 9 and 10? >> yeah, we often do. >> let's call 9 and 10. >> 9 discussion proxy voting report for calendar year 2017. and item 10, review and approval of proxy voting policies for the calendar year 2018. >> we report a summary report on proxies voted. this report provides summary level information on how the retirement system has voted. proxies across two categories, those put forth by management which include election of directors, appointment of auditors and mergers and those put forward by