when these projects go over budget, it does impact operational budget of the organization. while we do have limited resources, we can do a much better job with the resources we have. with that, i'd turn it over to c turn it over to cgr so they can do their presentation. >> thank you, commissioner and chairman. >> my name is keith kenneth. >> if you will and told us, we will do a little bit of trading back and forth as we go through the presentation. supervisor campos gave a good synopsis of what the motivating

factors were for the audit. what you see on this green hair is the mission that was put into the request for proposal that was issued to which we responded. and along with that mission were questions in several areas, and these are acted -- answer directly in the report in section 3. we used these as a handout in our interviews -- use thesso wed communicate exactly what our mission has been peer yen the timetable -- i would like to make note of the fact that we started in the end of

september/october ain 2010. we completed the first round of the report on march 16 of 2011. in keeping with this collaborative approach of sharing our mission with the mta we at that point gave them a look at -- recommendations we have come up with at that time, most of which carry through to the final report. they circulated that and gave us some feedback on what the recommendations were, which was very helpful in preparing our subsequent drafts of the report.

methodology. and it we're trying to an audit triangulate between what the agency says are the capital process procedures, which is called here on this slide stated practices, what the actual practices are as evidenced by the results achieved, and by what we learned from interviews and in doing things like flow charting and using other diagnostic tools. we also reduced -- referred to generally accepted good practice. supervisor campos used the term of best practice of the agencies

like the project management institute, f ta, which has a project -- handbook of project management refer to these themselves as generally accepted good practices that should be adopted to get away from the idea that any particular agency has to use them. that they should go through and sift out the ones that fit their situation. positive findings. and we had good cooperation. and we had interim report to you in november. -- we had an interim report to you in november. we have problems that caused a delay, but those were addressed, for those part there

was a lot of cooperation and openness from the agency. there were 19 recommendations -- there are more varied in the report, but these are the key ones -- more buried in the report, but these are the key ones. we offered two rounds of review. the first round produced 60 -- 76 comments, which are in appendix 8. with the finalized report we got either concurrence's -- we got concurrence on 17 of the 19 and partial concurrence on the others. we would like to point out some highlights but we think the procesthat we think filling ther with planning projects is a good

one and getting input and screening those, and also, we would like to comment positively on the public input that is giving to the public agency. >> a couple of other positive findings we should draw your attention to is the capital programs in the construction division initiated new control systems authorizing the development of a capital program control system, and this is absolutely the right thing to do, hibecause the current systes are outdated and do not meet modern standards. the second thing is we examined some projects in detail, and for example, at the subway project. we found that was within the constraints of the mta.

rather well organized. we particularly like the fact there have been substantial risk analysis. we liked the fact that they have regular meetings with outside bodies to give them their views, and as a result the fta has given them one of the two projects that have the highest ratings from the fta for grants. >> that said, we would like to discuss five areas of major concern. the first one is how well our projects managed -- are projects managed from looking at the

results they achieve or do not achieve? so we picked 13 projects that have been up history, were far enough along, so that we have visibility either an overrun or a potential over run projected to the end of the report. 12 of those were in that state. one was under budget. i think in a normal sample like that we would expect a higher percentage. so if you look at the figure -- the comparison was done again in costs, and bass line costs

include the design and construction, which is shown in the thermometer in blue. the internal cost, which would be essentially project management, a resident's engineers in the yellow, and then a contingency, which is usually added on to these projects during the conceptual engineering phase. conceptual engineering phase entails 35% of the activity required, so the requirements should be fairly well defined. what these three elements constitute is a baseline cost. what we found in reporting to the board, the fsmta board, and

their own monitoring, is they compare themselves to the approved budget, not the baseline budget. so a strong teaheme is they ougt to be going back and using that as a benchmark. so when you have a baseline budget, this usually becomes the basis for planning capital requirements. when you have an overrun like the average group that we look at of 32 percent signe%, it cree call a brittle portfolio were you have to find funds for the project, and oftentimes this may mean taking from other projects or sources.

so what we would like to see, we think the recommendations will do, will put more of an emphasis on the front end and the planning to first reduce the amount of contingency that may be needed, which is the 30% in the orange, and minimize the potential of overruns. and >second >> the second major concern was the reporting oversight. the first is that the board is not proactive in determining either standards or the information it needs to receive in order to have an oversight of the capital programs. the second is, as my colleague

has mentioned, that most projects do over run their baseline cost, and this is because the board keeps approving additional amounts and additional time for the projects to be completed. typically that early estimates -- the early estimates turn out to be low and the project's cost more. this affects not only the cost benefit, which was expected, but as jim has mentioned, it also affects other projects, which are either waiting to be done or needing to be done. the third thing is that ongoing information the board receives from staff that it is relying on is inadequate for the purposes of oversight. mainly this information comes in the form of a quarterly report,

and because this that are controlling to the approved budget and not based when budgets, they keep showing -- in fact, they have shown all projects as having green traffic lights, that is to say nothing to worry about, and members of the board commented they rely on those as a guide and other things are proceeding well enough. those are often not the critical information. for example, in the central subway, the fdta as a number of points, and one of the critical criteria for how well the project is doing is to examine a contingency that is left. you would think this is something that the board would be particularly interested me -- would particularly be interested in. so those are the of three

points that concern us most. another point, which is related, is there is a great deal up confusing financial information around the mta. -- great deal of confusing financial information around the mta. i will draw your attention to two or three of them. the first one is this body, this board, march 30, 2010 authorized were passed a resolution authorizing 1.57 8 billion for the central subway project. this is on a number of conditions. one of the conditions was the stick to the 1.57 8 billion budget. -- 1.578 billion budget.

the report shows the estimated cost was 50 billion above that. this did not attract comments from the board members, nor was unnoticed by the senior members of staff. in fact, when we inquired about it, we were told it was a typographical error. another example is the executive director, about a year ago, told the board that the mta expected to spend 455 million per year on the state of repair. just a few months earlier in march they had reduced capital budget for 2011 and 2012, which showed expenditure as less than half of that. the problem is that people possibly such as yourselves, see

these variety of figures, and really do not know what to believe, and can only be confused about it. >> item four addresses management processes. and one of the drivers for one of our recommendations was the fact that the agency has initiated a program for state of repair -- state of good repair. the ad iat is an fta-sponsored initiative that evaluate the assets and evaluate where they are in their lifetime, and they have set goals for what they would like the state of good repair to be. for example, if all of the

assets were up 50%, they would be halfway through their lifetimes, even though the assets themselves burievaried in vehicles and such. what we feel fits the responsibility for state of good repair, we will talk about it in more depth later, is the recommendation for asset management teams. right now the agency is functionally organized and thai the participants in the capital projects such as resident engineers and profit controls people, project engineers, are in separate sections.

we have a recommendation for an idea of satisfying the intent of state of good repair. we found it lacking process documentation. we looked for an appendix 8. we identified several areas where this should be improved. one thing is that we are recommending taylor processes for different kinds of assets -- tailored processes for different kind of assets. for example, vehicles should have a different process than construction. there was a steadudy as is stud

for capital processes. the study cited ripe for abuse exciting projects. we think that asset management team will provide closer control on who charges those. at other clients we have seen thus contribute significantly to overruns. i>> the fifth area of major concern is the one we complemented before, which is the cps projects, the capital control projects. this is a difficult project in that it interfaces with numerous other systems, and needs to be very well managed. we believe the initial estimates

provided by the vendor were overly aggressive and would not be met, and we have subsequently had discussions with the staff, and we still believe that. we are concerned that that system, which is important for the future control of projects, is going to run behind time and cost more than it originally was expected. >> ok we're shifting now from our concerns and to recommendations. we have already talked about the asset management teams. we would like to make them, asset managers be responsible for some of the areas that were

questions in our charter like state of good repair, what the delivery method is. would we design or build with the contract manager or so on? and measuring performance against baseline budgets, putting more time into the front end, which is item b. and making sure the project manager has cradled the grave responsibility for projects. that is all we're an estimate is required. -- that is where an estimate is required. we have talked along the lines of the improvement of board reporting. in an appendix 5, there are

examples of various metrics that can be used in that type of reporting. they would provide a portfolio look so that they can look at overall howled the collective management of the portfolio is performing. right now each project is reported individually. >> the task of eliminating the confusing financial information will be a difficult one, because the information is communicated from many different sources within the mta. nevertheless, it would be possible to do a better job than it is being done at the moment, particularly at a higher level when the executive director makes comments about cost and time and so on.

>> [inaudible] life is anything but normal. i>> we think a special effort should be put into improving processes, and a single organizational units should be made possible. there is adequate resources made to improving the use of information technology. for such a large agency, it has a very small i t stuft.t. staff. they're not providing the communications development that they need. another theory, which is critical, is one of the cpsccs

integrating to avoid duplication of effort, and the project operations manual needs to be improved to reflect all of these changes, and also to talk of the i mtiming of information. old information is generally not useful in managing projects. finally, our recommendations are that management of the improvement projects themselves needs to be sharpened. we believe the cpcs is slipping, and ultimately will take longer than originally estimated. we also think there are areas that need attention, and finally

come out we have also set out principles for the design and implementation, which need to be adhered to to see you do get value from the system. >> any questions or comments? supervisor mirkarimi: i very much appreciate the detailed information you have. this is the first time that this information is being presented to the transportation authority, and i think there were so many useful nuggets of this information that the time that it might take to really appreciate some of this information may not be afforded perhaps in this body, so i will suggest to this chair of plans and programs that this not be the last conversation based on

the details that are being presented today, and that this be carried over into committee, but i know it was important for all of us to hear this at first swoop, based on what has been presented, especially with the new director of mta and chief executive director that is also present. i also want to highlight that as supervisor, supervisor campos remarked that this is the first time and audit had been funded, for the most part, by the transportation authority as a way to help foster a level of interconnection between ta and mta. and earlier in the time when this was introduced as an idea to members of the mta, especially members that are no longer with the mta, this was not well received, and i believe that was a mistake or misperception, because i think with this level of

interconnection, the ultimate goal is to a line agencies as best as possible to help each other, especially when the mta so richly deserves it. i appreciate the information being presented today, but if people feel a slight pinch of thyme, i do not want that to get in the way of a conversation not being pursued in plans and programs and offline. supervisor campos: thank you. 'there is there is a lot of matl in substance here, and the reason we wanted to bring this presentation is because the level of importance of this matter. we wanted to make sure that the entire board heard directly on what the findings and recommendations were, with the understanding that a lot more work needs to be done and a lot of the work needs to be done in committee, but it was important

for us that every member of the board heard this. i just have a couple of follow- up questions, if i may. i wanted to say that some of these findings are very disturbing, and one general question that jumps out with respect to the cost overruns and the fact that you looked at 13 projects and 12 of those were above the baseline budget by $87 million, which is about 32% of the estimated cost. how does that compare to what is generally perceived to be of best practice? how does that compare to what would be expected in this kind of operation? >> well, i will take the first crack. first, the procedure is to do 35% engineering of the project.

these are not the initial capital improvement project estimates. these have some rigor behind them. there is also then a contingency, which i think by practice is 30%, that is our understanding. you have another question. given those things, we think it is certainly a poor performance. supervisor campos: in light of the poor performance, i do not understand how it is in terms of oversight that the board of directors provide, that when all of these projects came before the board there was at traffic, a green traffic lights if you will that made it sound or look like everything was going