that it was not a good presentation. i am inclined not to support this because either there is shortsightedness in terms of presentations, or there is some reason for not telling us that there is information. if we have this, great. i am glad we are ahead of the game and have evaluations. but that is not what we heard last week. so i am very upset with this. president mendoza: let me remind my colleagues that this is clarifying questions. you can share your comments after we have public comment. are there any other clarifying questions? i am going to wait for everyone else to hold other comments until after we have public comment, if we have any public comment. is there anyone that wanted to

make public comment on this particular item? ok. commissioner wynns, did you have any clarifying questions? commissioner wynns: i appreciate what the other commissioners have said. i hear their frustrations. one of the biggest problems with this is that we have said from the beginning -- i think we should think about what would be a logical way to do this. the budgeting process is, from the school district point of view, my impression has always been that what you are asking us to do is to basically report a status quo budget for next year. but it is really just a proposal, because then it goes to the board of supervisors, and then we start negotiating.

while i agree that we should be far more constant in our own ability to signal a good decision and -- a good data- based decision this evening, i would like for there to be forthcoming to us, not necessarily this evening, how the process could change or get better, or if it works for us. that is kind of my frustration with the budget as it is being proposed to us. it basically says, "for purposes of this budget and beginning the outline process with the city, we are proposing to roll over all the programs we have." given the requirements for the ability of the city to not give us all the money, that is a problem. i do not think that was

clarified or clear about the steps in the process and whether it works for us the way it is now. president mendoza: thank you. there was no public comment, so we will keep going around. student delegate, did you have any comments? commissioner martin -- norton? commissioner norton: i share some of the frustration that has been voiced tonight about the need for us to have a clear and were established format for how we will evaluate these programs. some of this description makes me feel like -- what measurements are we using? it just feels a little bit haphazard, i guess, based on what i am seeing. but the other thing that i think is even more troubling to me is that cac has note -- i know that they are meeting regularly and

have done work. but there were not at our meeting last week and are not here tonight. that is troubling. it makes me wonder how engaged -- these are our appointed representatives from the community that we are supposed to be relying on for their explanations. if they are not there, what that is saying to me is that for some reason they feel disconnected from this process. there is not -- but never work they are doing -- whenever work they are doing, if there were proud of it and felt it was important to us, there would be here to show it to us, and they are not. that really bothers me, and i would like to know more about that. i will talk to my stuff about why that is. i would hope there would be representatives from the cac here. the other issue i have had, and i have spoken to you about this privately but wanted to bring it up publicly, it is the whole

issue of the carry over. i understand there is some good reasons for that. i do not think the decisions that were made, the various policy decisions, the issues for why we have a carryover are necessarily a bad thing, but it does trouble me that almost 10% actually was carry over. it went to other things than what we originally approved when we approved the spending plan. i suggested to step earlier that the board needs a little bit -- to staff earlier that the board need a little bit of progress reports that some of these things are going to change. we are in a complex environment. king's costs less or more than we thought they were. people aren't available, or whatever it is. but the board needs a progress report. this money you have budgeted for this use is not going to be used for that this year because we

made a policy decision to do something else with it. that would be sufficient to me. but looked at the plan and look at what we did last year. a million dollars we thought we were spending on one thing is going to something else. that is bothersome to me. i think what commissioner wynns said about the way the budgeting is working -- it is not working for us. i think this discussion, although it feels a little bit painful because we are voicing frustration with staff, i think it is healthy. hopefully, it can be the basis for figuring out a better way to do things in the coming year. commissioner murase: i want to thank you for the side-by-side comparison of the previous year's proposal to this years. that is very helpful. in terms of the outcome, i do not see what it was we requested

at the committee of the whole. i think one statement that comes close to it on one of the handouts is the question "is there is a significant difference of secondary school students all attendance rate in 2008 between students who participated in the clinic and students who did not?" we are talking about a $30 million intervention, $30 million. we should not be asking it just participation in athletics makes a difference. what is the difference of $4.80 million to that? we want a comparison. what couldn't we do without this money? we really need to have a system statement of what difference is this money-making to the strategic goals that we have to student achievement.

i hope this gives you a clear direction in the kinds of outcomes we are looking for, the statements we are looking for. i think it is great that in this example it says those students who participated in athletics had better attendance. we want to be able to make the case that this money is essential to athletics, and therefore better attendance. finally, on the carry forward, i think we have a lot to answer for, $1.10 million. that is a lot of money that was not plowed into these programs. the causes of that are identified as overestimated fte benefit costs. that is an awful lot of estimation. i think that a lot of refinement can be done. president mendoza: thank you, commissioner. >> thank you.

commissioner maufas: commissioner chang? i would like to request of dr. khana come forward. i think you are unfairly -- not that the doctor should take a beating either. i think you are unfairly being browbeaten. this is the first time i am looking at you. you should not be taking the brunt of this. that is my perspective. it should be somebody who is getting more money to take the beating. [laughter] that is what it is. i have to say, really you are looking at the end of the rope for diplomacy. i think in the plainest way that

the board can put it, we have asked for certain things. and what it feels like is the runaround. it feels like the runaround. to be here today, asking for a vote, when we cannot get very basic information -- it seems like the runaround. if we put you off long enough, we have deadlines, and we have to make them and get over to the board of supervisors. whether that is the case or not, that is what it feels like. and i do not believe that our deputy superintendent and our superintendent want you to do that to us, as the board of education, because we have to go up there and get a beating as well. that is what happens. we go outside of this board room and take a beating for the money

that is spent, a beating for the money we are not spending on other necessary items, and we cannot account for it. even off the top of my head, i know i have asked for these members to be here. why they are not here today -- i am baffled. that is basic. that is really basic. this is one of the rare times they can speak to the board of education to report the work we have appointed them to do, and they are not here today. that is another level of the runaround. if i go to my appointees and they tell me they did not get notice, that is not right. these very basic things need to be covered by you all. let me ask, dr. connor. is this a proposed process that

we have in place that we are now looking at? >> thank you for letting me speak and defend walter, because i do not think it is right for him to get, you know, come to the first board meeting and take such a beating. the key here is walter has been religiously producing the comptroller's report, which is what he was told was all he needs to do. so the comptroller's report was that this was the inputs that go into the budget, and he has to collect from every program manager the outputs on those activities. when were they done, and what happened? he did push for evaluation to occur, at which point they had to try to come up with the evaluation measures. most of the programs, as we went

and looked at them, only had process measures. they did not have outcome measures that we could go to. at that point, we started to set the stage for evaluation. that means you have to first define the outcome measures. and so the outcome measures still continue not to be defined. we worked with criteria. now we talked about criteria in order for us to evaluate. we need to have four basic things. that is when the criteria went up. we assessed every program on the criteria. the athletic department did meet the criteria. we were able to look at student impact with the athletics department, because they had criteria. it is easy for us to crunch numbers. if we had numbers of participation, we have all the data.

it is very easy to do statistically. but the key is for the programs to define this outcome measures -- these outcome measures. it has been difficult. commissioner maufas: ok. it should not be difficult at this point in time, deputy superintendent, mr. armitrage. the you have something to say before i ask my next question? i am listening. >> just to respond to your comments regarding the carry over. within that 9.2%, you are right. that is $1.17 million. but it is important to point out for the record that we are, as a district, conservative in how the budget. for example, we place budgets at the highest step.

when we calculate benefits, we also are conservative about that, because we are not sure what retirement costs are. it is better to air and have extra than the reverse. that is our district policy. there are usually adjustments made to reflect that. our policy, given the state of the budget crisis and our already dedicated reserve, it was a policy decision to not allow that to be redirected within the program. in terms of what we promised, programmatic, speaking to the inputs, and not the outcomes, which is exclusively rpa's domain, in terms of what we

promised, we met our goals of what we said we would do. there was a total promise of 85 full-time positions. i am sorry. 86 that we said we would do. by the end of the year, all but one of those positions was filled, which was an academic coaches, which was eventually discontinued. the remaining is about three to five positions toto, the word, for various reasons, people on maternity leave, things delayed on hiring. in terms of program goals, that was essentially achieved. that is not to dismiss any of your concerns for the important of this -- the importance of this or how this needs to be as transparent as possible, or how critical this has been to the district and how much we need to be accountable. this is probably the one program within the district that has to be the most accountable, given

the structure we have in the city. that is driven home. we get it. we really do. i think that as commissioner norton and i had spoken about prior to the meeting, we do issue our quarterly report and show our burn rate of the funding. we are able to give a more augmented report to the board to keep you informed and make sure that you have that, so you are empowered to be able to speak with us and are kept informed. i appreciate how important it is an want to to understand we take it seriously. commissioner maufas: let me understand something for clarification purposes. this is directed to all of you. what you are saying is, in regards to the outcome and the ability to have access to that

information -- you are saying to me that is something that is in sort of the silo of the research. it is the planning and accountability department. is that correct? >> i am sure doctor khana can speak to this in detail. >> we are trying to say that a $30 million investment will now be ordered around. there are -- there is a procedure where we have approached this before on evaluation, at which point the inputs and outputs are of

primary importance, and what the city requires. the outcome measures are there in the evaluation. for those, we can only show criteria, guide the program, help them in coming up with them. commissioner maufas: what i am trying to figure out is how we can trigger from one department to the next to get this information to the board of education. i have heard two different departments explain how they do their thing, but nobody has the ability to pull the trigger to get this information from a certain department. this is to both of you. how do we get what we need here

at this stage? >> if i may, i think the big picture conclusion is that the work around this general area, with respect to the whole dimension of program evaluation data, measuring whether it is inputs, outputs, outcomes, student data -- i think everyone at the staff table would agree that we understand the importance. we do not under appreciate the importance, not just for us making good decisions, but to make recommendations to the board based on that evidence, and also to take the next steps to communicate responsibly and effectively with the members of the larger community, whether it is stakeholders are round particular spending plans or elected officials that are considering the recommendation

to the board of the education. all of that is well understood. to be honest, we have to acknowledge that that work has been slow. we do not have as much progress to show in that general regard as we would have envisioned a couple of years ago, even five years ago, when this was undertaken and the whole initiative was launched. i think what the staff has gone through may hopefully give you a sense of the process, and i do appreciate that some of this has been shared, at least with the cac, if not fully with the members of the board of education. but the various flow charts and logic models hopefully give some indication of the thinking behind our approach collectively to evaluation. i think what we need to do next

is to be more of the specific details, the evidence. line by line, how far have we gotten as far as progress on each of these steps? i think we have probably gone further in some areas than others. in some cases, we probably have not gotten very far. as a staff, we do need to accelerate our efforts. from my perspective, and the deputy superintendent can chime in as well, i do not think it is our preference to come on such an occasion as this meeting, to be going through the organizational boxes of the stuff department charts and things like that and say, "where does one department stop and another pickup." as a department, we understand what the need is, and it is our

responsibility to deliver on that. it is a little awkward, given the timing and the feedback that we have received. i know we are asking the board to consider approving a spending plan, in spite of the reservations you may have on this particular set of issues. all i can say is we will redouble our efforts on this whole issue, going forward. commissioner maufas: i want to hear if the deputy superintendent had anything to add, because i was addressing my comments to him as well. >> i think the bond with in all of our programs and all of our spending is something we need to do, particularly in the current budget environment. if i am hearing correctly, you want to know what we are spending these funds on, and what is the impact. the impact is to student

achievement. is that what i am hearing? president mendoza: and accessing the equity and accountability in our strategic plan. >> that is something that we are going to meet with staff to build into our evaluation process. it will build into all of our budgeting process, including the priorities we have spoken about publicly and privately. how does that fit into our accountability plan? that has been our effort. some of them are easier than others, especially when evaluating programs and decisions that have marked from when we initially went down this road to where they are now. the shifting our priorities are clear to us, what they want us to focus on. the message is very clear. commissioner maufas: my last comment. in all of my conversations, i

hear that our staff and cac members are following the latest iteration of the resolution. but that last iteration gave staff more flexibility to do what you will need to do, and we are less micromanaging and more a policy-making body that lets you move high you need to move in accordance with city partners. but if you are following it that closely and it is not giving you what you need to help us, let us know that as well. i am certain we are going to hear the in any type of conversation. but i have mixed messages. you are following the resolution very quickly -- very closely, but it does not allow you to provide us feedback. i would like to hear feedback.

maybe not today, but more around that issue, if it is an issue. it's sort of keeps coming up as one of those "and by the way." it is hindering us, but it was not meant to. if that is the case, you have to let us know. that was my last comment. thank you so much. vice president yee: let me back off of what i said earlier, when i made some comments. just listening to the discussion, i am realizing more and more the issue here is almost everyone is in agreement to supporting evaluation. all of us sitting here have a different definition of what this evaluation would look like and the levels of the valuation. it is probably true that we have

one level of the valuation when it comes to the process part of it. so some of the evaluation that we are thinking of -- a think what happened is that the board sort of devolves over the year into thinking the evaluation should be more of an impact evaluation. and it is true. we start getting into that depth of evaluation. it is very time-consuming, costly. you need a certain type of expertise for certain types of evaluation. i almost have to go back to my original conclusion last week, which is that staff sit down together, talk about the valuation -- the evaluation, and come up with the definition you all agree on so we are not talking about different things. in the time you need to do that,

whether it is a month or two months, come back to the board and the public to say what the evaluation should look like. for certain programs, it will be at a certain level to have impact. certain activities voters supported -- we do not have to prove that art makes a difference. we know it makes a difference. i do not know if that evaluation would be seen as support for another program. please come back. i really want for us to get this right and the on the same page, instead of everybody talking -- i know i am not talking about the valuation in the same way everybody else is talking about it. i am confused as well as everybody else. i would appreciate if we could

do that. president mendoza: thank you. i have a few comments. i always feel blessed we have this money. if these are the kind of challenges we have, so be it. i want to remind folks that when we set this up, there were very clear areas that we wanted to fund. if we think about the different areas -- if you look at sports, we are looking at basic services. we are not looking at doing anything that will make our children smarter or brighter because they play a special kind of sport. we are talking about basic sports, providing them with basic sports. i think the same goes for our librarians. are enhancing -- we are pro--- we are trying to put a librarian in every school, and to try to make their lives a little bit easier with technology. easier with technology. when it comes to art and music,