the things that has our e.r. rooms overflowing. it's this issue at and when people are left out on the streets and they end up in the e.r. rooms, it just causes i'm not a doctor. i don't pretend to be. i'm not saying i just know what i saw and i know what i heard. so there's many reasons to address this issue. and we don't want people to end up in the e.r. room when if we can get them in placement before or it becomes that type of crisis, the users, even when they are arrested, unless they have warrants, they're not getting significant jail time. usually it's a matter of hours. sometimes it's more if it's sobering, they're released. so and that's part of our challenge. well, that is that is a challenge. that's that's a real challenge. but at least there's an opportunity for somebody to reach out to them and an opportunity for case management after the fact. any

progress on enacting the lead program and obtaining funding for it or revising, revisiting that strategy? it's a strategy that is has been talked about. we haven't made any movement with a formal reentry introduction of lee. you know, i am a proponent of lead. i think it has value. i think part of that. but even that there's a law enforcement component to that when it's done the way it's designed, you know, people are given an option. you're either go to treatment or, you know, the hammer is waiting on you and the hammer is incarceration. and that is one element that we're not necessarily fully offering right now. right when you have the option on, i mean, treatment especially harm reduction, that is actually the model, right? that if you have an opportunity to access treatment in lieu of and isolation, punishment, probation, parole, whatever the case may be that those folks are then have the agency to self

motivate into treatment and yet that's not necessarily the strategy, as i understand it. it's that that we're using. and i, i commend the department for being out there and sometimes reversing overdoses. i mean, i think that's a positive number. and that's something that helps balance out the challenge. but at the end of the day, i think it's been said before, that's not what our officers are necessarily here to do. right we're we're trying to improve our public safety outcomes. we've done a great job. i think , and it doesn't get talked about enough that our violent crime numbers are down right. and we are facing other issues. but i think that not having that focus solution outcome oriented investment is really going to get us to that place. and as vice president carter overstone said, there have been other solutions proposed. and i just i

feel it's we're stuck in a place of kind of just, you know, hoping that this strategy is going to get us to a to a better place. and it just it seems like it's also going to damage a lot of lives in the process. yeah. thank you. and i do, i, i respect that. i do. i do want to say, though, i want to highlight the fact that we are at the table really working hard with our public health officials, people that are assigned to this, you know, drug market agency coordination center. they're there every day. people all working in the service side of this. the homeless and supportive housing, because a lot of our people that we're talking about are unhoused and human services agency trying to connect people back to their their support systems. so there's a lot of work being done. you know, in collaboration with what we're doing. and i

don't want that to be lost in this conversation because i know when we only talk about the rest statistics, whether it be drug users or sellers, we're not talking about that piece of what this is about. this this being this agency coordination center. there's a lot of work being done and we're trying to move the needle on bringing these services and so it's not all policing, but that takes collaboration and. to vice president carter stone's comment about the report, some of the people who had part in that report are working with us to try to make this work. so i just want to make sure that we don't lose sight of that. it's not all about the arrest. i know that's what the questions that we're asked, but it's so much more to this than just that. and we're trying to bring all this stuff together. thank you. thank you. sergeant, could we go to public comment for members of the public that would like to make public comment regarding line

item five, the chief's report, please approach the podium. i'd like to use the overhead again. my son, aubrey abkhazia, who was murdered august 14th, 2006. i have this where mayor gavin newsom said i know who killed her son. mayor gavin newsom said thursday. da know who killed her son. the police know who killed her son. he can name individuals and their addresses. i bring this with me all the time. of all the names of the people that was involved murdering my son the last report we talked about how much money has been paid out for the unsolved homicides. and then you put on your agenda about about, about the unsolved homicides and finding someone else to pay. i mean, finding someone else to pay tipsters to solve our cases. and still, i

don't know what's going on with that. if you hired anyone else to solve our cases, i bring these pictures of all of the unsolved homicides. we all know about them. they're not solved. again i bring my me standing over my son. this memory will never leave my mind. i come here every wednesday to bring awareness to the unsolved homicides. this is what the perpetrators have left me. a lifeless body. i'm still looking for someone to solve these unsolved homicide ads and pay tipsters some kind of money. again, i bring up, we got to look back at our history so that we can move forward. that is including my son. what can we do about it? i come here every wednesday. it's not putting on a show. i'm not here to entertain

. this was my child. this is our children. something needs to be done. thank you. members of the public that have any information regarding the murder of aubrey abacus, you can call the 24/7 tip line. at (415)!a575-4444. hello. good evening, commissioners. good evening, chief. i firstly want to thank you for that report you've given . and while you all were discussing it, i did want to mention that seattle bill passed with a majority of measure 110 and 2020 and what the measure 110 did. was it decriminalize the possession of small amounts of hard drugs and establish a drug treatment program funded by the marijuana tax. now, the police officers can only cite the users and has been an abysmal failure in 2021, there

were 745 documented overdoses. well, in the year later, in 2022, there were 13 and nine 1309 an overdose and no treatment center. so i think the problem we face here in san francisco is the board of supervisors is responsible to get the treatment center and it's sfpd's job to ensure we get the user from point a to point b and ensure the safety of the community. but we're expecting the sfpd to do the board's job and their own. so like i said before, thank you, chief, for the work you've done so far. and we recognize the work and we want you to keep up with it. thank you. hello everyone. my name is joe cirillo. i'm a licensed clinical social worker here in san francisco. i actually came to

talk about the dolores hill bomb because i'm a skater, but i will say really quick, thank you for reviewing what's really happening on the street. i'm a therapist on the street with the harm reduction therapy center, definitely seeing lots of overdoses. so just really putting my support behind more coordination rehab centers can be like waitlists of up to like two weeks. so while people are waiting the arrests don't seem to be helping folks. and yeah, hopefully there can be more resources put into support and medical support on the street and therapy. but my other hat is i'm a board member of the san francisco skate club, which is a after school program. um, i just wanted to read a few points that our members wanted me to address to the board. so we know that a lot of, most of the charges for the 117 folks who were arrested in july at the dolores hill bomb

were dropped. but we just wanted to see if the city and the commission can bring a little bit more clarity to what's happening to those folks who were charged, especially the youth, like what happens with their mug shots, their fingerprints. what are the implications if they're arrested or cited? again and just one other point tied to that, if i can use i don't know if this is still working. so basically, i don't think it'll show up very clearly, but the san francisco skateboarding laws and the enforcement around the laws don't seem to be updated in a lot of the city government websites. i'm just pointing out the board of supervisors analysts website says that traffic code section 100 says that skateboarding in most areas, aside from skate parks and aside from neighborhood sidewalks, is illegal. i've personally been.

there's no further public comment light on him six directors report discussion an report on recent activities and announcement commission discussion will be limited to determining whether to calendar any of the issues raised for a future commission meeting. executive director henderson. thank you. so we have opened up over 600 cases so far this year. that's a jump in the number of cases that have come in this year from last year. we've closed. 552 cases, which is more cases than we took in at this time last year. we have 304 cases that are still active and we've sustained 44 cases so far

this year. we've also mediated 31 cases and we have 22 cases. who is investigate? actions have gone beyond and a nine month period of those 22 cases, 17 of the cases are told and we still have seven cases that are pending resolution from the commission and 85 cases that are pending a resolution in with the chief's office in terms of the past two weeks, we've received about 36 cases have come in since the last police commission. when i gave this report and the largest percentage of cases that came in, which is 14, were for allegations involving a neglect of duty, meaning allegations where an officer failed to take a required action by complainants. the full list of all of those complaints can be found on our website in terms of

the precinct, the largest number of cases in terms of complaints came in from the past two weeks. the largest from central station. we had five allegations that came in regarding central station and those allegations were for complaints alleging, again, that officers failed to properly investigate or to take action when information was reported to them. the full breakdown by every precinct, including the airport, can be found online for the allegations that have come in in terms of outreach, we conducted a mediation conference. we'll talk about that next week. i'll have a summary for you guys outline the information that we presented in terms of the audit we received an exemption from the police commission on 8.10 for calendar year for 2022

activities. and i just want to clarify, you'll see the information in the audit, but i'm just reiterating it just to remind folks on why we needed that exemption, because there were no investigations governed by 8.10 and 2022, and because revisions to 8.10 are underway as well. so you'll hear more about that as that unfolds. we have nothing in closed session this evening, but present in the hearing room today, in addition to myself, is chief of staff sarah hawkins. also present is senior investigator brant bagian and also present. is one of our attorneys on our policy team. jermaine jones. for folks that may wish to get in contact with dpr for any reason, you can contact us at sf gov. org or you can contact us. at

(415)!a241-7711. i'll reserve te rest of my comments for the agenda items as they come up that concludes my report. i thank you, director henderson for that report. at our last meeting, we agenda and discussed . sfpd's auditing practices as it relates to the audit of traffic stop data and director mcguire presented on the current practice rs which are just just checking when there's a when there's an entry made that's not completed. the audit flags it, but it doesn't actually look for any signs of, for example, data falsification or anything else that would go to the integrity of our stop data. i just wanted to i know this is an issue that you've spoken about and

something that's at the core of a dpa's work, so i just wanted to invite you to, to, if you had any thoughts on sfpd's current auditing practices and how they could be improved. yeah, i think you raised some of the really important issues on the inquiry about falsification or data analysis and those were specifically cornerstones i think we were missing. i did see that report and i reviewed that document that was presented on the what was referenced as a small data. and i have a number of concerns. and again, i can't discuss any of the pending cases is that i have specific only or those specific investigate actions. but in reference to the overall information and the bigger issue, which is what i think you're asking me about, i , i think there's some real challenges that exist that are actual even from the audit that

we were presented small a audit from the charts. it still shows a number of outstanding entries that go back as far as 2020. i real concern about the information that identified in the charts that were numbers were deleted. that's a third of the cases were deleted. 13 of those cases from 2020 are still in progress. for me, these are just numbers on a page. there is literally no analysis or explanation that speaks to the issues that we were investigating and continue to investigate as it relates to stop data. it's not in compliance with any of the audit steps that were committed to under collaborative reform. um, the cri or even the bureau orders. that's not what at least what i saw. that's not what this is. that doesn't speak to what was mandated or to the data that

we requested or tried to raise and evaluating the stop data we has raised the issues about stop data and the reliability of that data for over a year now. but we have i haven't seen a change either in the response or in the data. and this this report did not answer any of those questions as well. i've said this before that we are a small department and we have limited staff. we are working diligently on the individual cases that we have, but we have real concerns as we have real concerns about the information that we have that speak as well to the individual discipline without having a broader understanding or a full measure of both scale and scope of what the problems can be. we've seen that problem in other jurisdictions, and i'm

not saying that we have those exact same problems in san francisco, but we have the same indicators of those problems here in san francisco. but it's difficult to drill down on individual transgressions without a broader understanding of more data and broader analysis. and those are the two things that we need. i would suggest, and i think we're moving in a direction where the best option would be a review, which is comparable, but something different from an audit. we've done audits in the past as everyone is aware, award winning, i might add, and so i'm going. to begin a review too, which is separate from an audit because it's a little more nimble in terms of the analysis. but that that review is going to call for us to either work with outside agencies and or with the

department, but will absolutely need more access to data and analysis, which is what i think this is where i think this process and where the project needs to go. so that's what i'd like to do. but i'd be calling or asking for someone from the commission to continue that work with us as we unfold. failed to confirm or deny by the problems that i think we've uncovered thus far. i know commissioner benedicto has worked with our audit team extensively. i don't want to expand his role anymore or yes, i do. i don't even need to dance around. it is we i think that's that's what i would like to see. that's what i think would be helpful in addressing this issue. i think the individual cases that come up are a real concern. but if we

really want to speak to the problem in ways that confirm or deny the problems that we've seen in other jurisdictions is i think that's the step that needs to take place. i don't know if that answers your question, but that's that's very helpful. i think that i think that does answer the question. that's helpful. and i think, as we said last time, we i think we need to agendize this issue again to coalesce around a specific course of action and the thoughts that you laid out certainly makes sense to me. um commissioner walker. thank you. now i have a question on i think we've talked about this before and i, i think it would be helpful when we look at this in light of sort of the discussions that happen every week around should we be arresting drug users or what should we be doing

? the information that we're getting around complaints that come in? i would like to see kind of the complaints around that issue of people calling, because there's folks who are doing stuff out in the street. and when officers come, they don't respond. i mean, i would imagine since i've been on the commission, most the largest percentage are in action rather than action. and so that's real . i mean, there's a lot of what police are doing, especially as we ask for community policing that is about crime prevention, not crime fighting per se. and so, you know, it's the if the complaints are around a mental health crisis or if they're around drug use in the streets or if, you know, the drug use accelerates to something else and i mean, i think we have had a policy in the last few years

of not doing anything, letting people be and making sure there was no violence happening. but i think that the last few months, there has been an effort to stop the action. yeah. so i would say and i think which is why i report those statistics as in as much detail as possible. and i try to make sure that it's clear that as a disclaimer, these are the allegations because these those exactly are the things that people are calling to complain about. but without the specifics from the subsequent and following investigation ones that come from it. yeah, but this you know, the commission asked me to report on these things. that's kind of why i do it. it's harder for me to drill down on in all of the specifics, although i do to 100% on everything that comes to and through the agency every week. and that's why i put it online just so there's complete transparency about what people

are asking for here with some of the specifics. and i focus on just the general ones. but you are correct that just in terms of a review, and i think this is part of what you were asking, is what we're seeing is that that trend in terms of folks complaining that they've called to have someone arrested or that they've called to have a police report generated for something and it's not happening, continues to be the top allegation. so separate from all of the other analysis and complaints and investigations that come into the agency, that has not diminished what i've seen in terms of diminished or changes week to week is the percentage get higher or lower, but still remains the majority. it's rare that that that that allegation or that complaint is not the top complaint at the agency. i think it's a it's part it's a relevant part of the

conversation we have about alternatives. and i think knowing that knowing the need out there is going to help us find better solutions as an alternate response when we can. so i don't disagree and to me, when you guys are having the conversation, the balance on the other end of the conversation is just this, that the calls are still coming in. and i think the department, you know, as the chief has addressed in the past , the calls come in every day for everything. and these these allegations, i think, reflect that the majority of those calls that come not just to the department, but to the agency themselves, asking for a demand for services, a demand for a response with out a broader breakdown of the specifics of what those calls are. but those come subsequently, later, and then you see them in my further investigations and then those are broken out quarterly and annually. i mean, i think it's also important to note that the care court is starting up around

the severe mental health issues and the referrals from family and loved ones that that have more impact on the system. at some level, these numbers are a measurement of the need of that efficiency or if it's working in terms of the public's response to it, great. yeah. thank you, commissioner benedicto. thank you very much. vice president carter overstone. i definitely agree with you. director henderson, that i think that some sort of formalized review i think makes sense. it's something that that i've discussed with steve flaherty, who does audit a cpa. i think it's an important an important specific step so we can achieve that transparency. i wonder, chief, if you had i know it's being floated and talked about live, but if you had a reaction or wanted to respond to that, to a possibility of having to do some sort of directed review about about the about the stop

data. yeah or response, because i know we heard from director mcguire that one reason why the audits as they are right now are really only able to look at are so limited. i we talked about how they're only looking at started and not completed. and one reason was that there's just one. so there's just not the resources. but if it were something that uh, to at least give us a snapshot of and better insight into what, you know, the scope of the problem is, because we've seen this in other departments. if that'd be something that you'd be open to working with the dpa on. yeah, i'm open to working with dpa on that. i just want to say also we have to sustain the effort. you know, i think mr. rosenstein read part of the substantial compliance and one of the issues was we got to staff it. we have to staff this. you know, the audits are good. they're great. but audit recommends actions have to be implemented. and you all know that because we've been held to account on that and we

have to staff it, you know, and that's kind of the bottom line. the scope is fine. i am open to that, to answer your question. but we have to staff this to where audits are sustainable because as absolutely it has to be an ongoing thing. absolutely. and i think that they aren't mutually exclusive. i mean, i think that a one time review that, as director henderson said, is maybe short of an audit. so it doesn't carry with it some of the same requirements might provide us a roadmap. and then long term. absolutely. to add more to that unit sustainable, i think i think is good. i think then if i'm i'll continue i'll reach out with rector henderson's. i'll reach out to steve flaherty to see. and i think what would be, you know, i think that we've talked about this a lot, chief. i think that the system works best when dpa and spd sort of have a unified front and they're collaborating on something instead of instead of in a more adversarial nature when it comes to policy changes. i wonder if i

see i see director mcguire there and i want to director mcguire on the spot. if you could ask if we could work collaboratively with dpa to come up with some sort of a specific action proposal to satisfy what the vice president has asked for. so we can present at a future commission meeting. this is the one time review proposal. there's buy in from dpa. there's buy in from sfpd. there's this is the data sharing they've agreed to. and so we're not getting and if we could present that to my fellow commissioners for a vote and then that action can be taken. yeah. i mean, i'm always willing to work with staff. steve. i'm go we go way back to the controller's office together and i'm also always willing, willing to work with janelle or whomever is appointed or assigned the work for sure. perfect. thank you. director henderson. yeah, i was just going to say, i think one of the big issues is just in speaking to one of the concerns that the

chief raised in terms of personnel, a lot of this can go almost out. the majority of this conversation is really focused on access to data, less than the personnel to evaluate it. what specifically is a problem or not a problem? the issue is more access to the data and the raw data. even for the small audit information that was presented, going through and evaluating that information is, i think, a big part of the solution that confirms or denies whether or not there are a pattern in what we've identified are clear problems, whether or not that's individual or systemic. that's, i think, a big issue. i know you said you wanted to agendize the issue and we can come together then on what the plan is. but i'm mindful of limited resources , both from dpa. we're very small department in comparison to the police department, and

i'm mindful of the chief's concerns about staff and personnel for this work as well, which is one of the ways is one of the reasons, one of the very reasons why i think a review is the best option that is open to working with third party agencies like an academic institution or third party agency or third party department to help leverage both the analysis and the data to make sure that we have something that's accurate and reflective of what i think the concerns that are raised from the individual cases that we've seen . great. thank you. and yeah, i think the we'll get to talk about this more later. but the d.o.j. cops report specifically suggest using an outside organization. i think that would hopefully satisfy the chief's concerns about staffing. and i am surprised to hear that access to data has been an impediment since by state law, all of our stop data is publicly available except for personal identifying

information. so i'm hoping that any issue to data access can be resolved with dispatch. okay. seeing no other names in the queue, sergeant, could you take us to public comment for members of the public that would like to make public comment regarding line item six director's report, please approach the podium. there is no public comment. line item seven commission reports discussion and possible action commission reports will be limited to a brief description of activities and announcements. commission discussion will be limited to determining whether to counter any of the issues raised for future commission meeting. commission president's report. commissioner's reports and commission announcements and schedule of items identified for consideration at a future commission meeting. commissioner benedicto. thank you. vice president carter hubberstone. a couple of updates from me. um let's see where to start. earlier this week i spent some time with with the policy

development unit, which has been sort of restructured to work within our written directives unit and the way the department is handling its policy development. i believe that the vice president and commissioner walker have also attended similar meetings in the past. there are an incredible group of officers who are trying to who are spending a lot of time trying to get our policies in line. we heard from some members of the public today that there's a disheartening number of our policies that still date back to the 1990s. you know, i want to acknowledge the i talked to the chief about this before, about the progress. this commission has made. i think we've passed more general order updates in the last year than in the preceding significant number of years. so a lot of progress is being made, but a lot of progress still needs to be done. and so it was really educational to sit in with along with some of the commission staff and as well as assistant chief flaherty with in that meeting. um, i also

want to acknowledge i believe they left, but the members of the public who raised the issue of our public alert policy as one of the policies that the date back that's that dates back to the 90 ends. i've asked them to send i believe it was the oakland policy that they both worked on to our commission staff so it can be posted for members of the public to see and hope to see that among all the policies that are hopefully updated, going forward as well as fellow commissioners know, we've continued to search for a our policy analyst position. commissioner walker and vice president carter and i will be conducting interviews for that shortly. and so hopefully that long saga of that vacancy will be coming to a close as well. um, and i also lastly, i wanted to talk a little bit about the open letter that the police officer association actually sorry, before i do that, i also wanted to acknowledge captain o'brien noted his domestic

violence survivors month. it's also hispanic heritage month and filipino american heritage month . so you know, a lot of celebrations in this month as well. but i wanted to close to briefly talk about the open letter that the police officer association sent to the commission office and shared with the media. i know lieutenant mcrae spoke about it in public comment. i don't see her her in the gallery anymore, but as she said, she's easy to find. so i'll if given the opportunity, i'll share these thoughts with her as well. i think it is important to note that every single member of this commission takes confidentiality incredibly seriously. it is. we are bound by charter by state law. half of us are attorneys. we are bound by our own professional responsibilities and take and take confidentiality and our duties as the discipline body for our officers. very seriously. and that is true of every single commissioner. we sometimes have disagreements up here on policy

, on strategy, on tactics, but we do not have disagreements on our serious commitment to confidentiality of officers under our disciplinary proceedings. and it was disappointing that that this letter questioned that commitment, but also accused a commissioner specifically accused vice president carter stone without any evidence s of violating what is, again, a very significant and serious duty. there's agreement, i think, among this commission to get to the bottom of how confidential information made it into the public. and i think that there is agreement that we want to get to the bottom. and we heard commissioner walker talk about agendizing that and i think that is 100% appropriate. and called for. you heard the vice president call for that to what i don't think is called for is to jump to a conclusion, to an accusation, a very serious accusation without any

evidentiary basis. and that was that was disappointing to see. i understand that the poa has a responsibility to protect its officers. we as commissioners have a responsibility to protect officers confidentiality as well. and i think i've said this to a lot of people, it's important that when we have disagreements on policy, we don't have disagreements on motive and we don't have disagreements that we have similar goals. and it was it's been overall, i think, a productive year of working with the poa on a number of issues and it was disappointing waiting to see what an accusation. it didn't have any evidence when we've seen in the reporting that it was apparently we've seen from the publication that published the story that it was there was a document redaction issue and nothing at all tying it to the malice of any individual commissioner. and so i personally would like to stand up that i don't believe any of my fellow commissioners would violate a confidentiality of

officers. i believe that every single one of my fellow commissioners takes that commitment seriously. and i defend my fellow commissioners individually. i defend our commission as a body. and i think that it was out of line to make that accusation without evidence. and i joined my commissioner's commissioner walker and the vice president and hoping that we'd get to the bottom of how this confidential information was produced, but that we should stick to facts and not baseless accusations. thank you. thank you. commissioner benedicto. and, you know, i echo all of those words, all of those sentiments. and i will be responding to the poa's letter in due course. just to updates for me. i like commissioner benedicto was invited to attend an orientation session with the policy development unit. i very much appreciate the invitation and a chance to meet some of the new

employees and a chance to get a better understanding of how they view the policy making process and some of their thoughts and challenges. so i appreciated that and also like commissioner benedicto, i will be sitting on the interview panel interviewing applicants for the commission's policy analyst position and look forward to hiring someone and filling this this vacancy that's been open for quite some time. commissioner yanez. thank you. vice president carter overstone a quick report from me. had a good meeting with commander parra from the community engagement division, put them in contact with some folks that reached out to me about contributing and offering some youth development training for the department officers. and i'm and also contributing to

community policing strategies. the connection was made and they will be moving forward trying to coordinate calendars. as i, i had a conversation with juvenile probation department president margaret brodkin, who invited me to go to the youth commission on sorry, to the juvenile probation commission to provide an update about our progress towards initiating the pre booking program and that update they requested in november. so it gives us a little bit of time to get a little more information in. thanks, chief, for reaching out to the policy folks to get us the revised juvenile. dgo 701 draft which which i know is a work in progress. and it's good to be able to get my eyes on it so that we can provide some feedback and suggestions and lastly, i am working to

coordinate with the youth commission in who has some questions around in the pre booking program that we are working on developing. and lastly, i do i did around this data conversation, i had a meeting with a professor from san francisco state university who is interested in offering and leveraging some of their staffing to kind of comb through all these data points points. i know that it may be a challenge to engage a third party, but i think when somebody's leveraging their expertise, i think we should seriously take a look at that and see how it is that they can contribute. but considering what a large and heavy lift that's going to be, so i know that this is going to be agendized for us to figure out what the path moving forward will be. so i'm very interested in participating in that conversation and that's my report. thank you. commissioner walker, for thank you. so i will

just reiterate again that we are in ongoing discussions about the recruitment of women into the department with the assistant chief lady over there for the issue of child care has has been raised. and we also i've been discussing this with at the state level with our senator and our the governor's office to actually help with. i know that the state gave money to help with recruitment. and this is one of those issues of that could really help with these kind of services available for folks in our department. so i also discussed it with the sheriff's office, who who is also very supportive of having the service available. so i'm really excited about that. and also look forward to the update about the lactation stations and how we better improve those

situations. and also, there's ongoing conversations about the patrol specials project. i talked again with our city attorney and we are just the next meeting, i think is october 10th, where we're going to start discussing very specifics about it, what it would take to do some pilot programs with our community benefit district partners, the specifics of that and not just the rules and regulations and training requirements, equipment, but also, you know, what we have to do to amend current agreements with the community benefit districts. there's a lot of support from folks who are already doing private security, who really want to partner more . i've also spoken to the folks in department of emergency manager that are running the heart program, which is the sort of the parallel of parallel infra structure creation to get folks is not aimed at jail or

hospital oils into wherever they're going. and it needs a whole other infrastructure to support the bringing all the parties together. so i really am looking forward to that discussion and seeing how that all works. the departments are already speaking together, but really having the kind of infrastructure we have that allows us to transport people who are arresting and who are in an emergency. it needs to be as effective. so hope really we can i'll update and bring people in on that. i think that's it for me. oh, i did want to mention that there is a october 20th, 26. there is a pink brunch at bill graham auditorium supporting women's health breast cancer supported by the sfpd, especially the women's caucus.

or is it the caucus of the women's group within the department? i'm happy to share information and maybe i'll give it to the to sergeant and he can let everybody know about it. but it would be great to have everybody there. it's a it should be well attended. i think they're anticipating 14 to 1500 people. so okay, october 22nd, commissioner yee, thank you very much there. vice president carter, also, i just want to report that last month, september 22nd, i was able to attend the safety meeting with cic and cadc. that's a community youth center and the chinese american democratic club that held a safety forum for in richmond district with cap captain chris canty. some of the concerns with the merchants was

that on their there have been quite a few break ins, so i replied to them that they should also check their locks and making sure it's properly the right type of locks for their for their door fronts and also hopefully they have cameras there to keep them safe. cic was able to pass out some safety package information while i was there briefly to just introduce myself and to listen to their concerns. one of the merchants asked me is have you ever tried for filing a report online? i said, yes, i have. and he says his question is, is it difficulty? and i say, yes, it is difficulty. go through it as as i experienced it myself. and the last one was, is there a translate options for other languages and i did not know that answer. so maybe i'll talk to the chief after this meeting

and get that information. if you have it. that's all i have to report. thank you, sergeant. could we go to public comment, please? members of the public would like to make public comment regarding line item seven commission reports. please approach the podium. and there is no public comment on line item. eight discussion and presentation on benchmark. first, sign a case management system at the request of the commission discussion on. there you go. you can just use. all right, thanks. good afternoon, commissioners. my

name is ron huberman, chief director. i am the ceo of benchmark analytics and we are grateful for the opportunity to be able to present to you. today, i'm joined by my colleague nick montgomery, chief research officer at benchmark analytics. hello, commissioners and chief. great. so we thought it would make sense. we'll give a quick overview and of course you'll interrupt with any questions you may have or leave them, whatever the appropriate protocol would be. but i want to just start by thanking the chief and all the members of the san francisco police departments who have been our partners in this endeavor and have been incredibly forthcoming and hard working and get us getting us to the current position that we're in. so thank you for all that work. great there's fundamentally three parts of this program, and today we're going to be focusing on two parts. one is first sign, first sign is the brand name or the public name for the product that is an early warning and intervention system. and we'll be talking a lot about that product. what the data that goes into it and how we come up with

the information that we do as well as system call care. so stands for case action response engine, and it's what records the follow up actions after an officer has been identified for incremental support. so let me just start by saying a few quick things about benchmark. we were founded out of the university of chicago where the company was first formed. it was initially a research, initially a research project that that spanned over ten years that try to understand a fundamental question, which was can you use basic police operating data to understand and the likelihood that an officer is going to have a problematic incident rate and then use that data to try to get in front of it and help steer that officer and provide interventions and support to help get them on a better and different trajectory . the result of that research from the years of chicago is that the answering is overwhelmingly yes for police

data does clearly show some officers need incremental support. it is identified, able and knowable, and there's a way to interact and do that. our work is funded by the joyce foundation. it's a non-for-profit organization that is based out of chicago and works primarily in the great lakes region, as well as the american institutes for research. the american institute for research is one of the largest non profit research organizations in the country, and they do a lot of work with municipalities, counties and others. in this particular work. and so the three things that we fundamentally focus on as a company are how we get and how police departments manage all of their people. data we do not touch the crime side of the house. we focus only on people data. so internal staff, data and information of what we do. we then use that information to help police departments understand which officers need incremental support and intervention, and then we help

them track and manage those interventions. ultimate to lead to better outcomes for those officers and the public that they serve. just a few quick notes about us. we operate the world's largest multi-jurisdictional database. why that's important is it's very hard for any one police agency to put together all of the information you'd want to understand police performance globally. but because we do this across the country with agencies of all sizes, we're able to take what we learn from any one agency and port those learnings from the data and analytics to other agencies that we work with and then the last thing i'll quickly say is we operate late. we host, i should say, through the university of chicago, something called the national police early intervention outcome research consortium. it's a mouthful, but ultimately it's a place where police departments come together and can share best practices and understanding around what interventions work, what do you do, what are the best model policies around how to intervene, how to intervene, how to identify problematic conduct

and the like. just a quick other note here. trying to look while i talk here. we're the only research based product on the market and so up until this point in really the history of law enforcement has been trigger based systems. if you get three of this over y window of time, as an example, they alert we have a research paper from our colleagues at the university of chicago that shows that those generally lead to a 70% false positive problem, meaning you're alerting a ton of officers that shouldn't be alerted. and typically a 40% false negative problem, meaning you're missing officers that you should be identifying. we came about this and said there's got to be a better way and a research based approach is what ultimately ended up mattering the most to do this. so what we can tell you is we've been very deep in this work with sfpd to date, and we've developed a model that is deployable and ready and

pending, authorized version of a policy for its deployment. but ultimately, the data science results of it show that the model has 81% efficacy. see what it does that number mean? it means that 81% of the time when we identify by an officer within the system, within the next 12 months, they'll go on to have an investigation for a major event. it is about what we find in policing across the us. our average efficacy rate is about 85. i believe it's 81% today. just because we're not live, because the system gets smarter over a window of time. in the sf context, we used over 79 different data variables to come up with the early warning system . so there are 79 different variables that are involved. they fundamentally come from four different data sources. as officer attributes, arrest records, internal affairs data and use of force records. we find about 5. and this is a national number of american

policing. when we engage in police departments to be at risk . what we can tell you about american policing globally or nationally is it's never evenly distributed. so don't think about 5% of an agency being at risk. that's spread out evenly. typically when we find risk, we find it in pockets so that any one community might be experiencing more risk than others might be, and that typically that 5% is creating 66% of the disproportionate force events. i'm now going to pass it on to my colleague nick to talk a little bit about how the system works. and we can get into all the details as well. thanks nick. thanks, ron. so the way that the platform works is that it takes data fed daily from the sfpd data systems, brings all that information in and converts it into an assessment of those officers and their activity levels. it's looking at sworn officers who have at least one year of activity to begin that assessment process. and any officers included that has a one

year or more as it's looking at the data on a daily basis, it compares those officers behavior patterns over the course of their their tenure to other patterns that have existed inside sfpd historically. so it's looking for those other officers who have been investigated in the past and looking at those patterns as to identify similar trends for individual officers. those officers are identified and selected by the system. and then there's a secondary system that looks to compare officers to their peers. so not only is the platform identifying compared to prior patterns, but saying are the individual officers selected different from officers who are deployed in similar areas, similar ranks, similar times? once it makes it through that secondary gate, the officers are released to a dashboard for prioritized intervention. those officers are categorized into an actionable state and advisable state or a minimal state for supervisors to go through and begin the review process to

determine next steps. so what does that actually look like? and what you'll see here on the screen, as well as in your printouts, it's providing detailed information on that officer's risk history. so have they been alerted on before? has it changed over time? essentially is this a recurring pattern for the officer? it's looking at information for arrest patterns, complaints, investigation and discipline and use of force. and for each of those, it's looking over the course of the past years to compare that officer and whether or not they have similar or greater levels of force investigations. et cetera. compared to their peers and the overall department, those individual areas that may be issues for the officer, they're flagged as either elevated or high, telling the supervisor and the user that the officer is in the top 5% or top 1% overall for the for the department, and then they can follow down for detailed information about the specific events that the

officers were involved in over their recent history. this part of the page is what's the main place for supervisor to interact and engage with to understand their individual officers who have been alerted on they get a broader view or they can see all of the officers that they're working with and that is based on their permissions and their scope within the department. so sergeants would just see their unit. and as you move up in the chain of command, they would see other officers that are underneath their purview. all triaged for them into that actionable, advisable and then minimal state for them in terms of the officer risk with that, they can also track whether or not they've begun their interventions, how long and click in to see additional courses of action and that's really where the meat is. that course of action process is where supervisors working with support from across sfpd, sfpd, can identify exactly what an individual officer needs and where their issues might have been and supports mentoring or coaching, coaching, training. et

cetera. to correct their behavior. that process is then reviewed and submitted to command staff for approval so that everyone's aware of the officers and the supports that they're receiving. so as we look at the system and the way that it's being implemented, i want to echo ron's statements earlier that we've had strong partnership from across the police department in terms of implementing, getting all the data in and building and developing policies. we've got the data in there currently for arrests, complaints, use of force and assignment history, and that as additional information is added over time, it can be added to the algorithm to enhance the selection process as well. so that overall is what we're seeing with the platform. and as ron had mentioned earlier, the system is ready to go. and right now and it's waiting policy approval. and we know there's been i would just

say as well, tremendous amount of work that's gone into that policy research and work to try to identify what makes the most sense for san francisco. really, really great work by the whole san francisco team. great thank you both so much for that illuminating presentation. just a couple questions for me. can you just at the very outset, you said we don't look at crime data , we only look at in-house data. can you just clarify what exactly you mean by that? yeah so we look at officer behavior. we don't unlike a traditional case, a case management or a record manager system that captures crime data. right? our whole focus is a company and the work that we do is focused on officer attributes. so because of that, there's we're not capturing crime. what we do capture is activity, though, right? officer based activity. so arrest, use of force, vehicle pursuit, internal affairs, complaints that might be alleged against the officer. all of that

data is brought in, but we're not capturing otherwise criminal data inside the application in using that in the in the course of the evaluation right. okay. that makes sense. thank you. and then you mentioned the 81% efficacy rate. does that include both false positive and false negative error? no what that statistic speaks to specifically is the likelihood that says from the moment someone is alerted. right. so let's put it this way . if 100 officers were alerted, 81 within the next 12 months would have an investigation on for, i think, the definition we use here is a major adverse event. so would will there be some investigation that's launched? i i see. so i think maybe i'm not understanding this then. so you mean you tested it against historical data and you're saying if we you know, if we had known what we knew at the time, these officers would have been flagged before their event,

basically? that's correct. okay. and then but then so i guess my flip side. so that's not capturing, i guess, false positive error. right. but then do we have a sense for the number of officers who we wouldn't have caught, who nevertheless would have had a who then go on to be investigated? right so we've the way that we've optimized the model and this is a decision made in in partnership with the department is to optimize for making sure that we get the when we do flag someone, that we're flagging the correct person so that we gain buy in and trust with those officers. so we're maximizing or minimizing false positives, right. and so that's we're at that 5% is what we're seeing in terms of officers across the department overall. we're not optimizing for the false negatives. and so do we have a sense like have we quantified the type two error for this model? we don't have the specific type two error on that, but that's something we can provide. and just out of curiosity, that seems like something one would want to know. i'm just curious why that why that isn't something that

was i mean, i understand why you might want to focus on one versus the other, but i'm just curious, like why the assessment of the model's efficacy didn't involve assessment of type two error? it's included as part of our detailed review of what's actually happening inside the model. but our selection criteria for have we achieved sort of the goals, the model focus on the false positives. great. okay so you mentioned there were 79 variables and one question i just occurred to me is, will it be transparent to the person looking at the dashboard and why any particular officer was flagged? i mean, nowadays we read about ai, where the ai is doing a great job at whatever it's supposed to do, but you just don't know what happened behind the curtain. and so with this number of variables, i'm just curious, will it be reasonably clear why you were flagged? yeah if we could you know, we've got screenshots of the system that you can interact with right? would it fundamentally allows the streets supervisor to do and

it's really designed for a supervisor right is to take a look at an officer for a window of time and understand what what is the nature of arrests that are occurring, what is the outcome of those arrests? are they are they resulting in force now? force the system doesn't treat force negatively, right? the system is measuring what we would call as proportionality of force. so relative to how much resistance the officer encountered, how much force was ultimately used to overcome that from a pattern perspective, we never look at one individual event right? we're looking at a pattern of conduct. and so ultimately what we are what the system is optimizing for and what it's assessing for is to help supervisors understand what are the nature and types of arrests, right? what is the nature and type of force that that officer is using. what is the nature and type of allegations or investigations that might be occurring with that officer and then it localizes it for them. so, you know, we had the chance to

listen to some of the dialog today. so, for example, an officer who is working in the tenderloin, compare them to an officer that might be working in a community that is very low crime would be an unfair comparison, right? there's going to be fundamentally different levels or necessity of force utilization in. so it gives that supervisor the immediate peer group comparison across all these domains so they can say, okay, if i'm a midnight sergeant in the tenderloin, how is this officer relative to other officers who are performing the same job function in the same neighborhood, interacting with the same same members of the public? and so what we typically see is it's pretty obvious when you look at that dashboard and you look at these things to say, okay, this officer may have a problem with verbal engagement because it's the nature and type of allegations they get. this officer might have challenges with de escalation. so the system provides enough data and descriptive descriptions to do that. we as a company don't

include any analysis that can't be explained. so we are not a we are not an ai company where we just run, you know, random forest or any other, you know, machine learning algorithm and says, aha, someone's at risk if we can't explain it in a way that and we use focus groups of supervisors from across the country that an that a supervisor can't look at it and say ah i think i get the problem . so then it's we're not using those measures. and that's a big part of why those peer comparisons are in there. and that secondary screening part is to make sure that it's interpretable. all right. and on that, on that point of kind of normalizing or adjusting for other factors, what kind of work went into ensuring that kind of we understand which direction the causality is pointing into. so for example, you gave the example of normalizing for where which district station the officer is assigned to. but let's say, for example, in an

officers assigned to a specific unit and that unit experiences kind of higher use of force, how do you know that that should be kind of controlled for because that unit is just put in more situations where they encounter those types of situations owns or it's the opposite. it's that officers who are more likely to use force are disproportionately assigned to that unit. how does the model kind of understand what's causing what? so i mean, that's a great question, right? so there's as we think about the trade off between in the model itself, purely identifying officers who will be investigated in the next year versus selecting down to just those people who are different from their unit. so it's a it's a two part process intentionally for that. and it's not it does not entirely rule out the possibility that if everyone is high in the unit, that it's going to allow those officers to not be selected for. so that's definitely something we monitor for inside the system as part of our ongoing processing within

the data. but there are certain patterns that even in a unit that so when we said that there's, you know, a small percentage of a police force that we identify, i'm going to speak nationally right? that may have problematic conduct that that 5% is never even there in a unit, right? it's a commissioner is what you're describing. but but even within problematic units patterns appear that we identify. so for example are you do you have a large amount of on view. right. discretionary arrests associated with the arrest, resisting arrest where there's injury. okay that that pattern repeat id is almost always super problematic. and if you have a unit involved in that right that has a lot of that, then even though the unit might have a lot of utilization of force, it's still going to be identified in our system and to

be very clear about one thing, our system, we expend a lot of analytical time thinking about the fact that the system should never be alerting for high activity, right? so you can have an officer who's got a ton of arrests and a ton of other activity and never get flagged in our system. it is not a bad activity. is the outcome of that activity that the system is measuring against. right. that's helpful. last question for me. this is more of a straightforward question. as you described kind of this one year probationary period before you get provisionally kind of sorted. and part of that involves comparing your stats to historical stats. are those historical stats just sfpd, or is that relying on your other jurisdictional databases? it's purely the sfpd patterns and trends. the national database helps inform where the model starts, but everything about what's happening inside the model for sfpd is sfpd specific . great. thanks so much, commissioner benedicto. thank you so much. vice president

carter overstone thank you for that presentation. i know it's been rescheduled at least once. and so thank you for your flexibility on that. this seems like a program that has a lot of potential. i mean, we've been calling for a modernized early intervention system program. i've been calling for that since long before i was on the commission, since it was in the d.o.j. report, it was in the blue ribbon report. and so i'm glad to see that that we're moving forward on that. i want i think my question is for chief, when they mentioned that all that's left is, is the policy implementation stage, do you mean that there's a general order that needs to go to the commission or like what's what's the what are the next steps in terms of implementation? so we so we've put out we've already put out the first phase of training. we put out some videos to socialize this with the department as we work through. and i won't say troubleshoot, but whatever issues come up in

this, in this year, we will take that. and if we make it need to make adjustments to the training that we've already put out to anything that we've already put on paper. as far as the protocols that would be during that period. so not, not dgo, i think it's not necessarily a lot of this is operationally setting the right protocols based on whatever we find during this year. and if i could just add to that, just with vice president carter orbison's question, you know, there's always i mean, this is the obvious, but i'll state it because it has not been said. there's always a human element to the analysis. you know, these systems, as good as they are, only are effective when the reviewers understand what to do with the information. so like the unit you use. question it's a great question, by the way, but somebody has to dig deeper to determine whether these things that might be red flags are, in fact problematic

issues. the system is can only alert. and so part of this is incumbent upon the department to make sure our folks are trained. they understand on what to look for, what questions to ask, where to dig. when these alerts come up. and that is vital to the success, in my opinion, of this working the way it's supposed to. thank you for that. and so as i understand it, then there's nothing that is needed on the commission level to enable the continued implementation of the benchmark system, not at this point. i mean, there may be changes afoot as we dig into again, what this what this offers. i think this is going to open up a lot more ability for us to do what is designed to do. so that might come with requests for the commission to weigh in on if it touches any of our policies as it stands right now is isn't governed by there's not a general order on is correct. no there's not a specific order on.

yes. okay. so it's not like they would need to amend any general order to enable this to continue to continue to move forward. right. that's correct. but i'll double check that right now because i can look real quick. okay. yeah. and i know that i know commissioner yanez has has showed great leadership on this and pushing it forward. so i know there's great attention to the commission. so i know we we'd love to be kept abreast as things update or if there are any, if there are any roadblocks that we can help plow through. i think there's the interest on this commission to do that. you know, i'll note that you talked about kind of socializing this among the department. i think that is that is an important step. it reminds me a little bit of the what you saw in this department and others. you know, gosh, almost a decade ago when you saw the widespread implementation of body worn cameras where there was at first a lot of skepticism. but what you've seen now is unanimously to an officer, every officer i've spoken to is a huge fan of the value add that body worn cameras provide to their policing, the value that they

provide to accountability ity to making sure that they're able to do their job to the best of their ability. so i am hopeful that when we look at modern early intervention systems, as in the in the longer lens, a couple of years from now, it will look like what we saw with body worn cameras, where after officers were socialized and understood the value that they become an independent, an indispensable tool for policing. so i hope to see that. thank you. so, commissioner, i stand corrected. so 3.19 is the icu at this point, i don't. it has not been touched in since 2007. so i'm sure there will be things that come up that might need to be revised. and this is probably i don't know where it is on the schedule, but it's due for revision anyway. okay we should keep that maybe at the top of the commission's and i'll just mind, i'll just chime in that there is actually like a parallel process taking place. and now with is also participating in that. we just

didn't want to put a timeline because there had been some delays with implementation and with developing the actual platform so that is a work in progress. thank you, commissioner martinez. and again, thank you for your tremendous leadership on this issue. thank you. commissioner benedicto and so i do have a couple of questions there had been a request at some point from dpa to see if the if the system was capable of sharing information in real time with dpa when it came to certain alerts and general failures investigations, is that taking place as. no, not to our knowledge, but it's not a systems question. it's a policy question, correct. so i'm sorry , but i would say the way that our platform is designed is, is it's not meant to be a gotcha system, right? not that dpa is a

gotcha system, but it's not it's not meant to be a disciplinary tool. it's meant to be a preventative tool by its design. so it's all about incremental. so i'd say from a policy perspective, our position as a company and as a research organization is this particular tool is most effective at driving behavioral change. when it's developed from the ground up as supportive, non punitive way to help those officers who are struggling out there. got it. and the reason i ask the question not necessarily to criticize or try to contribute to changing your program model because it is a policy question, right. and for us, when we see such, you know, important technology, we finally reach our department. it is something that becomes useful across the board for our city to benefit from it. chief, would you have any issues with sharing that information as far as there are certain thresholds that lead to an actionable risk as far as the

languages here? right or an actionable, advisable risk alert ? and sometimes those initiate internal affairs investigations that that my understanding are some are not necessarily always as initiated with the knowledge of dpa or with dpa being involved for there to be a parallel investigation action, if in fact there was some type of, you know, dpa type of incident. but as far as there being you know, non compliance with a with a dgo or some type of behavior that's deemed, you know, in need of some type of action as far as disciplinary action is concerned. so if i understand your question, i that would be problematic if information if it's a

disciplinary complaint that dpa has and there's something an incident that also triggers an alert, dpa should have access to and will have access to the incident. but the information from the system is not meant to be used as disciplinary unless the investigation uncovers that there's discipline involved. so i don't know if i answered your question, but the answer is if there is something that triggers an alert that dpa is has the jurisdiction for, they would get that information. but the data from an alert being triggered just on the surface, i don't think that is that is what this system is, not what this system is designed to do. so the way i understand the system, once an alert is leading to what is the actual language here. a the action of alert. let's see. yeah, there's actionable alert

and advisable alert and minimal risk. minimal risk obviously doesn't issue anything actionable alert. is the system saying that there is a strong pattern that needs review? and i think the chief's point on the human has to look at it. we've got police experience, actionable, advisable suggests, hey, the patterns going in the wrong way. and so someone needs to look at it, but it doesn't necessarily rise to the level of actionable. but the system itself isn't using any new information. so any investigative information, such as an allegation or other data that's feeding it is already in the system. it's already there. right. there's not. it's just using existing data or existing outcomes of investigations. for example. but it's not it to help identify to say, hey, someone needs extra support or training or intervention. right. but it's not there's no investigation from our alert. right? it's not there's no independent there's

no independent investigation other than the identification of a pattern that says that officer may need some incremental support. and so when that alert reaches, the supervisor, i know that there's like a risk manager team that will send that alert to the supervisor, then that's up to that supervisor to determine whether that actionable alert will actually lead to a performance improvement plan or a disciplinary action. correct. in this. so let me speak about this in general. outside of the sf purview, because we don't speak policy for anyone. so i'll talk about it nationally and just to give you a sense, we just came off last week. we had a national summit talking about this very issue across the country. so and we're very lucky that we serve everyone from very different jurisdictions dallas, charlotte , nashville, new york city, nypd, baltimore and the doj.

d.o.j. just brought us on for baltimore's consent decree. so we've got a lot of experience in lots of jurisdictions around this particular work and there are two general models us that that have that have grown from the work. okay? the original model is that it would get pushed down to the immediate chain of command. let's say a sergeant and lieutenant in the immediate supervisors of the officer. okay what we've generally learned about that model is it generally failed and it generally failed because when it would go to the unit, i'm talking about nationally, there would be little to no action. it would it would get dismissed first, right? someone would look at it and say, oh, this is fine , and it would be very hard to get action to occur. the second model, which is a centralized risk management model, which is it's only centrally managed, right? and we just had a major city that did that and then undid it. the challenge was, is

that the folks who lived in the in the risk management bureau equivalent would say, hey, this officer on paper needs intervention. right. but i've never i've never met nick, so how am i supposed to evaluate nick? because i've never been on a call with him and i've never been with him. and so that model struggled. so the ideal model that we have and we have a model policy that we publish is a hybrid model, which it comes into a central risk bureau like organization, okay? they review it and then they partner with the local chain of command collectively to determine the right course of action and to help get that officer back on track. and it's that marriage of the of the involvement of the immediate supervisors on the street. but with the central risk management together is what really makes it powerful and where we see the most behavioral change to the positive. i hope that answers your question, commissioner. it gives you some it answers how your system

works, for sure. yes. and the question is back to the chief. if at that point that supervisor iser determines this pattern of behavior merits some form of discipline or activates an internal affairs investigation, then can that trigger be shared so that in real time we have a parallel investigation, which is what is charter amended to do? well i don't believe that is the case or should be the case. dpa investigates complaints from members of the community. so let's say let's say the pattern is excessive force and officer has, you know, for excessive force. but there's no complaint from an individual. and the excessive forces are all deemed to be within the san francisco police department's policy that that trigger would not out right

yield necessarily. it would that would not yield discipline. but let's say one of those actually was out of policy. if i hear you correctly, you're saying that information should be shared to absent some dpa complaint is the question correct? yeah, i don't think that's appropriate. dpa do . director henderson. well, let me say, would there be a benefit if dpa has a relevant investigation, they will get the discipline history of the officer or if they have an investigation that's connected to this. but the idea of just i mean that that's not the idea that every time internal affairs sustains a complaint that that information gets flagged by dpa . i mean, that's basically what this would be. i don't see the relevance and i don't see where that's appropriate. if it's a dpa investigation that's connected to this. and there is

a there is a connection where that information is relevant to their investigation, absolutely. but just on the surface of every time a case that or a situation where a pattern of conduct is being investigated and let's say for whatever the circumstances are, it turns into a disciplinary situation. there needs to be some connection, in my opinion, to an actual dpa investigation. director henderson. okay. so two points. one on the information that's being collected and analyzed and disseminated can be helpful in terms of our practices separate from aghacha and separate from discipline, which are still relevant but is absolutely relevant to some of our policy conversations, legislative conversations and the mediation practices that we have. but the bigger issue and the ultimate issue that i think is the key here is rather than notification from the information being disseminated to us in real time , and this is what the

conversations have been with benchmark. and i think with the department as well have been asking for a portal of access to the information for concur cases, not to create new jurisdiction, but where there's concurrent jurisdiction and open cases or from the dpa's open investigations, if they're shared information about concurrent cases that are going on from internal affairs and or from items or information that's been flagged from benchmark to have access to that information . and that's what we don't currently have access to. that's what i believe we're trying to work towards is does that make framed it so much better than i did? well, it sounds like we're saying either the same or similar things. i believe so. and i'll just add a national perspective. we don't have a single we work with 200 agencies in seven statewide contracts. and i just give that for context. and every jurisdiction obviously needs to determine its own outcome. but we don't have a

single one where this is disciplinary based or it leads to anything to do with discipline because it undermines the it undermines the buy in from officers and from supervisors who we need, who ultimately will make or break a system. right. they're either going to take action or they're not. and when we get them to take action, it's because they they feel it's safe to engage officers to say, hey, this is not right. i got to i got to help you be better at this difficult craft of policing or we need to support you in different ways that live outside of disciplinary track. right? that is a that is a supportive angle. and so just for context, every other jurisdiction that has done this or we've studied it or we've worked with has ended up in that framework work for the ais program that we provide. great. thank you for that. and i guess in there is one of the questions that i have for our current it department chief. this system, um, i think

relies on fidelity to a model that requires documenting discipline at and i feel, i know i sound like a broken record every time there's a report or a quarterly report provided where i see that there is one performance improvement plan for the whole department. but what are we going to do to, to shift the culture of our department to actually, when there is an advisable or actionable or, you know, risk recommendation or an alert for those supervisors to actually be put things in writing in a performance improvement plan that can then be revisited to ensure that what we are hoping for the prevention of an incident is actually taking place. well, things are are put in writing and i just read one today, you know,

performance improvement plan where it was everything was in writing, including whether that performance was was improving. so the things are put in writing. i think i believe what i'm hearing. your question is, are we taking the appropriate action when we have something that's actionable, i believe is your question. so the performance improvement plan is one way to get at that. if, let's say, a an alert is triggered, the review is done like it's supposed to, and the support is, hey, this officer needs to be on a performance improvement plan and they're overdriving. they hadn't gotten in an accident. they hadn't killed anybody. but we know they were overdriving because of whatever the alert they are not talking to people the right way. and maybe that's the reason that they're forced threshold is high. the force is may all be in policy, but the real reason is because the officer's not talking to people the right way. and, you know, that's causing

some of the issues. those are things that that i have seen in performance improvement plans. that's been documented. but the question i believe your question is how do we determine whether the performance is improving? right well, yeah. so one one of one of the things is when you stop seeing, let's say that the example i gave with force, you see a reduction in the use of force. you you, you have a reason to review body worn camera and see that the person is actually treating people with the dignity and respect that we expect him to. so there are measures out there and if those things aren't happening and the force continues, then we have to go to another measure. it may be reassignment, it may be. and that's happened before. so there are there are there are things, concrete steps that have been taken with performance improvement plan. so i just want to say that because this will

still be a part of at least the support that we can give officers when they trigger an alert. and if it's a discipline case that triggers the alert, that discipline case will be investigated and whatever is appropriate, it will happen there as well. but the officer still may need support, even if the discipline case is investigated and sustained. and there's whatever the appropriate penalty, it doesn't mean that the officer that that that performance improvement plan goes away. does the benchmark system or ongoing support provide guide development in those areas on how to, you know, create a successful performance improvement plan, how to, you know, gradually work towards improvement in certain areas? yeah, there's built into the action plan itself which there's a screenshot there in the in the document. it helps the supervisors and the risk team

first assess what the problem is , right? so what aspects of the data did you look at? and then it allows them to, it provides suggestions around mentoring, coaching, training, intervention, support so that they can select from as part of building that plan. what's your follow up timeline going to be? how often are you going to check in with the officer? and then there's the capability for after, for after, after action plan follow up. so for example, some agencies will schedule 30 day journal entries where you have to report back on what you've been doing relative to the plan. so right. the plan itself helps the supervisors construct the way of doing that, to be supportive to the officers. great thank you. uh uh, director henderson, thank you. i'll be brief because i made my main point, but i just wanted to respond in part to what commissioner yanez was saying. one of the issues that i think you would expressed concern over was what those results will be if they're not

shared phd from d.p.a. and again, i've raised this issue before. i think if the statistical reporting and data were commenced, right? so it's not just dpr even, for example, the things that i report on weekly, monthly, quarterly, annually, there is a commensurate report from internal affairs. you would see those things and be able to measure the efficiencies of bringing an agency like benchmark into the picture. and that was part of the original conversation justifying the budget to bring them here. and i think that that's a report that leverages having them actually speaks to some of the problems that we repeat frequently here about the lack of resources having benchmark at the table speaks to those lack of resources because the technology will be a tool if we use it effectively so that we have broader, more transparency and more efficiency in terms of what the data will be, because that's what we originally had spoken about was the benchmark was

going to be and serves as more than a representation of just early intervention systems, but was also a tool to be used and now paid for. that's both going to collect, analyze and disseminate information. so i just want to make sure that we're still talking about the same things and not a narrow framework of just the reaction on to the actionable alerts. there's a lot more information that's being shared to and through benchmark. and as long as we have the commitments to be expansive about it, that shared data and the collection of data, i, i think it's a positive thing. so it's and i know that you guys already have a benchmark or best practices because you're in other jurisdictions, both in terms of what what the capabilities are of benchmark and what you have done, not just with the departments, but with all oversight agencies and independent agencies, civilian

oversight agencies as well. and you've worked with jurisdictions that have those agencies. his. yes, in the past, yeah, absolutely. and there's a whole aggregate statistics piece to the platform as well, right? that there's i think there's two pieces to it. obviously, there's individual alerts, right. which is at the officer level. but then there's aggregate information that the system provides, which is pretty broad. and maybe what would be helpful when you guys are presenting in the future if there's a whole separate component, just to speak to the total of what the commission has jurisdiction on or over, if some of those presentation options speak to some of those shared data analysis or shared data capabilities with the civilian oversight agencies, that would probably be helpful and specific and then we wouldn't have this drill down to make sure that we're not forgetting anything that we've built. and paid. for all right. commissioner walker, thank you. thank you for this

presentation. i mean, i see this as an incredible personnel tool that is really going to take some time away from in human analysis and i think that i really understand and see the importance of there being a trust factor in what it is. it's really helping managers manage their staff and i mean, i think it's important to see aggregate numbers of how many triggers there were in one given time around certain things and, and compare and see if this is working. and again, i want to i'm really reluctant about sharing specific data because it does then sort of cross over the line into personnel issues and feeling punitive. and, you know, it just it's going to kind of defeat the purpose of having it.

so i really understand that it and i really i love this because i think it's really going to be helpful. so all right. thank you both so much for the presentation. thank you, commissioner yanez, for your leadership on this. sergeant, could we go to public comment? great thank you. thank you. members of the public that would like to make public comment regarding line item eight, the benchmark presentation, please approach the podium. there is no public comment line item nine discussion on collaborative reform initiative quarterly update second quarter 2023. discussion on. here we go. getting in here. i thought i was going into the white house. did you did you. good afternoon or good evening,

commissioners. vice president carter overstone chief director henderson. i'm catherine mcguire, executive director of the strategic management bureau . we weren't expecting to present tonight necessarily, but but we do have i mean, meaning our unit, but we do have deborah kirby here from jensen hughes, who is our our contracted consultant, that is doing the monitoring and providing monitoring on behalf of cal d.o.j, who is doing the oversight piece of our collaborative reform. so with that, i'll hand it over to her. good afternoon. as i pull up my powerpoint, my name is deborah kirby. i work for jensen hughes . so good evening. my name is deborah kirby. i work for jensen hughes. i've been asked to provide a brief overview of

where we are in the collaborative reform initiative and then also to be available for any questions or concerns we may have at this time of the process next slide, please. so in. in 2016, the us department of justice came into san francisco at the request of the then mayor and chief of police to look at issues within the within the city and within the department following some highly publicized officer-involved shootings, the last of which was mario woods, our team conducted that initial investigation in and we came forward with a series of recommendations, 272in all that were focused on five strategic reform initiatives. as you just heard a presentation here that identified that there are concerns that are consistent across policing agencies in the united states and the concerns and the objectives that we identified here were consistent right? so use of force bias in

policing, community oriented policing, how the organization holds itself to account as well as how it hires and retains personnel. all issues that i've heard discussed here tonight and i have heard throughout the years here in san francisco. next slide, please. so we published our final report under phase three in february of 2022. at that time, the department had accomplished almost 90% of the overall goals. the objectives of the recommendations at that time, our report in the final three identified what were some of the areas and some of the concerns going forward at that time, what had occurred was that the department negotiated through and with the california department of justice to go into what we call phase three plus. and the goal of this program or this phase of collaborative reform was intended to help transition the department from a traditional monitor oversight role of some outside agency to

one where they can self sustain and identify not only what are their organizational goals, but how do they want to go forward in the transformation. so consistent with that, what we wound up doing, if you go to the next slide, please, is that the focus here began in september of 2022. there are five project plans are aligned with each of the strategic recommendations with the exception of personnel, although that plays into one of the plans with the goal of coalescing the recommendations and the way that they were originally envisioned that they came in under strategic areas and they were necessary linked to help the organization build iterative practices that helped them to achieve the goal outcomes. and so under these five project plans, the department engaged and agreed with the california department of justice. and two, on a timeline that's rather aggressive that ends in april of 2024. our contract with this project extends to allow for final review and an assessment.

it's no surprise here, but we know that there are always end of game or end of project issues and resolutions between the various stakeholders. and so that time expansion will allow us to deliver a consistent report that reflect all of the input of the stakeholders. there's also a portion in this phase that allows for sustainable review. and so what that means is that of the recommendations that were awarded substantial compliance by the california department of justice, we will dip sample and review to a percentage of those. right now, it's about 10% and waited in terms of their priority and their prominence in terms of the reform occurring here in san francisco. so some of these were awarded early on the work on use of force was significant. the department was identified as having a leading policy in 2016. and since that time there's been additional work occurring. and so looking at that and saying, you know, is the organization capable of maintaining a level of transformation and accountability to what it's promised to do? further, what

we're looking at in next stage is that the independent monitoring review will assess not only the work done on the recommendations, but also the organizational structure as it can go forward in what we call to infinity and beyond. right. because as police issues are iterative, they they grow, they continue every day. and the question is, is there the organizational framework and capacity to address these issues in a way that is consistent, transparent, focused on community service and allows the department to demonstrate that it is following its policy and guidelines. next, one. so when we talk about the project plans , there are basically two addressing use of force, right? and in 2016, this department had a significant issue with data, even though there are complaints about where the data is today, it has significantly advanced since that time. and what you just heard was a presentation by an organization that's coming in

to help the department coalesce its disparate data systems into what ideally will be a managed management dashboard. and i'll talk about that in a minute. but it what the use of force aspect of this is, is really to be able to start to measure, analyze and report consistently on the force that officers are using. there's the sustainability aspect of that in terms of looking at what's going on around use of force and then also review of what the practices are and the policy practices of this organization. i need to note that the 2016, 2017, the department of chief's substantial compliance based on its policy 5.01. since that time, it's gone under subsequent review and revision of the department's own accord. and frankly, the disengagement policy that they've recently published is probably a nationally innovative policy and one that we don't see happening in other jurisdictions. so that's a positive outcome, right. the other thing that we've noticed and we were just

here last week on a site visit as part of the work under phase three plus was that you're seeing the training division, specifically the field tactics unit taking the recent officer involved shootings and looking at and breaking down what's happening relative to the video , the data, the evidence and identifying whether or not there are ways that training can help the department overcome a force situation or improve the outcomes of a force situation. so that's truly innovative and impressive work that the department continues to do around the use of force. next slide please project plans three and four really are the ones i think have the most interest and part what we're talking about under project three is advancing the community policing practices of the organization. and this thing called covid really put the end of phase three a little bit off of the rails. the department did a good job in terms of trying to maintain community outreach at a time when it was not necessarily feasible, but this phase is intended to allow the department

to identify and define what its structures are for community engagement, to demonstrate that they have the capacity and the willingness to do so, and that they can take forward the early work of the stakeholders and collaborative reform and push that into a process that will allow them, the stakeholders and the department to work together prior project plan four is probably the more interesting one and definitely intriguing in terms of what the department is trying to do in terms of identifying bias and being able to assess and address whether or not there are bias practices and what does that look like in san francisco? so you just heard the presentation of the vendor who will be working on that system and that is directly linked to some of the outcomes in terms of the recommendations. and so what we're seeing within the department is that not only are they looking at this concept of bias, but as you heard here today, really looking to develop a management dashboard that will allow the department to identify, assess what officers

are doing, but also make it transparent to officers in terms of their performance and what they're doing, and then allow for the opportunity for managers, leaders to have real time data to assess are there variations, are there issues, are there things that i should manage? because the concept of management within law enforcement is not one that is widely inbred. and law enforcement is really focused on discipline because generally officers arise through a system of collective bargaining and they understand that. so the ability to have this data and move it forward, i think really will put the department on a national best practice standard and then finally, the fifth project plan, which isn't addressed here, is really about the department looking at its internal internal affairs data and how again, how they look at discipline and what does discipline mean within the department and frankly, whether or not there's disparity in discipline practices. and so these five project plans together, you can go to the next slide. i really want is going to bring the department. i think,

into a wrap up of what collaborative reform is about. but also allow the department to demonstrate what its commitment is to the community forward. now that being said, you know, there are challenges and issues for every plan, right? so i've discussed the promising right that a management dashboard. but the one issue that i think is imperative to understand is that in 2016, when we came here, the department was really in a reactionary mode and wasn't necessarily inclined to look at practices, things that were identified as not being appropriate. what you see in this department today is one that is learning, focused and really at iterative improvement . and so you have commanders now, you have deputy chiefs, you have chiefs who were leads on the development of the project and the recommendations in the early days. and so that allows the department to operationalize it, challenges. all right. so resourcing so we're hearing discussions today about who's addressing narcotics and who's doing this. and some times the

actions of data collection, analysis, getting reports, dip sampling and audits are not seen with the same fervor that having police on the street are. and so in a self driven reform program, which is what san francisco has and is unique nationally to any agency that i'm aware of, resourcing continues to be a challenge because there's not a judge, there's not some lawsuit that's forcing behaviors. it completion timelines on it projects are always notorious for delays, but also that can the it plan deliver what is anticipated and is expected by the department? and the timeline is a challenge. we have until april of 2024 for a completion and there are pressures that will come up against that, part of which are resourcing and it but it will also continue to go. and then we continue to hear some discussions around policy development. it was an issue in the original report that we're

not deep enough in the woods right now, but the revisions of 3.01, the policies that we've seen promulgated have shown that the department is able to work within the system, but just to make sure that there aren't any backslides in terms of any of the policy applications, as we haven't seen it yet, but that will be part of this process. next slide, please. back one. thank you. so the one thing i do want to say and i've said it is that san francisco is a unique department in that it voluntarily, although somewhat less so. right. undertook a reform approach which when the us, doj exited this situation in 2017, the department in the city could have stepped away right? but they elected to continue to move forward with the reform initiatives. and so that is something to be commended. you've had california department of justice as the oversight partner for a period of time and you've also got organizational investment in this program. i've

talked to people recently, like i said last week, we work with the teams, we work the project plans, the goal and the focus of how to improve is present invisible. and so this is not going to be easy. there's 1300 officers out there. each one can make a decision at any time. and we're answering questions and issues regarding that officer. but if that structure is there, if that stakeholder engagement and the transparency remains as this department is well positioned to be successful in the future and so for me, that's a positive end to this presentation and frankly, to several years worth of work. so i'm here now to answer any questions you may have. thank you, miss kirby, for the presentation and director mcguire for being on hand. and a couple questions for me. we've recently discussed we discussed a little bit today, but at the last meeting we fully agendized the issue of the department's auditing of its stop data. and

we discovered that in essence, the department isn't auditing its stop data, not not any audit that would catch anything that would call into question the integrity of the data auditing of stop data is something that the cops report comes out and says it's very important. it's specifically recommends engaging . a third party like a university. and i think the words that you use are that kind of like the bare minimum is to get an independent outside partner that that's considered best practice. we're obviously not doing that. we're where do you see where we are now on auditing of stop data and do you have a. jensen you have a view on what steps we can be taking to the commission is going to be actively considering this. so i just curious if jensen hughes has a view on kind of what direction we might consider going in? yeah. so there's good and bad news here, right? the submission of data, the stop

data for how many years hasn't been basically audited by most agencies who are engaged in it. shocking to me. i come from a state where that was audited. so i think it is a problem throughout california. i also believe that with the work that's being done currently in terms of the management dashboard, the ability to link the various databases will help inform. but our original call investigation review assessment identified that there were significant issues and i don't know that there's been strength placed in developing those those standards. now, there's challenges here, too, right? so what type of organization comes in and how do they do that? and frankly, whether or not the department is in a position to be able to continue and maintain a level of audit. we've talked to stakeholders to this process is short answer. it continues to be a challenge. it's not one

that i see a near-term solution to, but it is one that is part of the closeout of this project. we've been having conversations. there have been some outside agency brought in to look at various aspects of the data, as you're aware, but as it relates specifically to stop data, i think that continues to be a challenge for the department and one that we'll be working with them. great. thank you for that . i think it was last week or maybe it was last meeting or maybe a couple meetings ago. i asked the chief if he would be amenable to putting on our website all bureau orders, bulletins, maybe some department notices, but but putting up all significant policies that are not department general order, general orders. i'm looking at my notes. i think at page 149 of the d.o.j. report that was called out as kind of, again, something that was basic best

practices, the chief's response was that i wasn't prepared to commit to doing that and cited staffing issues. i'm just curious if you know, jensen hughes has a view about the necessity for transparency around all all policies that govern how officers in the field carry out their duties right? so i think the issue that we saw first on was that department notices were being used to circumvent the due process. right? and so that's why it was called out initially because it was unique and frankly, probably counter mounted. the goal of the process established in the dgos when we get down to bureau orders, unit orders. so when you look at a large law enforcement organization, there are multiple orders and the ability to keep those aligned and consistent with the change in policy that you were just speaking about, a policy that hasn't been revised

since 1999, i came from the chicago police department. i'd go into units where i was commanding. i'd be like, when was the last time somebody touched this book? so i think there's two things here, right, is that the department has committed to transparency around its dgos that's consistent, right? but but to be able to build a process internally where they're able to maintain those standards becomes part of, i think, you know, transformation and beyond. is that what are those auditing? and when i say audit, i mean something different here because of los angeles. so i'll use paul's term of review. what are the review practices around the update of unit policy is and i think there's several things that you have to focus on initially because it is iterative. what are your risk issues? right? you know, are you do you have different use of force policies? i'd be looking at swat right. and what are your positions around stop what data are you maintaining that's pii and is there anything that's not consistent with the current organizational rubric? so the

long term answer here is, is that i don't know that it would be helpful to put every unit order division order up on the site. but i do think that the organization itself needs to be able to establish a rubric under which those policies are reviewed, are consistent with the practices and where they impact community concern. there should be transparency to them. great. thank you. yeah. you spoke primarily to two policy revision, and i do want to ask you about that as well. but just to this issue of transparency, is there is there any reason why we shouldn't, for example, post publicly every bureau order, or is that not what what the cops report directs us to do as well as a recommends that we do as a best practice? yeah and i can't answer what is contained in the unit orders now because. no, no, i'm sorry. just bureau orders. bureau orders. yeah i don't even know what's in the bureau orders, but to my understanding

, there shouldn't be anything that's inconsistent on, you know, there's always a concern about whether it directs a specific tactical action. but i don't think that would exist at the bureau level. but i can't speak to that. but conceivably, there should be transparency to the orders. great. thank you. on this larger policy issue that that you raised, it does jensen hughes have a view on kind of what best practice is are in terms of policy making? i think, you know, we've talked about this commission, the need to revisit 3.01, which sets out our policy revision practices. and i think many stakeholders perhaps for different reasons, also agree that revision of 3.01 would be appropriate for my perspective. we've had recently the issue that you've flagged the department using bureau orders improperly to circumvent the geo making process. we just they just issued two bureau orders that clearly should have been dgos. in one case, it

directly conflicted with an existing dgo and in the other case a geo revision was well under way and the department front ran that and just issued its own bureau order. we've had deputy chiefs blow owing deadlines under 3.01 and not even taking the time to request for an extension of time. so we just lose track of policies and where they are in the process and we've had some new very innovative interpretations of 3.01 from the department's policy unit claiming, for example, that that that geo affords them unlimited time. now to revise a geo despite setting out strict time limits. so that's those are some of the issues from my perspective. certainly the bureau order issue is spoken to directly many, many times in the cops report. i'm just wondering, in jensen's hughes practices or experience whether you have recommendations about how to make our policy

revision process work better for. yes. so you're raising issues that i'm hearing anecdotally. and so we really haven't had a chance to look at what is the current status of what's going around the policy review. i think that there's a couple of things in play here, right? is that policies are meant to guide the actions of police officers and sometimes those requirements require some sort of swift intervention or, you know, if something's deemed illegal, if something occurs that the department knows that it needs to switch it has to have the flexibility to do that. however, when it comes to those community facing concerns, what is the measure that's in place to hold the department to account to ensure that there is then a public view, approval and implement of those policy requirements? and so that truly san francisco is a good practice in a lot of ways in terms of its community involvement. i mean, that is something that is, you

know, continuing to be progressed across the country right now. but you also face a challenge in california that's a little bit unique. is that most agencies are lexipol, right? and so this concept of policy and how to build policy and how to actually manage policy remains a challenge for a lot of these agencies coming out of lexipol practices and in san francisco, you know, it is a major city, right? and so the size of the bureau and what are the specific needs of the bureau? how do those drive those policies? so it's this concept of community facing issues. and then, frankly , how do you get operationally what you need to get done and where do those two intersect? and in some regard, this idea of time, you know, it goes to both ways and how does that get done? and so to me, 3.01 was developed by this group with a goal of improving policy development. but it has some tweaking, it seems, because we hear comments

from various stakeholder ears regarding that. so it's best practice we're working on that. i can't tell you that there's one solution. san francisco is unique and we're trying to work for that solution. and one other thing on this on this issue of policy making, one thing about 3.01 is it gives a lot. you know, department leadership get a lot of input throughout the process. but there's no built in requirement that the commission consult and solicit actively the thoughts and feedback of line level officers who will actually be charged with implementing the policy. do you see other jurisdictions as engaging in involving officers more soliciting feedback and when revising policy? it depends upon the policy right. so and it also depends upon the agency. so there's a lot of policies where they have the smees, which are the officers actually doing the work in some circumstances that review and inform the policy. and that's a practice like a policy working group that has

various levels of officers working on a policy. there's also ways that, you know, i have agencies that are using the system similar to the department and how they send out orders that they send out notice to officers to make comment, and that those comments then are received and evaluated by the policy unit. and so there's ways to expand the visibility and the input of officers to the policy process. and other agencies are doing that. great. last question for me. you you aptly described a period of time that you referred to as infinity and beyond. and i'm just wondering what what does that look like concretely? once jensen hughes moves on to its other, more important work in cal d.o.j. that that mou expires who is or maybe it's just the folks up here but but what do you envision as the process for ensuring that we maintain

compliance with what we've already reached that we are meeting compliance for the outstanding recommendations and that we don't backslide on recommendations that we that we have completed? yeah. so there's one thing that i just really want to put out here right now is that this is a significantly different department and frankly process than what we saw in 2016. and so it's been iterative and sometimes when you're sitting in the chairs, you don't see that. but coming back from where we were in 2016 to where we are today, i think that there's a lot of good here in this organization. the city and in its stakeholders that should be recognized. i think that, you know, and beyond really addresses the issue that you're raising. can these stakeholders work collaboratively to address the real issues and concerns? because you may disagree, but you all believe in the process. you believe in this department, you believe in this city. and so each of you need to work in that. i am by no means pollyanna, but i do believe that

the ability to create an operational framework that looks at one, you know, what's going on with the stakeholders, what do we need to do? and three, how do we improve? it is key to ongoing success. there's always going to be some issue that's the nature of policing. it's the nature, nature of where we are in terms of crime, right? but the ability to apply the principles of for the community by the community and are we doing the thing that we want to do under our own goals, i think are critical to continued success. and that's why it's important to recognize that this is more or less self driven. this this city, this department asked for this. and this is where it is. and so when i say when dad's out of the room, how this city and how this department goes forward, i think the framework is here now. and it's important because a lot of officers, a lot of lower level supervisors have been exposed to

this program. they have grown up in, you know, in the department owning it. and they're now at places to make this continue to go on. now, there's always going to be challenges, but it's how you address it. and i think that that framework has been instilled in this organization and now we're working on it under three plus to make sure that it's fully there. but there's hope there. great. thank you for that. commission benedicto thank you, mr. vice president. and thank you, miss kirby, for that presentation. and for the continued commitment out of johnson hughes to this work. i wanted to ask most of my questions are focused around the sustainability review for already completed recommendations. the dip sample would this process allow as part of that review? if there were if there were backslide observed, would that include johnson hughes changing the designation from substantially incompletion to something else? or would it would it just like what would that look like if you if in one

of the recommendations you saw some backsliding the ag has final authority on saying whether or not this department has been seen in substantial compliance. and so the award was at a point in time. right. and so now the goal is to say is the is the department maintaining those goals that were identified and awarded at that time. if they're not, there'll be a conversation between the department, the cal d.o.j. and frankly, a framework of discussion as to what that looks like, whether or not it might be a mediation plan or some aspect of that regard, and don't necessarily walk back the initial award, but we can identify what that framework is now and where the department may be deficient and then identify what would be the expected outcomes as a result of that. that makes sense. so since the award is a snapshot in time, it wouldn't change that award, but the ag could not note that a snapshot in time now would no longer find that in substantial

compliance. right. what you're saying? right. so if, for example, you like take use of force, it would be a totally unlike the hypothetical. but if this commission suddenly decided we wanted to unanimously revert to the 1995 version and, you know, obviously the award was because that version was was superseded that the doj would be able to note that that in in the report that comes at the end of phase three plus. yeah and it also and you just raised an issue there too is if there was a significant deviation from what was deemed as a successful outcome. you know, if you look at the overarching five areas, if the department completely walked away from its use of force goals, i would anticipate that there would be more than a discussion around mediation. i would as well. okay, wonderful. i have so the dip sample will be about 10% of the recommendations. so that's about . is that right? you said, yeah, we're looking at about 35, right now. that's about what we'll be looking at. have the 35 specific recommendations that'll be part

of the dip sample been identified by jensen hughes, or is that still not at this time? we last week. that's why we were here. we're looking at some weighting, obviously certain areas will be of more interest than others in terms of so. so you've got a couple, you've got some distinct issues here. you have several that were done very early on. all the work around use of force. and then you had a, you know, several recommendations that came in at the end of phase three. so there'll be a little bit of examination on that. and then the rest of it will be distributed through the other strategic areas. so is it the case then that it will be weighted based on that and then randomly selected within that or. yes. so it's not the case that they'll be sort of hand selected? no, no, no, no, no. there'll be yeah, there'll be some weighting and then random selection. a couple might be hand selected based on their prominence or what we're seeing emerge. but for the most part it really is weighted to what, you know, what the issue is and how it's perceived in terms of its

overall compliance. and for example, a lot of the work on bias is being done now. so we wouldn't weight that as heavily because there's current work happening that will inform the outcome. understood is it the case and not being like, would it be possible know we talk a lot about a lot of the recommendations here. if you know between the commission, between and the department, if there were a discrete number that we would want hand selected in the dip sample, would that be something that jensen hughes would be in a position to accommodate concerns about specific recommendations? i would ask that you'd forward them to me, i mean, we're we're looking we may we may have already selected those. and so we could we could adjust that. but clearly, if there are concerns that are being identified and by all means forward them. okay. i wonder i wonder if that'd be something that we could discuss as a commission along with the chief and director. henderson because i think that and just one clear thing i have to approve any plan with d.o.j. but it is helpful to

recognize if there's areas in particular. and so as we're selecting recommendations to review, it would be helpful. i mean, i think one example just off the top of my head is i know we've had extensive discussion about the about the stop data audit recommendation in this commission. and so but i think it might be helpful to help to provide that input for cal d.o.j. consideration. and on that issue, i mean, it is a problem across california. of course, a lot of agencies were delivering data without really having that firsthand knowledge of what was being delivered. so it's something that's being worked out and there may be solutions forthcoming from other jurisdictions as well. perfect. thank you so much. and i do want to acknowledge something that you had said. and, you know, for members of the public that are watching that are less well versed on the broad picture here, you know, you had this this 2016 report that was it's been it's been nothing but years of upheaval with the department of justice pulling out with covid, delaying a lot of the

community input stages. but you have had i know even your entity has changed names but have had jensen hughes consistently working throughout this process. and to have sfpd in the strategic management bureau, working consistently on that process, it is a lot of incremental changes that have added up to a lot. and i don't want to understate that when we talk about the progress that we all agree still needs to be made. so thank you, miss kirby, and thank you, director mcguire . commissioner walker, this is really been helpful. so thank you for being here with the report. um, certainly there's always something institutions can do to get better, but it does feel to me in this report that we have a we should be proud of what we're doing with with all of our partners out in the community. um, i, i think that, i mean, i know it's hard to compare, but it does feel to me like a lot of the models that

we're engaged in are our models across the country. so i just want to thank the chief, thank all of our command staff for engaging in this and the commission's pressure, which happens a lot. um, it's you know, it really i think our city should be proud of our police department and a lot of it is because we're engaged in this partnership. so thank you very much. early on, we identified that the san francisco police officers were all well informed. many are highly educated and really had a good natural way with the community. and so we continue to see that even today when i was trying to get in here. so that was helpful. great thank you again. thanks so much, miss kirby. appreciate you staying late with us. thank you . oh, oh, we had a last minute addition. i'm sorry. yeah thank you very much there. vice president carter. so, again, i want to thank you also for all

the hard work we're putting this through. so i know this has been a long process and thank the chief, the commission and all the members that went through it with us. my question is, the time how much time was, you know , for a staff of 1500 or how many officers we had? what was the time spent on the training for these or to bring to bring the members up to i guess, to the policies and to meet d.o.j. requests and then the second one behind it, i guess it's an annual or continual process. and what it is that in man hours in a yearly base this my thoughts. yeah so what i can tell you is that our job was in the weeds, so to speak, part of the success of this program is that the chief did develop a standalone bureau that helped guide and

direct that right. and so when you're talking about training, i would argue that it continues, it's ongoing and it's been significant, right? every time there's a switch in use of force to meet legal requirements or to meet policy goals, that invokes training the department is also mandated to deliver training through the post out training, as they call it. and so the i guess the good thing for this department is that it's been committed to training. one of the success outcomes. i think, in terms of use of force is this recognition of the connectivity to persons and mental health crisis and so early on, the department invested in training officers on that coming out. also trained all of its supervisors. and so you had an informed response when you're dealing with somebody in mental health crisis. but that also included changing and dispatch codes. so this, you know, for lack of a better term, wraparound view of what an issue is and what can be done to help improve that has been, i think,

a significant investment by the department in terms of training , i would argue that you're seeing the value of that investment and the decreased use of force and frankly decrease the severity of use of force. okay i guess what i'm looking at is, is that additional resource that is needed in the police department that adds to the like the command staff to have overlooked the policies and the changes? well, i you're asking my opinion, so i'll give you my opinion. i think it's critical. so the challenge that we see with law enforcement is that there was a disinvestment nationally in training. and frankly, here with like, you know, canned policies that organizations fell away from identifying goals and how they wanted officers to behave. and so what you're seeing nationally as a result of certain issues arising out of the george floyd and breonna taylor is a

reinvestment in what we want our officers to do. and that includes policy. it includes training. it includes field supervision. but it's not organic. there has to be an engine that drives that. and so to be able to do reviews, audits , dip samples, make sure that people are doing what they say they're going to do, sometimes is at that field level. and you heard benchmark talk about that is that units are focused on what they're dealing with on a day to day. the structure that allows people to visualize, look across the organization and say, does this need some help? is it still spinning, has to rest with somebody for success in terms of what we want in our police and what our communities ask of police and that's true here in san francisco. thank you very much, commissioner benedict. i thank you for recognizing me. mr. president. i did have one more question about the dip sample, which is i know when these are done in other contexts, depending on what you

find, it could potentially be expanded, right? if you see irregularities in, you know, when you do these sort of dip samples, is that something that is provisioned for in the way johnson hughes envisions this? again, i'm hopeful that it wouldn't be. but if you were to do a dip sample and you said, oh, of these 3517, we felt like there were deviations. we have this mediation plan. would that merit an expansion or an additional inquiry? so at this point, the department, through the bureau of strategic services, is, as i just butchered that right, is currently overseeing the implement portion of the recommendations that have always already been substantially compliant. and so they've got a process by which they're measuring the adherence to the requirements. and so it's our anticipation that we're not going to see significant gaps. and what the department is doing, you know, might there be something where we're saying, okay, this is not trailing? might it be, you know, there's possibilities everywhere, right? because there are 272

recommendations. but i don't anticipate a significant gap. the process itself doesn't address if there is a significant backslide and what would occur. but i can anticipate that there will be consequence to any serious backslide. but i honestly at this point, we don't anticipate that the recommend actions were necessary interconnected. there were a number of them, but they were iterative and task based. and so based on what we're seeing in terms of the ongoing compliance review, we anticipate not to see significant issues. but if something unexpected occurred, the ag would potentially we would anticipate a significant response. yes. okay that's all. thank you very much for recognizing me again. um, all right. seeing no names in the queue. thank you again. uh, and sergeant, could we go to public comment? thank you. members of the public. they'd like to make public comment regarding line item nine cry initiative. please approach the podium. and there is no public

comment. line item ten presentation and discussion of the disciplinary review board findings and recommendations. first quarter 2023 discussion. we'll go you. good evening, commissioner. chief scott. director henderson, members of the public. we are here to present on the recommendations and discussions that were had at the disciplinary review board. i'd as a way of background, this

is a board that meets quarterly to address aggregate trends and sustained cases of policy failures and training failures to try to more nimbly address any issues in real time as they're coming up. so all right. so aggregate trends identified by iad in q one of 2023. so so, so my apologies to the commission. i was presented with the quarter two for 2023. so that's what i was prepared for. and then he said quarter one. so i'm going to slide right into this. so for quarter one, we had conduct unbecoming an officer or member neglect of duty, specifically body worn camera and improper search. dps trends for the same time frame also included neglect of duty, specifically body worn camera violations in proper search or seizure, inappropriate comments or behavior and failure to properly investigate. one of the

things i want to highlight, and i believe we talked about it the last time we presented, is there is also an ongoing discussion about what what actually a trend means. and i like to remind this group when we talk from disciplinary review board, it's not statistically significant. it's more of identifying trends as they're happening. so we might say that the trend is search or seizure. that doesn't mean we've had an overwhelming number of search and seizure cases. it's just that at that moment in time, those are the most prominent cases on our caseload. and if i can just add specifically, we're looking at information that supports a timely identification of any issues that would warrant updated guidance that would give us change effectiveness or a corrective action at the same time. so that's that's the core of it. not so many how many, but what does that lead us to in the information in an. and so from there, we'll go into policy failure findings identified by

iad. okay so iad had three cases closed in first quarter as a result of policy failure. one case has been presented to the third to the third quarter, 2022 drb meeting and is regarding an incident report not being assigned to the appropriate investigation unit. the second case was presented in the fourth quarter to drb meeting and regarding a case where an a psa was unclear if they were able to take a specific type of incident report at a district station, both cases resulted in a recommendation to update the relevant department policy. the third case was regarding an ois. the case led to several policy recommendations and changes, including updates to dgo 5.018.02 and the creation of the f.t.f.o unit and the vr training course. dpr had two policy failure findings. the first involved plainclothes officers

not wearing body worn camera at an incident as at the time that it happened, they were exempt from wearing body worn cameras. the policy regarding plainclothes officers wearing body worn cameras is currently being updated to address this issue. so it's nice when we kind of see that the cases we're seeing align with the policies that were developing. so that's an example of why drb is working to do what it set out to do. and the second case where we found a policy failure involved an incident where a person in custody was held at the district station for an extended period of time due to the county jail not accepting custody. this is something that we have seen. the department's aware of it. we get a lot of complaints about it. and so it's really beyond sfpd's control. it's really a sheriff's issue. but i know everybody is working on that, and that is something that we continue to discuss at these meetings. so we have training failures identified by both dpr and

excuse me and iad in quarter one of 2023. there were no iad cases in the first quarter that resulted in training failure findings. dpr had one case that resulted in a training failure, as well as a policy failure finding officers froze a location pending the issuance of a search warrant but still entered the location. dpr found the department's training does not provide adequate guidance as to what freezing a location allows for as well as what constitutes exigent circumstances to allow a warrantless search of a location . and again, that speaks to training. that's pretty specific, right? so there's general fourth amendment training that's happening. and because the law has changed so quickly, we there are very specific instances where officers need more guidance. and so this is an example of us having those conversations more quickly than what would happen before, where we would finish our whole investigation and then issue the finding wrong as opposed to being able to talk

about it more nimbly. okay so out of this particular drb, there was one recommendation and i am happy to report on what that was. that the department needs to update training regarding fourth amendment specifically around this issue of what actions can or can't be taken when freezing a location to get a warrant. the updated training should also include more guidance on what defines exigent circumstances and what allows officers to make warrantless entry into a premise while they are trying to get the warrant. that concludes our presentation on. unless you want to hear a chord or two, i'm ready for that. one great. thank you both for the presentation. i, for the first time tonight, i think i see no names in the queue. you're welcome. oh and i was about to throw my pencil at

it. right. okay. thank you very much. okay. thank you so much, sergeant. members of the public that would like to make public comment regarding line item ten, a presentation, please approach the podium. and seeing no public comment line item 11 discussion and possible action to adopt revised department general order 3.05 department weapon return panel discussion and possible action. so we have so we have our subject matter expert here for that one. and let me grab my presentation. sorry. 1994 was the last revision of this. one

i have. yeah. okay. so commissioner, i'm going to just be very brief on this. there there weren't a whole lot of changes on this particular policy. there were some some tweaks to basically update the policy to our current structure for in terms of the members that sit on the panel. when you look through the red line version, which is attached to your agenda, most of the changes is were wordings and. not a whole lot of substantive changes. so if you have any questions, please, i can answer them. i was

involved in this conversation even though we don't have our deputy chief here, but i can answer any questions that you have, chief. i see no names in the queue. and i just want to say the way you jumped into action, there was truly, truly laudable. and remarkable. thank you for the presentation. um, i . i will. i will make a motion to adopt the. dgo second from members of the public who would like to make public comment regarding line item 1103 .05. please approach the podium. and there is no public comment on the motion. commissioner walker, how do you vote? yeah, commissioner walker is yes. commissioner benedicto. yes. mr. benedicto is. yes. commissioner yanez. yes. commissioner yanez is yes. commissioner yee yes. commissioner yee is yes. and vice president carter overstone yes. vice president carter overstone is yes. you have five yeses. line item 12 discussion and possible action to approve revised department general order 8.04 critical incidents response

team for the department to use in meeting and conferring with the affected bargaining units as required by law. discussion and possible action. thank you, commissioner. we do have sergeant art howard here, who is our subject matter expert on this one. again, it's a fairly short two page ego and i'll just open it up for questions. i mean, i think a lot of the work on the critical incident response team on behalf of behavioral science unit is really laudable work. again, not a whole lot of substance substantive changes, but we do think this policy is adequate in in line with what this department is trying to do with our critical incident response team. so if you have any questions, i or sergeant howard or both of us can answer. director henderson. all right. so i don't want to take away from any of the substantive work that was here, and i'm excited. this is i'm happy this is moving forward, this was identified,

though, as a joint response. and that there's some stuff going on back and forth as to whether or not what qualifies as joint. i don't want to go into the weeds with all of that. but i do want to say that this is an ongoing issue in terms of working back and forth with the department that both myself and my staff is working with the department on. i've had conversations with the chief. i know my chief of staff has had conversations with the assistant chief back there because we're really we're both concerned about preserving our resources in terms of how we move forward and maintaining this process to make sure that it's something that's efficient and timely and i'm flagging it now. how about dpa being in the role of maintaining challenging , preserving deadlines and

accountability within the process that is, i think is moving towards a new a different , more collaborative process than what we've been fighting back and forth with. so i just wanted to flag those again. i've already had conversations with the chief and we're in conversation about what some of those changes might look like, but i just wanted to flag it with this because this was one of the issues that came up without going into too long or belaboring the point. that's it. great thank you, director henderson. and just to clarify what what exactly that means, that means that the joint responses are listed. those explanation options in the grid listed as joint responses to public comments. you're saying some of those do not actually reflect deepa's response to the public comment. i'm just going

to say yes because i don't want to go into the whole detail of it. but yes, correct. okay. well, we might have to subpoena you to get the full answer. so that'll be for next week before the subpoena comes. i'm hopeful that we have of a broader solution that doesn't that that doesn't necessitate a subpoena. subpoena. okay. we'll we'll we'll we'll keep the ink dry for now on the subpoena. okay. seeing no one else in the queue , i will make a motion to adopt and send to meet and confer, subject to our resolution, the number of which i'm forgetting that commissioner benedicto drafted, which directs the department to negotiate only over those matters that are subject to meet and confer. i think it's resolution 2330, but i could be wrong. i'm sure our staff will correct it. second, for members of the public that would like to make public comment regarding line item 12, please approach the podium.

there is no public comment on the motion. commissioner walker, how do you vote? yes mr. walker is yes. commissioner benedicto yes. mr. benedicto is yes. commissioner yanez. yes. commissioner yanez is yes. commissioner yee yes. commissioner yee is yes. vice president carter was still. yes vice president stone is. yes. you have five yeses. line item 13. excuse me, line item 13 public comment on all matters pertaining to item 15 below closed session, including public comment on item 14, vote whether to hold item 15in closed session. if you would like to make public comment, please approach the podium. there is no public comment. line item 14 vote on whether to hold item 15in closed session. san francisco administrative code section 67.10. action moved to go into closed session. second all right. on the motion, commissioner walker, how do you vote? yes commissioner walker is yes. commissioner benedicto. yes. mr. benedicto is. yes. commissioner yanez. yes. mr. yanez is. yes. commissioner yee yes. mr. yee is. yes vice president carter ulverstone. yes

vice president stone is. yes. you have five yeses and we are going into closed session 16 vote to elect whether to disclose any or all discussion on item 15 held in closed session. san francisco administrative code section 67.12. a action motion to not disclose closed session with the exception of factual updates in item 15 c that will be discussed closed in the minutes. second, for members of the public, they would like to make public comment regarding line item 16, please approach the podium. there is no public comment on the motion. commissioner walker, how do you vote? yes, commissioner walker is yes. commissioner benedicto yes. mr. benedicto is yes. commissioner yanez. yes. commissioner yanez is yes. commissioner yee yes. commissioner yee is yes. and vice president carter oberstar. yes vice president stone is. yes. you have five yeses. line item 17 adjournment.

>> bring up person that [laughter]. for me it was we had neighbors growing up that were fold my dad he is raising me wrong for having me pursue the things that are not traditionally female roles. and i think the biggest barrier to anyone in general is when you have cultural norms that make you feel like you can't do

something that make you doubt yourself and make you feel you should not be there i don't belong. those other big efbarriers i think that is the thing to focus on the most is belong everyone should belong here. [music] >> wishing we trained women grow in production. and recording arts and so we have everything from girls night classes for middle and high school girls. we have certification academy program. that would be women and gender [inaudible] adid you tell us. progress in the internship frm program where they are working in the studios. they are helping to mentor the

youth in the youth programs and the job place am component. most of the time we hire interns instructors in our programs and engineer in our studios here. we have conferences we do all overnight country and we have concerts that we feature bay area women and gender artists. [music] [music] >> an education forward organization. and so advocacy organization. dedicated to closing the gender gap and the audio and production industries.

>> started out of the lead answer, why is there a critical gender gap in this industry that started at city college. why are there so few in this

class i was ashamed i did not have the answer being a feminist. why have i never thought of this i have been in the industry for decades and why have i accept today of all people. it was out of that and unraffling it. actually started the infernship last fall and just fell in love with all the things about women's oshg mission because we are diverse and so many aspects of audio i did not know and i feel like eyes opened up and i gained a lot of confidence in myself and other fells and queer people in the industry i felt there was more connection and community. ironically my time in the industry is all pretty good. i think what happened is i was raised by a father who is an engineer. i was comfortable being strounlded by men all the time in his lab i was used to technology. when i got in industry my mentors were men and i saw i had a unique importance that got mow in the place i could be fluent and navigate something difficult and it was the norm for me. what if it was not woman was createed provide it for everybody. have this environment you are surrounded by technology and people that are going to support you and get you in this industry in a good way. i have been interested in audio i was never trained in music took piano when i was a kid. i never pursued it because not a

lot of women doing that. and my family is not musically inclined. when i want to davis the first time i took a music class there were few females in the class. like a rodey for my dayed was load you will the mixers and monitors and the giant speakers and gigs and help run out the cables and take things down and set up mics i did all of that growing up and never occurred to mow that that was a field they could at all. and then one i could pursue i didn't nobody else was doing temperature my dad and then i go with him to studios and see -- the men in the studio. dj for 5 years now and comments you get like wow you are a girl

dj that is crazy. that is wild. and i have great moments where it does not happen. and they treat me like easy. telling mow what to do they correct mow in ways that make me feel less i sprjs the opposite and i notice hand's on like you don't know what you are doing rather than asking me. not consistent times it happens. it is like when i talk to other females they are like say the same things it is like funny i know that nice men don't experience tht main thing triggers me when i experience different treatment and that happens a lot in the audio world. industry is changing slowly. there is still that issue making the places that are places belonging for everybody. i don't think so.

having a studio where it is not all run by white men like most studios. the studios are only in the word built and run by women. it has been super normalize thered are opportunity for girls and nonbinary people. you go in school and middle and high schoolers know that this is a field. this is a thing there are many jobs you can have in this field. some producing pod casts to setting up live shows. there are so many things you can do >> wee go in and teach the audio skills and give them equipment. i pads and then teach them how to make music and they get to come in here and will getting the tools to people who don't have t. that is really important to me. that's why i was like wow. i want to be there for other fell and queer people who don't

have the opportunity and also to be a mentor for them to really push them to experiment and not going to break it. does not matter if it sounds bad that is the point to try it. i think it is the goal to see confidence what they are doing and passionate and asking for hymn and excite body learning and excited about making music and it changed my life to realize i'm callented in the field i can make music without being trained to it it is amazing to be able to be part of that process and -- ushering women to the field. we can entirely transform how -- the technology part of what you hear every day. we can put xhg something in women's points of view in this every time. it affects the store and he

messaging. think our best example is how we transformed an entire city. place that major artists on tour one of the men looks likeip don't get it there are woman every where i go and the person was like you are in san francisco. you like oh , you are right it is here. most venues have graduates we are grateful to the city for that reason because than i supported us at the beginning. following your curiosity and interest and don't let anybody get in the way what is presented to you, go for t. no matter what! we are here for a reason. find what it is. don't let somebody else tell you what it is. you are the oldsmobile one that have been can know when you are

supposed to do. go do it. television

(background noise.) >> (music).

>> back in 1981 while the navy base was a homecoming and celebrate and she started free foods in san francisco and i'm originally born initially mother was in the military i have been in the navy for years a planning commission over in alabama recently and brought over to us and the main ships visually part of this the ships theoretically

the coolest part opening- through shallow waters. and at a decent time and the main trip when i was in the military and a street name down to the 345i7b and people are very nice and ship will be named after the city they supported us. i think the main thing just for the- ships nice to have those but outside of (unintelligible) navy completely most of stuff on the inside is very, very- fast. >> the first one the blue angles represents our active

refers or reserves to integrate with the civilian population and remind us where everyone is and the aircraft carrier around the world have no idea they were created in 1946 and actually do that outside of that we- have three 5 show sides we have performers and ironically i grew up two blocks south of here i have contact from iraq and my whole family was gathered an hour after serving the country one of the first memories i had. and a lot of opportunities as far as education and traveling some of the honors it is inspiring and really no difference between us and the jets the difference we were

represent of the marine corp i feel great for my brooks to go around the world to fly this aircraft. >> kind of a push to revise so in 2 thousand 10 in san francisco meaning the stipulates and planes but he did leadership in the navy and marine corp, you know, what the marine corp. has a massive disastrously response and what we should do take the patrons and partner up with the governors and started a whole suppressor preamble civilians have response training and education and actually activated and work with the a organizations. >> the part of this for a

decade we have been partnering with local governments basically as long as we've been around. how to mitigate those for earthquake and we participate in exercises throughout the training i guess our plans- one of my main functions to let the departments know we are the neighbors and we have been here to help you. and make a friend we want to make sure people know what we are and where we are and the needs what you are capable 6 and so we can help to fill that. >> (music).

>> my hope that people stack shake hands and welcome the men and women no uniform we where their from i think how the cream of crop is involved with the u.s. carriers and how brilliant the men and women are. and how to help other people with the generosity and the unbuilding help that is important. >> (music).

>> united

>> good afternoon and welcome to the october third, 2023 meeting of the san francisco board of supervisors. madam clerk, call the roll. >> thank you. mr. president. >> supervisor chan. >> present. >> supervisor dorsey. >> present. >> supervisor engardio. >> present. >> supervisor mandelman. >> present. >> supervisor melgar. >> present. >> supervisor supervisor peskin. >> present. >> supervisor preston. >> present. >> supervisor ronen. >> present. >> supervisor safai. >> present. >> supervisor