and will do say a panel interview and a rather informal interview which is different from say an oral examination or they may do a written exercise or performs exercises it's it's rather flexible at that point and the department does have some discretion on the way that it makes those selections and then finally there is the on boarding process and that's the process by which a department was finally kind of made its selections of who who it thinks is the best match for a position will go through a check of references background of work all you know conviction histories which which is something that the department of human resources requires. and then the final point in processing and signing of paperwork at that point. yes. in the work of the the panel interview and taking the list it sounds like there's

it's not like you have to take the first person on the list in ranking you you have a criteria who develops that criteria so let me take a step back and there. with each eligible list there is what is called a certification rule attached to that eligible list in many cases the default certification role is the rule of three scores and that really does address your issue of do you have to take them from the top down at that point where the certification will says it's the rule of three scores if there's one position you do need to select from the first three scores on the eligible list people may have typed scores which means that you actually may get to interview more than three people but basically you get to take a look at the three scores as as the number of positions increases that you're filling and the referral is issued to more positions you actually get to reach more

people or more scores on the eligible list i'm not sure i answered your second part of it isn't just that you you interview it and within those three you can determine who is the best fit for the criteria you develop that's correct so long as songs whatever criteria you have for making that is fit the criteria it's fine yes. commissioner's only questions were very public post wondering if there was any provisions for examination and. scoring of. candidates who are bilingual or for the actual language capability so i can adjust that with regard to kind of the position part of it my colleague john cross can address that better with regard to the actual examination part of it but when it comes to the position itself if there is a bi lingual requirement so for instance if there is interaction with the public or

sector of the public that actually does warrant a bilingual speaker speaker in that position then what we do is we actually tag the position itself and the requisition with what we call special condition and whoever fills that position must meet that special condition now you've kind of given me the the easiest special condition to explain with regard to the what we call the list because as as i mentioned before if you have the rule of three scores and so you've got say three people on the list who you're entitled to interview if none of them meet this bilingual requirement you can go down the list until you actually have and can consider three people who do meet that bilingual requirement there are some pretty strict standards for actually meeting the that requirement and i think

john cross can actually answer some of those questions of a bit better than i can. thank you for the presentation it sounds like a very fair process but i have a question on the length of time it takes to process all this ok so i think what i'm going to do is defer that to later in the presentation after you've you've heard the presentation on the examination portion from from my colleagues could i ask a question so. we talk about this list that you hand over to whatever department yes do you keep you know there's a there's probably you know x. number of positions that we have in the city that you're familiar with do you proactively make sure your lists are active we i mean i'm i'm just saying that because we've oftentimes had to wait for

a list to develop and it seems like that might be something to look at and i'm sure that will come up again but i can tell you from my own experience having been doing this for twenty years now and in the city. we are so much better at making sure that we actually have active eligible this than we used to and i think there's a real push to do that so you know there may be room for for improvement but we really have some great strides in that direction ok carrie thank you just. passed me off of somebody else one sense of this question great we're going to stand to me the big word here is the authority to fill in the requisitions right. when you are to raise the recreation on the ark. do you wait for. who told you that we want to fill the positions to where are we supposed to contact you then and say once you say ok you're right for the job you wait until you hear

about a month or so once a department gets positions into its budget basically it's the department's move to say ok we're ready to feel we're ready to issue a requisition issuing the requisition is actually department function once the requisition is issued and in the system where the mayor's office the controllers office and department of human resources can actually see it then begins the back and forth of getting the approval for that requisition ok ok to answer your question but so you're still negotiating at that point still after the requisition that that's true because you know the mare's in controls office will look at funding issues the department of human resources may still have questions about the appropriateness of the classification and there may be back and forth with the department for more information to ensure that the position is actually being filled in a class that matches the duties and responsibilities commission and so what i'm

hearing is that when the wreck recession submitter all three parties know the mayor the controller's office and department of human resources know at the same time going to iraq researcher has been put forth that's correct. i did have i did have another side on requisitions but i think we basically kind of covered most of most of that information so i will turn it over to my colleagues and we'll get back to if we have any outstanding question thank you for your presentation coming here this morning thank you. good morning commissioners president mccarthy my name is john kraus. the work of personnel selection can begin once the requisition process has been approved and in a moment i'll talk about some of the phases or steps that are involved in personnel selection but before i do that i just want to say that what we do is an arbitrary

just as d.b.i. i'm sure works under a body of coode was in regulations we must in personal selection particularly in public employment be guided by a number of laws regulations principles standards that kind of thing i don't know how to get this back up so for example we must be guided by local guidelines and regulations. as shown by the city charter in the civil service commission we have state regulations that we must follow we have federal regulations that me also must follow so for example civil service commission rules specify types of eligible with types of announcements types of certification rules and all the procedures that are related to those particular. there was areas. to california regulations.

civically to local government personal standards. cover the same processes as the civil service rules but they may put some limitations on what we can do for example we must issue announcements for at least five days and we can we can have an announcement up there for just two days would be a violation of their standards we can certify more than ten individuals on a referral. if we go beyond that they consider that no longer a competitive process and of course we have the civil rights act of one nine hundred sixty four particular title seven. and basically says. an employer doesn't have to have any discriminatory intent if there is. in order to have a violation. this act. if there is adverse impact against

a protected class the employer better prepared to defend whatever selection instrument they used that resulted in that adverse impact and in the seventy's we had a couple landmarks of supreme court decisions in there been numbers of them afterwards but these were the two big ones which was griggs versus do power company and the album or paper company versus moody and they basically said in essence that your selection process needs to be job related and you better have a job analysis that demonstrates that the process is job related and from a very technical standpoint we work on the american psychological association and the society industrial organizational psychology principles for validating examinations they address principles for designing constructing administering and scoring examinations so there are

a variety of other court cases and what have you brought in the next. slide what i tried to do is basically boil down all these rules and regulations and principals into. these points here. in merit system testing we're focusing on merit and fitness and what does that mean we need to look at. that applicants in these qualifications need to be related to what's required to do the job so how do we know it's required to do the job we do a job analysis and a job analysis is a generic term for the process of critical examination of the components of a job and the interrelationships of the elements of those components and what's required for measurement so the test content must be job related it actually must be representative of the job you can't just single out one little component of a job and focus on that you should try to cover as much as you can about the job in

your selection instrument and what you're testing must be defended from the standpoint of and business necessity and by that i mean there needs to be a legitimate purpose that's related to the fair safe and efficient operation of a business so if it doesn't meet that criterion you shouldn't be testing for that either and the other principle here is that testing must be competitive in that's where we really need to have some quantifiable way to separate people on the basis of their qualifications and you know you rank order people on the list on the basis of those scores and those scores need to have some meaning in terms of describing who's are superior worker who could be super worker and somebody who's maybe just a marginal candidate. obviously candidates

and applicants need to be treated in fair manner. so for example we need we need to have that announcement that job posting up for reasonable period of time in order to be fair we need to give applicants a reasonable period of time to file for that announcement the test should not discriminate against protected class members. a test can be valid and have adverse impact. and if that's the case you better be sure that the test is the most valid instrument out there in order to defend that selection instrument the law basically says if you have two instruments that demonstrate adverse impact you need to use the instrument that is more valid. and then we also have under civil service rules what have you candidates have the right to challenge your protest all kinds of actions and procedures throughout. the personnel selection

process and and you know they have due process rights so so next slide basically outlines all the different phases or steps that are involved in personnel selection the big one is the job analysis process because. often i hear and why can you just grab a test and administer it you can't do that. this is the work that basically all happens even before the announcement is issued it's like the behind the scenes work it's kind of tedious nobody really enjoys doing this kind of work because what you need to do is. analyze the job in terms of all the tasks that are required to do the job or all the knowledge skills and abilities we call them worker characteristics that are required to do those tasks we have to link those those

knowledge is and skills to the individual tasks we need to bring in subject matter experts supervisor incumbents to sit down and rate these tasks and k s a's skills and abilities we work out how frankly they're performed how critical the are in terms of the knowledge skills and abilities we only really are focusing on those knowledge skills and abilities that truly differentiate a superior worker from a marginal worker we need to have the incumbents in the supervisors tell us what that is and we only need this is important because sometimes we hear candidates say well you know this test doesn't look like my job well doesn't look like your job because the test is only measuring what you need to have when you enter the job what you do in your job much of that you've learned on the job wouldn't be fair to test people on those knowledge is that you already know that you aren't on the job

the law basically says you need to test people on what they need to know day one when they enter the job so the test may look a little different than what somebody might expect and we spend quite a bit of time doing our calculations to determine. you know what are the very most important knowledge skills and abilities how are we going to measure them what kind of instruments are we going to use how many instruments are we going to use what are the weights going to be assigned to those instruments or to the measurement of the specific knowledge skills and abilities or the simulations that we're going to do whether we're going to rank people on the basis of their scores or we're just going to you know a pass fail kind of situation and then the was very clear that we need to document all that information because we need to basically show that there's a rational logical relationship between what the test is measuring and what the job what's required on the job and the job analysis is the intermediary between those things. so we're back to the.

very people you know we're back to this diagram or showing the the phases so the top tier there is basically all the job analysis let's assume we get that all done we're down to the point we've documented everything and we look at the job specification which is really the official classification description and we say lo and behold the job analysis doesn't align perfectly with the job description we've heard from these subject matter experts the incumbents and supervisors that workers are required to do these duties in these duties are not reflected in the job specification so we now need to update our job specification or the subject matter experts may say to us you know the minimum requirements and that's what we used to screen applicants to determine who is eligible to participate in the examination

process you know they're no longer appropriate we really want three years of experience doing this or we want a specific educational degree or what have you. that means we need to revise the job specification it means we need to post the specification and that begins the first point where people can challenge what we're doing so if they have a problem with the duties that we've we've changed revised or the new minimum requirements they can challenge it and this can really slow down the process for a long period time particularly if it goes to the civil service commission but assuming everything is fine and dandy then we move onto the job announcement and that's where we basically outline everything for the applicant we tell them these are the minimum requirements you need to get to participate. these are the test procedures that you're going to have to go through. the list is going to be active for x. period of time this is the certification rule that we're going to apply etc etc and

we post announcement typically for two weeks but if we were really in a rush to get an announcement out there we can go maybe as short as five days in some cases once we get that announcement up there we can actually start to begin working on the exam and the development of the exam really the time that's required really depends on the type of instrument that we're going to use the most time consuming process of course is when we need to develop a new instrument from scratch let's say it's a very technical job the analyst is not very familiar with the technical aspects of the job so they must rely on the expertise of incumbents and supervisors who know their job so they may actually help. develop the test they might write test items they would participate in developing reading criteria it's safe it's an oral examination or it's a written examination they may help in a stablish in

a past point for the examination determine what's the appropriate point where we say who passes and who fails what have you but it. entails quite a bit of time and it's a back and forth between the analyst and the subject matter experts somewhere along that time now we've collected all the applications and then we compare the qualifications of the application of the applicants to the minimum requirements that are required to participate in examination process those who do not meet the minimum requirements we notify them that they've been rejected they are ineligible and again we have another point of protest so if they wish to challenge there's another back and forth process and that can actually go to the civil service commission in some cases as well. next is now that we've established who is eligible to participate in the selection process we schedule the candidates and that usually means giving them at least two weeks notice. before the test itself

so you know that's a big chunk of time right there just the notification period giving them less time and that it's not really fair to the applicants. and then we have minister the test and we allow candidates to challenge the test the manner in which it was administered you know we received all kinds of protests from the rooms too cold it was noisy outside that kind of thing. and we need to resolve that that's a period of time again back and forth and then we score the examination and we in our scoring you know we consider those weights they talked about and what have you and then we factor in promotive points and pull city employees who have six months of experience get extra points and they're added to their examinations score. and get as many as sixty points if you're a veteran you can get thirty five extra points if it is able veteran you get

seventy five points or seventy points i'm sorry. and then we issued a list then we issued a list we notify candidates of the results they have an opportunity to come in many times to inspect the examination. or at a minimum they get to inspect how they were scored to ensure that they were scored accurately and if they're not happy with that then there's another point of protest where they can challenge that so as you can see there's many different phases in the process and there's many points at which the process can get bogged down for a variety of reasons committee i like little graphic there next to the response to protests is there a minimum amount of time that is required each of those respond to process stages not not for each face except for the announcement period we want to have the

announcement up for a minimum of five days. the referral process kerry talked about the certification there is we need to get the the referral out in the case of position based testing exam within fifteen days of the list adoption if it's a class based examination i'll explain that in a moment we have a thirty day time timeframe to issue this the referral ok which brings me to actually an explanation now of our two major testing programs in the city one is the position based testing program we call that the p.b.t. program these are basically examinations that are primarily done by the individual departments under delegation orders they have the authority to.

announce an examination and minister examination and then we have the class based testing which generally d.h. are is responsible for. the class based tests usually involve classifications where the title is used across multiple departments and agencies are out the city it's really a large scale kind of thing so when we do a job analysis we're not doing a job analysis it focuses in one department we're bringing in subject matter experts to reflect what is done in that classification and all the departments so the scale is much much larger. but the class based testing process is a little longer than the position based testing process so we have some criteria here that kind of no one of these criteria is exclusive exclusively determines whether it's going to be beauty or c.v.t.

but a number of these may factor into into that decision so generally if we're focusing on us specific position in a department then we would go for the p.p.t. i mean the focus basically to get it and examination a list out there that's going to fill a particular position but if there's lots of vacancies that cover lots of positions within that classification number going to do the quest based test if there's an urgent need to fill because that p.v.t. process is much faster you know we may decide to do that usually when we have large kind of the populations in the hundreds and thousands it's usually a class based examination and and so on. the next slide actually question because. on the slide before and on that particular slight there you there is a line it talks about citywide activities in order to fill

a particular class here in the city as i've listened we are particularly concerned about migration. and is there any class or specific citywide activity that your department is plugging in to allow reversing the out migration. no. i'm not following you there but i think the reference here for the city wide hiring is what we're really trying to say is that the classification is used for multiple departments and. they're hiring into these positions if so most multiple departments are not responding at all to the out migration matter that the mayor in the city of as embrace. how does that relate to this hiring trying to get hiring spread out people of diverse backgrounds oh yeah we have a number of programs that were trying to outreach you talk about outreach outreach

that oh yeah yeah we have programs that we're trying to do is to lower the minimum requirements as low as we can possibly go so that it's not an artificial barrier we have apprenticed programs to bring people in that way into the system we have job training programs human services as a number of job training programs and we're trying to use those training programs as substitutes for meeting the minimum requirements so there's a number of different activities going on around that all right well that's good to hear yeah thank you. ok so the next slide. compares to two different programs and i basically want to. show you how the time frame is different the position based testing program started around two thousand and six and actually there's a whole set of civil service rules that define this program and its intent was really to speed the selection process and in particular dress the large number of

provisional employees that existed back at that period of time so the goal is basically to have a list adopted within sixty days of the announcement that and that's built into the rules for this program so i already mentioned that the class based testing has a broader job analysis so that requires more work requires more time the next item is class based test we have a ten day minimum window for posting the announcement where is in the position based testing we have a five day minimum so we save you know about a week there and then when they come in to inspect their papers to see whether they were scored correctly. there's a five day a longer period of inspection for the class space versus three days for the position based testing program i mentioned about the referral notification thirty days for the class based and fifteen days for the position based in the class based

at the. candidates and applicants basically can appeal anything and i can go to the civil service commission and the position based testing program and many of the appeals can be addressed at mickey callahan's the h.r. director is level so we don't have to have that extra time going before the civil service commission and the type in number and times when you can appeal as much more restricted for the position based testing program. when we issue a class based wist we first issue a tentative list sort of a draft list if you will and that's to give candidates an opportunity to look at everything because if there's a an error or you know we scored somebody improperly or something's wrong we'll make that fix before we officially adopt the list we don't have that with the position based testing program one reason is because we have limited opportunities to appeal but we go straight to it.