|
I frequently get questions about how police promotional examinations are developed. The central issue usually deals with whether paper and pencil tests followed by an oral testing process are the best way to determine whether a person should be promoted to a higher rank. This article discusses some of the issues surrounding police promotional testing and the answers to frequently asked questions. To understand why police departments still use civil service tests we need to go back in time to when there wasn’t a “civil service” testing process. In the 1800’s cities organized police departments by selecting and promoting officers based on nepotism and politics. It wasn’t until 1883, with the passage of the Pendleton Act, that a Federal Civil Service Commission was established to eliminate political interference in the hiring and promotion process. This was the beginning of using “merit” as the basis for hiring and promotion. However, the Pendleton Act didn’t really begin to be enforced until the early 1900’s and dealt only with Federal positions. In 1902 the International Association of Chiefs of Police was formed and its first president—Richard Sylvester—began a systematic campaign to remove politics from police selection and promotion. This was followed by the emergence of two leaders in the movement to professionalize departments: August Vollmer (Chief of Police, Berkeley, California from 1905-1932) and his student and protégé O.W. Wilson (who served under Vollmer and then was Chief of Police in Wichita, Kansas from 1928-1939 and Chicago, Illinois from 1960-1971) were the driving force behind merit examination for entry level and promotional examinations for police officers. Other factors in getting politics out of police appointments and promotions was the formation of the Fraternal Order of Police in the early 1900’s and the unionization of the majority of police departments throughout the United States. Federal and State court decisions and laws have resulted in some form of civil service testing for most police departments and many union contracts and/or personnel rules and regulations have language requiring the use of content valid testing for selection and promotion. Constructing a Police Promotional Examination The purpose of the police promotional examination process is to objectively predict who among the candidates has the knowledge, skills, abilities, and personal traits (KSAP) to most successfully perform the position for which the candidate is being tested. In order to determine what those KSAPs are, a job task analysis is performed. Simply put, a job task analysis is a systematic methodology of defining tasks and/or behaviors needed to perform a job successfully over a period of time. The job task analysis provides specific, job related information with which to construct the KSAPs for which candidates would be tested. This is intended to insure the examination process is job related and valid. In other words, the person designing the test has to have a rationale that can withstand industry scrutiny as to what to test candidates for any why. There are many methods of constructing a job task analysis, but all include efforts to determine what a sergeant does (for example) and what job tasks are important to successfully perform the job. Once that’s done then a further analysis is done to determine what a person must know in order to do the job and what skills, abilities and personal traits are most required. The mistake made by many test vendors is constructing a “generic” examination they believe can be administered anywhere. This does not take into account that a Sergeant’s (for example) duties and responsibilities in one department may be quite different then what a Sergeant’s duties and responsibilities are in a smaller or larger department only a short distance away. These variables must be taken into consideration if the test is going to be content reliable and valid. Test Validity In constructing a police examination evidence must be gathered (job task analysis) that the evaluation instrument (test or series of tests) content is the same as the content of the job/position. The more the tests ask questions about what is critical or very important to successfully doing the job over time the more content valid the test is as a predictor that candidates scoring higher than others are more likely to succeed. One method of ensuring the test is content valid is to identify subject matter experts (a representative sample of those successfully performing the job) to rate a series of variables—probable cause for example—on a nominal scale from least to most important. The next step is to write test items to examine a candidate’s knowledge of the work behavior or task being tested for. The question then is whether an assessment center approach (putting candidates through real life scenarios and judging how they perform them) is more valid than a “knowledge” test such as a 100-question multiple choice or written exam. It has been my experience that assessment centers are more content valid because policing is a “doing” profession. Correctly answering a question on hostage situations does little to prove that you know how to handle a hostage situation in the real world. Candidates must be able to demonstrate they can actually do it. So, although I do administer paper and pencil examinations followed by oral boards, it’s usually because contract language requires it. In an effort to make certain a city or town will play by the rules, language has been negotiated into a union contract specifying the type of promotional examination that will be used. This is complicated further by contractual language citing the weight of each part of the test (60% written, 40% oral) that is required. Although I appreciate efforts by unions and others to ensure the examination process is fair and equitable, it’s the job task analysis which should determine the type of testing (and the parts and weights of each different part) of the examination process. What type of examination process is best? Over the years I have constructed and administered every type of examination process there is. If we can agree that police promotional examination are not a precise science, but a prediction that candidates who come out higher in the process (numbers 1, 2, and 3 out of 100 candidates) will be better able to do the job than candidates scoring lower, then I believe the best method to reliably make that prediction is a modified assessment center. I recently created and administered a modified assessment center for the rank of sergeant in the town of Simsbury, Connecticut. After completing a job task analysis, I presented its results to Chief of Police Peter Invertsen and recommended an “Assessment Center” approach be used to test candidates for the position. Although, the department had never used an assessment center before, Chief Invertsen is all about what’s best for the Town and the department. After hearing my proposal Chief Invertsen took the risk in a union environment to start using a new type of testing process. Here’s what the creation of the Modified Assessment Center (rank of Sergeant) consisted of: 1. Completion of a job task analysis involving all police personnel at the rank of Sergeant and above.
2. A comparison of the results of the job tasks analysis with similar cities and towns and a literary search to identify KSAPs requisite for job performance. 3. Assigning a number on a nominal scale (critical=5, very important=4) to the KSAPs and creation of a prioritized matrix of what to test candidates for. 4. Identifying a reading list (books, state statutes, police department policies and procedures, etc) for candidates to use in preparing for the examination process.
5. Meeting with candidates to discuss the testing process. The candidates were provided with a copy of the job task analysis.
6. Competency based test creation. In order to test candidates for the competencies identified in the job task analysis the following tests were developed: a. A 75-question, knowledge based, multiple-choice examination with five essay questions
b. A traditional oral examination
c. A situational-type oral examination--unlike a traditional oral examination, in a situational oral examination candidates are provided with situation (hostage situation, building collapse, etc) and are required to make a presentation to the panelist of how they would handle it
d. A supervisory/subordinate examination with a role player acting the part of an officer
e. A leaderless discussion
In all, twelve assessors (police lieutenants and captains) took part in grading each candidate. A training session was held with the assessor to lessen subjectivity in grading and squared deviations were applied in insure assessment validity. Using five different types of evaluation instruments in the testing process not only evened the playing field (a high level of expertise in paper and pencil testing was moderated in the process), but provided the ability to test for competencies paper and pencil tests cannot test for. The more you know about the testing process, the more likely you will succeed on test day. If you are in a position in your department to recommend or hire people to administer a promotional examination, ask them what they know about creating a job task analysis that will ensure test content reliability. If they provide you with a vague answer, choose a firm or individual who will create an examination process geared towards your individual department. There is too much at stake to do otherwise.
|
