By Dr. Brian Kinnards
As trainers, administrators, and officers, we are often subject to conducting research or to validating the results of existing research in an effort to provide data that helps us with policy, procedure, training outcomes, and/or duty effectiveness. There are typically four levels of results or “information” that can be determined from our audience in capturing such data:
- Reactions and feelings;
- Learning (enhanced attitudes, perceptions, or knowledge);
- Changes in skills (learning was applied to enhance behaviors);
- Effectiveness (improved performance because of enhanced behaviors)
Obviously, the fourth level of “effectiveness” is what we’d like to consider as most significant in the scope of our projects. We can be researching the effectiveness of certain fleet vehicles, citizen perceptions of effectiveness of a local police department, or conducting a survey on the effectiveness of polo shirt uniforms instead of Dickies. The possibilities and outcomes can be endless but who should really be responsible for carrying out such research: the department or a contractor?
Ideally, the organization's management decides what the research goals should be. Many agencies then hire a research expert to help the organization determine what the research methods should be, and how the resulting data will be analyzed and reported back to the organization. My opinion is that departments can be poised to better train their own people as organizational “assets” in conducting the research function. If no outside help is obtained, the agency can still learn a great deal by applying the methods and analyzing results themselves.
The only point of caution in conducting your own, agency research is that there is a good chance that data about “effectiveness” will not be interpreted fairly if the data are analyzed by the people responsible for ensuring that the product, service or program is a good one. This caution is not to fault those in-house researchers but rather to recognize that strong biases exist in trying to objectively look at, and publicly report about, effectiveness of a service or product.
Therefore, if at all possible, have someone other than those responsible for the research to look at and determine research results. You don’t have to go out and hire a professional researcher but you can do something as simple as visit with researchers and faculty at your local college or university or have a trained researcher from another department or agency in your area assist with the project!