By Gregory L. Walterhouse

What is benchmarking? Simply stated, benchmarking are points of reference from which measurements are made. There are several types of benchmarking, the first being corporate style benchmarking, predicated on the belief that superior results are the product of best practices that can be emulated from others. The second type is visioning initiatives whereby a vision is established leading to the creation of results-oriented targets. Visioning initiatives are like strategic planning. The third type of benchmarking, and the focus for this article is the comparison of performance statistics. In this type of benchmarking, an organization compares their own statistics to either national standards or data sets from other similar organizations.

Benchmarking can identify top performers within a data set and highlight relative strengths and weaknesses within an organization but does not identify best practices. However, there are some challenges to benchmarking. Relative to the public sector, the inconsistency in the ways that municipalities measure and report their data can make benchmarking challenging. Another challenge is turning benchmarking data into actions to improve service. This will require data-driven decision-making and transitioning from traditional models of service delivery to more innovative models, in other words, change, which is often difficult for some, particularly in the fire service to embrace.  

Another challenge is the interpretation of data. For example, a higher cost per incident rather than a lower cost may seem counter intuitive. Nevertheless, a city that has fewer incidents will have a higher cost, perhaps because prevention efforts receive more funding indicating a more efficient use of available funds. One report suggests that comparing one fire department to another may not be the most accurate metric due to different demographics and that comparing response service to prevention service within the same agency may provide a more accurate measure of efficiency.

Benefits of benchmarking
Several benefits can result from benchmarking. First, benchmarking helps develop standardized metrics against which a department can evaluate their performance. Second, benchmarking assists in developing performance expectations for the department. Next, benchmarking helps establish a culture of continuous improvement for fire departments and provides a basis for department administrators to identifying and correct performance gaps. As elected officials are becoming more data driven, benchmarking can provide the data needed to support budget and staffing requests, equipment purchases, and new or expanded programs. Benchmarking is also a useful tool when developing strategic plans.

National standards
The National Fire Protection Association “Standard (1710) for the Organization and Deployment of Fire Suppression Operations, Emergency Medical Operations, and Special Operations to the Public by Career Fire Departments” provides national consensus standard metrics that departments can use for self-evaluation.

NFPA “Standard (1720) for the Organization and Deployment of Fire Suppression Operations, Emergency Medical Operations, and Special Operations to the Public by Volunteer Fire Departments” provides similar metrics for volunteer and on-call fire departments. Some examples of standardized metrics contained within the NFPA standards include alarm processing time, turnout time, travel time, total response time, staffing and more.

Another of several examples is NPFA “Standard (1410) on Training for Initial Emergency Scene Operations.” This standard provides required performance metrics for deploying hand lines, master streams, automatic sprinkler system support, and truck company operations. Bearing in mind that national standards are minimum standards of performance, comparing a department’s performance to similarly situated departments at the regional, state or national level may be more informative regarding efficiency of operations.

Data
Compiling a data set for comparison is complex, as well as time, and labor-intensive. Employing an existing data set will simplify the process of benchmarking.

One source is the ICMA that offers free open access benchmarking that is a software-neutral data set with over 20,000 data points that facilitates comparing local government performance metrics. ICMA’s benchmarking service focuses on key performance indicators, with corresponding definitions. Some examples of operational performance indicators for fire and EMS include total BLS and ALS responses, average response times, total expenditures for fire/EMS personnel and operations, percentage of residential fires confined to object or room of origin, and percentage of cardiac patients with pulsatile rhythms upon delivery to a hospital.

Demographic performance indicators include, residential population served, square miles served, median household income, percentage of population below the poverty level, percentage of vacant housing units, respondent ratings of fire service quality and more.

There are also performance indicators related to fire service human resources including, hours paid including overtime, sick leave hours used, turnover rates, and worker’s compensation days lost due to injury. 

Though less focused compared to the ICMA data set, the National Fire Incident Reporting System (NFIRS) at the U.S. Fire Administration offers a free data set of more than two million fire incidents annually. The data sets are available on CD by request and include categories of all incidents, fire incidents and fire and hazardous materials incidents. Though the data will facilitate a comparison to national data, it is not conducive to making comparisons to individual departments. Users of this data must also be aware that participation in NFIRS is not mandatory and therefore is not a complete census of all fire incidents and NFIRS is prone to reporting errors, though the National Fire Data Center performs internal quality checks to identify and correct errors. 

Third parties
With many departments struggling with reduce staffing and budget constraints, finding the time and resources to perform benchmarking can be challenging.

One solution is a benchmarking analysis performed by a third party. The Emergency Services Consulting International (ESCI), which is the consulting arm of the International Association of Fire Chiefs (IAFC), has teamed with ISO to produce the Community Fire Service Performance Review for Structural Fire Protection. This is a peer-review benchmark analysis that fire departments and municipalities can use to make comparisons. The review compares a department to 15-25 accurate peer organizations along with regional and national averages based on 32 data points. ISO compiles the points into four main sections: emergency communications, fire services, water systems, and community risk reduction. The focus of this third party benchmarking analysis is narrower in scope focusing on how departments can improve their ISO Public Protection Classification rating. 

Summary
In summary, benchmarking is important to the fire service. As data is increasingly driving decision-making in the public sector, comparing fire service delivery to both national standards and similarly situated municipalities is critical to providing cost effective and efficient service. Benchmarking identifies areas of needed improvement, strengths, and helps restore trust in government by using data to make decisions and uncovering innovative ways to elevate performance. Benchmarking also provides information that may lead to mid-course adjustments or terminating programs that are not producing intended outcomes. Data-driven decision-making leads to greater accountability and transparency, encourages continuous improvement and improves consistency.