Introducing the Essential Metrics for Alternative Emergency Response Programs

“We can look at as much alternative response program data as we want, but what really matters is making improvements in service delivery to support better outcomes based on that information.”

This quote from Aloka Narayanan, project leader on the Criminal Justice Team at the Harvard Kennedy School of Government Performance Lab (GPL), perfectly encapsulates the action-oriented research that the GPL specializes in. And while that applies to the full portfolio of the Government Performance Lab, from procurement to homelessness and housing to child wellbeing, Narayanan’s expertise is in the Alternative 911 Emergency Response initiative. While Data-Smart City Solutions has previously featured examples of alt-911 programs in cities like Denver, New York, and Sacramento — as well as research on program efficacy — this is one of the first times that insights from pilots and programs have been collected, analyzed, and published in an easy-to-use data tool, the Essential Metrics for Alternative Emergency Response Programs. 

What Does this Solve:

This tool marks a significant step forward in aiding jurisdictions seeking to replicate alt-911 programs by providing 29 common, actionable metrics for everything from emergency call triage and dispatching to project performance management to measuring community impact. Based on over three years of work with more than 30 jurisdictions, the GPL team created this tool to address three common data challenges. According to Narayanan those challenges were: 

1. Identifying what types of data to look at to understand program impacts and next steps.

2. Asking the right questions of the data to assess what is driving these results.

3. Prioritizing what data to regularly review and where to target staff and program resources. 

For the first challenge, one of the biggest issues with this field is the relative newness of alt-911 programs and resulting dearth of insights or reporting. Cities or counties interested in developing an alternative response now have a comprehensive set of guidelines for measuring program performance based on GPL research; each of the 29 metrics that jurisdictions should pay attention to is “paired with a common data source used to measure it, a recommended frequency of review, and a priority of either essential or supplemental.”

This leads to the second point, which is asking the right questions of the data. “This is very much a GPL way of thinking,” said Narayanan “knowing what questions that you can ask to understand what data points to pull out and collect.” The GPL has other tools and dashboards that follow this method of data-driven performance management to help local government decision-making. The new alt-911 tool focuses on how to make decisions around alternative response pilots, testing hypotheses, and facilitating meetings to drive action based on this information. Which feeds directly into challenge number three; many of these local governments have limited time to review data and facilitate course corrections or redirect resources. Providing this roadmap eliminates a lot of the guesswork and lowers barriers to trialing alternative response programs. 

Why These Metrics:

Each of the 29 metrics is categorized under one of six Core Questions. For example, the first question asks, “Are programs successfully triaging and dispatching calls to the right responders? Where are there opportunities to respond to additional calls?” Metrics under this include call volume, defined as the number of calls taken by the alternative response team. The common data sources for this metric are the jurisdiction’s 911 calls and the alternative response dispatch system. Collecting this data is an essential priority. 

According to Narayanan, these metrics were informed by both an initial scan of what jurisdictions with alt-911 programs publicly reported, and insights from jurisdictions who have received technical assistance and applied research support from the GPL. By researching the broader field, the GPL team could test what data was possible — and feasible — to collect more broadly. The GPL identified core questions and metrics as they developed the tool over time, with an eye toward what data would help advance the field and further research.

Cities or counties can use these metrics whether they already have a pilot program or are interested in creating one. The recommended frequency is important for the data tracking and review process, even with established alt-911 programs; the demarcation of supplemental versus essential was based on the GPL’s previous work with the other jurisdictions. “There’s no point at which jurisdictions should stop collecting and reviewing the data,” said Narayanan, “jurisdictions still need to do reactive troubleshooting and maintenance beyond the initial phases of a program.”

How to Utilize:

Jurisdictions interested in setting up an alternative response program should review the tool in advance to help set them up for success. Reviewing the essential metrics and identifying which agencies hold different types of data can help guide who should be at the table to collaborate. Narayanan recommended “sitting down with a working group of agency leaders to help secure buy-in on data-sharing and continued collaboration on making improvements using the data before a pilot.” This typically includes law enforcement agencies, public health departments, and social services.  

Cities or counties that have established data collection processes should consider setting up a data structure that allows stakeholders to look at all the information, over time, in one place. Data dashboards are particularly useful for this and some of the leading governments in the field, like Durham, North Carolina, have a corresponding public-facing dashboard. This allows the community to see (anonymized) metrics like calls, service recipients, and service outcomes from the city’s Holistic Empathetic Assistance Response Team (HEART) program.

And while any jurisdiction with an alt-911 pilot should be collecting and reviewing data, a rigorous evaluation is also important to consider when looking at program expansion and sustainability. Adopting an alt-911 response is a significant change to the established system of emergency services, and “as with any new pilot program that significantly affects how government functions, it’s always helpful to think about evaluation,” according to Narayanan. Internal data-analysis capacity is an issue with some jurisdictions, so partnering with a local university can help.

Another important stakeholder and source of data is the community. Many cities and counties have established community feedback mechanisms, that both intake and report out data. “Data transparency engages community members,” said Narayanan, “and helps them understand the impacts of the programs.” While some jurisdictions have formal community advisory boards, others utilize online comments or public meetings to allow community members to express opinions and advocate for changes or additional work.

Looking Ahead:

“Jurisdictions should consider this a guide but not set in stone,” Narayanan said, “It’s a North Star, but jurisdictions can explore proxy metrics or other ways to get to this information.” While this tool should be considered one of the foundational pieces for developing a successful alt-911 program, there are additional tools the GPL has already published and others that are forthcoming that build the foundational infrastructure for helping launch and iterate. The GPL is introducing a complementary tool during a public learning call on May 14, along with a panel of two alt-911 program managers from jurisdictions the GPL has supported. Previous publications include case studies on lessons learned from San Gabriel Valley Council of Governments’ multi-jurisdiction program and how to select calls relevant to alternative response, drawing from GPL technical assistance in Long Beach, CA.

Interested jurisdictions are invited to join GPL's Alternative 911 Emergency Response Community of Practice, which is a free monthly learning series open to any government staff working on alternative response management and implementation, and sign up for the GPL’s newsletter to stay updated on new tools and research.