• October 15, 2013
• Performance Measurement

This is the sixth in our performance measurement series. Read the first, second, third, fourth, and fifth articles.

Performance reporting based on internally generated city data is valuable, but it only reports part life in a city. Integrating external data and benchmarking – as is done in Colorado Springs – can help flesh out this picture. However, both models still exclude one immensely valuable input: the direct views and opinions of city residents.

The District of Columbia is one of the first US cities to attempt to report citizen feedback and complete the performance reporting picture. The city aggregates citizen feedback and publicly discloses ratings of city services through Grade.dc.gov. Letter grades are given to each of 15 city agencies based entirely on how the public views the effectiveness of each.

The grades are generated with the assistance of an external vendor that aggregates and classifies the data. More than half comes from an online web form hosted on the city’s website, with the remainder coming from a variety of social media streams. Twitter provides the vast majority of social media data (around 98%), but data is also pulled from Facebook, Yelp, Google+, FourSquare, and other platforms. Submission through text messaging is also available, but more rarely used. Feedback reviewers at the vendor assess each city related post, classify it by administrative agency, and color code it based on the content of the message. Green is a positive review, red is a complaint, and yellow is something in between. Scores are then aggregated to form a real-time grade for each agency. As of now, this ever changing grade is available only internally. Grades are posted to the city’s external site on a monthly basis.

Feedback is aggregated in this way for external reporting purposes, but the results are also used internally. A daily email with detailed feedback is sent to agency leadership and those directly responsible for customer service functions. Employees are expected to follow up directly with citizens on any posts that include specific complaints or requests for service. At a more aggregate level, data is periodically reviewed to assess trends in what citizens are saying about city services. The combination of direct interaction on specific complaints and aggregated performance assessment means that DC is engaging in a type of dual feedback loop with citizens. Those providing feedback benefit directly from having their individual needs met while the public at large benefits from the use of this data to fix systemic issues.

The external “report card” doesn’t provide feedback parsed by specific services or functions, but in many cases, the implications of the overall grade are easy enough to surmise. If the Department of Transportation’s ratings are steadily climbing, then it is a safe bet that the city’s impressions of the public transit system as a whole is getting better. However, it is less clear how opinions may be trending between train and bus systems, for example.

Matt Desjardins, Communications and Initiatives Specialist for the District of Columbia and project lead for Grade.dc.gov, says that the city is continually looking for ways to improve the system, finding new ways to leverage the data it produces. The city is also continuing to formalize processes around how they interact with the public using this information. For example, minimum response time guidelines for agency representatives responding to complaints and service requests via social media are now being established.

The advice Desjardins provides for those starting in the field of citizen feedback collection and reporting is similar to that provided by Colorado Springs – start small and non-controversial. Grade.dc.gov began with a handful of pilot agencies that were the most amenable toward participation in the program. Later, it was ultimately a mix of power and persuasion that brought additional agencies on board. Many in city government were enthusiastic about the opportunity to leverage the tool to optimize performance, once this vision was communicated. Others required a push from city leadership, a strong commitment at the executive level that citizen feedback reporting and transparency was the way the city was moving.

The next and final article in this series will examine opportunities for refining performance reporting models as tools and technologies continue to develop.