![Podcast microphone against a blank background](/sites/g/files/omnuum10826/files/styles/hwp_21_9__1920x825/public/datasmart/files/podcast_microphone.jpg?itok=3Q9qYvou) # StatGPT: The Future of City Performance Episode Seventy-Eight August 27, 2025 [ Betsy Gardner ](/people/betsy-gardner) In this episode, Professor Stephen Goldsmith speaks with Santi Garces, Boston’s chief innovation officer. They discuss Goldsmith’s new StatGPT paper, which explores how generative AI can transform city performance management. Garces also shares practical examples from Boston and insights from the recent Bloomberg Center for Cities stat and AI workshop, where 14 cities gathered to explore real-world applications of generative AI in government. Embed *Listen here, or wherever you get your podcasts. The following is a transcript of their conversation.* **Stephen Goldsmith:** This is Stephen Goldsmith, Professor of Urban Policy at the Bloomberg Center for Cities at Harvard, with another one of our podcasts. This one focuses on generative AI. I'm with one of the country's most-expert users at the government level of generative AI, Santi Garces. Welcome back. **Santi Garces:** Thank you so much for having me again. **Stephen Goldsmith:** I guess I should say what you are other than expert. You're the chief innovation officer and head of the mayor in Boston's innovations cabinet. Have I got that right? **Santi Garces:** Yes. The head of the Innovation and Technology Cabinet for the great mayor, Michelle Wu. **Stephen Goldsmith:** And I think you've won the award for chief innovation or chief information officers of the most U.S. cities of anybody in the last decade. **Santi Garces:** I have had the pleasure of working in some great cities and for some great mayors. **Stephen Goldsmith:** Yes, South Bend and Pittsburgh, and if you're lucky, you'll work your way up to Indianapolis. Santi, let's spend not too long today. The subject of this conversation is applications of generative AI that have or promise to make big differences in the way cities perform. A lot of people are talking about generative AI, they're talking about how to regulate it, they're talking about how it might displace workers or increase their productivity. Today, let's just focus on the specifics of applications that either have been done or are at our fingertips. And pay particular attention to the fact we had 14 cities to our offices at the Bloomberg Center at Harvard about 10 days ago where you, Mitch Weiss from the Business School, and with some support from Zencity and NTT, led cities through a pretty interesting session. Let me start with this question first. Name the three coolest generative AI applications in Boston you've already put in place. **Santi Garces:** Absolutely. I will do as I was told. The first one that is interesting is we've been piloting AI search on boston.gov. What that means is that we are using some of the underlying foundational elements of generative AI, so these embeddings, which is a way of representing words based on context. So someone's able to find information about small businesses if they're typing "hair salon," for instance. One quick thing about that that is interesting is that the preliminary results showed that people are 30% happier with the results of the generative AI search compared to the traditional search. The traditional search uses literal search. It just goes and tries to find a word that looks like the word that people are looking for. And again, that requires the content to be organized the same way that people are looking for. So 30% increase with our prototype and we're continuing to evaluate to understand a little bit about how much it would cost and how to scale it. The second one is some work that we've done with a procurement tool that we call Bidbot. It is a tool that allows employees to gather information about the procurement process. We have about 60 different documents, including the regulation that's set at the state level, city policies, best practices, we have some templates. And it allows employees to be able to ask questions like, "What is the best procurement mechanism to buy something?" It helps them build drafts. It is not intended, nor will it allow you to grade or evaluate any actual procurements, but the idea is to help people build procurements faster. And we're working with some great folks at Harvard, with Mitch Weiss, with Karim Lakhani, with Jeff Liebman on evaluating the performance of Bidbot. And nothing to share publicly yet, but it was very exciting and we got some really great results in this randomized control trial that we did to evaluate Bidbot. And the third one, I would say let's go back to the oldies. We've done this, again, simplified version of exposing roll call. This is the actions that the council members have taken and what are the votes? How did they vote? And we created 20 word-summaries for each of those. So 19 years' worth of data, we were able to do it for about $4 in credits. And the feedback that we've gotten is really positive. Some initial results that we got with older versions of models were a little bit worse. And then as we started to use newer LLMs, the performance of the results have gotten much better and that's a great opportunity. And I could talk about some things that are cooking that are in progress. They're not yet in production, they're not things that we've put in, but that are in development. I'm happy to talk about those too. **Stephen Goldsmith:** Yeah, let's come back to those in a second. Let me just ask you a question about the first one. I was deputy mayor of New York. If you want to open up a restaurant, you don't get a restaurant license, you get license one, license two, license three, license four. If you're still alive at the end, you get a certificate and of occupancy. And even though we set up an ombudsman to help you negotiate the system, there was no digital ombudsman. So take us back to your first suggestion, and does your generative AI read all the rules, regulations, forms, and permits required for a hairdresser and then create the form for the person to fill out, tell them what forms to fill out? Just talk to me a little bit about how you're using generative AI to reduce the transactional friction of doing business with Boston. **Santi Garces:** Yeah. I think that for us, we're actually trying to do this in a way that's a little bit more incremental, but we're hoping that by virtue of being incremental, it means that we're actually on sure footing and moving actually faster. Our AI-driven search, instead of creating content and trying to interpret vast amounts of regulation, what it does is, and this is the main difference between a search and a chatbot, is what search does, it returns to you what is the content that is most relevant to you in a ranked order. It just gives you a ranked order of what is the information that's more relevant. And additionally, the other thing that we're doing is creating a summary because sometimes the information that you are actually looking for might exist across a couple of different articles. We're able to create a summary of information that is across different articles. The advantage of the approach that we took is that we're not producing any information that is not anything that's already publicly on the website. And we put a lot of effort in. We have a lot of infrastructure in making sure that the information that we have is relevant, that is current, that is correct. And by virtue of the evaluation mechanisms that we've put in place for the search function, we're actually also identifying people telling us, "Hey, you're missing information about this program," or, "this is not clear." We're actually creating these virtuous cycles because we're getting better data about how people think the quality of our content is, and that just helps people that are responsible for those pages to make it better. Again, generally speaking, we are considering what you're describing around permitting. There's some other work that we've been doing and we have a really great team of people working on this. One of the things, I'll give you the ambition of what we're trying to do, and we think that AI will have a role to play, but nothing to announce just yet. The permitting experience is one that is usually described in administrative terms. We have all of these different types of permits that are done by all these different departments and agencies, but that's how we've been built by the legislators, regulators. That's how we've been designed to operate. But the thing that people want to do is they want to remodel their kitchen. They want to open up a restaurant. And in some sense, what we think that AI could be really powerful at doing, and obviously supported by things that we know that are maybe a little bit more old-school at this point, but user research, data analysis, actually just good technology building and process changes, is being able to map out the processes as intended by the constituent, like what is that end-to-end experience of getting the thing done? And then being able to translate this language, this perspective of our constituents into the administrative language of the city and make a lot of the internal pieces that are important, they're regulated, we're required to do them, they help people make sure that they can trust the fact that the building's not going to fall, that when they go to a restaurant, they're not going to get sick, but without having to become experts in deciphering what we are trying to do and how we work. **Stephen Goldsmith:** Okay. Makes sense. I saw some interesting software that San Antonio and Dallas were customizing on building permits where they had worked through at the point of filing for a building permit, the city's software, I guess it's their vendor's software, would help the applicant identify what they need to do to finish, what's missing, where the plans are missing, et cetera. It just feels to me like what you're doing is on the way to vastly improving customer experience with the City of Boston. **Santi Garces:** Yeah, and I can tell you because it fits a little bit on this theme about how agents can help people do and how is it that they can help drive better decisions? And again, this is still in an exploratory phase, but one of the things that we're doing is we have over 20 years of permit data in boston.gov and our open data portal in data.boston.gov. And again, the permit type is a field that is structured, but that's more along the lines of how we process the permit. The comments field actually tells you what people were actually trying to do. They're trying to update an existing permit that they had that had expired. It has so much rich substance. So one of the things that we're going to be doing is using large language models, using some of that same infrastructure, to translate that comment into these normalized intents of what was the person intending to do, and then do a cluster analysis to try to understand what are the discrete types of things that people try to do with our construction permits, then go back and apply those new constructs, these new kind of ontologies of what the intent is, to the existing permitting data so that we could understand how frequently people try to do those different things in time and how do those match with the administrative stuff? And anyone can do this. Anyone can go and get the data from the open data portal. They can generate code that does machine learning, and they could run it on Google Colab or on their own computer. So it's really amazing because things that used to be really sophisticated and complicated, research projects, are things at the fingertips of really anyone. We're lucky at the city. We have some brilliant data scientists. This amazing person called Bianca was helping spot-check some of the AI-generated code that I was putting together to try to do some of these. And again, these are prototypes, some initial ideas that we're exploring, but it's just so powerful because it's so accessible and it's so easy for people to get started and it's all at our fingertips. **Stephen Goldsmith:** This is a podcast and not a video, so I think that the listeners should realize you raised both hands and put your fingertips up in the air for emphasis. It's at the fingertips. Thank you for- **Santi Garces:** There you go. Thank you. **Stephen Goldsmith:** ...that visualization. Santi, let's go to the meeting. You are spending some time at the Bloomberg Center trying to help us evangelize, if you will, about how government could be better through the applications of tools. And I know you've been an advocate of do the simple practical stuff first, don't get all tangled up in the more bizarre. So in part based on your encouragement, we brought these 14 cities to Harvard around the issue of what used to be called stat programs, the CompStat, CitiStat, the way cities measure what they're doing in terms of performance. Tell us a little bit about the afternoon sessions. What was the purpose of those sessions? We broke the cities up into a set of eight-person round tables, but what assignment did you give those cities? **Santi Garces:** I think that what was interesting about this is, again, we had 14 cities and it is actually coming from a place of humility. It's going and saying, "No one city has figured this out, but we're all trying to understand the same thing," is when and how could these tools, could generative AI, and we call them agents, but in reality just trying to figure out how is it that a large language model when it's referencing a piece of information and configured with a system prompt, when can it do something that is useful and not only something that is useful, but something that if it happened at a broader scale, it would really have an impact on government? Yeah, the city of Boston has over 20,000 people. We have about 18,000 employees, but we have like 25,000 people that access our systems. If every employee was doing things a little bit better, if they were able to reduce a step in a process, if they were able to better understand how to benchmark their work against others to solve problems, to create project plans, to do all these things, the city would be doing better as a whole. So what we did with the 14 cities was a provocation was asking them, based on the kinds of things that we know that are helpful, and these are some experts from great cities, what are things that we know that are helpful that help us solve civic problems? And tried to turn it into a product by virtue of creating these little custom GPTs of these little agents. Happy to talk a little bit about what happens next, but that was the provocation is, can we think about something that we know has been useful that, if we made it into a tool, it could be replicated across the entirety of government and potentially outside of government as well? **Stephen Goldsmith:** Give us an example. They sit down at the table, Santi tells them to make government better in a practical way. Tell us what Seattle's table did. **Santi Garces:** Yes, so that team came up with a couple of really great ideas. The first one was a “chatalyst.” One of the things that is difficult in government is data analysis skills have been typically limited to a number of people and a few teams in city government. The first thing that they started to do is building a tool that allowed people to ask questions from data. It would automatically generate a data analysis about, I think that they were focusing on 311 data, so it could go on, give some descriptive analytics. It was able to go and create charts and graphs. It was able to create maps about questions related to 311 data. And again, that's something that typically would have required an analyst some amount of time and only a few people would be able to get some of that time to inquire. **Stephen Goldsmith:** Does one of these solutions allow somebody who's not a data analyst, who is a supervisor in the sanitation department...what's the name of the sanitation department in Boston, for example? What's the name of the department? **Santi Garces:** It is under the Streets Cabinet and it is the Waste Management team, I believe. Dennis is, he's a wizard. He makes all this stuff disappear from the curb, you know? **Stephen Goldsmith:** Yeah, that guy. Let's say the wizard of Waste Management has a deputy who works in a part of Boston who wants to look at these things. This agent that you spoke about that was built at this table, how would that relate to his or her work? **Santi Garces:** Yeah, so again, I'll give you an example. So our Waste Management team in Boston is actually really brilliant in thinking about not only how do we pick trash faster, but it's how is it that we reduce the total amount of trash that we put out because we're recycling more, we're composting more, or just throwing less things out? Some things that would be obvious that you could do is you could go and see how many missed trash pickups do we have? Do trash misses tend to be more frequent depending on whether it snowed or not, or when we're close to the beginning of the school year or whatever? You could start seeing if there's things like rodent reports whenever we miss the trash. You could start doing those kinds of correlations. But again, because we have this really brilliant and forward-thinking team, all of a sudden you could take that 311 data and you could start putting it in context with overall trends around recycling. You could put it in the context of composting. You could take one piece of data and then starting to think about some broader strategic goal that would be hard to start thinking about without... You'd need to wait to get access to these really brilliant data analysts that had big, complicated PCs in City Hall. **Stephen Goldsmith:** Based on observing these cities do a set of pretty practical things, what, in two hours, right? They developed a number of agents. What would unlock the most applications of generative AI that would produce value in cities? What's holding us back? **Santi Garces:** Again, the hypothesis that we have right now is that, and it is, you put it really greatly in [the StatGPT paper](https://datasmart.hks.harvard.edu/transforming-city-operations-statgpt) that you wrote based on some of these conversations that we've been having, Stat wasn't just about one thing. It was the fact that when we put attention to issues that matter, there's a large number of little tools and things that we can do that make it better. Any problem that is worth solving will require a combination of policy change, process change, people change, technology change, but we don't have to wait to have all of the ingredients at the same time to make change. The part that gets me excited is actually all of the little ways in which improvements can be made and not having to wait for this really static, not very articulate or agile approaches to management because that's, in my experience, what ends up helping. And I think about something, for instance, like what Brian Elms did in Peak Academy in Denver that he's grown to other cities. It was by virtue of bringing the right knowledge and the right tools to a really large set of employees that you could go and make massive amounts of change. Top-down change is important and it is valuable and it is impactful, but when you have the entire organization humming with little bit of change that is incremental, you can actually be sometimes even more innovative because you can be thinking about problems that no one else has seen. Like the point that you just made about sanitation, maybe the truck driver that is picking up the trash knows about an issue, but if they knew how to write a professional memo or they were able to create a really compelling PowerPoint presentation or do something that would be able to convey this idea, they might be able to convince their boss or their boss's boss to invest in a different kind of truck or changing the route or something. But the power of decision and the power of influence is one that tends to be restricted in these big, complicated bureaucracies, and I think generative AI has the promise of potentially giving back power and resource and skill to people that generally don't have the ability of driving change in that way. **Stephen Goldsmith:** How much, a little off the subject, but how much training does the person who's not Santi Garces and hasn't been a CIO in three cities, how difficult would it be to more broadly train city managers on these tools? **Santi Garces:** Well, I think you pointed out we had a very diverse group of city officials, and in about two hours we ended up with about, what, 15 different custom GPTs? So again, I think that it is a matter of creating the space to try. The intent, and we said it from the beginning, the intent of this exercise was not becoming something that was going to be in production we were going to give to everybody. We actually said, "Over the next six months, we're going to be thinking about what are the things that might be useful so that we can curate which of those things are worth investing in and trying to scale?" But what I would say is expertise comes from experience and experience comes from experimentation. Trying a little bit without much consequence, taking 15 minutes to try to do something can be really sometimes the best way of learning how something works. And then you can figure out if it's worth going and doing more of it. There's some really great resources that are freely available, like InnovateUS that has a ton of material on generative AI and how it can be used in a number of public contexts. And there's a lot of training material on this stuff. And at the city of Boston, we're going to be working with InnovateUS to deliver training to our workforce. But again, I think that nothing beats getting your hands dirty. And then I think that for us as leaders in cities, creating this space for people to learn and to try so that they can go and decide how is it that it might fit within their work. **Stephen Goldsmith:** Santi Garces, CIO for Boston, the leader, as you can tell, in change management, the use of technology, helping cities operate more quickly and better and more effectively. Thanks, Santi, for your time today. **Santi Garces:** Absolutely. Thank you for having me again. ## About the Author ![Headshot of Betsy Gardner](/sites/g/files/omnuum10826/files/styles/hwp_1_1__100x100_scale/public/2025-05/Betsy%20Headshot%20resize.jpg?itok=k2OsSp1g) ### Betsy Gardner Betsy Gardner is the editor of Data-Smart City Solutions and the producer of the Data-Smart City Pod. Prior to this, Betsy worked in a variety of roles in higher education, focusing on deconstructing racial and gender inequality through research, writing, and facilitation. She also researched government spending and transparency at the Lincoln Institute of Land Policy. Betsy holds a master’s degree in Urban and Regional Policy from Northeastern University, a bachelor’s degree in Art History from Boston University, and a graduate certificate in Digital Storytelling from the Harvard Extension School. See also:- [ Artificial Intelligence ](/topics/artificial-intelligence) - [ Civic Analytics Network ](/topics/civic-analytics-network) Share on:- [ Facebook ](#) - [ Twitter ](#) - [ Linkedin ](#)