Generative AI and the Possibility Government

In this episode, host Stephen Goldsmith is joined by Harvard Business School professor, former mayoral Chief of Staff, and faculty affiliate at the Bloomberg Center for Cities, Mitch Weiss, who talks with Pr. Goldsmith and Boston CIO Santi Garces about the transformative power of artificial intelligence and GenAI. Using Weiss's framing of possibility government, they discuss current and future uses of AI in city government and why cities should focus on transformative problems.  

Listen here, or wherever you get your podcasts. The following is a transcript of their conversation.

Stephen Goldsmith:

Welcome back. This is Stephen Goldsmith, professor of Urban Policy at the Bloomberg Center for Cities here at Harvard University with another episode of our Data-Smart City Pod. Today we’re with one new guest and one returning guest; we have Santi Garces, who listeners know is the CIO for the City of Boston, and Mitch Weiss, who is professor of Management Practice at the Harvard Business School where he focuses on public entrepreneurship. He's also, importantly, was former Chief of Staff to long time Boston Mayor Thomas Menino, prior to coming to the business school. Welcome gentlemen.

Mitch Weiss:

Good to be here. Thanks for having me.

Stephen Goldsmith:

It is my pleasure. So before we get to the topic at hand, which is AI and generative AI and how it's going to affect city government, Mitch, how about a short introduction of your background, how you got to the business school from City Hall. Our listeners know already a lot about Santi, but tell them a little bit about yourself please, Mitch.

Mitch Weiss:

Well, I had arrived in city government in Boston, actually just out of business school, in the Mayor's Office working on a handful of projects, but they included a series of technology projects. One was to bring GIS into the city. The other was to bring GPS into the city. And I always joke that either dates me or gestures at sometimes how long it can take for certain technologies to make their way into governments. But that was a while ago. I came back in the city government as Chief of Staff, as you mentioned to Tom Menino in 2010 in part to help him lead the city, but also most especially to help bring around an innovation agenda. I had the great privilege of co-founding the Mayor's Office of New Urban Mechanics with Chris Osgood and Nigel Jacob, one of the first big city innovation offices, helped lead those efforts for the mayor's fifth term office before coming to Harvard Business School to teach about invention inside government, and for, by private govtech companies. And of course it's that interest in new things in government and new things for it, which have taken me to this interest in AI. And I've had the great honor of knowing Santi in several city incarnations starting in South Bend and later Pittsburgh and now Boston. And you, Steve, I guess I'm now dating you and not precisely when you were in Indianapolis, but certainly in New York and since. And so it's a delight to be with both of you.

Stephen Goldsmith:

Thank you. I mean, this is a real treat. I could talk to you guys for hours. I mean, you are two of the leaders in using technology to improve the way government operates, city government in particular. Lots of thoughts here. I want to spend the few minutes we have today thinking a little bit about applications of generative AI, particularly into Stat and performance management. A subject that you all came over for a session with large cities that we did late last year. But Mitch, before we get there, it feels to me like part of the issue here is technological, but most of the issue here is the structure organization, bureaucracy and imagination in city hall. So just to give you an opportunity to plug your book, why don't you connect “We the Possible” with the possibility of using GenAI, how is it we need to think differently? We'll get to the technology in a second, but just organizationally, if you're inside city hall, pitch your book a little bit and apply it to generative AI.

Mitch Weiss:

Well, it's nice of you. I mean, how about if instead I'll pitch the ideas in the book. The main idea, the better idea, of course, is this idea of “possibility government,” of not always sticking with what we have, which quote/unquote works. But if we're being honest about it, it's been inclined towards maybe a mediocre middling outcome and instead, pursuing in government new things that only might possibly work. But if they did, they'd be transformative. And I think AI honestly fits quite squarely in that spectrum, which is it's a little scary to move towards. We could avoid it and keep doing what we're doing, but I don't think we'd be able to solve the biggest problems that we have without it these days or certainly not as swiftly or as efficiently without it these days. But it also has its flaws. It's not perfect and organizationally and technologically it might not work.

And so I think if fits squarely in this idea that we need to chase after possibility, but also I write in the book that we need to move from probability to possibility, but not all the way to delusion. We need to avoid things that are wasteful, distracting, corrupt. And so, I think with AI, that's another important admonition, which is we want people in government to be exploring it, but to be exploring it responsible and productive ways. And I hazard to say right here upfront, there are massively productive ways to be deploying this in governments and there are hazardous and delusional ways, and we should chase one set and not the others. So, I think it's always you write these books and in some ways, they become…you hope they will hold the test of time. And I think this admonition to chase possibility but not delusion, even though the book's now about four or five years old, is as important now as ever.

Stephen Goldsmith:

I'll help you do your best not to be delusional. I think that's a good goal. It does feel to me though, like the first half of your definition, right, that the ‘art of the possible,’ ‘the imagination of the possible’ is often a limiting factor in technology. I want to come back to that. I do want to get to the technologies involved, but Santi, let's take Mitch's frame. You work in a City Hall where you've got a lot of authority and a lot of opportunity to innovate. I would suggest that the environment in which you work is highly unusual. So just talk a little bit about not your actual use of generative AI, but what is it that gives you the authority – other than the mayor herself of course, but how do you get the authority and how do you radiate that authority across the city as an enterprise? So take Mitch's We the Possible…you are the possible inside a City Hall that's known for that. How is that organizationally set up? How'd you get all that authority?

Santi Garces:

Yeah, I think I would start by saying that I don't think that I have a lot of authority and maybe that is what makes whatever authority I have be well used. So as we started thinking about the use of generative AI or other innovations, I think we start thinking about what is our mission to serve and what are the incentives that we have to deliver our mission? So I think that for us, there was this sense when generative AI left center stage November, 2022, there's this sense of ‘this is something that seems big.’ The number of people that were using Chat-GPT was growing faster than even what social media had done. So it was like, this thing is here and it's either going to happen to us or we are going to be able to be in a position of knowing how it happens and what do we want about it and what do we not want about it.

So I think a lot of people in technology and a lot of CIOs over time, for good reason, start building antagonistic relationships with their users, the people that they serve, because our job is to keep people safe and secure and maintain things that are reliable and innovative. Things that usually don't fit well with secure and reliable and scalable. But as precisely we said, let's engage people in this great learning experience. Because ultimately when people are like, “what is the greatest use case of generative AI?” And I tell them, “The person who's going to know what the best use case for generative AI in Parks is someone in Parks is someone who knows about the pains of what it is to run a Parks Department.” And it is our job to create an environment where people feel safe that have the right resources to do the right thing anyways.

So I always try to think that if we're doing the right thing for the right reasons and we're empowering people to make those choices, I think that it can be really powerful. So anyway, so that's why I think whatever authority I have is mostly just trying to stay inspired by Mitch's book. And what I love hearing from Mitch is there's also a failure of imagination and a failure of imagination to deliver services is just as toxic as all of the failures from bad use. It is harder for us to find someone who's calling out that failure of imagination in people. And we see it in translated into a political sense…people go and are willing to break things up because they get tired, they get bored. And it is our job also to try to push the limits and keep people engaged and try to figure out new ways of solving problems that seem intractable.

Stephen Goldsmith:

Thanks Santi. Mitch, let's talk about imagination a little bit more, but connected to generative AI. So I have this theory, I guess maybe not a theory, but hypothesis that Stat programs are outdated, CompStat CityStat, that they're important, but that they're outdated in the way they're managed. And one of the reasons I think they're outdated is because they are hierarchical in the way they were designed, right? A really smart Chief of Police, Bill Bratton, would tell his lieutenants, ask a bunch of questions and some data nerd would go kind of analyze and they'd come back and do it, at any rate. But generative AI, to be truly transformative, should allow the application of data to decision-making much more broadly in government. When we did our session late last year for all the large cities, you did this slide with kind of the personas of the bots. And to me, each one of those little icons on that slide showed how we could improve the data literacy and imagination of various people throughout the organization. So my question, which is way too complex, but I'm still excited about the slide, is how can you use these agents to improve imagination, not just improve the answers, but to improve the imagination challenge that Santi raised?

Mitch Weiss:

Well, first of all, I love the notion that we could use these tools to really empower many more people in the organization to ask really good questions of the data and then imagine new solutions from the answers those challenges generate. I love the idea that performance moves out of basically being a system that's delivered from on high to a formal quarterly or monthly even cadence and to something that it is everybody's and all the time. I think on the imagination front, that's certainly true. We can turn everybody in city hall into problem identifiers and problem solvers with the help of some of these tools. First of all, you can build AI and tools and bots and even agents. We could get into the semantics if we need, it's, I'm not really here to hype any of the particular nomenclature, but you can build AI tools that certainly would help you interrogate the data, like ‘look into this data and tell me where there are potential risk areas,’ or ‘look into this commentary from residents and tell me where they have frustrating sentiments’ and out of those risk areas, out of those frustrations, you as a city worker can begin to maybe even conjure up some ideas on your own. 

You can certainly, but also to your point Steve, enlist these tools as imagination helpers; if you were to give the tools a set of potential problems, they can brainstorm some potential solutions. They can help you create a set of questions for a hackathon. They can actually begin to get you to even prototyping, and these days all the way to practically building out an already working technological solution that you might deploy right away to see whether it's got any merit, any purchase. I mean the agents can help you on the imagination front in problem identification, in solution identification, even in prototyping. And so for us to move faster from “I think we have a problem” to “we actually might have a solution,” I think is quite important.

And the second point I would make is they can also just help you think more broadly. If you tell them, help me think more broadly, they will push you in that way. If you ask them to suggest ideas, which might be outside of the typical boundaries that we do or from this other field or from the perspective of Jane Jacobs or some other urban planner you admired, they can do that. If you ask them to tell you from the perspective of a 12-year-old living in the city, they can do that also, and not perfectly, but in ways that will do things like stretch your imagination.

Stephen Goldsmith:

I going to go back to Santi, but Mitch, your last sentence, I’m trying to figure out how generative AI would help with contextualization. I like the way you said, well, show me what this would look like through the eyes of Jane Jacobs or a 17-year-old who lives on the street corner. Give us a sentence or two more about how contextualization, and the way you said it, would improve city services, right? Because too often a city bureaucrat, and I don't even mean that in a negative sense, I just mean a person that operates in a set of rules, can't quite understand what life looks like, figuratively or literally in a community. So how could we use these tools for contextualization?

Mitch Weiss:

Well, I mean, so you can use them. You can create essentially synthetic focus groups of people and whether they're teenagers or whether they're small business owners or whether they're people who live on this street or that street and glean their perspective, you can make those better by augmenting those groups with real data that's rooted in those people's lived experiences and their behaviors. And you can try to gain some of that context even before you go out and meet with those groups in person. I would say, I don't want AI to substitute for our actual interactions with the residents who live in our cities. That would be terrible, but it can be a precursor. It can be a prelude. The fact of the matter is, as you referenced in government for reasons of resource constraints and otherwise, often the amount of people reached out to was zero.

And so this is better than zero. It's not better than people, but it can certainly be a preamble, prelude, supplement to that kind of outreach. You can go with a more refined set of questions and then get more out of your actions with the residents who live in the cities. And you can, as I said, augment that with real data from the kinds of calls that they've been making or the kinds of experiences they've been having based on your city. And really over time if you had for example, synthetic focus groups, you could refine and refine and refine and refine and it would learn how to be more like the people in your city and maybe be a place that you could bounce ideas off of before putting them the world and getting you and them in hot water.

Stephen Goldsmith:

That's a great answer…unfortunately, Santi, I have one more question for Mitch as a result of this great answer. So what do you think the city's responsibilities could be in regard to your last answer for helping…let's call it registered community groups? Like the official community groups use generative AI on open data to improve the quality of their insights. Because I think too often our community groups don't have the advantage of full understanding of the data before they are asked to express their opinion. So how could one use generative AI to improve the quality of community insight or even for the real meetings?

Mitch Weiss:

What's interesting, I mean, I would mention two fronts and Santi may have other suggestions, but the first is the data itself, right? If you make data available openly or it is available openly and you train community groups on how to use generative AI tools with data, whether it's just to be good prompters with it, whether it's to use tools that are the new sort of data analysis tools that are coming out in AI, they could make more use of government public data and generate their own questions and answers. I think that's one front. The other thing that your question makes me think, Steve, is if you also augmented models with essentially the policies of government, the strictures and structures of government, then the residents could also be more informed about the opportunities and limits inside of government to solve the problems of identify with the data instead of it being where there's this wall where citizens don't know what goes on inside city halls unless on occasion we deign to tell you because you’ve come to a community meeting, they could really have a much more fluid sense of: what are the policies of city hall, what are the resources, how is it functioning inside there?

And then begin to really act as almost true co-partners in designing solutions. So I think if I were really trying to design a new performance model in the spirits you're describing, I would start with a data layer. I'd want the data made available, but I'd make sure that that was not just city data in terms of ‘here's the potholes and streetlights and graffiti,’ but also the policies and the regulations and the org structures. And then I would give some training and tools on top of it to leverage both, and citizens could be really quite fruitful partners.

Stephen Goldsmith:

What a great answer. After all, We the Possible does exist inside some legal framework of rules, regulations, zoning, setbacks, other things that would help inform the discussion. Santi, I don't think anybody in the country has done as much as you have in City Hall on these issues. So we have a few minutes left and before we end, let's give you a chance to brag. What's the most interesting application or innovation you've done with generative AI in Boston City Hall?

Santi Garces:

I don’t know if I could answer that because you can't pick which of your children is your favorite, but I would mention two that are interesting and germane to what Mitch said. So, these are little experiments, but they seem very promising. So, one of the things from a performance management standpoint that we want to do, to Mitch's point, how do we use the synthetic agents? And the challenge is you don't want to replace people, but there are times in which getting data from people is unfeasible, unrealistic, but synthetic data…one of the biggest critiques about generative AI is that it won't produce things that are truly novel because they're going to just create different permutations of data that exist. But that's not a bad thing. And I'll give you two examples of that. So the first one is, and again this is kind of preliminary research, one of our team members, Sebastian Olascoaga, has been able to use synthetic agents to replicate surveys that we administer to our constituents and basically creating synthetic versions of the people that would take the survey questions and kind of replicating the demographic profile of who takes the surveys.

We are able to predict pretty well what are the answers that we would get from the community survey as a whole. So it's pretty cool, just showing that in some sense kind of these biases and these intrinsic information that the model has about how different people view and live seems to be consistent with the way that we experience people responding. But you can see how this might be really powerful when we're not able to gather data because part of the survey results were missing or we have a population that's hard to reach. Imputation and data analysis is a big problem. It's a big area of need and this allows you to do imputation that is a little bit more powerful. The second one that is very interesting, and this was done by academic partners of ours, so Esteban Moro who's now at Northeastern, but he was previously at MIT and his group have been using synthetic control.

So basically using synthetic agents to help us get better counterfactuals to be able to test the quality of policy. So when we want to see whether a policy worked or not, you want to not only see ‘what was the difference in outcomes before and after,’ you ideally are looking what happens with a group that got the treatment versus a group that didn't get the treatment, that was in the control. But often it's really hard for us to find really good controls because the example they're looking at is ‘what happened when the Mayor made Bus Route 28 free,’ they're trying to see what is the social impacts, but there's no bus route that's exactly like Bus Route 28. So what they were doing is creating the synthetic data out of data that they had from other bus routes to be able to amplify the signal so that we could get better counterfactual. So again, I don't know if they're the coolest, but I think that they're pretty cool about how is it that we're using data to get a better sense of really being able to measure and know whether the things that we're doing have the impact that we think that they're going to have. And hopefully that builds trust of people. And they're not maybe the first thing that comes to mind when you're thinking about generative AI, but they’re really powerful tools that in a broader context can be really, really helpful.

Stephen Goldsmith:

That was an interesting answer. You sounded a little bit like a Harvard Business School professor I think, but it was a pretty good answer. What you just described with synthetic data, is that different from, or the same or part of what might be called a digital twinning effort?

Santi Garces:

Oh my god. All right. How much time do we have? This is like a particular pickle of mine. So I think it is part of the world of digital twins, but I think part of the issue with digital twins is that they tend to overfocus into trying to create scenarios for things that haven't happened yet. And this is where the critique of GenAI falls similarly with the critique of digital twins. It's really…we would have never predicted what happened in terms of mobility, economic response about the pandemic without data that ever included the pandemic. It's just so unprecedented that there's no amount of modeling that would've led us to really get any accurate prediction. So I guess it is a digital twin to the extent that it helps us model how things were happening. But I guess the part that I'm excited about is it's not only about having a crystal ball to predict the future. That's interesting. It's actually having this ability to create multiple versions of the same universe to be able to see what worked in one and didn't work in the other that really helps us learn even without having that predictive power. And if you want, I'll spend another 45 minutes on a rant about that.

Stephen Goldsmith:

No, I don't think so. I think your last answer was sufficient, thank you. We'll do the other 45 at some other time. Mitch, a couple more questions to you. And then a final question to Santi. I know from listening to you, because of your keen intellect, that you could give us dozens of examples of potential applications of generative AI. It feels to me though, like we're still at the bell and whistle stage and not the transformative stage of generative AI. So if you were back in City Hall as Chief of Staff advising a mayor, even a mayor as talented as the Mayor of Boston, what would you advise? Understanding that there's all sorts of issues about privacy and security, which are not the subject of this podcast, but are legitimate issues. But what would you advise about how to unleash the power of generative AI to actually make government more responsive and work better?

Mitch Weiss:

I think I would say two things, and let me ground them in an example. I just returned from Abu Dhabi actually, and was meeting with officials there and their city government and more broadly on what they were doing on AI. And one of the use cases that they have out, this is public knowledge, is a product if you will, that essentially can ingest building plans and tell you whether they accord with the permitting rules of the city. And that output could be really helpful for people who are developing those plans, architects, developers, homeowners, so that they know ahead of time whether they're in compliance or not, and can also be useful for government officials so that they can get some help and aid in knowing whether they should render an application essentially compliant or not. This is a potentially transformative, to use your word, transformative use case.

We know what an amount of housing shortage there is around the world, housing affordability in all of our US cities. And so if we could more swiftly decide whether or not to build this thing, it could be actually quite transformative, this apartment, this building, et cetera. So my first piece of advice is if you want to do transformative things with generative AI, focus on transformative topics. In this case, like housing, housing affordability, housing creation. Okay. The second thing is their efforts on AI in Abu Dhabi have been enabled in large part by a more than decades long push to essentially organize and connect their data. So if you look at all the various data sets, it's either digital native or it's been made digital, and there are all sorts of connectors via API and otherwise, so that their AI systems that now sit on top and can relatively efficiently and easily access various data and pull different streams of data together. And in most cities, I would say that's not the case. We still have lots of stuff on hard to reach….maybe they're on databases that aren't connected to the cloud, maybe they're not connected to each other. And so the second piece of advice I would say to mayors, if you want to be really, really transformative in terms of AI, you've got to look at the base data layer and really work to make it more ingestible and make it more connected.

Stephen Goldsmith:

Thank you. Santi, let's take Mitch's answer and maybe make this the final question. So you're a CIO, you've been one in three different cities, you're now touching generative AI every day. When I was with you, I don't know, maybe a year ago you had outsourced coding to Chat-GPT. So on this issue of potential, give us an idea in terms of potential about how humans can gain opportunity by outsourcing the commodity to the bot and redeploying their discretion to solve real problems.

Santi Garces:

Yeah, I think, I wouldn't say that I outsourced my coding. I just maybe would admit that I was never as good of a coder as I ever wanted to be. And the issues that most people don't know that when you're not as good at coding as I am, you spend most of your time reading the documentation or going through stack overflow, you put trash out and then you get a bunch of errors and you just spend your time trying to fix the error. So you have some intention of the thing that you want to do, and then you crawl through the mud until you get to the other side. But with generative AI you have this amazing opportunity of being able to get working code. And again, I think to Mitch's point, when you embrace the possibilities partially where you can get even better results, so if you ask the tool to give you professional grade code, it often does a better job of structuring the code in the right way, making sure that you have good documentation, and that makes the code better to be used by other people that might have to review the code and then decide whether they're going to put it into use or whatnot.

I think that for someone like me who, again, I have experienced coding so I know what I want and how I would go about it and getting it, it's enabled me to build prototypes that then I can give to our developers and then they can take pieces of the code and whatnot. So if I understand the business problem to solve, I can get something that looks closer to the answer for the person that's actually going to have to put it and build it in production. And then we see people that have those jobs being able to do other things that are generally kind of tedious, but they're really important, like creating tests to make sure that the code is doing what we think that it should be doing, and then to build documentation so that other people can support and maintain the code. So I think that again, in the context of the pieces that the three of us keep talking about, this notion of agents, you don't have to substitute an entire person's work, but if you're able to supplement specific tasks, you can make it so that you're focused on doing the things that have a higher value and that also enable better interaction between people because the code is clear, the code is better documented, and those are things that usually take time and take a little bit of friction.

Stephen Goldsmith:

Thanks for the answer. I'm so old that I've seen all sorts of technological breakthroughs. When I was mayor of Indianapolis, I think we were the first city to put services up online. We actually owned the URL e-gov. And then when I worked for Mike Bloomberg, we had an opportunity to create the first (didn't work so well because it was too early) data analytics center of a major city. I think this generative AI opportunity is as transformative as anything in the last a hundred years. And there are no two individuals more at the forefront than you Santi and you Mitch. I appreciate your joining us today and for your enthusiastic leadership on behalf of cities. Thanks for your time.

Mitch Weiss:

Thank you, that’s your overly generous. Thank you. It's been fun to work on these things. There's lots of people around the world that are starting to, and it's been good to be connected and see what folks are up to.

Santi Garces:

Yes, thank you.