The Savvy CIO: Episode 8 – “The Goldilocks Zone: How Much AI is ‘Just Right’”

Dr. Swatee Singh: There are some things that you should never pass on to an agent, even if they can do it. And this is a bit of soul searching that most enterprises have to do. Think about the biggest pain point, think about how that work is done. Think about the new workflow, truly end- to- end. Don’t have a workflow which has agent, human, agent, human, agent, human. You really want, as much as possible, the agents go off and do a bunch of things as far as possible, but then the humans come in because that is still going to be a very long end- to- end cycle time versus if you could just redesign the workflow overall.

Bradd Busick: You’re listening to the Savvy CIO, modernized wisely by Park Place Technologies. I’m your host, Bradd Busick. This is the show where we talk about budget pressures, AI adoption, security, and the art of keeping the lights on without burning everything down with the people who are tackling these problems every single day. Insurance runs on precision, actuarial models built to withstand decades of edge cases, regulatory frameworks that leave no room for guesswork, and a culture where getting it right the first time isn’t just a slogan, it’s a matter of survival. So when the industry that invented risk management starts deploying AI agents, the question isn’t whether they’re being careful, it’s what careful actually looks like at scale. Dr. Swatee Singh leads data, AI, and corporate technology at TIAA. Dr. Singh has spent her entire career building systems that have to be both trustworthy and innovative.

Today she’s going to talk us through what it looks like when the most risk aware industry in the world decides they can’t just react to AI agents. They need to be the ones leading them. Dr. Singh, welcome to the Savvy CIO.

Dr. Swatee Singh: Thank you, Bradd. My pleasure to be here.

Bradd Busick: I was so excited for today’s conversation, particularly given the area that you’re leading in. Given that there’s no manual on how to run AI in corporate tech at an organization like TIAA, what was the moment that you realized that the old governance model that I think you and I have always adhered to from a CIO or IT leader perspective just no longer played? It wasn’t going to hold up and that something better was actually possible?

Dr. Swatee Singh: If I take a look back at the insurance industry and if I think about, this is an industry that has thought, ” How are we going to break, and how are we going to protect ourselves in the case of an accident, even before we start the engine?” Obviously machine learning, which was the precursor to everything that’s happening in AI is not new to the industry. As you said, we’ve been using it across financial services. It’s been used in risk underwriting, that was one of the first places. But then after that, frankly in every aspect, if you look at broader financial services, it’s pretty much pervaded every aspect of how they do business and how they do operations. So obviously it’s something which is not completely new in the sense of how to prevent client harm. It’s something which the entire industry has thought about for a while, but I think what’s changed now is just the speed.

So in the past, even five years ago, if you were about to build a new machine learning model, maybe you were updating your old fraud model in terms of how fraud is detected. The process was somewhat manual. There were people involved, there were slower gates along the way. You perhaps went to your internal model review management committee and the committee then looked at your model, at your input and figured out whether there was bias or no bias and you kind of chugged along. What’s happening now is to get the real value out of agentic AI, one has to think about true end- to- end workflows. So when you start doing that and as the agent is making decisions, every decision point in the past would’ve been a whole manual process. And that’s what’s making it so difficult for companies across the broader financial services spectrum to actually ensure that they do so in a controlled manner.

So the big change happened as agentic AI came into play. This was not even generative AI. In 2023, 2024, it was still the old processes worked. It’s when they really want to redesign work, when people want to think about how workflows will happen, that’s when the light bulb went on.

Bradd Busick: Well, and tell me how you’re navigating this, right? Because your role, which I love, sits across many functions that used to have really distinct and separate rules of engagement.

Dr. Swatee Singh: Yep.

Bradd Busick: How has the convergence of these pieces changed the way that you’ve even organized your own team?

Dr. Swatee Singh: I had the benefit of starting at TIAA just a few months before the ChatGPT phenomenon-

Bradd Busick: Explosion.

Dr. Swatee Singh: … came and the whole explosion happened in November 2022. Obviously in the research community, I had been reading papers. I think the previous version of GPT had been out, at least in my memory, at least a year, year and a half. So we knew that there was big things coming. This was fundamentally changing how natural language is processed by these systems and meaning is understood by these systems. I think what Sam Altman did in terms of making it go public for the whole world, it made you realize the volume and the vastness that is going to come hit us. And one of the first things I did, Bradd, my team heads the AI Center of Excellence and it has the client tech labs and a few other things underneath it, but we think a lot about client- facing AI applications done safely.

So what we immediately started working on at TIAA, and we were amongst the very first few ones, remember what I talked about machine learning error in the past and how there were all these manual processes?

ブラッド・ブシック：そうですね。

Dr. Swatee Singh: Each of the different departments within an organization looked at machine learning risk individually. So IT risk had its own policies and they would think about controls independent of say cyber and cyber risk, and they would think independently of privacy and maybe ethical risk. So one of the first things TIAA did was got these, what we call control partners. So these are folks at TIAA who, from various perspectives look at a problem and say, ” What’s the risk associated with this?” So this would be cyber, IT risk, this would be MRM, et cetera. So what we did was we got all these control partners, privacy, ethics, et cetera, along with the COE, and we drafted a single policy at the company level, at the enterprise level, and we called it our Responsible AI Policy.

Bradd Busick: Love it.

Dr. Swatee Singh: So it didn’t try to redo what, let’s say, maybe IT risk had done previously, we referred to it, but it actually standardized the definition of risk across the company. What is low, what is medium, what is high risk? And so today we think collectively, I joke in the company, it’s a King Arthur’s table in the sense that there is no head of the table, but everyone has veto rights. But at the same time, everyone comes with a sense of responsibility to the table knowing that they’re coming in as deep experts within their field and they are trying to solve the problem as a single unit.

Bradd Busick: I think about a lot of organizations that it’s not King Arthur’s table. It’s almost like the Last Supper where there is the head of the table and someone is making the call. And when I think about this idea that insurance has spent decades building guardrails for human decision makers, I’ve got my underwriters and they do what they do and I’ve got adjusters and they do what they do and then the claims team. But now again, with your leadership, it sounds like the industry’s starting to figure out what it looks like to let agents navigate those lanes and systems autonomously. And it feels less like a widening of a road and more like discovering that the road that we were riding on before was built for horses and now I’ve got really fast cars on it. In practice, what does that look like when AI agents are deployed with the same rigor that insurance applies to everything else? And what does that actually look like on a day- to- day basis and how do you guys make that possible by way of measurement and governance?

Dr. Swatee Singh: It requires an investment at an enterprise level to build enterprise capability. So here’s what I mean, and I’m going to take it through the journey of agents as it came along. So initially, as I would call, the blast radius was relatively limited. So maybe the agent was going and fetching some data. Usually that was one of the first use cases in most companies where there was a set of unstructured documents like maybe PDFs, Excel files. Folks had a lot of data insights and deep expertise sitting somewhere in their ecosystem, and all they wanted was to bring the insights to the user that makes the business decisions and to bridge the gap between the two. So what would’ve happened in the past is the business user would’ve said, ” To make this business decision, I need to know this, this, this.” Then the analytics team would start working on it, perhaps work with the tech team, build a dashboard, build a report.

And then after a few weeks, the business user would get an input and say, ” Ah, so I can enter this market.” Or, ” I cannot enter this market.” Based on what the data is saying. And what really the holy grail of course is how do you bring those insights to the person who’s making the decision in natural language so that all these hops in the middle and all the other handoffs that happen in terms of process approvals get knocked out of the way. How we think about this, and I think generally that’s a best practice in the industry at this point is, first of all, we treat every one of these agents, whether they’re external agents, so these are agents that we have bought or partnered with companies to get into our ecosystem or agents we’ve built internally, we think of them as untrusted digital workers, not just plugins.

So first of all, you have to recognize that they can independently possibly operate. What I would also say is as a company, we’ve evolved our thinking based on how the tech has evolved. So beginning of this year on Jan 1st, 2026, if you would’ve said, ” Swatee, as we think of these agents, and okay, so they’re untrusted. How do we think about what do they have access to? What data do they have access to? What tools do they have access to?” I would’ve said whosever agent this is, it should just inherit the same rights. So if Bradd has an agent, then if Bradd has access to three data sets and whatever other decisions, the agent should be able to do that. But what we’ve come to realize is, actually no, it needs to be much more narrower because it has to be task specific. So what does that mean?

Practically that means that at an enterprise level, now we have to think about… So just take these two things, digital workers access. Now it means in the backend, HR systems that think about how we think about our employees, they have to think about how do we think about agents? Tech systems have to understand, they have to have a registry of all the agents that are out there, so they need controllability of the environment. Tech systems need to build an entire identity and access management system for the agents separate from the humans. It needs to have its own review cycle and its own control measures such that they don’t go ahead and act in ways we didn’t expect. We have to think about, and this is the most important thing at a company today if they’re looking to get value out of AI, is how do we put the guardrails where the context is constrained?

So the contextual layer, some people call it the orchestration layer, the layer in which the entire context of the company sits and these agents are operating in. So Bradd may have the context and he knows why he’s asking for some data. So I want to prepare for an upcoming client meeting, which is where we discuss strategic topics and here are the three topics I’m looking for. That may spawn off a few different agents. Each agent may not realize what’s the context? ” Why am I being asked to go fetch certain types of data sets, let’s say, about your client and our relationship with that client, but the contextual layer does know it.” So how do we prepare for it? I think you have to think at an enterprise level, build enterprise capabilities for keeping track of what agents are there, what access do these agents have, what context will they have?

You need to build capabilities that observe these agents, observability and telemetry tools that can have kill switches that can stop agents when guardrails are crossed. You need LLMs as judges, so you need agents judging other agents. So a whole ecosystem around that to say, ” Did this agent end up doing something you didn’t want it to do?” And in all of this, of course, you want to do this at breakneck speed because that’s where the industry is headed. So how do you create an environment and processes where you can prototype very aggressively, but you deploy into production with very tight parameter controls?

Bradd Busick: Yeah, that’s so well said. I am really thoughtful of all of the CIOs and IT leaders that are listening to this in their cars or at the gym or out walking the dog who are probably thinking, ” Holy cow, Swatee’s done the work on the structural side with humans to ensure that we have success.” You’ve actually done the work on our back side of the house, so I’ve got role- based access controls and observative ability. However, when you zoom out, if you adhere to this idea that today’s the worst it’s ever going to be, what’s the thing that actually earns trust at TIAA with these autonomous agents and systems? How do we actually know that we’re moving the needle and at what point do you guys say, ” Yeah, let’s give it a little bit more. Let’s expand that blast radius?”

Dr. Swatee Singh: I think to build that trust, firstly at an enterprise level, hopefully your control partners that I mentioned like risk, cyber, any AI, COE equivalent you have, privacy, ethics, legal, all of these control partners have to be co- architects of these technology platforms that I mentioned, such that it’s seamless, it’s behind the scene. This is where you will hear things like governance by design, policy as code. It is not about, ” Okay, you’re going to have to get an approval email and now it sits in somebody’s mailbox, you send six reminders, then they look at it, then they ask you a question, then something gets approved.” So it is inbuilt into the ecosystem. I think that’s number one.

And number two, you have to have observability. And of course, then in the end, all that observability has to result in some telemetry and some reporting for folks who can look at trends over time and see what’s happening.

Bradd Busick: When you’re building agentic AI on top of an environment that still has legacy tech, I mean, insurance, as you and I both know, is a lot like healthcare.

Dr. Swatee Singh: True.

Bradd Busick: The last time we had a full CapEx cycle was like 1987, and so we’re running off of just really antiquated technology. When you think about that relative to this new foundation you’re building on top, where’s the biggest opportunity to modernize both the environment and the organization at once instead of them treating them like competing priorities?

Dr. Swatee Singh: I think it has to be around data. So when you mention all that legacy tech debt that exists in most companies, any company more than 20 years old probably has some level of tech debt. And if you’ve been a hundred years old or more like us, then you have tremendous amounts of tech debt potentially. But hopefully like us, you’ve been slowly chipping away at that debt through programs, et cetera. But I think in the end, it does come down to that data component, at least for me, because how much control do you have in your data? How good is it? Is it well documented? And by that I mean traditionally, again, five years ago, Bradd, I would’ve said for any company having a good metadata, whatever tool that you use, having good metadata, which tells you what is this data, who owns it, and is it an integer? Is it a text? What is it?

But today, if you haven’t built a data ontology on top, so data ontology is what’s going to tell us or is going to tell the agent the context, which sits in my head and your head if we are employees of a company. So yes, this is somebody’s date of birth, but you use this for these purposes only and not these other purposes. For these other purposes, you don’t need date of birth, go use something else, as an example. If you don’t have that ontology built on top of your data and you don’t have well- governed data where data quality, data ownership, fixes to data quality issues is something that’s happening on an ongoing basis because data problems never end, they’re ongoing. So if you don’t have that backend and you haven’t built the new backend needed in the world of agentic AI, which is around data ontology, and of course the security model around access to data, I think it’s all going to come to a screeching halt.

ブラッド・ブシック：その通りです。

Dr. Swatee Singh: So you will prototype very fast, and then it’ll be the classic thing that happened with all the machine learning models. Remember, I think back in 2015, there were talks that I went to in conferences where people said building the model isn’t the problem. Now the problem is pushing it into prod. And most companies were averaging between two, three months to as much as six to nine months to deploy models into prod. Where did all that lag come from? That lag came from A, the manual approvals through the various control partners, but B, also by data, building the right data pipelines.

Bradd Busick: As you think about the audience that’s listening to us today, what would you say to a CIO to convince them that maybe going wide and structural, like what you guys have done at TIAA is the right call instead of providing value or trying to do innovation spikes in little pockets, what can you gain by going bigger from the start?

Dr. Swatee Singh: I mean, you will end up frustrating your broader organization if you force them to go through this beast that needs to be built in some ways. So what we’ve done is we’ve created experimental environments. So this is where, as long as you don’t use PII or other constraints, you can start a POC within hours. So you have an idea, maybe you found an excellent external vendor partner whose tool you really like. Maybe it’s an existing vendor that you’ve had a relationship with for 10 years and they have created a new capability you want to test out. We have test environments that are isolated and safe for them to go experiment. We give them 30, 60, 90 days to go ahead and experiment and figure out, ” Is this valuable?” So what we have observed is as soon as we allowed that, that took down, frankly, a lot of the noise around, ” Oh, this is process again and this is going to stop us.”

Because as people experimented, I’ll give you real examples. There were people who experimented with external tools, which even a year ago, maybe a year and a half ago, some of these language models and vision models from Frontier Labs were terrible at extracting data from a table or a chart that is embedded in a document. They hallucinated more and God forbid, if that thing broke over two pages, it handled that page break really badly. So there were folks who said, ” Oh, but that is such a key thing when we have multi- page contracts as an example. So we’re going to look at this other tool and we really want to bring it into our ecosystem because that’s a key enabler for us to unlock what’s sitting in all these contracts.” Okay. And they were doing POCs, they really liked it. And kid you not, I mean, while they were in the middle of doing this, these models came up with another iteration.

Bradd Busick: Totally.

Dr. Swatee Singh: Fix these issues. They were like, ” Oh, we didn’t need to do that.” So dumped the POC, moved on, said, ” We don’t even need it because now this is out of box.” I think if you don’t allow for that innovation upfront, it’ll create a lot of frustration. So you need to have safe experimentation and then only when you’re ready, only when you know that there is true business value. Because the last thing we also want, and we’ve all gone through this perhaps in the last year or two years across the board, is that we had cool ideas, but they had no business value. So allow all of that to happen upfront before you put all this process in place, before you tell them, ” We are going to have these controls in place.” And you have to define the guardrails and access controls. None of that is a conversation to be had unless they’ve gone through those filters.

Bradd Busick: We’ve talked about this idea on this podcast and I’m going to send you a shirt with the acronym that says CBNU, which stands for, Cool But Not Useful. And in the early days of this work, you and I saw so many CBNUs. I mean, it’s hard to even quantify it. If you had a CIO that wasn’t sold and is worried that agentic AI is going to be a compliance headache waiting to happen, or they’re going to end up on the front page of the news and they’re staying cautious and waiting to see what happens, what would you tell that CIO? I mean, what separates the technology leaders who translate technical risk into something a board can actually act on from those that struggle to make that case or just want to wait on the sidelines and see how you and I do in this space?

Dr. Swatee Singh: Yeah, sure. I mean, first of all, I’ll say hopefully everyone has recognized that the real value is not going to come just because you enable some version of a Copilot, whatever your favorite tool is, if you enable that, that’s not going to give you any productivity. That’s going to give you maybe faster speed, but that’s really going to be hard to quantify, hard to say what’s the business value of this? So that’s not where you’re going to get it. You’re not even going to get it just because you put a chatbot in front of something. So maybe again, it creates slightly better customer experience or even internal employee experience potentially, but that’s not really what’s going to drive value. Value would be what this moment enables us to really look at. What is the company’s biggest pain point, internal or external? And really look at it end- to- end.

So work redesign has to be a key part. We all started with, ” Oh wow, I do these things and you know what? I can actually take some agentic, or even back then, a basic generative AI component and I could actually stick this in my overall process and it makes things better.” But again, value was very minimal. The real value is over a few iterations we’ve learned where you keep humans either in the loop all the way in the end and make agents, allow them to go as far as possible safely without any human intervention. And then you have whatever decision making that the human has to do at the very end, that makes for much faster end- to- end workflow. So hopefully you’ve begun with redesign process, but then in the redesign process, there are some things that you should never pass on to an agent, even if they can do it.

And this is a bit of soul searching that most enterprises have to do and understand, what is the core of their business and what do they want to keep? They think of it as just simply operations in the backend versus perhaps some client facing components which they want to keep with a human touch. So think about the biggest pain point, think about how that work is done, redesign that work, think about the new workflow truly end- to- end. Another lesson learned is don’t have a workflow which has agent, human, agent, human, agent, human. You really want, as much as possible, the agents go off and do a bunch of things as far as possible, again, safely, but then the humans come in because that is still going to be a very long end- to- end cycle time versus if you could just redesign the workflow overall. Hopefully as you try to do that, you are going to see the value.

We are also looking at all our AI programs at an enterprise level. So we have a structure called APEX, AI- powered excellence. So across the enterprise, we look at all these opportunities that are coming up through sort of a, I wouldn’t call it single lens, it’s a highly federated environment, but it allows you to see what’s happening across the company at the executive table. And I think that structure has really helped us because lessons from one part of the organization have certainly permeated very quickly to the other. We’ve started seeing lots of trends. Some things perhaps should never be agentic. I mean, that’s just a classic case of, ” That’s just automation. Let’s just go do old school RPA on that.”

I think if all of us think of it a little bit more holistically, because when something is new, everyone wants to try their hand at it, they have great ideas, you don’t want to damp their enthusiasm, but at the same time you don’t want a thousand experiments that are low value, take up organizational capacity and the air in the room and yet yield virtually nothing of importance.

Bradd Busick: Nothing of value. I agree. Dr. Singh, super grateful for your time. Thank you so much for joining us today on the Savvy CIO.

Dr. Swatee Singh: It’s my pleasure, Bradd. Thank you for inviting me.

Bradd Busick: I think so much of what Swatee said is important, but here’s some of the biggest takeaways that I think I’m walking away with. I know many of the IT leaders that are listening today are walking away with. First, she described this awesome image of King Arthur’s table. Take that image of all the thought leaders in your organization and imagine what kind of table your leaders are sitting at. Are they facing each other? Is it rows? Are they looking at the backs of each other’s heads? I think in the world of AI, the Savvy CIO understands that King Arthur’s table is probably the best design. It facilitates thought partnership. It facilitates engagement. It facilitates buy- in. It also facilitates conflict and intellectual dialogue.

The second piece that she brought up, what I think is super important and most of us miss, which is you can go deploy any model that you want and most just stop there. TIAA has taken it a step further with observability, measurement, and the identification of high value pilots that then became enterprise operations. So these agents aren’t just working in a vacuum, they’re actually integrated into the workflow and they’re constantly being measured and governed. And I think that’s the difference between good and great.

That’s all for today. Thanks so much for listening. Please follow us so you don’t miss an episode. This has been the Savvy CIO brought to you by Park Place Technologies. If you want to learn more about Park Place, go to www. parkplacetechnologies.com. And now, one last word from our guest. This show is called The Savvy CIO. So Swatee, what is the savviest choice that you have made in your career to date?

Dr. Swatee Singh: I think I picked the right team to work with. So who you work with is the most important decision you make.

Bradd Busick: Spoken like a true servant leader. I’m your host, Bradd Busick, and as always, IT shouldn’t just be at the table. IT is the table.

Swatee is a hands-on technology executive with deep expertise across AI/ML/GenAI, Cloud, Blockchain, and enterprise-scale data and analytics ecosystems. In her CIO and shared services roles at major global firms, she has consistently delivered revenue-generating, customer-centric, AI-powered applications while leading strategy, technology development, and execution end to end.

A recognized thought leader in Data, AI/ML, Cloud, and emerging technologies, Swatee has filed or been granted more than 50+ U.S. patents. Her contributions also span numerous industry presentations, panels, and published works.

Currently, she serves as the CIO for Data, AI, and Corporate Functions at TIAA, where she leads the development of enterprise-wide data and AI strategies with measurable business impact. She is focused on building industry-differentiating, next-generation capabilities in data management and GenAI. In her CIO remit, she also oversees technology for Finance, HR, Legal, Risk, and Compliance, transforming these domains through automation, GenAI, and modernized platforms to streamline operations and reduce costs.