BCG x Hyde: Scaling the Enterprise AI Adoption Curve

Akash Bhatia leads BCG's global technology sector. Anirudh Sanga is co-founder and CEO of Hyde. They sat down in Hyde's New York office to talk about where enterprise AI adoption actually is, why cost is becoming a board-level problem, and what it practically takes for an enterprise to own the intelligence running its business.

Ani: Thanks Akash for being in our offices. Welcome!

Akash: My pleasure. Thanks for having me.

Ani: Let's start with a quick round of intros. Would you like to introduce yourself?

Akash: Sure. I'm Akash, out of the San Francisco office. I work for BCG. I lead our tech sector. Tech for us means hardware, software, and services. I've been doing tech personally for over 20 years, mostly focused on enterprise software. And nowadays it's all, as you can imagine, AI, AI, AI. There's no other topic.

Ani: We're super excited to be partners with you and doing some of the work together.

We've been having this discussion on and off every time we meet, on where enterprise AI is going, what enterprises are thinking about right now, what owned intelligence actually means. So maybe I could ask you what you're seeing across the spectrum of clients you work with. Where are they in AI adoption? What does owned intelligence mean for them?

Wide spectrum of enterprise AI adoption today

Akash: Let's start with the adoption piece, because I think that's an important one. I'm glad that we're doing this in New York, which is a little bit outside the bubble.

If you think about adoption, first of all it's very dispersed. It's not uniform. If you just talk to the AI native companies, which I do a lot, it's very concentrated. Everyone's red pilled, et cetera.

If you look at a normal enterprise company — tech or software companies, or retailers and banks and things like that — most of them are pretty nascent in their adoption. And what does that mean? Obviously everyone's been on this AI journey for a while, but most of them are probably at the stage of, okay, I have Claude Code or Claude Cowork installed on every desktop now, or OpenAI, or whatever the tool of choice is, or Copilot. I think that's there.

People are starting to see what I call individual productivity. That means Bob is 10% more productive because he or she can do something with their emails. They can summarize stuff, maybe create some documents. What I don't see is uniform functional level productivity. Is my whole sales organization much more productive now? Is my engineering organization much more productive now? And then if you macro it out even more, is the organization as a whole much more productive? Can I go faster, higher throughput? No.

To summarize it: first of all it's pretty diffused. But it's also that the productivity is at an individual level rather than an enterprise level.

Ani: Across the spectrum, we work with a lot of AI natives that are at the frontier. If you think of the AI spectrum as a four-year evolution curve, they've been building at the frontier for the last four years and they're doing certain things right now across the AI stack, how they're deploying AI in their company, how they're building products around it. And then there are people on the other side of the spectrum that are using it for individual productivity and maybe starting to scratch the surface of what organizational productivity looks like, or what a workflow that is completely done by AI looks like.

Most of the companies on the left hand side of that spectrum are still in the application layer and they're using applications that are out there. These are point solutions, your Claude Codes, et cetera.

What's the next step from using Claude Code, or being power users of Claude across the organization, to then taking that step to organizational productivity?

Failure modes of AI adoption

Ignoring cross-functional workflows

Akash: I don't know if there's a clear path that people see. The way that it's practically done right now is the CEO will say, "Hey team, we need to come up with some scenarios. We need to come up with some use cases." People come up with use cases. People prioritize the use cases.

One failure mode is — not to pick on IT — but they will send it to IT and say "here are the use cases, can you please go implement?" I don't think that's the right approach. But in general they'll come up with some use cases in sales and marketing and finance, and then say we're going to be working on them.

The realization that people have is, one, what I talked about: the cross-functional productivity, the cross-functional workflows are not present. People really don't think about that. There are some companies out there that are trying to tackle that, some open source, some proprietary.

Blowing-out cost

Akash: The other one is people are starting to realize cost. This thing costs money. Up to this point, maybe if you had the conversation six months ago it'd be like let's just get Claude Code in there, let's just get OpenAI in there. And now people are realizing, oh my God, some of these costs are actually fairly significant. And some of the numbers are pretty big. For the first time you hear people saying, where do these costs go?

I'll give you a very practical example. I had this one friend of mine who works for a tech company and he runs a large portion of the go-to-market operations there. He said the cost is pretty massive for the company. It's a big tech company. But right now it's centrally funded. Next year what's going to happen is they're going to allocate the funds to him and say, either you get rid of people and then you use those token costs — you have to pay for it somehow. That was his point.

He's coming to a realization and he's like, dude, I don't really see the productivity here. So that's one realization. Cost is an issue, no question. How you put it into your budget is a very, very real thing. And how can people help control those costs?

Maintaining control

Akash: The third one, which I think happened because of the Fable thing, the famous 19 days — it's off the market, now it's back on the market. People are like, okay, do I have control over this? Why did they just stop? They stopped giving me access to this "intelligence." What do I do?

What I see, and I'd love to hear your perspective, is that AI natives are like you said a year ahead of everyone, or maybe two years ahead of everyone. But for normal tech companies, and God forbid normal enterprises, people are starting to realize sovereignty is an important thing. Control is an important thing.

Why enterprises should own their AI models

Ani: It does answer the question, and I think there's a critical path of evolution here. Irrespective of where the company might be on that evolution spectrum, they do take the next step, which is deterministic. If you're moving from an individual productivity agent that everyone in the org is using, you would then start optimizing different layers of that, or maybe start thinking about entire workflows you can do with AI.

The companies that we work with — we started Hyde a year ago, and back then the only companies that were thinking about training their own models were the ones already feeling the burn of depending on the frontier model API. So the same things that you mentioned: cost, sovereignty, having higher control over the derivative of your own growth.

If you pin your growth on the evolution of the underlying model or the underlying technology, you're not really fully owning your growth, and there's a lot of dependency risk inherent in that.

AI natives started doing this maybe six to nine months ago, and we're seeing the early stages of that happen in enterprises as well. The early adopters of AI in every vertical in the enterprise are thinking about this now.

We worked with a hedge fund and they are going as far as owning the entire stack. They're buying their own GPUs, so they're going to be paying for electricity, and everything else will be insourced. That's an edge case, because in the hedge fund world you really care about alpha, that's everything to you. The concept of depending on an external API provider, external intelligence provider, or even hardware provider is very dangerous to them.

But some part of that actually applies to every enterprise. You do want to own the intelligence that's running your business and giving you competitive advantage, or the right to win in the future.

Akash: I guess it depends on the where. Again, not to pick on HR, but if you're in HR and you have to optimize the process and you're a hedge fund, maybe that's not the thing that's going to give you the alpha. Maybe you don't need to own your intelligence in that space of it.

But if it's core to your business — it's core versus context — if it's something core to your business, it's what makes you special, what differentiates you, then I can see that.

How do people actually practically do that? From what I see, most enterprises are pretty far removed from that. That's my sense. Maybe I'm being mean to people, but my hunch would be AI natives, or even the hedge fund example that you gave, by definition they'd be able to attract really, really smart people, pay them a lot of money and tell them there's a bunch of data, go. How does it happen? Can you explain how they actually do it practically?

Compute and talent can be bought. Data cannot.

Ani: Model training is a function of this famous line: compute, talent and data. Compute and talent can be bought. Data cannot be bought. The most valuable data for training actually lives inside enterprises, especially big enterprises, because they have a long history of winning in their industry. All the decisions that led to that are captured in some tacit knowledge base somewhere. Maybe these are onboarding manuals for new grads or new hires, maybe these are historical memos presented by the execs at some point. But all this information lives somewhere in these organizations, and that is the source of alpha for them.

Historically it has been in the AI native world where you're competing on model capability rather than human decision making capability. The same source of alpha actually applies.

So how are these companies doing it? They've got talent. They have been building their own training rigs, their own MLOps.

Akash: Can I stop you for one second on that? What kind of talent do you need? Very practically — is it the gajillion dollar pay packages that Meta was paying for people, is it that caliber of people that you need in order to do this? Or is this mere mortals, or somewhere in between?

Ani: Three years ago maybe you just needed a PhD researcher who was very deep into the transformer architecture to train these models. More and more there's AI leverage in building these models that's becoming available as well. Part of what we build into our platform is to package that AI leverage so that a median data scientist sitting inside a Fortune 500 today can actually train their models. The talent bar is, you don't need the researchers anymore. You can have just data scientists or ML engineers do this.

On the compute side as well, there's been massive leverage that AI and some of the libraries that have come out recently have given you in making training cheap. And because training is cheap you can make it train frequently, and you can eventually get to continual training where every time a new edge case is encountered or a new piece of tribal knowledge is captured by the business, you can convert that to the weights of your own models.

So the bar is coming down, and it's becoming more and more of an approachable technology for enterprises now.

Mining the mess

Akash: Interesting. So the data is internal.

First of all I agree on the comment about — having seen so many enterprises from the inside — the data is messy, to say it kindly. It's all over the place, in people's heads, it might be in notes, it might not even be written down. Given the messiness of the state, how do you actually collate it? How do you mine it? Do you go into companies? How does it work?

Ani: That's the tradecraft that we've built over many cycles. You just have to do it and get enough reps to actually understand how do you build a good training data set? How do you extract that knowledge? Where do you need human tagging versus just structuring existing information being enough?

Broadly speaking there are a few steps that we boiled this down to. Number one is observe: how is the company using AI today? There's a ton of really valuable information just in that. The prompts that are going out to frontier models, how the models are responding — you can mine these traces to create a set of tasks that a company is using AI for.

That itself is massively valuable for an enterprise that's at the stage you mentioned, where they're using these applications for individual productivity or for workflow automation. Because now you're not just seeing the overall spend, you're seeing the contours of that spend. What are the 5,000 tasks that my company really cares about using AI for? And then I know the scope of AI for my company. I don't need a trillion parameter generalist model to do those 5,000 things. I just need models that can do those 5,000 things really, really well — maybe the best in the world.

Akash: Just so I understand, is a task equivalent to a prompt? It's like, what's the weather going to be in New York and I'm using Fable for that. It's hot. But is that a task, or is that not a task? It's a collection of something?

Ani: A task would be one level above a prompt. You asking for weather, or directions, or your flight schedule for when you're flying back to SF — all of those would be under a task category, and that's what we'd define as a task.

Akash: So different people in the organization could be looking for flight stuff and then you would say that's a task. I'm just mentally trying to understand it.

Ani: That could be a task, or a calendar lookup could be a task. So you're looking for flights, you're looking for meetings, you're looking for holidays. All those things get grouped as: the AI is looking at your calendar and giving you some response back.

Akash: Got it. Sorry I interrupted you. So you look at the traces, you form these tasks, and then what do you do?

The model router

Ani: That's one way of forming tasks, and most companies already have this data today. If you can build that task set you can effectively start baselining your AI. So you can start seeing, okay, for 5,000 tasks that I use, 2,000 of them are using Fable which don't necessarily need Fable. And because you have the tasks and you have the accepted responses, you can start testing incrementally smaller models, or the smallest possible model that will give you the same performance. That gives you this right-sizing notion, where every task is routed to the exact right model, or the smallest possible model that'll give me the right performance for that task.

That's what we call a model router, and we can deliver that in two to three weeks to an enterprise customer just by looking at their existing traces. And that reduces token spend by over 50% immediately.

Akash: That's huge.

Ani: It's huge. For large customers that could be tens of millions of dollars.

Akash: So just to be clear, 50% token costs — is that still 50% dollars as well, or it could be?

Ani: It would be 50% token cost in dollar value, not in number of tokens. In dollar value of tokens.

Akash: Three weeks, you have 50%.

Ani: Everyone is overspending on AI right now. Because the exec function that's owning the budgets and owning these relationships with the model providers, they're not the ones deciding which model gets used. Those are the people at the front lines, and if I'm a coder I would want to use the best possible model for whatever task I'm doing because I really care about the outcome of the task. I don't really care about the minuscule spend that's going into Fable from my task. But it adds up over thousands of people doing the same thing in an organization.

The first thing that we do when we go into a new customer is baseline intelligence: see what are the tasks that this company is doing, and what is the right model to execute that task with a high degree of accuracy but in a way that's economical.

Akash: One question, if you don't mind me asking. There are companies out there that have "router" and "open" in their name that claim to do model routing. How is it different to what you're talking about? Should I get — I'll just say it — OpenRouter and then plug in my traces into that? How would that work? Is it different than what you're doing?

Ani: Ours is much more tailored to the organization. We actually deploy in the customer's VPC. We ingest their traces. We create the baseline intelligence stack that is tailored to them in a way that is data driven implicitly, because you're looking at their traces from their last n months of data. And then the router you get as a result of that is the exact router that they want.

The commodity products do that over a large corpus of data, and it's opaque why routing is happening to a certain model versus another. You still don't get that confidence score in why my task is getting routed to model x versus y.

Akash: Quality would be better, right? If you're doing it your way because you've trained it on your enterprise, that quality would be better. I think I get that.

Ani: The quality would be better. You get more control. You actually own the router. You can change things manually if you wanted to. You can also trace down to the task level why a task is being done by a certain model versus another through the router. So a lot more owned by the company, rather than delegated down to a model that you don't control.

Training your own models

Akash: Then on the whole train your own models — talk to me about that, because the one fear that you hear is, okay, I'm going to use Kimi or GLM or whatever. It's a Chinese model. I don't feel safe with that. But now you have American-based open weights models as well. How does that work? Is that related to what you just talked about? Is it a completely different exercise? How would an enterprise do it, what are you seeing?

Ani: There are really strong open models now, and post-training them with private data gives you massive outperformance compared to open models or closed models, because you're training them on private data. So you're hill climbing the hill that you really care about, and that hill is private because only you know about it. The frontier models are just not trained to do that.

It's a natural progression from what I just said. You baseline your intelligence, you figure out the tasks that you care about, you create the sense of correctness of what is the correct output of each task that my company cares about and what's the correct process of getting to that. This could be a multidimensional thing of correctness. And then you can train models to be really good at those tasks. That's how you start insourcing intelligence versus using frontier models.

Akash: That's different from just routing. Routing is picking between Fable and Opus or something like that. And then this would be just training your own models.

Ani: That's right.

Akash: How long does that take?

Ani: A few weeks.

Akash: That's it?

Ani: We've built a lot of tooling to make training efficient, fast, and non-clunky in the way that a data scientist would manage the training pipeline and the environment. A lot of that is delegated down to software, and the data scientist workflow becomes a little bit higher level, where what they're doing is preparing the environment. Your evaluation data sets, your evaluation rubrics — the sense of correctness — your reward functions and so on. And then you give it to the platform, spin up GPUs.

Akash: Is it like vibe coding? Is it that level, or is it not that level? It's not like me as an idiot vibe coding an application? Someone still needs to understand the innards of model development, model training? Or is it really at the Akash stupid level?

Ani: It's very close to being a seamless experience, but it does expose all those controls. It comes with really good defaults, but you can also open the hood and change certain things if you're an ML nerd like me. So it gives you a spectrum of capability in terms of technicality of interaction with the thing.

Longer term it would just be a CLI tool for a Claude Code to use. You can imagine a future where ML engineers are AI agents, and they're using your training rig as a way to on-demand train models as new evaluation data sets are being generated, or the business is learning new things.

Akash: I was trying to get to the kind of human that an enterprise would need. You're saying it's a median human ML engineer, not a PhD with 700 degrees?

Ani: The ML team that already exists in companies can very easily use tooling to train models now.

Why not a neocloud, or a boutique consultancy

Akash: Can I ask about this? It's super confusing and there are so many companies doing this in the market, maybe you can help me understand. Without naming names again, you have neoclouds that are saying hey, you can run workloads on my GPUs, but they're all increasingly getting into post-training — "I can custom train a model for you." There are also other AI native companies going around, almost like AI-native boutique consultancies, effectively going in and saying we can do that for you. And then there's a mixture, and I'm sure there are others in this mix as well. It's a very frothy space, as you can imagine.

How is your approach different? What's different about it? Why wouldn't I just go to a neocloud and say okay man, you guys know this stuff, you guys do it — or to a boutique consultancy? What's the difference?

Ani: A couple of things. One is we deploy in the customer's cloud, so we're not selling them the entire stack of buying GPUs that we're going to make 80% margin on. We're not saying just give us access to your data and we'll come and build everything. We're providing a software layer that makes training extremely efficient and meets the enterprise where they are.

Where most enterprises are today is they have a ton of traces from where they've been using frontier models for a number of use cases. They may have some sense of what use cases have hit product market fit for them, that they want to scale up. That's where we meet them. We deploy our product, but in their environment, so nothing leaves their environment. We're not selling them GPUs — that's not the business we're building. We're just building the best possible toolkits and workbench for the data scientist in the organization to be effective in enterprise maturation, which is getting to owned models and right-sized intelligence.

Akash: I get it. It's a more Switzerland approach, rather than saying I have a GPU cloud and you have to be on my cloud to actually run it. It's a more agnostic layer, and it doesn't have to be on cloud. It could be on premise or in their VPC, right?

Ani: We're not trying to be a bull in the china shop and just add a new cloud to their already existing, probably multi-cloud setup. Or try to sell them a lot of humans to come in and do this.

It's much more of a giving-them-what-they-actually-need approach, which is a thin sliver of what's actually missing from the stack.

Future of enterprise AI deployment

Akash: To close up, I'd be curious on your viewpoint on a typical enterprise — you can take any size you want, let's say a billion dollars in revenue, been around for 30 years, I'm making it up. How many models will they have? How many custom models do you think they will have? Tens, hundreds, thousands? Or is that even the wrong question to ask? Where do you see this going, and I know it's hard to predict anything in this market?

Ani: I think it goes back to the pre-AI era where companies had ML models and they were building a ton of ML models. They had certain use cases that they identified. But now ML effectively is reasoning, and it's a substitute for tasks that would happen in a company. You think about all the tasks that a company does — some of them would be just models. It would be digital reasoning that you would build yourself and put into production. So I think that would be hundreds, thousands of models that you build.

But it won't be some technology that falls into the company. It's something that you're building in house, and just like you were building models before and it was doing some statistical thing behind the scenes, it'll be the equivalent of that.

And I think there's still space for frontier models. There would be some strategic bifurcation where certain kinds of workloads go to frontier models because they're very good at open ended reasoning. But the vast majority of use cases that a company cares about and is a source of competitive advantage — where maybe they have a ton of proprietary data on decisions in that space — will become self built, self hosted, sovereign models.

Akash: The analogy that comes to mind for me is when a SaaS company first came up, a software company. What does it do? It defined what to build. It goes and writes code. It tests it and then it ships it — with CDs before, but now the internet. But then after that SaaS came, and in a SaaS operation you actually have to run the damn thing. You have to run the operations, you have to have SREs, you have uptime, think about all the things. And that became core to the definition of what a software company was, where it wasn't before.

I can see this. This is beyond software and tech companies, but I can see enterprises having the capability to own your intelligence, or to capture your intelligence and shape it and use that in your decisions going forward.

Ani: You work with a lot of SaaS companies as clients. They have seen a little bit of dislocation as well in the last four years, where they have tried to become AI native or create products around AI. And a lot of them are just using frontier models. Do you see that as a threat? How are they dealing with that right now?

Akash: It depends on the SaaS company, the leadership and how they see it. Everyone's responding to this. Most companies and software companies in general will be like, hey, any data moat that I have, I need to protect it. This is why you see all the protections going up, and pricing models saying you can't get out my data. I'm not sure how many of them have actually taken the time, or have the expertise, to go and do what you're suggesting — which is almost to productize or codify the intelligence that they have in their company and put that in there.

I think it will happen, but the part that they don't have, unfortunately, is that all the cool kids want to go work at AI native companies rather than some of the incumbents. So that's going to be the challenge. But I wouldn't knock them out, because they have distribution, they have trust. They have all these things that some of the AI native companies won't, and this could be a good differentiator for them to build over time.

Ani: It's also a way for them to re-architect their financial profiles. Pre-AI they were selling software built on top of cloud providers, so their costs were cloud costs and then the revenues were SaaS license revenues. Now it's cloud plus AI, which is consumption based, and then the revenues are also some proportion of consumption, which is where a lot of these companies are going. Effectively they are becoming token resellers, or have removed themselves from that traditional SaaS financial model and moved to more of a usage based model — but then the cost profile is also usage based.

Akash: I agree with the point about bringing it in house. I don't know if I would classify them as — what did you call them, token resellers. I think that's too harsh. I wouldn't call them token resellers, because they have inbuilt workflows. They've done a lot of work over the years to understand a certain domain, in the finance domain, HR domain, supply chain domain. I don't think that's trivial.

The part I do agree with is I don't think any company, and let's click on software, really knows what the P&L looks like in the future. I don't think they know it, because in the historic parts it was very clear: R&D is this much, sales and marketing is this much, SG&A, et cetera. You could have analysts pore over any SaaS results and say you're messing up here. But now with these token costs, I don't think they know how to really think about that. That part I agree with. This might help reduce the cost — you're saying 50% — and then give them more control. Control over their destiny.

The equivalent would be back in the day building on Windows versus a Unix system, or in the cloud era building on Azure versus GCP versus AWS. Wouldn't you want to have the fungibility to build on anything? The answer would be yes. But it's even more profound than that.

Ani: Thanks again Akash for coming. Great to host you here.

Akash: Anytime.

Ani: I look forward to working together more closely with our new partnership.