Three IT teams at three different companies discovered their AI API spend the same way: by opening an invoice. The amounts ranged from surprising to serious. In every case, the spend had been accumulating for months in accounts that were not connected to the company's central cost tracking. This is how you catch it before that conversation.
Why Shadow LLM Spend Is Different from Other Shadow IT
Shadow IT is not a new problem. Teams have been spinning up SaaS subscriptions outside the IT procurement process for years. What makes shadow LLM spend different is the cost growth curve. A team signing up for a project management SaaS at $15 per seat per month generates a predictable, linear cost. A team deploying an AI agent that calls GPT-4 with a 4,000-token context window, triggered 2,000 times per day, is looking at costs that can cross $3,000 per month per agent without anyone noticing until a billing cycle closes.
Token pricing scales with usage in ways that surprise teams that are not used to thinking in per-call economics. An agent that works well in testing with 20 requests per day can behave very differently at 2,000 requests per day in production, especially if the context window grows as the team adds more system prompt instructions over time.
The other difference: LLM API keys are easy to create and easy to embed in code. There is no procurement workflow, no IT ticket, no approval chain. A developer can have a working API key in four minutes from the time they decide to try a model. That is a feature for developers. It is a detection problem for IT.
Where the Spend Hides
Shadow LLM spend typically accumulates in three places:
Individual developer accounts. Developers sign up with their work email or a personal email, attach a corporate card if one is available, or expense requests against a T&E budget. The API key lives in an environment variable in the application. IT does not know the account exists until someone escalates a reimbursement or the card gets declined.
Team-provisioned accounts without IT visibility. An engineering team lead creates an account, provisions a key, and manages the cost out of a departmental budget line. IT knows in principle that AI spend is happening. They do not know the specific models in use, the request volumes, or whether data classification policies are being followed.
Bundled cloud spend. Azure OpenAI and Amazon Bedrock charges appear in cloud bills as line items that can be easy to miss if the team responsible for reviewing cloud spend is not specifically looking for AI service charges. A team using Azure OpenAI may have the cost absorbed into a general Azure subscription where it looks like compute.
The Detection Methods That Actually Work
There are four practical approaches to finding shadow LLM spend before it becomes a budget problem. They have different coverage levels and different setup costs.
Cloud bill analysis. If your teams use AWS or Azure, pull a detailed cost breakdown by service and look for OpenAI-compatible API calls, Bedrock, Vertex AI, and Azure OpenAI line items. This catches spend that went through managed cloud providers but not direct OpenAI or Anthropic API accounts. Useful as an initial sweep, not sufficient as ongoing monitoring.
Network egress inspection. At the infrastructure level, outbound traffic to known LLM API endpoints (api.openai.com, api.anthropic.com, and similar hostnames) can be logged. This shows you that requests are leaving your network to those destinations, with approximate volume. It does not show you cost, team attribution, or what data was included in the requests. Getting this into a dashboard with enough granularity to act on requires dedicated effort.
Expense report and procurement review. A one-time audit of recent expense reports for AI API subscriptions and T&E reimbursements will surface spend that went through personal cards. This is a lagging indicator and works once, not as a monitoring system.
Routing through a control plane. The most reliable ongoing approach is to require that all AI API calls route through a centralized endpoint. This gives you request volume, cost, model usage, and team attribution in real time, not after the billing cycle. Teams point their agents at the control plane endpoint instead of calling the LLM APIs directly. The first-day visibility into cost by team is the primary benefit, but the secondary benefit is that you are now also in a position to enforce data policy and set budget alerts.
Setting a Detection Baseline in a Week
If you want to go from no visibility to a reasonable baseline quickly, here is a practical sequence that does not require a large project.
Day one: pull your AWS, Azure, and GCP bills for the last 90 days. Look for anything under AI, ML, Cognitive Services, Bedrock, or OpenAI service categories. Note the amounts and the accounts they appear in. This gives you a partial picture of the managed-cloud portion of the spend.
Day two: send a short email to team leads asking for a list of any direct API subscriptions to OpenAI, Anthropic, Cohere, or similar providers that their team is currently running. Frame it as a cost consolidation exercise, not an audit. You will get 60 to 70 percent disclosure from this step. The remaining undisclosed spend will show up in other ways.
Day three to five: if your network team can give you DNS or proxy logs, look for outbound requests to known LLM API hostnames. This surfaces traffic that was not disclosed in the team lead email.
From there, the ongoing detection question is whether you centralize routing or continue periodic manual sweeps. Manual sweeps work for organizations with low AI adoption rates. For organizations where five or more teams have active AI agents, manual sweeps will always lag.
Budget Alerts Are Not the Same as Visibility
Many organizations' first response to shadow LLM spend is to set budget alerts on individual API accounts once they discover them. This is better than nothing but solves the wrong problem. A budget alert on an individual team's account tells you when that team has spent more than a threshold. It does not tell you the aggregate across teams, it does not enforce a consistent data policy across agents, and it does not give IT the routing layer visibility that lets you answer questions like "which agents are calling GPT-4 versus GPT-4o-mini and does the quality difference justify the cost difference."
The distinction matters because budget alerts address spend containment within a known account. Shadow spend, by definition, is in accounts you did not know about. The detection problem is different from the containment problem, and solving only the latter still leaves the former unaddressed.
What We Are Not Saying
Centralizing AI cost visibility through a routing layer is not a way to prevent teams from building AI agents or to impose approval processes on every deployment decision. Teams should be able to move at their normal development pace.
The goal is not to make AI harder to ship. It is to give IT the instrumentation that enterprise infrastructure needs to function responsibly. You would not run a compute cluster without knowing what workloads are on it or what they cost. AI API spend is the same category of infrastructure cost. It belongs in the same monitoring context.
If your current state is discovering AI spend when invoices arrive, the gap to close is instrumentation, not policy. Get the routing layer in place first. The cost attribution comes with it. Policy enforcement and budget controls can follow once you know what you are governing.