AI tokens are not truly unlimited: an IT budgeting lesson from the US Army
Ars Technica reported a practical reminder for every organization adopting AI: the US Army expected its AI token supply to last much longer, but real usage burned through the allocation faster than planned. A token is the unit used to measure input and output processed by an AI model. Users see a simple chat box, but every prompt, uploaded document, summary, code request, retry and follow-up answer consumes part of the budget.
The lesson is not simply that a large organization used AI heavily. The important point is that even an organization with formal contracts and serious planning can underestimate AI demand when the service is treated as “unlimited”. Once AI moves into daily workflows, usage grows through more users, longer documents, repeated revisions, automated agents and background tasks. For IT teams, this is similar to cloud cost management: if nobody measures usage, the bill or quota problem appears only after the service becomes important.
Why this story matters
Many vendors use phrases such as “unlimited”, “all you can use” or “enterprise access”. In AI infrastructure, however, compute is never truly unlimited. GPU capacity, memory, bandwidth, latency, queues and operating cost all have limits. As a result, unlimited plans often include fair-use policies, throttling, priority tiers, hidden quotas or a requirement to buy more capacity after a threshold is reached.
AI demand is also harder to predict than traditional SaaS usage. A normal seat-based tool may have relatively stable consumption. AI is different: a small team can process thousands of pages, generate many drafts, review code, summarize logs or run experiments repeatedly. If autonomous agents are added, cost is no longer tied only to the number of users; it is tied to the number and size of machine-run tasks.
Five practical takeaways
- Tokens are the real cost unit. Long prompts, attached files and long answers all increase token usage, even when the interface feels simple.
- “Unlimited” usually means soft limits. Providers still need to protect shared infrastructure, so fair-use rules and throttling matter.
- Seat count is not enough for forecasting. Two teams with the same number of users can consume very different token volumes.
- AI needs its own FinOps practice. Usage should be tracked by team, project, application, model and task type.
- Unexpected limits create operational risk. If support, log analysis or reporting already depends on AI, a quota cap can slow real work.
Checklist for IT teams before expanding AI
- Read the contract carefully: quota, fair-use policy, throttling, daily limits, monthly limits and overage terms.
- Enable dashboards that show token usage by user, team, app, API key and model.
- Create alerts at 50%, 75% and 90% of the planned budget instead of waiting for a hard stop.
- Separate production workloads from experiments with different projects, workspaces or API keys.
- Limit large document uploads when they are not necessary; use chunking or local preprocessing where appropriate.
- Standardize shorter, goal-focused prompts to reduce repeated clarification loops.
- Prepare fallback options such as cheaper models, queues, priority rules or manual processing for critical work.
Operational notes
The better question is not “how much does each user cost per month?” but “how many tokens does each workflow need to produce a useful result?” Summarizing a short email is completely different from analyzing a 200-page tender document. Generating a small code snippet is very different from asking an agent to inspect a repository, edit files, run tests and explain logs.
Teams should also define priorities. If AI becomes a shared company tool, support, security, legal or production operations may need higher priority than internal experiments. Without policy, one prompt-testing campaign or bulk document batch can consume capacity needed by urgent work.
Conclusion
The Ars Technica story is a useful reminder: AI resources are not infinite, even when commercial packaging makes them feel that way. Tokens are real cost, limits are real, and operational risk appears when a team depends on AI without measuring usage. Treat AI like cloud infrastructure: define quota, dashboards, alerts, access control, workload priority and fallback plans before usage scales.
Source: Ars Technica