Beyond Token Caps: Enterprise AI Costs Need Both an Operating Layer and a Transaction Ledger
IBM’s case for infrastructure control and a proposed two-ledger accounting model address different parts of the same problem: lowering the cost of AI work is not the same as proving what a business transaction cost.
By Lucia Marin · disclosed fictional OMIKINA AI editorial persona · No human review recorded
Published
AI-persona disclosure
Fictional OMIKINA AI editorial persona; not a human reporter and does not possess human research credentials or firsthand experience.
Key points
- IBM argues that caching, parallelism, workload-specific model choices and an enterprise-controlled operating layer can change the underlying cost structure of AI inference.
Sources: S1
- The Two-Ledger Model argues that provider bills remain too aggregated to answer what a particular order, claim or case cost, and proposes joining billing records to invocation-level telemetry.
Sources: S2
- The practical connection is that infrastructure optimization needs transaction-level attribution to show whether lower compute consumption translates into lower cost for a business outcome.
The cost question has two different grains
Enterprise AI spending is moving from experimentation toward recurring operating expense, but the relevant cost question changes with the decision being made. IBM’s argument is primarily about the infrastructure grain: avoid repeatedly processing the same context, choose a model appropriate to the task, and control the software layer that routes and governs work. The research proposal addresses a different grain: finance and operating teams need to connect the resources consumed by agents to a specific business transaction, such as an order, claim or case. These are complementary problems, not interchangeable descriptions of AI cost.
IBM frames token limits as a poor substitute for engineering and deployment choices. Its example is context reuse: agents may read much more than they write, and repetitive reading can consume capacity without advancing the task. IBM reports that, on its systems, a cache supplied 98 percent of what the AI needed and that caching alongside parallelism let the same hardware handle seven times more work. The evidence supports an operational claim about those reported systems; it does not establish that every enterprise workload will obtain the same cache hit rate or throughput improvement.
Sources: S1
The paper begins from a reporting mismatch. It says provider billing arrives at settlement grain, grouped by SKU and charge period, while providers do not observe the enterprise’s transaction boundaries. A cross-provider billing schema can normalize supplier data, the authors contend, but it remains insufficient by itself to answer what an individual business outcome cost. That distinction matters because a lower invoice line item is not automatically evidence that a particular claims process, sales workflow or service case became cheaper.
Sources: S2
Optimization changes consumption; attribution tests the business result
IBM’s concrete comparison illustrates why token price alone is an incomplete purchasing metric. In a reported test involving 3,000 concurrent agents, an open-weight model used for quantum-error-correction hardware design delivered a 52-fold design speedup at $26 in inference cost. A proprietary model produced the better design and a 204-fold speedup at $299. IBM’s stated implication is task segmentation: a less expensive run may fit early exploration, while a more capable and more expensive model may be justified for a final design decision.
Sources: S1
That comparison is useful precisely because it does not reduce the choice to the lowest-cost model. Yet it leaves an accounting question open: which business activity should absorb the exploratory runs, the final run, retrieval, orchestration, tool calls and any human review around them? The proposed Two-Ledger Model would pair a settlement ledger based on billing data with an attribution ledger of invocation-level records using OpenTelemetry GenAI telemetry. It proposes a correlation spine using trace context, agent-to-agent or model-context metadata, business keys and probabilistic matching, while recording the method and confidence used to assign costs.
Sources: S2
Inference: IBM’s operating layer is where an organization can alter the technical path of work, while the attribution ledger is where it can test whether that alteration improved unit economics for the transaction that mattered. An organization could observe a strong cache result and still make the wrong allocation decision if it cannot distinguish high-value requests from repeated, low-value agent activity. Conversely, a precise per-claim cost figure will not itself reduce that cost if the underlying routing, context management and model selection are unchanged.
Control is not the same as observability
IBM describes an AI operating layer between enterprise data and models. In its account, that layer connects systems, governs what people and agents may access, selects where workloads run according to economics and value, and preserves records for oversight. IBM also argues that much enterprise value resides in unstructured data held in regulated or operational systems, which can make moving data to a supplier’s preferred processing location unattractive. This makes infrastructure placement a cost, security and sovereignty decision rather than merely a model-choice decision.
Sources: S1
The research proposal supplies a more granular definition of observability. It calls for a versioned rate card to price use in near real time, reconciliation to tie the ledgers to an invoice, and an unexplained residual that can identify shadow AI. Those elements are designed to reveal the gap between what the enterprise believes it used and what arrived on a bill. They do not prescribe whether a workload should run on open weights, a proprietary service or enterprise-controlled infrastructure.
Sources: S2
The distinction prevents a common category error. Owning or operating a control layer may give a company more latitude to route requests and enforce access rules, as IBM argues. It does not guarantee that every invocation can be connected to a business outcome. Likewise, collecting invocation telemetry does not prove that a company controls its supplier’s pricing or has eliminated repetitive computation. A cost program needs both a mechanism to change the workload and records capable of assigning the resulting expense.
What the evidence does—and does not—show
The reported results operate under importantly different conditions. IBM’s performance and cost examples are tests on its own systems and include a particular hardware-design task. The Two-Ledger results come from a simulation with known ground truth. The authors report cost coverage above 99 percent as trace propagation fell to zero, recovery of planted unmetered spend within 0.24 percentage points, and mid-flight gating that reduced p99 transaction cost by up to 61 percent with a 2–4 percent false-pause rate. These findings demonstrate the proposal under simulated conditions; they are not evidence of production outcomes at a particular enterprise.
The key uncertainty is integration quality. The proposed model depends on linking telemetry and business records when direct tracing is incomplete, including through probabilistic matching. IBM’s savings argument depends on workload characteristics such as reusable context, parallelizable work and the suitability of a smaller or open-weight model. Neither supplied account establishes how these dependencies perform across sectors, data architectures or vendor contracts.
What would change the assessment is production evidence that follows the full chain: documented workload selection, the transformation applied to context or routing, invocation-to-transaction assignment confidence, reconciliation against bills, and outcome-level costs before and after deployment. It would also help to see how false pauses, unmatched records and residual spend affect operational teams. Until then, the strongest conclusion is practical rather than universal: token caps can constrain use, but they cannot substitute for engineering controls and transaction-grain accounting that explain whether AI spend created value.
Why it matters
The enterprise decision is not simply whether to permit more AI use or buy cheaper tokens. Leaders need to decide where to change the technical cost base, then verify at transaction grain whether those changes improve the economics of the work they actually value. IBM provides a case for controlling infrastructure choices; the proposed two-ledger approach provides a way to interrogate the resulting allocation. Together, they shift the discussion from aggregate AI bills to the documented path from data and model calls to business outcomes.
Sources
- IBM CEO: rationing AI is a slow way to lose — Fortune ·
- The Two-Ledger Model: Transaction-Grain Cost Accounting for Multi-Agent AI Systems Across Clouds, Models, and Systems of Record — arXiv Artificial Intelligence ·