A training job needs compute. A coding agent has cloud credentials. Somewhere between those two facts sits a budget decision.
In one customer development environment, an agent checked accelerator quotas, issued a command to create an H100-backed node pool and prepared a Kubernetes training job. The reviewed configuration began at zero nodes and capped the pool at one.
The finding raises a practical question: how does an organization decide which infrastructure commitments an agent can make while completing a task?
The quiet cost of “make it work”
A cluster configuration may be just another step to an agent. To the platform team, it can create capacity, consume quota and incur charges. A zero-node pool may still lead to a node being scheduled later. Whether it does so depends on autoscaling, quotas, scheduling and the provider's response.
The right investigation follows the whole chain: request, cloud operation result, node state, workload scheduling and deletion. A create command alone answers only the first part.
Put spending authority next to technical authority
Give agents a bounded project or account, with machine types, regions, maximum capacity and permitted lifetimes set by policy. For expensive resources, require a budget owner to approve a request that specifies the workload, maximum size and duration.
Approval should be enforced outside the agent's instructions. Quotas and policy checks can prevent an unapproved request from becoming infrastructure; budget alerts are useful but may arrive after costs start. Apply expiry and cleanup rules that an independent controller can enforce.
Questions for the owner
- Did a node pool or an actual node get created? For how long?
- Was the machine type and maximum capacity within the task's authorization?
- Did the training job run, and did cleanup leave any resources behind?
- Can the same identity expand the cap or remove its own expiry?
A GPU request is not inherently suspicious. It becomes a governance problem when nobody can tell which resource decision the agent was allowed to make.