Skip to content

Agent architecture: where compromises happen

LLM APIactions · resultsUsersubmits tasksScaffold serveragent loop & stateparses actionsInference serverserves the LLM APIholds model weightsmodel weightsExecution serversandboxed code runstools · GPUsInternetthe outside worldmonitor · audit · deferweights (if given)PRs deployed tothe serversown agent looprogue scaffold'sAPI queries

1. The basic architecture

Three responsibilities almost surely live on logically different computers: the inference server that serves the LLM API (and holds the weights), the scaffold server that maintains the agent loop's state, and the execution server where the agent's code actually runs. The user talks to the scaffold.