Reliable AI Agent Architecture: Code for Guarantees, Models for Judgment

The dangerous part of an AI agent is not that it sometimes gives a bad answer. It is that a bad answer can become an action inside a loop. The model chooses a tool, changes state, reads the result, and tries again. If the surrounding system has no hard boundaries, one uncertain judgment can turn into missed work, repeated calls, an invalid edit, or an expensive retry chain.

A reliable AI agent architecture separates two kinds of work. Deterministic code handles guarantees: scope, permissions, ordering, validation, budgets, and stopping conditions. The model handles judgment: interpreting context, resolving ambiguity, choosing among safe options, and explaining a result. You need both. Asking either side to do the other’s job creates avoidable failure modes.

Alibaba’s open-source OpenCodeReview project is a useful case study because it makes this boundary explicit. It combines deterministic review pipelines with an LLM agent instead of asking a general-purpose agent to discover and enforce the entire process through natural-language instructions.

OpenCodeReview, Alibaba, and GlideflowAI are independent projects. This article examines the published architecture; it does not imply a partnership or endorsement.

The difference between a workflow and an agent

The words often get mixed together, but the distinction is practical.

A workflow follows code paths you defined in advance. It can branch, retry, and call a model, but your program decides the sequence. An agent lets the model direct more of the process: it chooses tools, decides what context to inspect, and determines which step should come next.

Anthropic uses the same distinction in its Building Effective Agents guide: workflows use predefined code paths, while agents dynamically direct their own tool use and process. The guide recommends beginning with the simplest system that works because agent autonomy usually adds latency, cost, and new failure modes.

That does not make workflows primitive or agents inherently better. They solve different parts of a problem.

The design rule is simple: if you can express a requirement as an invariant, enforce it in code. Give the model discretion only where discretion adds value.

What pure model orchestration gets wrong

A prompt can describe a process, but a description is not an enforcement mechanism. “Review every changed file” communicates intent. It does not prove that the model actually reviewed every file.

OpenCodeReview’s maintainers describe three problems they observed with general-purpose agent review:

• Incomplete coverage. An agent may focus on a few interesting files and skip others in a large changeset.
• Position drift. A finding can describe a real issue but point to the wrong line or file location.
• Unstable quality. Small changes to prompts or context can alter what gets reviewed and how the findings are presented.

These are not arguments against using an LLM. They are signals that the LLM has been assigned responsibilities that code can handle more precisely.

The same pattern appears outside code review. A support agent should not decide whether a user is authorized to access an account. A purchasing agent should not invent its own spend ceiling. A document agent should not decide that processing 18 of 20 required files is “close enough.” Models can advise on those tasks, but the application must enforce identity, limits, and completeness.

OpenAI’s practical guide to building agents makes a related point: guardrails should be layered with standard authentication, authorization, strict access controls, and ordinary software security measures. A model-based safety check is one layer. It is not a replacement for the rest of the system.

How OpenCodeReview divides the work

OpenCodeReview publishes a hybrid design called “Deterministic Engineering × Agent.” Its code pipeline handles the parts that should remain stable; the agent handles dynamic context and judgment.

On the deterministic side, the project describes:

1. Precise file selection. The pipeline calculates which files require review and which should be filtered.
2. File bundling. Related files can be grouped into isolated review units rather than mixed into one unbounded context.
3. Rule matching. The system selects rules based on file characteristics before the model reviews the code.
4. Finding placement and reflection. Separate modules check where feedback belongs and review the proposed comments.

On the agent side, it uses scenario-specific prompts and tools to inspect context and decide whether a change represents a defect. The agent can read full files, search the repository, and inspect related changes when the diff alone is not enough.

The interesting idea is not limited to code review. The deterministic layer turns an open-ended task into a bounded decision space. The model still reasons, but it reasons over work units, tools, and permissions that the application has already constrained.

That architecture also makes failures easier to diagnose. If a file was not reviewed, inspect the selection and scheduling logic. If a finding is semantically weak, inspect the prompt, context, tool trace, and model behavior. A single agent prompt does not have to carry every policy, workflow rule, and quality requirement at once.

A reusable production-agent pipeline

You can apply the same separation to support, research, document processing, coding, and back-office automation. A practical pipeline has six stages.

1. Accept and classify the request

Parse the request, identify the user, and select an allowed task type. Reject unsupported operations before the model sees them. Do not ask the model to decide whether an unauthenticated request should be accepted.

2. Build a deterministic work plan

Code determines required records, files, approvals, budgets, and tools. The plan can contain optional branches, but it should state what “complete” means.

3. Let the model resolve ambiguity

The model reads relevant context, selects among approved tools, and proposes actions. It can ask for more information when the plan exposes a decision that only the user can make.

4. Validate every proposed action

Check schemas, permissions, current state, allowed paths, spend limits, and idempotency before execution. Treat model output as untrusted input to the action layer.

5. Execute with bounded retries

The program records attempts and stops at a fixed threshold. High-impact or irreversible actions pause for human approval.

6. Verify the outcome

Run tests, reconcile required records, or compare the result with an acceptance rule. Store the model ID, tool calls, validation results, token usage, and final status for later evaluation.

Here is an illustrative TypeScript shape. It is not OpenCodeReview source code:

const plan = buildPlan(request, policy);       // deterministic
assertAuthorized(user, plan);
assertWithinBudget(plan);

for (const unit of plan.requiredUnits) {
  const proposal = await agent.decide({
    goal: unit.goal,
    context: await loadAllowedContext(unit),
    tools: allowedTools(unit),
  });

  const action = validateProposal(proposal, unit.schema);
  assertAllowed(action, user, policy);
  await executeIdempotently(action);
}

const result = verifyCompletion(plan);
if (!result.complete) {
  throw new Error(`Incomplete workflow: ${result.missing.join(", ")}`);
}

The model never determines its own authorization, budget, or definition of completion. It contributes the part that benefits from language understanding and contextual judgment.

Guardrails are more than another prompt

“Do not delete production data” is useful as an instruction. It is inadequate as the only safeguard.

The OWASP AI Agent Security Cheat Sheet recommends least-privilege tools, input validation, schema-validated outputs, human review for high-risk actions, and explicit token, retry, cost, and tool-chain limits. It also recommends separating decision-making from execution for irreversible operations.

Turn those recommendations into application controls:

• Give read-only tools by default.
• Expose narrow functions such as createRefundDraft, not unrestricted database access.
• Require a typed payload before a tool can run.
• Re-read current state immediately before a write.
• Use idempotency keys for retried external actions.
• Cap turns, tool calls, elapsed time, and spend.
• Require approval for destructive, financial, or externally visible actions.
• Log decisions without logging credentials or unnecessary private data.

Some guardrails can themselves use models—for example, identifying off-topic input or potential prompt injection. Keep deterministic checks around them. A classifier score should not grant permission that the authenticated user does not have.

How to evaluate a hybrid agent

Agent evaluation must cover the path, not just the final paragraph. Anthropic’s guide to agent evaluations notes that agents act over multiple turns, modify state, and adapt to intermediate results. A final-answer grader cannot reveal every tool error or unsafe near miss.

Build an evaluation table around observable outcomes:

Run the same task from the same initial state more than once. Preserve failed runs. A system that succeeds once and takes a different unsafe path on the next attempt is not ready merely because the final text looks good.

If you compare models, keep the deterministic layer fixed. Change only the model ID, then record completion, retries, corrections, and total usage. An OpenAI-compatible model interface can make that swap smaller; the Codex CLI integration page shows the connection boundary for one developer tool. Compatibility reduces integration work, but it does not make model behavior identical.

Where a model gateway fits

A gateway belongs at the model boundary, not at the authorization boundary.

Your application can keep one model interface while routing bounded decisions to different model IDs. The surrounding code still owns permissions, validators, retry policy, audit records, and acceptance tests. That separation lets you evaluate another model without rebuilding the operational controls that make the agent safe.

Start with a small model shortlist from the current /models catalog. Use one route for the baseline and one alternative for the same frozen evaluation set. Record input and output usage separately, then apply the dated rates on /pricing. Do not select a model from token price alone: a route that needs more turns or more human correction can cost more per completed task.

For a direct SDK experiment, keep the model ID configurable rather than spreading it through business logic:

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["GLIDEFLOW_API_KEY"],
    base_url="https://api.glideflowai.com/v1",
)

response = client.chat.completions.create(
    model=os.environ.get("AGENT_MODEL", "glm-5.2"),
    messages=[
        {
            "role": "user",
            "content": "Classify this bounded work item and return valid JSON.",
        }
    ],
    max_tokens=512,
)

This verifies an API shape, not a production architecture. Validate the returned JSON, restrict the tools, and run the result through the deterministic pipeline described above. The /docs/code-examples/ page contains additional connection patterns.

Build the boundary before adding autonomy

Choose one agent workflow you already understand. Write down the properties that must always hold: who may act, which data may be read, which tools may run, what counts as complete, and when the system must stop. Implement those properties as code and tests.

Then identify the decisions that remain genuinely ambiguous. Give the model only the context and tools required for those decisions. Run a fixed evaluation set, inspect the entire trajectory, and add a new guardrail when a real failure reveals a missing boundary.

The next concrete step is small: take one instruction currently buried in your agent prompt—“review every file,” “never exceed this budget,” or “ask before sending”—and turn it into an assertion the model cannot bypass. Once that invariant is enforced, create a restricted key at /start and test two model routes against the same bounded task.