Building BoxLite With 13 People: What Coding Agents Taught Us

TL;DR

Coding agents are part of everyday work at BoxLite. Our most useful lessons concern review reflection, visible evidence, and decisions that still need human understanding.

Thirteen people, a micro cloud to build

We’re the team behind BoxLite. Our company is called Polygala. There are 13 of us.

Our ambition is to build a micro cloud. The work reaches from microVMs and guest execution to fleet orchestration and serverless lifecycle, networking, storage and images, databases and coordination, security and tenancy, developer tools, and operating the service.

A box is only one visible part of that system. Behind it are a control plane, a runner fleet, persistent state, network access, and telemetry. Around it are SDKs, APIs, a CLI, a console, documentation, and the machinery for testing and releasing changes. The runtime overview and cloud architecture show that breadth.

This includes integrating and operating existing systems. For example, the cloud architecture uses PostgreSQL for durable state, Redis for queues and coordination, and object storage for persistent volumes. Database and distributed-storage work are real responsibilities even when we use existing engines and cloud services. Architecture and data paths.

We also have to understand demand, find potential users, onboard and support customers, and handle pricing, billing, and recurring business operations. Figure 1 maps these responsibilities within our ambition; individual capabilities differ between the open-source runtime and hosted Cloud.

For a team our size, automation is a practical necessity. We use coding agents throughout BoxLite work, with shared tooling around Claude Code and Codex.

But getting code written was only part of the problem.

An agent would implement a change. A reviewer would find an issue. The agent would fix it. The next review would find another issue. Sometimes the next reviewer would challenge advice from the previous one. We had automated more work, but we still had to understand why the process kept circling.

That became the more useful question: what must change around an agent so that its work becomes easier to trust, review, and finish?

Figure 1. The work behind a micro cloud: eight engineering work areas, plus the customer and business responsibilities around them.

What happens between an idea and a merged PR

Our agent-tooling repository contains shared instructions, skills, hooks, and independent model audits. Repository setup also installs Git hooks. Together, these carry expectations through the work instead of relying on someone to repeat them in every prompt.

The pieces have different jobs. Instructions describe the rules. Skills explain how to do a task. Hooks run checks at particular moments. An independent auditor evaluates the work. A gate checks whether its verdict applies to the revision about to proceed.

A gate cannot write its own passing verdict. Changing the relevant code or design invalidates the old approval. That separation is a core architecture rule.

Here is how it appears in a normal PR lifecycle.

Research and design. A person brings the problem and system constraints. The agent investigates and proposes a design. Before supported repository edits, tooling requires a registered design and an independent research review of its current content. This gives implementation an explicit rationale that later reviewers can examine.

Implementation and tests. The agent works against that design. For bug fixes, we require a test that fails with the production fix reverted, then passes when the fix is restored. Hooks record supported test runs so that evidence can reach a separate reviewer.

Commit and push. An independent model audits the change. Git hooks check that the verdict matches the work being committed or pushed. A valid receipt can be reused for an unchanged audited commit. The benefit is continuity: approval follows the code it actually evaluated.

Figure 2. Before the PR: make the intent explicit and preserve evidence for independent review.

Prepare the PR. The agent explains the behavior, design, and verification. Tooling checks supported PR operations for writing and size requirements. Above our 400-added-code-line guideline, the human chooses a bypass or a split. The point is to make review effort visible before handing the change to someone else.

Follow feedback and revise. PR monitoring brings CI results, review comments, and merge conflicts back into the workflow. Revisions need fresh evidence where the work changed. Opening the PR is an intermediate step.

Human review and merge. In BoxLite’s normal merge path, author acknowledgment and another human’s approval are separate requirements. Required checks and resolved conversations also matter. The repository rules include an owner bypass, so this is the normal policy rather than a claim that bypass is impossible.

Figure 3. Through review: follow feedback, renew evidence, and keep human acceptance explicit.

These are workflow checks on supported paths, not a sandbox that can prevent every possible action. Their value depends on their coverage, their evidence, and whether they interrupt the right work. Six recurring problems taught us that.

1. Every review found another issue. We added reflection.

A fresh reviewer is useful because it can challenge the author’s assumptions. But a succession of fresh reviewers can lose the history of what was already tried.

A finding gets fixed. The next audit introduces another finding, revisits an old one, or reverses an earlier recommendation. The agent keeps editing. Nobody has explained whether the underlying problem changed or the previous reasoning was incomplete.

We added reflection across audit attempts. After two failed or errored attempts, the agent must explain the diagnosis, the failed fixes, and how its next approach differs. It must also report an executed check with the expected result, observed result, and evidence.

The reviewer has responsibilities too. Findings retain their identity across attempts. A newly discovered defect needs an explanation of why the earlier review missed it. Reversing advice needs an explanation of what changed. An independent auditor evaluates the reflection.

Figure 4. A repeated failure needs a diagnosis and a changed approach, while genuine new defects still count.

The lesson: repeated review needs a memory of the reasoning, not just a longer list of findings.

Reflection does not mean asking the next reviewer to approve. A real new defect must still be reported. Exhausting the review budget leaves work incomplete; it does not turn failure into success.

We have not established a measured reduction in review rounds. The concrete change is that repeated failure now requires a diagnosis and a changed approach. “Try again” is no longer a complete plan.

2. Our own instructions were encouraging scope creep.

We wanted thorough agents, so we told them to fix similar problems wherever they appeared. We also wanted strict review, so a passing verdict required no findings.

Those instructions interacted badly. A narrow task could expand into nearby cleanup. A reviewer could identify something as a nonblocking nit, yet the verdict format would still make it block the work. The agent was following incentives we had written into the process.

Our scope and review change made two distinctions explicit.

First, fix the requested problem and every evidenced instance of the same defect. Adjacent improvements get recorded separately instead of silently joining the task. Second, distinguish blocking findings from advisory suggestions. A passing review can contain an advisory without asking the agent to start another editing round.

Figure 5. Keep evidenced instances of the same defect together; give adjacent improvements a separate destination.

The lesson: define what belongs in the task and what is serious enough to block it.

“Be thorough” is too broad on its own. It can mean finding the same bug in three places, which is useful, or rewriting a neighboring subsystem, which may deserve its own discussion.

The benefit is a clearer handoff to humans: the PR has a reason and a boundary. The trade-off is that some useful improvements wait. We would rather make that choice explicit than discover the expanded scope during review.

3. The reviewer couldn’t see the tests we had already run.

Independent review creates a context boundary. That is useful for reducing shared assumptions, but it can also hide evidence.

Our test-evidence issue described authors running tests that the auditor could not see. Some VM and SDK suites were too expensive to repeat inside a short audit. Similar changes could receive opposite verdicts: one passed by inspection, another failed because no execution was observed. An unchanged diff could remain blocked even after the author ran new tests.

We added execution receipts. For supported test commands, a hook records the command, outcome, commit, and working-tree state. The authoring agent does not write its own receipt. New execution evidence can trigger a fresh audit even when the diff has not changed.

A later correction made completion metadata explicit: a pending process or ambiguous output must not become a passing test result.

Figure 6. Execution records bridge the context gap. A meaningful regression test also has to demonstrate the original failure.

A receipt still cannot make a weak test meaningful. We also require evidence that the test catches the original defect.

One concrete BoxLite example was a concurrent sandbox-creation fix. With the production fix reverted, a 50-way request burst lost 20–27 creates. With the fix restored, the reported suite passed 23/23 tests. Those are different measurements; the passing suite is not a claim that the same burst achieved 50/50 success. The PR also recorded a trade-off: queued creates held database connections and delayed other queries under load.

The lesson: “tests passed” needs a trail—what ran, against which code, and what failure it can detect.

Our receipts cover recognized commands, not every possible test invocation. They improve what a reviewer can inspect; they do not replace judgment about test quality.

4. We required research—but checked it after coding.

“Research before implementation” sounds like a clear instruction. Its timing turned out to matter as much as its wording.

Our earlier edit check established that a design document existed and was readable. The stronger assessment of its research happened at commit or PR time, after implementation. That left room for research to become a justification written around code that already existed. Issue #253 captured the gap.

We moved the independent research review before supported code edits and tied its approval to the current design content. If the design changes, that approval must be renewed. Later reviews can compare implementation against the approved intent instead of repeating the entire research assessment.

Figure 7. A research check is most useful before implementation commits the team to an approach.

The lesson: put a check before the decision it is supposed to influence.

A late research check can still find a problem. It also arrives after effort has been invested in one approach. An earlier review makes alternatives, constraints, and trade-offs available while they can still shape the design.

This boundary needs care. Investigation itself must remain possible. The gate covers supported editing operations; it does not claim to intercept every shell action. The objective is better decisions before implementation, not a document that exists only to satisfy a hook.

5. Our safeguards were consuming the attention we wanted to save.

A check can be correctly installed and still ask the wrong question.

Our end-of-turn gate initially asked whether a response stated facts. Almost every engineering update does. PR #1164 reports that it blocked 18 of 22 sampled real turns, while the 14 audits completed in the same window found nothing.

We changed the question: does the response make a conclusion the reader must accept without seeing its evidence? A bare “the fix works” deserves scrutiny. A result accompanied by inspectable output or a source is a different case.

The PR’s measurement table reports two blocks when replaying those same 22 turns with the shipped change. These are the results recorded in that PR, not an independently reproduced measurement for this article.

Figure 8. PR #1164 reports results for the same 22 sampled turns before and after the change. This replay does not measure company productivity.

The lesson: evaluate the cost of the check alongside the risk it catches.

This was a small gate replay. It was not a company productivity benchmark, and it did not establish equal defect detection. The PR recorded a limitation: a plausible fabricated citation could pass the lighter check.

That trade-off is part of the story. A system that interrupts too often spends the attention it was meant to preserve. A system that interrupts too little can miss unsupported claims. We need to examine actual triggers and outcomes, including the failures of the checkers themselves.

6. We extended the agent’s job beyond opening a PR.

A PR can look finished from the author’s session while important work is still arriving elsewhere: CI fails, a reviewer asks a question, or another merge creates a conflict.

We added monitoring to bring those events back into the workflow. Then we found a second problem: keeping a watcher running was not the same as delivering its events to the agent. Delivery could still depend on another foreground turn.

The follow-up work added scheduled event delivery. Actionable events must be reported before they are acknowledged. A final event must be delivered before the watcher retires. An empty check should remain quiet, and a failure should make lost coverage visible.

Figure 9. An event must reach the agent before it is acknowledged. Host delivery still needs qualification.

The lesson: a handoff is complete when the next worker receives it—not when the producer says it sent something.

The benefit is continuity across the waiting periods of software development. The next CI result or review comment can become a concrete next action without relying entirely on someone remembering to poll the PR.

There is still a boundary to the evidence. Passing fixtures and a saved schedule do not prove reliable delivery and silence on every host. Host qualification remains separate work. Monitoring also does not authorize the agent to accept a change on another human’s behalf.

What people still contribute: sense

We call the human contribution sense: understanding turned into decisions.

In engineering, sense is knowing which constraint matters, which abstraction fits, and which trade-off the system can afford. Agents can propose designs and challenge assumptions. People bring their understanding to the choice, and another human still reviews the resulting code in the normal merge process.

Author acknowledgment is useful, but it is not that second review. BoxLite’s author-review workflow binds the author’s acknowledgment to the PR’s current commit. The separate reviewer supplies another person’s judgment.

Figure 10. Human understanding shapes design, review, and customer promises.

The same division appears outside engineering. We use agents to find potential users and draft outreach. They can gather context and prepare a message. People still need to decide whether the person is relevant, whether the promise is accurate, and whether this is the right time to ask.

Outreach is one part of the business workload. Understanding demand, helping people get started, answering support questions, and managing pricing and billing all need attention too. The figure below maps those responsibilities; it does not mean every step runs autonomously.

Recurring business work can follow a scope authorized in advance. Exceptions still need somewhere to go. A prepared draft, a confirmed send, and an acquired customer are different outcomes; automation should not collapse them into one success label.

Figure 11. Business work spans understanding demand, reaching users, helping them succeed, and operating the business. Human decisions and confirmed outcomes remain explicit.

This is why shared tooling matters beyond code generation. It helps make the state of work visible: what is proposed, what has been checked, what is waiting, and who owns the next decision.

What another small team can adopt

Start with the failure you keep seeing.

If reviews circle, preserve their reasoning and require reflection. If tasks expand, define scope and separate advisories from blockers. If reviewers lack evidence, capture execution at the point it happens. If a check arrives too late, move it before the decision. If it interrupts too often, replay real examples and inspect the trade-off. If feedback gets lost, verify delivery across the handoff.

Figure 12. Start with one recurring failure, change the workflow, and evaluate what happens in real use.

For us, this is how agent tooling grows: a repeated problem becomes a concrete workflow change, and that change gets examined in real use.

We are still a team of 13, building toward a broad micro-cloud ambition. Coding agents help us take on that work. The surrounding process determines whether a generated change becomes something another person can understand, review, and accept.

The tooling is open source. Each lesson above links to the change or discussion behind it. Start with the one that resembles a problem in your own workflow.