Experiment · EXP-005

Is code generation enough without a run loop?

Generate-once versus generate, run, inspect, and repair inside an environment.

Studying Bolt and building with agentic coding tools, I treated generate-once as a hypothesis for software-building agents. It fails as soon as the question is whether the app works. The environment — terminal, preview, errors — is part of the intelligence.

Hypothesis: A strong enough model can emit working software from a prompt without executing it.

Research question

Can a sufficiently capable model build reliable software by generating code once, without executing the result and observing what happens?

Why the hypothesis fails

Code can be syntactically plausible and still fail to compile, install, render, integrate, or satisfy the requested behaviour. A generation-only system has no direct evidence that its output works.

Agent loop

The architecture that repeatedly proved more useful was: inspect the task, generate or modify code, run it, observe compiler or runtime feedback, inspect the failure, repair the implementation, and verify again. The terminal, preview, filesystem, package manager, test runner, and logs become part of the system.

Software-building agent

Prompt
  → Plan
  → Generate
  → Run
  → Observe
  → Inspect failure
  → Modify
  → Run again

An agent that can run a compiler appears better at coding partly because the compiler tells it when it is wrong.

If I had to build something like Bolt, I would not start with the model. I would start with a sandbox, filesystem access, package management, command execution, logging, preview, and tests. The model would sit on top of this environment and control it through tools. A coding agent becomes more useful when it can inspect files and execute tests. A spreadsheet agent becomes more useful when it can execute code against a workbook. Sometimes you need a better model. Sometimes you need a compiler, a schema, a sandbox, or a pricing engine.

Closed agent loop

Observe
  → Plan
  → Act
  → Verify
  → Update state
  → Continue

Without verification, retries are just resampling. With verification, a failure becomes a signal. Different domains need different oracles: tests, schemas, rules, humans.

Result

I abandoned generate-once as a model for software-building agents. Generation remains valuable, but reliable software work requires a closed feedback loop.

Finding

The environment can contribute to apparent intelligence. A model with access to compiler errors and tests can correct mistakes that the same model would have no way to detect in a one-shot response.

Abubakar, Imam. "Is code generation enough without a run loop?." Imam Abubakar AI Research, 2026. https://imamabubakar.com/research/exp-005-generate-without-running

This archive contains independent research, experiments, architecture studies, and technical observations. Some findings are exploratory and should not be interpreted as peer-reviewed scientific conclusions unless explicitly stated. Architecture studies of third-party products are based on publicly observable behaviour, published technical information, and engineering analysis. They are not claims of access to private source code or infrastructure.