AutoDevs

AutoDevs is an agentic command-line tool. You describe a task, and it turns that description into a plan, then a working, tested pull request — with you in the loop at every stage.

It started as a frustration: existing coding agents treated my repo like a flat text file and produced diffs I couldn't safely merge.

The Problem

Agents that edit code live or die by context. A task like "add retry logic to the client" is trivial for a human who knows the codebase and opaque to a model that sees only a prompt. Real repositories have:

• Deep import graphs and partial modules.
• Tests that encode intent.
• Conventions that aren't in any doc.

Older approaches either read everything (too slow, too much noise) or read nothing and guess (fragile). AutoDevs takes a middle path: structured exploration, then explicit planning.

Architecture

The system runs three loops in sequence: read the codebase, plan the change, execute and verify it.

User Task → Repository Index → Plan Agent → MCP Tools → Test Runner → PR

1. Read — A cheap static pass builds a map of entry points, tests, and modules.
2. Plan — The model sees the index and produces an explicit, approvable plan.
3. Execute — Narrow MCP tools apply edits against a file snapshot.
4. Verify — The test runner gates the PR; failures roll back cleanly.

The loops are separated on purpose: plan first on discovered context, then execute against a file snapshot and verify with the actual test suite.

Engineering decisions

1. Repository index before first prompt. A cheap static pass builds a map of entry points, tests, and modules. The model sees the index before generating anything, which kills the floating-search behaviour.
2. Idempotent edits. Tool calls only apply cleanly or not at all; a failed step rewinds the snapshot instead of leaving a broken tree.
3. Verification is the gate. A PR is only proposed if the relevant subset of the test suite passes. This single rule transformed output quality.

const plan = await indexRepository(cwd);
 
for (const step of plan) {
  const changes = await apply(step, snapshot);
  if (!(await runTests(changes.target))) {
    await rollback(snapshot);
  }
}

Benchmarks

Challenges

The verification loop was the subtext of the whole project. Running a full, correct test build for arbitrary repos is hard — cache invalidation, setup scripts, and platform differences each need to be handled without the tool marring the user's own environment.

Tool scope was the second hard problem. A tool that can run arbitrary shell commands is powerful, but unsafe and non-deterministic. I constrained the shell tool to a sandboxed repl that flattens access and logs everything.

Lessons

• Putting the human in the loop is a feature, not a failure. A 5-second plan check catches 90% of wasted generation.
• Correctness is mostly about verification, not better guessing. The test runner is the product.
• Tools should be few and opinionated. 12 well-scoped MCP tools outperform 40 vague ones.

Roadmap

• Multi-file atomic transactions with a first-class review model.
• Persistent memory across sessions so the agent learns repo conventions.
• Native GitHub/Circle CI integrations to close the loop on external CI.