AI & Agents · Advanced
Autonomous Coding Agent
Build an agent that reads a GitHub issue, plans, edits files in a sandbox, runs tests, self-corrects on failures, and opens a pull request, then score it on real SWE-bench tasks.
You build a fully autonomous software engineering agent that accepts a GitHub issue URL, clones the repo into a Docker sandbox, uses Claude to plan and apply code edits, runs the existing test suite, and iterates on failures before opening a pull request with the fix. The project is grounded in real evaluation: you score your agent against SWE-bench Verified tasks so you can measure improvement concretely rather than by feel. This is the same class of system behind Devin, SWE-agent, and OpenHands, built from understandable open-source primitives. Completing it gives you a reusable scaffold for any agentic coding workflow.
What you build
- Parses a GitHub issue and retrieves the full repository context using tree-sitter to build a condensed file map
- Plans a multi-step edit strategy using Claude tool-calling with explicit reasoning before touching any file
- Applies targeted file edits inside an isolated Docker container so the host machine is never at risk
- Runs the repo test suite after each edit and feeds structured failure output back into the agent loop
- Self-corrects up to a configurable number of retry rounds using the test diff as the correction prompt
- Opens a pull request via the GitHub API with a summary generated from the agent reasoning trace
- Scores completed runs against SWE-bench Verified tasks and logs pass-at-1 metrics to a local SQLite database
What it teaches
- Designing stateful agent loops with LangGraph including conditional branching and retry ceilings
- Using tree-sitter for fast, language-aware code parsing to build compact repository context
- Sandboxing untrusted code execution with Docker volumes and the Python Docker SDK
- Structuring tool-calling prompts so an LLM produces machine-parseable file edits rather than prose
- Evaluating agent performance rigorously with SWE-bench rather than anecdotal pass/fail
- Managing multi-turn conversation state across plan, edit, and correction rounds
How it works
- 1
Issue Ingestion
- GitHub API
- Issue body
- Repo clone
↓ parse
- 2
Repo Mapping
- tree-sitter
- File map
- Relevant files
↓ plan
- 3
Edit Planning
- Claude tool-call
- Reasoning trace
↓ apply
- 4
Sandboxed Execution
- Docker container
- pytest output
↓ pass/fail
- 5
Self-Correction
- Failure diff
- Retry loop (max 3)
↓ publish
- 6
PR + Scoring
- GitHub PR
- SWE-bench eval
Sign in to open the build guide
Free account. Get the step-by-step build and every resource link.
Take it further
- Add a retrieval step using embeddings over the full repo to surface the most relevant files before tree-sitter condensation, improving context quality on large monorepos.
- Implement parallel hypothesis testing: generate two independent edit plans, run both in separate Docker containers concurrently, and pick the plan whose test suite passes more cases.
- Integrate a cost and latency tracker that logs tokens used and wall-clock time per SWE-bench task so you can plot a cost-vs-resolved-rate Pareto curve across model choices.
