Creating an Agentic Coding Partner with Nanocode

I'm thrilled to introduce nanocode, a library designed to help you train your own Claude Code model from start to finish. We utilize Constitutional AI, a method employed by Anthropic for their Claude models, to guide our training process. This involves defining an agentic interface for our model, generating synthetic data, and aligning the model with our SOUL through preference optimization. Nanocode is built entirely in JAX and optimized for training on TPUs, drawing inspiration from Karpathy's nanochat project.

## Getting Started

You can kick off your journey with nanocode using the Google TRC program, which offers free access to pre-emptible TPUs for a month. This is an excellent opportunity to experiment with nanocode without incurring high costs. For instance, training the nanocode-d24 model with 1.3 billion parameters takes about 9 hours on a TPU v6e-8, costing $200, while the smaller nanocode-d20 model with 477 million parameters can be trained in just 1.5 hours for $34.

## Training Process

The training process for nanocode is similar to nanochat, with some key differences to enhance agentic coding behaviors. We use additional coding data from The Stack-V2 to strengthen the model's coding capabilities and improve code tokenization efficiency. Our models are trained with a parameter-to-data ratio of 8, following nanochat's scaling law analysis.

## Agentic Behavior

Agentic models are designed to perform tasks by interacting with the world through tool calls. We define a templating system to structure these interactions, using special tokens to delimit user and assistant turns, as well as tool calls. For nanocode, we defined four main tools: Read, Edit, Grep, and Bash, each with specific arguments to facilitate interaction with a UNIX environment.

## Constitutional AI

To instill a unique voice and character into nanocode, we employ Constitutional AI, which involves two stages: Constitutional Supervised Fine-tuning (SFT) and Reinforcement Learning from AI Feedback (RLAIF). The SFT stage uses synthetic data generation to align the model with our SOUL, while RLAIF helps refine the model's preferences.

## Direct Preference Optimization

Instead of traditional RLHF, we use Direct Preference Optimization (DPO) to align the model's outputs with our desired preferences. This approach simplifies the process by eliminating the need for a reward model, focusing instead on binary classification over preference pairs.

## Experimenting with Nanocode

Once trained, nanocode can interact with your UNIX system through its agentic CLI, allowing you to explore codebases and perform coding tasks. Although it's a small model, it's designed to be hackable and adaptable, encouraging you to customize its SOUL and tool interface to suit your needs.

Finally, I encourage you to dive into the codebase, which is compact yet powerful, and explore how you can create your own agentic coding partner. Whether you're interested in JAX or want to instill a unique personality into your model, nanocode offers a flexible platform for experimentation.