Shipping faster: an engineering copilot grounded in two million lines of code

The problem

A B2B software company had grown to 300 engineers across four acquisitions. The codebase was two million lines spread over 140 repositories, with a decade of tribal knowledge sitting in Slack threads, half-current wiki pages, and the heads of six long-tenured engineers. New hires took roughly five months to become independently productive. Worse, the same questions circulated forever: who owns this service, why does this flag exist, what happened the last time this queue backed up.

Off-the-shelf coding assistants helped with syntax, but they knew nothing about this system. When an engineer asked why a retry policy was set the way it was, a generic model invented a plausible answer. Plausible and wrong is expensive.

The hypothesis

The bottleneck was not code generation — it was context retrieval. If we could ground a model in the actual repositories, architecture decision records, runbooks, and resolved incidents, most of those recurring questions would answer themselves with citations an engineer could verify in one click.

The build

• Multi-source indexing — repositories, ADRs, runbooks, incident post-mortems, and API docs, each chunked with structure-aware splitting so a function stays with its signature and a runbook step stays with its context.
• Hybrid retrieval with code awareness — semantic search over embeddings combined with symbol and path lookup, then a reranker. Asking about a class name retrieves the class, not blog-style prose that mentions it.
• Answers with provenance — every response cites the file, line range, or document it drew from. Uncited claims are suppressed rather than smoothed over.
• Deployment inside the customer VPC — an open-weight model served on their own infrastructure. No source code left their network, which is what made legal say yes.

Rollout

We started with a single platform team of twelve for three weeks, tracking two things: how often answers were cited correctly, and how often the engineer still had to ask a human afterward. Once the second number dropped below one in five, we opened it to the rest of engineering. The staged rollout was as much about building the eval set from real questions as it was about caution.

Results

New-engineer ramp shortened meaningfully, cross-team escalations for context questions fell, and the six long-tenured engineers got a measurable slice of their week back. The unexpected win was documentation quality: when a question got a bad answer, the fix was usually to write the missing ADR — so the tool quietly created pressure to document.

What we'd tell you

• Ground it in your artifacts or don't bother. A generic assistant is a syntax helper, not an institutional memory.
• Citations are the product. Engineers verify; give them a one-click path to the source.
• Build the eval set from real questions your team already asks — it is sitting in your Slack history.
• Deployment location is a legal question before it is a technical one. Decide it first.