Inside Knowledge Search: retrieval over your own codebase, not the internet

Ask a general-purpose coding assistant "why did the checkout deploy roll back on Tuesday" and it will, correctly, have no idea what you're talking about. That's not a model-quality problem — it's a retrieval problem. The answer to that question lives in your deploy logs, your incident channel, and a Slack thread from three people ago. Knowledge Search exists to make that context queryable the same way you'd query a well-indexed codebase.

Indexing more than files

Most retrieval-augmented systems chunk source files by line count and embed the chunks. That works reasonably for prose; it works poorly for code, because a fixed-size chunk routinely splits a function signature from its body, or separates a class from the interface it implements. Forge instead parses each repository into its AST and chunks along syntactic boundaries — functions, classes, and their associated docstrings and comments travel together as a single retrievable unit.

Alongside source, we index deploy events, incident postmortems, and PR discussions as their own document types, each carrying structured metadata — timestamps, associated commit SHAs, affected services — that source code alone doesn't have. This is what makes a question like "why did checkout roll back" answerable at all: the retrieval layer can join a deploy record to the commit that triggered it to the PR discussion where the risk was actually flagged.

Keeping the index honest

An index that's a day stale is worse than no index, because it answers confidently with wrong information. Every merge triggers an incremental re-index scoped to the changed files and their direct dependents, rather than a full repository re-embed, which keeps the index current within roughly 90 seconds of a merge landing on the default branch for repositories of any size we've tested against.

Why answers cite sources

Every Knowledge Search answer links back to the specific file, commit, or thread it was derived from — not because it's good UX (though it is), but because it's the only way to make the system's failure mode safe. When retrieval misses relevant context, the honest failure is an incomplete answer with visible sources, not a fluent, unsourced answer that sounds complete. We'd rather you notice a gap than trust a hallucination.

What's next

The current version indexes a single workspace at a time. The most requested feature from design partners with multiple related repositories is cross-workspace search scoped by team permissions — that's the next thing on our roadmap for Knowledge Search.