Brain index

INSIGHT 29: Fact Models Make Static Rules Agent-Usable

Static analysis for AI agents should not stop at "grep the AST." The useful product is a fact model: files, imports, symbols, references, types, call edges, control flow, dataflow, tests, coverage, generated-code markers, and ownership metadata exposed through stable APIs.

The reason is practical. Agents fail at repo work when relationships are hidden. A fact model makes relationships queryable and diagnostics repairable.

This insight is the bridge from the blog/talk argument into the polint / exlint line of thinking. It belongs in this repo because the article needs the principle, not a product document.

Source map

RefSourceLocal textRole in this insight
R30GraphCodeBERTpaper-text/graphcodebert-2009.08366.txtEarly evidence that data-flow structure is useful for code representation.
R43Type-Constrained Code Generationpaper-text/type-constrained-codegen-2504.09246.txtType facts reduce invalid generated code.
R49Chunking RAG Code Completionpaper-text/chunking-rag-code-completion-2605.04763.txtContext units need declarations/neighborhoods, not isolated functions only.
R51ToolGenpaper-text/toolgen-autocomplete-repo-codegen-2401.06391.txtAvailable-symbol tooling reduces dependency/static-validity failures.
R54CodeGRAGpaper-text/codegrag-2405.02355.txtAST/control-flow/data-flow graph views bridge text and code structure.
R55InlineCoderpaper-text/inlinecoder-context-inlining-2601.00376.txtCall-graph neighborhoods supply dependency-aware context.
R62CrossCodeEvalpaper-text/crosscodeeval-2310.11248.txtCross-file API context materially improves repository completion.
R63CatCoderpaper-text/catcoder-2406.03283.txtCode and type context improve repository-level Java/Rust generation.
R64A3-CodGenpaper-text/a3-codgen-2312.05772.txtLocal/global/library-aware API retrieval helps, but over-retrieval hurts.
D31-D33polint docsarticles/polint-*.mdLocal framework evidence for typed facts, diagnostics, baselines, and agent-oriented output.

The static-analysis ladder

For the article, "static analysis" should not mean one thing. It is a ladder of fact families, each enabling different kinds of agent feedback.

LayerFact familyExample agent-facing rule
File discoverypaths, globs, generated/vendor statusDo not edit generated output; scope checks to app code.
Syntaxliterals, imports, declarations, JSX attributes, branchesBan raw colors, forbidden imports, missing route middleware.
MetricsLOC, complexity, function size, assertion countsDetect complexity spikes in agent changes.
Module graphresolved imports, package exports, dependency directionEnforce architecture boundaries and public APIs.
Symbols/referencesdefinitions, references, re-exportsMigration completeness and dead public surface checks.
Type factschecker facts, public API shape, generated SDK modelsBan invalid/raw API calls and enforce contract usage.
CFGbasic blocks, returns, throws, loops, branchesResource cleanup, branch obligations, postconditions.
Dataflowdef-use, validation-before-use, source/sinkInput validation and taint-like policies.
Call graphdirect/indirect calls and callbacks where knownRoute-to-service-to-repository path checks.
Test/coverage factstest names, assertions, coverage reportsEvidence that risky branches or route handlers are tested.

This table is important because it prevents two bad simplifications:

  • "Just write a custom lint rule." Some rules need symbols, types, or dataflow.
  • "Just give the agent the AST." Raw parser output is too low-level and too unstable for normal rule authors and agents.

Cross-file API context is a fact-model problem

CrossCodeEval constructs examples where the current file is not enough. The completion target often depends on imports, definitions, usages, and symbols in other files.

CrossCodeEval data copied from the paper

MeasurementValue
Examplesabout 10,000
Repositoriesabout 1,000
Languages4
Python examples2,665
Java examples2,139
TypeScript examples3,356
C# examples1,768
References with names needing cross-file informationalmost 100%
References predictable from current-file context aloneabout 2%

CrossCodeEval retrieval data copied from the paper

SettingExact match
StarCoder-15.5B Python, in-file only8.82%
StarCoder-15.5B Python, retrieved context15.72%
StarCoder-15.5B Python, reference-assisted retrieval21.01%
GPT-3.5-turbo C#, in-file only3.56%
GPT-3.5-turbo C#, with cross-file context11.82%

Source trace: R62, paper-text/crosscodeeval-2310.11248.txt.

The immediate inference: imports, declarations, public exports, and references are not secondary metadata. They are the minimum fact model for repository completion. Agents need the same relationships static analysis computes.

Type and symbol facts reduce the API hallucination surface

Type-Constrained Code Generation, CatCoder, and ToolGen triangulate the same point from different methods: visible legal surfaces improve generated code.

Type and tool data copied from the papers

SourceData pointFact-model implication
Type-Constrained Code Generation94% of generated TS compile errors are type-check errorsType facts matter beyond syntax.
Type-Constrained Code Generationcompile errors -74.8% on HumanEval synthesisType constraints reduce invalid code.
Type-Constrained Code Generationcompile errors -56.0% on MBPP synthesisSame direction on another benchmark.
CatCoderJava pass@k up to +17.35% over RepoCoderType/code context improves repo generation.
CatCoderJava compile@k up to +14.44% over RepoCoderType/code context helps validity.
ToolGenDependency coverage +31.4-39.1%Accessible symbols reduce dependency mistakes.
ToolGenStatic validity +44.9-57.7%Static tools reduce invalid identifiers/members.
A3-CodGenglobal retrieval F1 k=5 0.601, k=10 0.526, k=15 0.479Curated API candidates beat unbounded context.

Source traces: R43, R51, R63, R64.

This makes generated SDKs and public API surfaces part of static-analysis design. A "no raw internal API call" rule is simple only if the analyzer knows what a raw call is, what the generated client is, and how imports resolve.

Graph facts: callers, callees, control flow, and data flow

The graph papers are the deeper end of the ladder. They do not all evaluate the same task, so the article should not merge their scores into one leaderboard. Their shared mechanism is still useful: code is relational.

Graph viewSource signalPractical use
Data flowGraphCodeBERT uses data-flow structure for code representationTrack variables, definitions, and value movement.
AST / CFG / DFGCodeGRAG builds graph views for retrieval-augmented generationRetrieve structural neighborhoods, not raw text blobs.
Call graphInlineCoder inlines caller/callee contextSupply dependencies around the current edit.
Requirement + code graphGraphCodeAgent dual graph retrievalMap task requirements to relevant code relationships.
Repository graphRepoGraph / Repository Intelligence GraphExpose build/test/dependency topology.

This is why an agent-friendly static analyzer should expose scoped graph queries, not dump a whole AST or whole repository graph into context. The rule author and the agent need a neighborhood:

flowchart LR Route[Route handler] --> Service[Service] Service --> Repo[Repository] Repo --> DB[(Database)] Service --> Client[Generated SDK] Route --> Tests[Route/component tests] Service --> Types[Domain types] Tests --> Coverage[Coverage evidence]

Typed fact views are better than arbitrary rule-body guessing

The local polint/exlint research direction is relevant because it turns this into an API design question. A rule should request the facts it needs. The engine should know those facts before analysis so it can plan work, validate setup, cache correctly, and explain unsupported capabilities.

Bad shape:

rust
fn rule(ctx: &RuleCtx) {
    // rule can secretly read anything from a broad database
}

Better shape:

rust
fn no_raw_api_calls(
    ctx: &mut RuleCtx<'_>,
    imports: Imports<'_>,
    strings: StringLiterals<'_>,
    // future:
    // resolved_imports: ResolvedImports<'_>,
    // symbols: Symbols<'_>,
    // types: TypeFacts<'_>,
) -> RuleResult {
    // the function signature is the fact contract
}

The insight for the blog is tool-agnostic: make the rule's data needs explicit. This matters because agent-facing diagnostics are only trustworthy when the analyzer can say which facts were exact, which were heuristic, and which were unavailable due to missing setup.

Precision tiers should be visible

Not all facts are equally strong.

Precision tierExampleDiagnostic language
Exact syntaxraw string literal, direct import path"Found this literal/import."
Resolved syntaximport resolves to forbidden internal package"Import resolves to forbidden module."
Heuristic structurebranch appears untested by naming convention"No local test evidence found."
Semantic/type-backedcall violates generated client contract"Call does not match approved typed API surface."
Cross-file inferredmigration appears incomplete"This appears inconsistent with the migration pattern."

This is not just academic caution. Agents optimize against the feedback they receive. If a heuristic warning is written like a proof, the agent may make weird changes to satisfy it. If an exact violation is written too vaguely, the agent may miss an easy fix.

What this does not prove

It does not prove every codebase needs a custom static-analysis framework. Existing tools cover many common cases.

It does not prove deeper facts are always better. A3-CodGen and chunking research both warn against indiscriminate context expansion. Fact models should enable selective queries, not maximal dumps.

It does not prove graph analysis is needed for every rule. Many useful rules are syntax-only. The point is to avoid pretending syntax-only rules are the whole architecture surface.

Blog visual candidates

  1. Static-analysis ladder: syntax -> module graph -> symbols -> types -> CFG -> dataflow -> call graph -> tests/coverage.
  2. Rule fact contract example: function parameters as capability declaration.
  3. Precision-tier table.
  4. CrossCodeEval exact-match chart: in-file vs retrieved vs reference-assisted.
  5. Relationship neighborhood graph: route -> service -> repo -> SDK -> tests.

References

  • R30: GraphCodeBERT, paper-text/graphcodebert-2009.08366.txt
  • R43: Type-Constrained Code Generation, paper-text/type-constrained-codegen-2504.09246.txt
  • R49: Chunking RAG Code Completion, paper-text/chunking-rag-code-completion-2605.04763.txt
  • R51: ToolGen, paper-text/toolgen-autocomplete-repo-codegen-2401.06391.txt
  • R54: CodeGRAG, paper-text/codegrag-2405.02355.txt
  • R55: InlineCoder, paper-text/inlinecoder-context-inlining-2601.00376.txt
  • R62: CrossCodeEval, paper-text/crosscodeeval-2310.11248.txt
  • R63: CatCoder, paper-text/catcoder-2406.03283.txt
  • R64: A3-CodGen, paper-text/a3-codgen-2312.05772.txt
  • D31-D33: polint docs, articles/polint-readme.md, articles/polint-agent-playbook.md, articles/polint-ignore-comments.md