Automating Code Review: How Neural Networks Are Taking Over Developer Grunt Work

Why Automation Is Necessary

Avito generates a thousand pull requests a week — up to 300 simultaneously during peak hours. This creates an enormous load on developers, who get pulled away from their core work to do code review. The result: PRs can sit in the review queue for days, time-to-market for new features increases, and the team gets fatigued and may miss critical bugs.

The LLM engineering team set three goals for automation:

• Speed up development — eliminate review queues • Reduce the human factor's influence — constant context-switching causes developers to lose focus • Unify the process — different reviewers give different feedback, often introducing subjectivity

System Architecture

The system consists of three components linked in a clear workflow:

1. Stash — the interface where a developer leaves an ai_review comment on a PR 2. Go service — receives the event from the Data Bus, parses the command, pulls the diff and repository metadata, stores the review in the database with status STATE_REQUESTED, and sends an RPC to the ML service 3. Python ML Pipeline service — processes the context and returns generated comments

The ML service responds with 200 OK immediately so it doesn't block the Go service — generation can take a while.

ML Pipeline: Three Stages

RuleChecker (generation). The model identifies all potential issues, drawing on a taxonomy of errors built from the team's real PRs. It's optimized for high recall, even at the cost of false positives. The output is a JSON with line numbers and recommendations.

ReviewFilter (validation). A second pass improves precision. The model works in a Conclusion First pattern — it receives the same context plus a specific comment and answers whether that remark is important or not. It filters out 90% of hallucinations and false positives.

CommentAggregator (deduplication). The system builds comment embeddings using BAAI/bge-m3 and computes cosine similarity. If similarity exceeds 0.85, comments are grouped together. Only the most substantive comment from each group survives. The output is a final list of unique remarks.

Model Choice and Performance

On the MERA-RuCodeReviewer benchmark (689 examples in Java, Python, Scala, Go), Gemini 2.5 Flash leads, but the team chose Qwen3-Coder-30B-Instruct-FP8 (ranked second) because it's open-source and can be hosted on their own infrastructure. This matters: the code never leaves for external APIs, and self-hosting is cheaper at scale (PRs require 2,000–10,000 tokens).

Performance: • Small PRs (up to 3 files): 10–20 seconds • Large PRs (10+ files): ~60 seconds • Maximum of 650 parallel reviews

Quality metrics are gathered from developer likes: • Precision: 85% of comments are classified as good • Outdated rate: 32% (comments closed by a developer fixing the issue). For Infra and DevOps, this reaches 60%.

Key Takeaways

The core idea: separating Recall and Precision works better than a single request. Even with a long context window, a model loses track of things, so a multi-stage system is necessary. Validation that filters out hallucinations is critical — it's the foundation of user trust.

The model needs to see the full code of files and the repository structure — a diff alone isn't enough for quality analysis. AI takes over the grunt work, but humans must still oversee the process.