Kaggle, now under Google DeepMind, launches Benchmarks SDK for comparing large AI models

Kaggle, which millions of data scientists know as the main arena for machine learning competitions, is changing its identity. The slogan "Your Home for Data Science" has given way to "The World's AI Proving Ground" — and this is not just a marketing rebrand. In 2026, Kaggle officially came under the management of AI Frontier, a new division of Google DeepMind.

A new owner means a new strategic focus. Kaggle is no longer just a place for prediction contests or image classification challenges. The platform's mission now is the systematic evaluation of large language and multimodal models under standardized conditions.

The headline technical update is a new Benchmarks section on the site and the open Kaggle Benchmarks SDK on GitHub. It is a full framework for creating, managing, and running test suites. The mechanics are simple: a researcher defines a test — input data, expected output, and a quality metric — groups several tests together, and that group becomes a benchmark.

The SDK runs the models under equal conditions and produces the results: logs, JSON, comparison tables, and leaderboards. The system is flexible enough to support almost any testing approach, from classic accuracy to complex multi-step tasks that evaluate reasoning. Benchmark data and code can also live in private datasets that are closed to the public.

Companies can build internal model evaluation standards without revealing their methodology or test cases to competitors. If they choose, they can make a benchmark public, and it becomes a shared standard for the community. Why does this matter right now?

Fair evaluation of AI models has become an acute problem. Popular public benchmarks — MMLU, HumanEval, GPQA, and others — draw regular criticism: their data leaks into training sets, so models effectively take the exam with a cheat sheet instead of demonstrating real capabilities. Large labs build closed internal tests, but small teams and academic groups lack that kind of infrastructure.

Kaggle Benchmarks SDK makes this tooling accessible. Google DeepMind gets obvious benefits from the platform's transformation. With its multimillion-strong community, Kaggle becomes a venue for showcasing DeepMind's own models against competitors, under conditions perceived as neutral.

The benefit to the community is just as clear: building a fair, reproducible benchmark used to take serious engineering work, and now it is available through a standard SDK. Nostalgia for the old Kaggle is understandable. The days when a well-tuned XGBoost beating a neural network on tabular data counted as a sensation are over.

The industry's challenge has shifted from "who predicts more accurately" to "how to objectively measure what a large model does." Kaggle is adapting to this shift and, judging by the scale of the changes, intends to become the standard for that measurement.