OpenAI launches Safety Bug Bounty targeting vulnerabilities in agentic AI systems

OpenAI has announced Safety Bug Bounty, a specialized track within its existing vulnerability rewards system. It targets not classic software bugs but risks unique to AI systems. Security researchers who discover and document vulnerabilities in the company's products can receive cash rewards through the Bugcrowd platform.

The key difference from standard programs is what researchers hunt for. Traditional bug bounties look for SQL injections, authentication flaws or problems in server infrastructure. The new program focuses on three vectors specific to language models: abuse of AI capabilities (bypassing safety filters, using the model for prohibited tasks), prompt injection, and leaks of data from conversation context or system instructions.

The focus on agentic vulnerabilities stands out. Over the past year and a half, OpenAI has actively rolled out agentic products: Operator, Deep Research, and the Responses API with tools for browsing and working with files. An agent that visits websites on its own, runs searches and executes code has a fundamentally larger attack surface than a chatbot. A specially crafted web page or document can contain hidden instructions that the model takes for commands from the user, and then carries them out. This class of attack is called indirect prompt injection.

The idea: the attacker does not address the model directly but embeds malicious instructions in content that the agent processes as data. For example, visiting a compromised site can make the agent send an email on the user's behalf, copy confidential data or change the settings of connected services. The attack works precisely because many models do not distinguish between trusted system instructions and untrusted external content.

Data leakage in LLM contexts also requires specific testing methods. This is not about hacking servers, but about situations where a model unintentionally reveals the contents of its system prompt, reproduces other users' data through memory mechanisms, or allows fragments of the training set to be reconstructed through targeted queries. Traditional penetration-testing tools are not suited to such tasks; they call for specialized expertise.

By creating a separate track with its own evaluation and payout rules, OpenAI effectively acknowledges that AI-specific threats require a separate methodology. This is consistent with the stance of leading labs: Anthropic regularly runs red-teaming before releasing new models, Google DeepMind publishes research on the security of agentic systems, and regulators in the US and EU are starting to demand evidence of systematic testing.

The program's practical value lies in scale. Internal security teams are limited in size and prone to blind spots. An outside community of researchers can find attack vectors that insiders missed, especially with unusual inputs. For users of agentic products, this means more systematic testing of systems they trust with access to their browsers, files and accounts.