How We Took Over ChatGPT Accounts
A security vulnerability in ChatGPT allowed attackers to access user data, according to Simon Willison. The issue has been resolved with a bug bounty payout.
Original reporting · simonwillison.net
Read “How We Took Over ChatGPT Accounts” at Simon Willison →How this was written. GDS wrote this article with a language model from Simon Willison’s reporting (model: @cf/meta/llama-3.3-70b-instruct-fp8-fast); it describes that reporting rather than adding to it. The original, linked above, is the full story.
Key points from Simon Willison’s report
- Researchers at Parameter Labs discovered a security flaw in ChatGPT's implementation of interactive custom HTML apps
- The flaw allowed attackers to access user data and perform actions on their behalf
- OpenAI has closed the security hole and paid out a bug bounty to the researchers
Researchers at Parameter Labs discovered a complex security flaw in ChatGPT's implementation of interactive custom HTML apps, Simon Willison reported. This flaw was related to the display of these apps in a sandboxed iframe, which is meant to prevent untrusted code from interacting with OpenAI APIs and accessing private data. The goal of this sandbox is to tightly control the environment and prevent any potential security breaches.
The attack chain began with the MathLive JavaScript library, which is used by OpenAI for graph support and includes a virtual keyboard. This library can be triggered in the parent page via postMessage() from the sandboxed iframe, allowing for potential manipulation. The message receiving code uses a method that can add custom HTML to the parent page, and there was no mechanism in place to prevent other sandboxed frames from calling this method.
This vulnerability allowed untrusted code in an iframe to insert custom HTML into the parent page. Although this method does not directly execute script elements, it can be used to inject a new iframe without sandbox restrictions. This created a potential pathway for attackers to access sensitive user data.
OpenAI's content security policy rules blocked inline scripts and only allowed script loading from specific domains. However, one of these allowed domains was YouTube, which had an oEmbed endpoint that could be used to reflect JavaScript and execute it. This provided a means for attackers to bypass security restrictions and execute malicious code.
The outcome of this exploit was that attackers could create a ChatGPT share URL that, when visited by another signed-in user, could exfiltrate their private data or perform actions on their behalf. This was a serious security flaw that had significant implications for user privacy and security.
OpenAI has since closed the security hole and paid out a bug bounty of $5,000 to the researchers who discovered the flaw. This payout may be seen as relatively low considering the severity of the vulnerability.
Questions and answers
What was the nature of the security flaw discovered in ChatGPT?
The flaw was related to the display of interactive custom HTML apps in a sandboxed iframe, which allowed untrusted code to interact with OpenAI APIs and access private data.
How did the attackers execute malicious code despite OpenAI's security restrictions?
The attackers used YouTube's oEmbed endpoint to reflect JavaScript and execute it, bypassing OpenAI's content security policy rules.
What was the outcome of the exploit and how did OpenAI respond?
The exploit allowed attackers to exfiltrate user data and perform actions on their behalf, and OpenAI responded by closing the security hole and paying out a bug bounty to the researchers.
