Search Engines Picking Up ChatGPT Conversations is the Latest Privacy Surprise for LLM Users

A cache of about 100,000 ChatGPT conversations that made their way to Google’s index is the latest surprise appearance of LLM exchanges in public. While OpenAI claims that users were properly informed, it has also since disabled the feature and is talking to search engines about de-indexing the conversations that they picked up.

Links created for personal sharing wound up indexed by search engines

Not all ChatGPT conversations were open to exposure, only those in which the user clicked on the “Share” feature at some point to create a sharing link. That essentially creates an archive of the conversation up to the point the button was pushed, at a URL comparable in type to the ones generated for sharing documents within Google Workspace. But the ChatGPT URLs can also be “discoverable” by search engines, something that numerous users appear to have not been clear on given the personal and sensitive information included in the chats.

The issue emerged about two months after Meta AI’s app was found to be sharing chats from random users via its “Discover” feature, something that also included sensitive information from users seemingly unaware of the possibility (and that prompted an overhaul of the feature due to general outcry).

These incidents reinforce the idea that anything put into an LLM can potentially emerge somewhere else in an unpredictable way, and that privacy is an ongoing issue that is yet to be fully and properly addressed by both the developers and by regulators attempting to play catch-up with the manic pace of development.

How safe are ChatGPT conversations, really?

Researchers were able to pull up the ChatGPT conversations simply by entering the common portion of the “Share” URLs into search engines. While this wouldn’t directly tie the conversations to a user identity in any way, the researchers say that some conversations contained enough information to either directly or indirectly expose the user.

The indexing does appear to have been intentional, with OpenAI describing it as an “experiment.” One that is now over, as confirmed by OpenAI CISO Dane Stuckey. Stuckey said that the company thought it might be a “helpful” way for useful conversations to be discovered, but that it needs to be re-examined given the way things unfolded.

Materials found by the researchers include medical information, personal conversations about intimate relationships, business contracts and even what appears to be a non-disclosure agreement for OpenAI employees. The incident once again demonstrates the urgent need for organizations to be on top of employee use of LLMs, both in terms of official company policy and “shadow IT” work assistance that may be conducted on personal devices or in some other unauthorized way. While this particular case is a simple matter of flagging things for indexing by search engines that should not have been flagged, anything entered into an LLM enters an opaque box and essentially moves beyond any ability to control or track going forward.