Engineers Thomas Dimson (ex-OpenAI) and Joey Flynn have launched a web service called In the Weights, which allows for assessing how deeply information about specific individuals or companies is embedded in the weights of popular large language models such as GPT, Claude, Gemini, and Llama.
What Happened
The In the Weights platform makes direct queries to neural networks in an isolated environment without internet access. This eliminates the influence of web search and RAG (Retrieval-Augmented Generation), focusing exclusively on the models' parametric memory. Based on the responses received, the system calculates a score reflecting the probability of certain data being present in the training datasets.
Context
This evaluation method allows for the isolation of knowledge embedded directly into the weights (parametric memory). The project demonstrates the complexity of the machine unlearning process—removing data from model parameters—as information that has entered the weights becomes virtually unretrievable from the digital footprint.
Why It Matters for the Industry
The project serves as a tool for auditing the aggressive data scraping methods used by large corporations. It opens up opportunities for forming new market niches at the intersection of AI data auditing, reputation management, and cybersecurity, and also allows for assessing the degree of model contamination with sensitive information.
Why It Matters for Users
The emergence of such a metric could turn presence in AI memory into a new indicator of public significance, comparable to SEO, but operating at the level of fundamental neural network knowledge. This provides users with a tool for a primary audit of personal data leaks and a way to check how deeply brands are integrated into the core knowledge of AI.
What Is Not Yet Known / Limitations
There is a difference in how the tool's readiness is assessed: experts disagree on whether it is purely a research tool (eval tool) or a foundation for commercial systems of automated privacy control and LLM-SEO.
Sources
Author
Look at AI, Editorial Team