A developer has introduced the millfolio project, which proposes a new architecture for interacting with artificial intelligence, where privacy is ensured by separating the cloud-based planning process from the direct execution of code on the user's local device.
What Happened
The millfolio project implements a 'local-first' AI concept. Instead of sending files to a cloud model, the system transmits only an anonymized data schema (metadata) to the cloud. Based on this schema, a frontier model (such as Claude) generates a program (code), which is then sent to the user's device. The code execution takes place in a local sandbox on a Mac, where a local model processes the actual data values. The technology stack includes the Mojo language and custom Metal kernels for high-performance GPU inference.
Context
Traditional methods of using large language models often require transferring sensitive documents to the cloud for analysis. This creates risks of leaking personal or corporate information. The 'send the program to your data' approach (sending the program to the data, rather than the data to the model) changes the standard for LLM interaction, emphasizing privacy through a hybrid scheme.
Why It Matters for the Industry
The project demonstrates the viability of an architecture that separates cloud-based reasoning from local execution. This paves the way for creating secure personal AI agents and could lead to a transition from purely cloud-based RAG systems to hybrid models, which may become the standard for processing confidential data. Furthermore, the use of Mojo to optimize inference on consumer hardware confirms the language's potential for developing AI infrastructure.
Why It Matters for Users
For everyday users, this means the ability to leverage the powerful capabilities of models like Claude for deep analysis of personal notes, financial reports, or other private documents without the risk of uploading them to the internet. Only column names and data types are sent to the cloud, significantly increasing the level of personal information protection.
What Is Not Yet Known / Limitations
Experts point to a lack of specific data regarding latency, system scalability, and the complexity of integrating it with existing corporate systems.
Sources
Author
Look at AI, Editorial Team