EPFL researchers have unveiled a new software, now turned into a start-up, that could significantly disrupt how artificial intelligence (AI) is utilized in everyday tasks. This innovation eliminates the necessity for data to be transmitted to third-party cloud services, a move that poses a potential challenge to the established business models of major tech companies.
The use of AI has surged over the past three years, increasingly incorporating sensitive data such as patient records, customer requests, and confidential work documents. Traditionally, when AI is tasked with an operation, it starts locally on the user’s device, subsequently sending the query to the cloud. Here, powerful hardware processes the request, generating a response that is then relayed back to the user. This entire operation, known as inference, places significant reliance on vast data centers that are also instrumental in training AI models, including ChatGPT, Gemini, and Claude. Currently, both inference and training capabilities are predominantly held by large tech firms.
Researchers Gauthier Voron, Geovani Rizk, and Rachid Guerraoui from the Distributed Computing Laboratory (DCL) at EPFL have developed software that enables users to download open-source AI models and operate them locally without the need for cloud support. The software, named Anyway Systems, orchestrates and consolidates distributed machines on a local network into an on-premise cluster, employing robust self-stabilization techniques to optimize the use of local hardware. This challenges the prevailing notion that extensive data centers are essential for deploying AI models.
Installation of Anyway Systems is straightforward, taking as little as thirty minutes on a network of local machines. Crucially, this ensures that no data leaves the local network, thereby guaranteeing privacy and sovereignty. For example, an AI model as large as GPT-120B, the latest and most substantial open model from OpenAI, can be downloaded and deployed on Anyway Systems within minutes, requiring merely four machines equipped with one commodity GPU each, costing approximately 2300 CHF each. This contrasts starkly with the previously assumed necessity of investing around 100,000 CHF in specialized rack enclosures to run an AI model.
Professor Rachid Guerraoui, head of the DCL, noted, “For years, people have believed that it’s not possible to have large language models and AI tools without huge resources, and that data privacy, sovereignty, and sustainability were just victims of this, but this is not the case, and smarter, frugal approaches are possible.”
Concerns surrounding data security and privacy are heightened when data is sent to the cloud, particularly regarding whether that data may be employed to further train or improve AI models. The reliance on major global cloud providers for AI services also raises questions about AI sovereignty, as it transfers control over vital national assets—data, algorithms, and infrastructure—from domestic entities to multinational corporations. Furthermore, the substantial computational resources required for cloud-based AI queries—estimated to consume between 80 to 90% of AI-related computing power—are triggering rapid growth in data centers that consume vast amounts of energy and water.
Guerraoui explained that while Anyway Systems excels in inference, it could also facilitate a reduction in the resources required for training AI models. Pilot testing has indicated that deploying a model on a network of local machines may introduce slight latency in response time but does not compromise accuracy.
The concept behind Anyway Systems builds upon earlier variants of the algorithm developed by the DCL, which has been engaged in distributed computing, fault tolerance, optimization, and privacy issues for years. The researchers explored the potential of applying self-stabilization techniques to AI, yielding promising results. “As a lab, we might be unique in working on robust distributed computing and machine learning together from both a theoretical and practical perspective,” Guerraoui remarked. “They worked even better, and the result is almost too good to be true.”
Recently, Anyway Systems was selected as one of six inaugural grantees of the Startup Launchpad AI Track, an initiative powered by UBS, which is Switzerland’s first grant program dedicated to AI. Chosen from over fifty proposals, these projects are receiving funding and tailored support to expedite their journey from prototype to market readiness. The software has surpassed the prototype stage and is currently undergoing testing in various companies and government administrations throughout Switzerland, including EPFL itself.
Professor David Atienza, associate vice-president of research centers and technology platforms at EPFL, stated, “Anyway Systems represents an interesting and appealing technology that optimizes resource usage while ensuring data security and sovereignty and could be an AI gamechanger.” Its sustainable approach aligns with the evolving needs of EPFL’s advanced computing platforms.
Although Anyway Systems is not yet feasible for use on individual desktop or laptop computers, the trajectory of computing history suggests rapid optimization is likely. Guerraoui concluded, “What we’re saying is that we will be able to do everything locally in terms of AI. We could download our open-source AI of choice, contextualize it to our needs, and we, not big tech, could be the masters of all the pieces.”
The implications of this technology, if realized, could redefine the landscape of AI deployment and control, placing power back into the hands of individual users while challenging the dominance of large tech firms.
See also
DeepMind’s Shane Legg Predicts 50% Chance of Minimal AGI by 2028
India Surges to Third in Stanford AI Competitiveness Index, Overtaking UK and South Korea
Nvidia’s H200 Export Decision Fuels Alibaba’s AI Growth Amid Geopolitical Risks
AI Breakthrough by DoMore Diagnostics Revolutionizes Colorectal Cancer Detection, Minimizes Chemotherapy Risks
