Exo is an open-source software project that connects a group of machines together, and pools their computing power to create a private AI cluster. Collectively, they can run powerful Large Language Models that can’t fit on a single device.
This opens up a variety of opportunities for both individuals and businesses, as it effectively allows enterprises to run local AI models across their existing hardware.
Exo is built on top of a framework called MLX, which is heavily optimized for Apple Silicon, and non-Apple hardware support is currently more limited. So what about Intel?
Estimates put Intel-based machines at around 65%–75% of all business-owned hardware, making the premise of a private AI cluster a compelling one.
In this article, we’ll explore alternatives to Exo which have robust Intel hardware support, and that work across both Windows and Linux machines.
What makes good Intel support?
Before getting into solutions, it’s worth understanding what makes good Intel support for AI inference in the first place.
Where Exo utilizes a framework called MLX for Apple, there are a few different pieces of Intel-based tooling for LLMs. The main two are OpenVINO and SYCL.
Splitting a large model across devices and running inference usually involves these tools as building blocks that are expanded to support multiple machines.
OpenVINO
OpenVINO is a toolkit and runtime developed by Intel which optimizes and executes AI models across a wide range of Intel hardware. It includes support for CPUs, iGPUs, and most notably NPUs, making it a good base for machines with Intel Core Ultra processors.
OpenVINO can work with models in most popular formats, but expects them to be converted, optimized, and quantized in a special format called OpenVINO IR before execution.
You can learn more about how it stacks up to other popular runtimes for single machine inference in our comparison article here.
SYCL
SYCL is a programming standard built for parallel computing: something extremely valuable for LLM inference, especially for GPUs.
SYCL in particular has robust support for Intel’s lineup of discrete GPUs, all the way from their consumer Arc series, to their data-center grade D-series GPUs.
As SYCL is a compilation target, it isn’t typically something that end users interact with directly to do inference. Instead, it’s built into many inference runtimes as a backend that a user can choose for execution, and Intel themselves use it in their oneAPI toolkit.
Distributed Inference Runtimes for Intel
When it comes to doing distributed AI inference for Intel machines, your options are a little more limited. Here are some of the best alternatives and the use cases where they might be applicable.
llama.cpp RPC
llama.cpp is one of the most well-known inference runtimes, and the llama.cpp RPC project allows users to split their model across multiple machines to run inference.
The downside is that llama.cpp RPC itself is marked as a proof-of-concept/in development, so it may not be the most robust solution to deploy for businesses.
Similarly, support for OpenVINO is either limited or experimental.
vLLM and Ray
vLLM is an inference runtime designed for serving AI models with high throughput. In its default state, vLLM is built to serve AI inference to many users, but from a single server.
Paired with Ray, a distributed computing framework built in Python, multiple instances of vLLM can be used, and large models can be split across them. Like llama.cpp, vLLM offers support for OpenVINO and the now archived IPEX-LLM runtime for Intel Xeon and Gaudi servers.
While powerful, the difficulty with vLLM + Ray is orchestration. This kind of setup is ideal for enterprises with their own data center grade hardware or dedicated servers. For those looking to run a private cluster across commodity hardware (like laptops and PCs), this might be a complex option.
Cascadia
Cascadia functions similarly to Exo: it allows users to pool multiple machines together into a single private AI cluster, and can run powerful AI models across devices.
Cascadia was co-developed with Intel, is built on top of OpenVINO, and is uniquely optimized for multi-device Intel setups. We even managed to get a frontier-level model running across eleven Intel AI PCs.
The runtime is open-source, similar to Exo, and available to use for individuals or those interested in self-hosting Cascadia. We also offer Cascadia Enterprise as a batteries-included version for businesses.
Need help running AI locally?
We’ve helped numerous teams get set up with local AI workflows, fine-tuned models for their use cases, and helped them reduce their AI spend. All while running inference completely on their own infrastructure using Cascadia.
If you want to run AI locally but don’t know where to get started, get in touch and we’ll help you out.
Related articles

Research
Why Private AI Matters for Businesses
Private AI protects your business data and your competitive edge. Here’s why it matters, and how running AI locally keeps your data on your own hardware.
Read more
Announcement
Use Apple Foundation Models as Claude Code Subagents
yeschef now supports Apple Foundation Models on macOS 27, so Claude Code can delegate smaller tasks to Apple’s on-device model and save on your Claude usage.
Read more
Research
Local AI on Business Laptops: What Hardware Do You Need?
Many business laptops and workstations can now run capable AI models locally. This post covers what running AI on-device actually involves and how much RAM you need for different kinds of work.
Read more
Research
Running Inkling (975B) Across Eleven Intel PCs with Cascadia
We ran Thinking Machines' Inkling (975B) across eleven Intel Core Ultra PCs using Cascadia, reaching a peak of 60.29 tok/s of aggregate decode throughput with 88 concurrent requests.
Read more
Research
How NPUs Help with Local LLM Inference
Most modern laptops now ship with an NPU, but few people use it for running LLMs.
Read more
Research
Building a Permissioned, Peer-to-Peer AI Inference Network
How a permissioned peer-to-peer system can turn a fleet of PCs into a single, on-prem inference endpoint.
Read more


