The idea is immediately appealing: take the computers already sitting nearby and let them run a model that will not fit on any one of them. exo is an experimental attempt to make that practical through device discovery and model partitioning.
A large-model demonstration shared in July made me look at the project. The connection to exo was speculative, but the engineering question was clear: what must cross the network when a model is spread across machines?
For a rough sense of scale, 405 billion parameters stored at four bits each already imply about 202.5 billion bytes of weight data before additional overhead. Quantization metadata, caches and runtime buffers add to the requirement. Two machines with large memory pools can make the capacity problem look more approachable, but their memory is not one uniform address space.
The network is part of the model path
Consider a simple partition in which one machine holds the early layers and another holds the later layers. The first computes an intermediate representation and sends it to the second. The second continues the calculation. The next generated token depends on the previous step, so latency can accumulate in places that are hard to hide.
That is different from sending two independent requests to two independent computers. Independent jobs can often proceed at the same time. A single model request has dependencies between its stages. More devices may increase the model size that fits without reducing the time needed for one response.
The type of workload matters as well. Processing a long prompt and generating one token at a time stress the system differently. A throughput number from one phase should not be treated as a promise about the other. Network bandwidth, latency, device speed and partition sizes all interact.
This makes the most useful first question surprisingly modest: does the model fit and complete a request reliably? Only after that would I compare speed, energy use or cost against a smaller model on one machine.
The appeal of exo is that it tries to manage some of the coordination rather than asking the user to hand-build every connection. Its July code and documentation describe an experimental system, which is the right expectation to bring to it. Automatic discovery is a helpful start; it does not make unlike devices identical.
I can see a place for this kind of setup in local exploration. A model that would otherwise be inaccessible becomes something you can inspect and use without sending each prompt to a remote service. The trade is additional setup and a performance profile shaped by ordinary networking.
The useful outcome is not necessarily the fastest possible chat window. It may be access to a particular model with the hardware already available. Keeping that goal explicit makes the experiment easier to evaluate. Fitting more and responding faster are both desirable, but they are separate achievements.