Open Source
Explore the latest AI open-source projects from GitHub and HuggingFace.
Explore the latest AI open-source projects from GitHub and HuggingFace.
Backburner is a llama.cpp fork plus an iPhone app that lets a Mac and a USB-C-connected iPhone share the work of running Qwen3.8-27B locally. For prompt reading, the Mac runs layers 1-40 of each 256-token batch while the iPhone GPU runs layers 41-64, pipelined. The project reports 29-44% faster prefill at 16k-48k of existing context, measured by reading a 2,000-token file into a saved session on Qwen3.8-27B IQ4_XS, a MacBook Pro M4 Pro 24 GB and an iPhone 17 Pro Max: 157 vs 109 tok/s at 16k, 130 vs 101 at 32k, and 113 vs 87 at 48k. The phone joins only reads longer than about 512 tokens, so small agent steps stay on the Mac. Writing speed below 64k comes from the fork's own Mac kernels (SME2, Metal fusions, DFlash2 speculative decoding), not the phone: on the same setup decode runs at 25.0 tok/s at 27-33k context versus 11.3 for stock llama.cpp, and 25.1 with the phone attached. Past 64k the phone holds the oldest KV pages and computes attention over them on its GPU and Neural Engine, so context can stay 8-bit where the Mac alone would switch to 4-bit. The server sizes total context from the phone's free memory (196k-229k tokens at 8-bit on an iPhone 17 Pro Max), with end-to-end tests to 128k at 8-bit and 140k at 4-bit; decode at 128k is 12.6 tok/s. Greedy output was token-identical with and without the phone in the reported checks. The Mac serves an OpenAI-compatible API on port 8080, one request at a time. Requirements are an Apple Silicon Mac, an iPhone 15 Pro or newer, and a 10 Gb/s USB-C cable; the app installs through AltStore or Xcode, and the setup downloads about 24 GB of models. Tested hardware is an M4 Pro Mac with an iPhone 17 Pro Max and 16 Pro Max; M-series iPads install but are untested. Past 64k the app must stay open in front, and the server stops if the phone is unresponsive for 15 s. The project is pre-release and MIT licensed.