Open Source
Explore the latest AI open-source projects from GitHub and HuggingFace.
Explore the latest AI open-source projects from GitHub and HuggingFace.
**Needle 2** is Cactus Compute's open **45M-parameter model for tool calling, device use, and structured extraction**, shipped as a **single 14MB binary that runs a full session in roughly 28MB of RAM**. The weights are compressed to CQ2-bit with Cactus Quants and baked directly into the inference engine, so there are no separate model files to manage and inference makes no network calls. The repository is the Python package around it — `pip install cactus-needle`, describe your tools, call them — with the engine fetched once from Hugging Face and cached. It is currently on GitHub Trending at **4,315 stars and 308 forks**. The architecture is what Cactus calls a **Simple Attention Network**: a Hadamard MLP in place of the FFN, GQA attention, engram key-value memory over hashed n-gram tables, and multi-lane hyper-connections, with the design and ablations documented in **arXiv:2607.18363**. The Walsh-Hadamard transform is a fixed matrix applied in n log n time with no weights to read, which is a large part of how the parameter count stays at 45M. The README positions Needle 2 against **FunctionGemma 270M, LFM2.5 230M, and Apple FM** — trading wins at **5x to 70x smaller and 2 bits against their f16** — though it presents this as a size-quality frontier chart rather than a published benchmark table, so the specific per-task scores are not stated in the repo. The operating contract is unusually strict, and that strictness is the product. **Every problem is solved as a function call**: the context declares what may be called, the model answers with calls, and a **byte-level grammar compiled from your JSON schemas constrains every emitted token**, so malformed output is structurally impossible rather than merely unlikely. A request no declared tool can serve returns the **empty call `[]`** — there is no free-text fallback at all. Arguments contain only values evidenced in the input; an optional field with no evidence is omitted rather than guessed. Constraints declared through `needle.Field` (ranges, patterns, lengths, item counts, enums) compile into that same decode grammar, so out-of-range values cannot be produced. Two mechanisms handle the practical problems of running an agent loop on a phone. **Tool retrieval** kicks in above five declared tools: a built-in contrastive head embeds every schema once at init, embeds the query each turn, and admits only the top five tools into context with the grammar rebuilt over that subset — an unselected tool becomes unreachable, not merely unlikely — and `tool_index_path` persists those embeddings keyed by a fingerprint so unchanged schemas load instantly. **Bounded memory** comes from a 256-token sliding window with the tools pinned as KV sinks, holding total memory near 28MB no matter how long the conversation runs. Every response also carries a **confidence score** that is the minimum of a calibrated post-hoc head and the decode probability of the call tokens, so the intended failure mode is escalation to a larger model rather than wrong execution. Structured extraction is not a separate mode: declare the record as the only tool and the returned `arguments` are the extracted fields, with a typed `extract()` helper for Pydantic models. An optional system turn carries environment **facts, never instructions** — `date`, `locale`, `device`, `battery`, `network`, `location`, `user`, `assistant` — and relative language like "tomorrow at 7" resolves to an absolute time only when a `date:` fact licenses it. Fine-tuning is LoRA on the frozen base, merged at export into another single `.cact` file that runs on the same engine with no recompilation, and `needle playground` serves a local browser UI for trying tools before committing. The limits follow from the design. There is **no conversational ability to fall back on** — off-topic input gets an empty array, which is correct for an on-device action router and useless for anything chatty. Optional data synthesis for fine-tuning requires an `OPENROUTER_API_KEY`, so the cheapest path to a tuned model depends on a hosted API even though inference does not. Weights live at **huggingface.co/Cactus-Compute/needle2** under Apache-2.0, with the Python package itself MIT.