Open Source
Explore the latest AI open-source projects from GitHub and HuggingFace.
Explore the latest AI open-source projects from GitHub and HuggingFace.
Kolibri 1 is Aleph Alpha's mixture-of-experts reasoning model for German and English, trained from scratch and published as FP8 weights under Apache-2.0. It has 78B total parameters with 3.46B active per token, 50 layers, 384 routed experts per MoE layer (6 selected per token, plus 1 shared) and a 4:1 mix of sliding-window and full GQA attention. A tokenizer tailored to German word structure is used, and the model supports an explicit reasoning mode (reasoning_effort low, medium or high, or off) and Hermes-style tool calling. Pre-training used 20T tokens (about 62.5% English, 23.9% German, 13.6% code). Native context is 262,144 tokens, validated up to 1,048,576; the card recommends at most 262,144 tokens for serving efficiency and complex tasks. Weights are float8_e4m3fn with an FP8 KV cache used in evaluation; the footprint is about 78 GB, with a minimum of 2x A100 80 GB, 2x H100 SXM5, 1x H200, 1x B200 or 1x B300. Serving needs the aleph-alpha-inference package (a vLLM plugin) or its container image. In the card's post-training table, Kolibri scores 75.5 overall in English and 70.8 in German, against 74.7 and 69.8 for Qwen3.5 35B-A3B and 72.3 and 70.2 for GPT-OSS 120B; the dense 27B Qwen3.8 scores 80.2 and 79.9. On GPQA Diamond (EN) it reaches 84.3 and on AIME 2026 (EN) 96.0. Agentic results are mixed: TerminalBench 2.1 is 27.7 and BFCL v4 overall 61.4, below several 3B-active baselines. The Apache-2.0 grant covers only the published weights and configuration files, not code, architecture or training method, and training data is not released; a public training-data summary is linked. A BF16 variant, Kolibri-1-BF16, is also published under Apache-2.0.