Open Source
Explore the latest AI open-source projects from GitHub and HuggingFace.
Explore the latest AI open-source projects from GitHub and HuggingFace.
TypeLLM brings type-safe generation to existing autoregressive LLMs without changing their architecture or weights, letting a model keep its native free-form thinking while producing schema-guaranteed outputs (string, integer, number, boolean, or enum choice) through JSON Schema. It is built on SGLang and requires a compatible model to already be served through an SGLang HTTP endpoint; TypeLLMClient then points at that endpoint and sends a context plus a dict of typed questions, each with an instructions field, an optional enum list, and an optional depends_on list that forms a dependency graph so fields needing earlier results run after them while independent fields run together and reuse a shared cached prefix. The project describes itself as inspired by TypeSafe AI's Jev, applying a similar idea of constrained, single-token categorical decisions to open autoregressive LLMs served on SGLang. Additional features include an optional thinking mode with a per-field token budget, image input for vision-language models tested with Qwen/Qwen3.8-27B, return_probabilities for enum and boolean fields, sampling with temperature and seed control, and per-question permutation averaging that evaluates multiple orderings of enum options to reduce order bias. Tested checkpoints include Qwen/Qwen3.8-27B, the Qwen3.5 family at 0.8B/4B/9B, openbmb/MiniCPM5-1B, and inclusionAI's Ling-mini-2.0 and Ring-mini-2.0. On 231 public JevBench tasks evaluated on 2026-09-23, using the RadixArk/Qwen3.8-27B-NVFP4-BF16-LMHead checkpoint with no additional training, TypeLLM scored 195/231 without thinking and 228/231 with thinking; the project's own results page notes this is one run per setting on the public subset rather than the full 534-task benchmark, and that the thinking run averaged 918.9 tokens per task with no explicit thinking-token budget in that evaluation. The project is licensed under the Apache License 2.0.