Open Source
Explore the latest AI open-source projects from GitHub and HuggingFace.
Explore the latest AI open-source projects from GitHub and HuggingFace.
**SIE** is Superlinked's open-source inference engine for self-hosting the open models an agent actually calls. Its premise is that a production agent rarely needs one model — it needs embeddings for search, a reranker, an OCR model to turn PDFs into markdown, an extractor for structured output, a safety classifier, and an LLM to run the loop itself. Teams usually end up with a separate model server per task. SIE replaces that patchwork with **one cluster serving 100+ models**, each loaded on demand. The API is **OpenAI-compatible** for drop-in migration, exposing `/v1/embeddings`, `/v1/chat/completions`, `/v1/completions`, and `/v1/responses`. The pre-configured catalog spans Stella, SPLADE, ColBERTv2, BGE-M3, Qwen3 rerankers, GLiNER, and SigLIP, with retrieval models benchmarked on MTEB. Multiple models are served simultaneously with on-demand loading and **LRU eviction**, so memory is reclaimed from cold models rather than being pinned per-server. The task taxonomy in the README is the clearest statement of scope: search (embed, match, rerank), document-to-markdown (LightOnOCR, GLM-OCR, MinerU, PaddleOCR-VL, Docling), structured output (GLiNER2, NuNER-Zero), content guarding via `granite-guardian-2b` with a thresholdable probability, and running the agent loop on an open LLM with streaming. Docker images are deliberately **bundle-specific** — the `transformers5` image carries the Transformers 5 OCR models while the `default` image does not advertise them — so dependency-incompatible model families stay isolated instead of colliding in one image. What separates SIE from a bare model server is that it ships the **whole production stack under Apache 2.0**: a load-balancing gateway, KEDA autoscaling with scale-to-zero, Grafana dashboards, and Terraform modules for GKE, EKS, and AKS, installable as a Helm chart. Python and TypeScript SDKs cover the same surface, and there are documented integrations for LangChain, LlamaIndex, Haystack, DSPy, CrewAI, Chroma, Qdrant, Weaviate, and LanceDB, plus an MCP edge package. One caveat worth knowing before deploying: SIE collects anonymous usage telemetry (version, OS, architecture, GPU type) by default, disabled with `SIE_TELEMETRY_DISABLED=1` or `DO_NOT_TRACK=1`. The project tagged **v0.7.0 on 2026-08-09**.