SkillOpt is an open-source optimizer from Microsoft that trains reusable, natural-language "skills" for frozen LLM agents. Instead of fine-tuning model weights, it treats a plain-text skill document as the thing to be optimized, using scored agent runs to iteratively rewrite it into a better set of instructions. Released under the MIT license, the project has passed 14,000 GitHub stars by targeting a problem that has become common as teams build on top of hosted models: you often cannot retrain the model, but you can change what you tell it, and doing that well by hand is slow and unreliable. SkillOpt turns that manual prompt-tuning into a measurable optimization loop. ## What It Is SkillOpt frames a skill — a Markdown document of guidance an agent reads before acting — as trainable state. The model itself stays frozen; only the skill text changes. The system runs an agent on a task, scores the resulting trajectory, and feeds those scored rollouts to a separate optimizer model that proposes bounded edits: small add, delete, or replace operations on the skill document. The full cycle is rollout, reflect, aggregate, select, update, and evaluate, repeated over many tasks until the skill converges. The output is a compact `best_skill.md` artifact, typically 300 to 2,000 tokens, that can be dropped into a deployment unchanged. ## Validation-Gated Optimization The most important design choice is that edits are gated by validation. A proposed change only updates the live skill if it strictly improves performance on a held-out validation set, which guards against the optimizer overfitting to a handful of lucky runs or drifting into verbose, unhelpful instructions. To keep the process stable, SkillOpt adds a textual "learning-rate" budget that limits how much the skill can change per step, a buffer that remembers rejected edits so they are not retried, and epoch-wise meta-updates. Because the optimized skill is just text that the target model reads, there is no additional inference cost at deployment — the improved agent runs exactly as fast as the original. ## SkillOpt-Sleep and Continuous Evolution Beyond one-off optimization, the project ships SkillOpt-Sleep, an offline self-evolution mode intended to run on a schedule, for example nightly, to keep refining a skill locally as new task data accumulates. This positions SkillOpt not only as a training-time tool but as a maintenance loop for deployed agents, letting a coding or search agent's guidance improve over time without a human rewriting the prompt after every regression. A WebUI dashboard is included for monitoring training progress and visualizing how a skill evolves across iterations. ## Backends, Benchmarks, and Transfer SkillOpt is provider-agnostic, with documented support for OpenAI, Azure, Claude, Qwen, MiniMax, and other OpenAI-compatible endpoints, and it ships six built-in benchmarks including SearchQA and harnesses for Codex CLI and Claude Code CLI. The accompanying paper reports gains across 52 evaluated (model, benchmark, harness) combinations, with double-digit point improvements on strong base models in several settings. A notable claim is transfer: a skill optimized on one model or harness often carries over to a different one, which — if it holds across a team's own tasks — would let expensive optimization be amortized across several deployments rather than repeated for each. ## Usability in Practice For teams already running agents on hosted models, SkillOpt is straightforward to adopt: install via pip, point it at a provider, supply a task set with a scoring function, and let it produce a skill file. The value is highest when you have a clearly measurable task — a benchmark, an eval suite, a graded search or coding workflow — because the whole loop depends on a reliable score to optimize against. The honest caveats are that it requires enough labeled or gradable examples to form a meaningful validation set, that repeated rollouts consume real API tokens during optimization, and that the published benchmark numbers are the authors' own and should be reproduced on your own tasks before being trusted. It is also a young, research-driven codebase that will change quickly. ## Pros and Cons The strengths are clear: SkillOpt optimizes agents without touching model weights, produces small and portable skill artifacts, adds zero inference overhead at deployment, and is gated by validation to reduce overfitting. It supports many providers, ships benchmarks and a monitoring UI, and offers a continuous offline-evolution mode. The trade-offs are that it needs a well-defined scoring signal and a validation set to work at all, that optimization runs cost tokens, that transfer and benchmark claims are vendor-reported and task-dependent, and that the project is early and evolving. ## Outlook SkillOpt sits squarely in a growing area: as more capability is built on frozen, hosted models, the leverage moves from weights to instructions, and systematic ways to optimize those instructions become valuable. Microsoft backing, an accompanying paper, and a concrete validation-gated method give it more rigor than a typical prompt-tuning script. The main thing to watch is how well the reported transfer and gains generalize beyond the built-in benchmarks to messy, real-world agent tasks. ## Conclusion SkillOpt is a credible, well-documented tool for teams that want to improve LLM agents they cannot retrain. It is most compelling where success is measurable and where the same optimized skill can be reused across models or harnesses. Anyone hand-tuning agent prompts against an eval suite should evaluate whether SkillOpt's automated, validation-gated loop can do that job more reliably.