llmshift MCP server

LLMShift

Switch AI models without breaking your product.

GPT-4o, GPT-4o mini and GPT-4.1 retire on 14 January 2027, GPT-5 and o3 on 11 December, Claude Sonnet 4.5 on 30 November. Does your code call one of them? Find out in seconds, with nothing to install:

uvx llmshift scan        # or: pipx run llmshift scan

_{A real run: <code>scan</code> on a sample repository, then <code>compare</code> replaying llmshift's demo traffic on Claude Haiku 5.5.}

Then test the replacement on your real traffic before the shutdown date: llmshift snapshot turns your logs into tests and llmshift compare --to <replacement> shows what changes. llmshift demo walks through it with simulated models, no API key needed.

Providers now retire models roughly every six months. When a model's behaviour changes, outputs can look fine and still be wrong, and nothing raises an error. LLMShift is built around the moment of switching models, from finding the calls to proving the new model works:

| Command | What it does | |---|---| | Find what has to move | | | llmshift scan | Finds every model your code calls (Python, JS/TS, Go, Java, configs, .env, notebooks) and warns about retirement dates. | | llmshift digest | One message (Slack, Teams, e-mail, ...) listing the models your repositories use that are retired or retire soon. | | Turn real traffic into tests | | | llmshift snapshot | Turns real usage logs, including multi-step agent runs, into a test set automatically. You write no tests. | | llmshift import openai-evals | Brings in OpenAI Evals tests. The Evals platform goes read-only on 31 Oct 2026 and its API shuts down on 30 Nov 2026. | | Choose the new model | | | llmshift compare | Runs the tests on the new model and shows exactly what changed: broken formats, wrong facts, different tone, different tool calls. | | llmshift pick | Tries several cheaper models at once and recommends the cheapest one that still passes, with the monthly saving. | | llmshift safety | Does the new model refuse more, follow injected instructions or leak private data? | | llmshift embeddings | Shows how search results change before you switch embedding models. | | Switch | | | llmshift fix | Suggests prompt changes until the new model matches the old one, and points to where each prompt lives in your code. | | llmshift upgrade | Swaps a retiring model in your code and opens a pull request with the test results ("Dependabot for AI models"). | | llmshift check | On every pull request, tests only the prompts it edits, and comments with the result. | | After switching | | | llmshift shadow report | Checks a new model on a share of live traffic, copied in the background (shadow mode). | | llmshift watch | Runs a scheduled check against the live model and alerts you (Slack, Teams, GitHub issue, e-mail, ...) when its behaviour drifts. |

AI coding assistants can use all of this as tools: llmshift mcp is an MCP server for Claude Code, Cursor and VS Code.

LLMShift is a single pure-Python package with no compiled code: its only dependency is tomli on Python 3.9 and 3.10 (to read pyproject.toml), and a copy of PyYAML's pure-Python YAML parser is included. It talks to OpenAI, Azure OpenAI, Anthropic, Gemini and any OpenAI-compatible endpoint over plain HTTPS, so there are no SDK version conflicts. You can also run it as a GitHub Action.

pip install llmshift
llmshift demo          # 1-minute offline walkthrough with simulated models (no API key)

It runs on Python 3.9 to 3.15 on Linux, macOS and Windows, and installs without a compiler on any of them.

Quick start with your own app