GENERATIVE AI · PART OF THE GCC & FDE GENAI PRODUCT SUITE

Model OffloaderGenAI-driven query routing that stops paying cloud prices for easy questions.

A Generative AI router that sits beside your copilot or chat client. It caches repeats, answers easy questions with a local generative model, and only spends cloud AI tokens on the queries that truly need a frontier model.

Your copilotCopilot / Claude / any chat
→
Cacheexact + semantic, SQLite
→
Classifycomplexity score 0–1
→
Local modeleasy → $0, Ollama
→
Cloud modelhard → premium
01 / LIVE ROUTER

Type a question.
Watch it decide where to send it.

This runs the product's real Generative AI routing heuristic, ported line for line from our production engine — not a staged animation. It scores the text, explains why, and shows which model would answer it.

offloader · classify() · live in your browser
ROUTE
LOCAL
short, easy cues: what is
score
0.10
Easy threshold 0.40 (default, configurable). Below it → local model, $0. At or above it → cloud model. The real router also escalates automatically when a local answer comes back empty or weak.
0$ FOR CACHED / LOCAL ANSWERS
5MCP + CLI TARGETS SUPPORTED
13AUTOMATED TESTS
0THIRD-PARTY PY DEPENDENCIES
02 / HOW IT WORKS

One router.
Three honest outcomes.

No black box. The classifier is a transparent heuristic you can read in one file, and the safety net — escalation on a weak local answer — keeps a wrong guess from becoming a wrong final answer.

01 — CACHE

Exact and semantic

Repeat and near-duplicate questions are answered from SQLite with no model call at all. Semantic matching uses Ollama embeddings when installed, with a strict trigram fallback otherwise.

02 — CLASSIFY

Transparent scoring

Length, hard cues ("design a", "refactor", "debug", "step by step"), easy cues ("what is", "translate", "define"), code blocks and multi-part asks move the score. Every verdict comes with reasons.

03 — ESCALATE

Cloud as the safety net

Hard queries go straight to the cloud model. An empty, refusal-shaped, or failed local answer escalates automatically — so "easy" is a starting guess, not a guarantee.

03 / USE IT WHERE YOU ALREADY WORK

Your copilot.
Two new tools, nothing else changes.

As an MCP server it gives Copilot or Claude Code two tools: a free local/cached answer for easy sub-questions, and a stats readout. It never calls the cloud itself — that stays your copilot's job.

Copilot / Claude Code chat
# Ask your copilot, once the MCP server is registered:
> Use offloader_ask to define idempotent in one sentence.

<- "An idempotent operation is one that can be repeated any
    number of times without changing the result..."
    [offloader: route=local, saved≈$0.0007]

# Ask again — served from cache, 0 ms:
> Use offloader_ask to define idempotent in one sentence.
<- route=cache-exact

# Hard questions are declined so your copilot answers them itself:
> Use offloader_ask: design a distributed rate limiter...
<- {"declined": true, "reason": "too complex for local"}
GitHub Copilot (VS Code) Claude Code Claude Desktop Cursor OpenAI-compatible proxy for anything else
04 / INSTALL

One click, once licensed.
Percent and time shown as it runs.

Once access is granted, the installer checks Python and Ollama, installs whatever is missing, downloads the models, registers the MCP server in every client it finds, and runs a real smoke-test question — with a live progress bar the whole way.

Windows
# after access is provided:
Setup.bat
# installs Python/Ollama if missing, pulls the models, registers the
# MCP server in Copilot / Claude Code / Claude Desktop / Cursor,
# shows percent + elapsed/ETA, then runs a real test question.
macOS / Linux
# one-click Setup.bat is Windows-only today; the CLI runs anywhere:
python -m offloader doctor
python -m offloader install --target claude-code

This is proprietary software, licensed on request — not open source, no public source access.

05 / GET ACCESS

Part of the GCC & FDE
Generative AI product suite.

Contact us to license Model Offloader for your team or platform.

Contact us for access