Sluice
The sovereign model gateway arbitrating all LLM traffic across the ecosystem. A thin LiteLLM proxy on Azure Container Apps that presents one OpenAI-compatible surface to every consumer, while centralising auth, rate limiting, semantic caching, and telemetry in one place. No client needs its own credentials, retry logic, or usage tracking.
curl -X POST https://sluice.celladoresystems.com/v1/responses - Drop-in OpenAI compatible
- existing clients point at Sluice with a base-URL change — no rewrites
- Per-workspace virtual keys
- RPM, TPM & spend caps enforced at gateway; kill switch doesn't touch consumer code
- Redis semantic cache
- optional — cuts repeat-prompt cost and latency without changing call sites
How Sluice Connects to the Stack
Celladore is engineered as a unified, composable toolchain where every tool feeds upstream contexts and downstream verification loops without friction.
→ Docket
Sluice emits OpenTelemetry spans and Azure OpenAI diagnostic events that Docket ingests for per-project spend attribution.
→ Retort
All Retort agent LLM calls route through Sluice, gaining failover, rate limits, and usage attribution without changing a single line of agent code.
→ Deck
Deck reads Sluice's unauthenticated /metrics endpoint directly for gateway health tiles and throughput sparklines in the Dashboard panel.
Technical Matrix
- Runtime & Engine:
- Python 3.12 / LiteLLM Proxy / FastAPI
- Deployment Target:
- Azure Container Apps (Dynamic Autoscale)
- Communication Protocol:
- OpenAI REST (/v1/responses, /v1/embeddings, /v1/audio/transcriptions)
- Licensing Tier:
- Proprietary (Private — Internal Infrastructure)
Invented from Real Scars
Rogue storytelling agent loops racked up thousands in token bills in a single week with zero budget visibility and fatal provider outages disrupting live inference.
Provider downtime, unmetered API key leakage, and runaway recursive model loops.
Architectural Features & Guarantees
Modular design built for high velocity, deterministic reliability, and strict enterprise isolation.
Centralized Auth & Credential Isolation
One shared gateway key replaces Azure OpenAI credentials scattered across every consuming project. A single revocation kills access everywhere instantly.
Rate & Budget Governance
Per-key RPM, TPM, and spend caps enforced at the gateway with a kill switch that doesn't require changes to any consumer code.
Built-in Observability
Prometheus metrics, OpenTelemetry tracing to Application Insights, structured Log Analytics logs, and optional Langfuse LLM observability — without instrumenting callers.
Semantic Response Caching
Optional Redis-backed cache cuts repeat-prompt cost and latency. Identical or semantically near-identical prompts return cached responses transparently.
Per-User Model Selection
FastAPI State Service lets each user select their preferred model from a governed catalog — backed by Azure Cache for Redis in production.
Multi-Environment Deployment
Independent dev, staging, and prod deployments on Azure Container Apps — each with its own Container Apps, Key Vault, and optional Redis instance.
Typed API Contract & Snippets
Live cURL, TypeScript, and Python invocation payloads for Sluice.
Unified Developer Interface
Route LLM completions through LiteLLM on Azure Container Apps with per-workspace virtual key metering and budget tagging.
import OpenAI from "openai";
const client = new OpenAI({
apiKey: process.env.CELLADORE_VIRTUAL_KEY || "YOUR_VIRTUAL_KEY",
baseURL: "https://sluice.celladoresystems.com/v1",
defaultHeaders: {
"X-Celladore-Budget-Tag": "project:celladore-web",
},
});
const response = await client.chat.completions.create({
model: "claude-3-7-sonnet-latest",
messages: [
{ role: "system", content: "You are an autonomous engineering agent." },
{ role: "user", content: "Verify PR quality gate requirements and summarize." },
],
temperature: 0.2,
});
console.log(response.choices[0].message.content);Status Notice: Sluice is live and carrying production traffic across the ecosystem as our internal model gateway. It's infrastructure, not a product with its own signup. Contact us to deploy sovereign gateway infrastructure for your org. Get in touch with the founder .