Engineer A’s GPT-4o feature was beautiful in the demo. 6 weeks on, the bill was £14K for the month with no per-customer attribution, and they were rolling the feature back on Monday morning.
Service OpenAI integration · GPT-5 + cost engineering · UK
OpenAI integration for the team whose bill. doubled overnight.
Rebuilt the way the Cookbook tells you to: strict json_schema with Zod or Pydantic, idempotency keys on every tool call, prompt-cache-aware prompts, tiered model routing, and an eval suite that catches regressions before your CFO does.
24hreply, from a senior
200+projects shipped since 2019
Senioronly, on the spine
(Why teams sign)
The OpenAI code your senior engineer would have written. If they’d done it thirty times.
We use OpenAI’s best models where they win, and wrap the provider so a price hike or a deprecation never holds your roadmap hostage. Structured outputs and per-tenant budgets on every production call.
0+
0×
0%
Three engineers, three OpenAI bills, three CFO conversationssame feature · £14K vs £3.8K · the gap isn’t the model
Engineer A shipped it in a long weekend. 6 weeks later the bill was £14K.
Engineer B shipped the same feature with a per-call token cap and a model router that uses GPT-4o-mini where it can. Bill: £3.8K. Same accuracy, same customer experience, a very different conversation with the CFO.
The gap between A and B isn’t the model. It’s the engineering discipline around the model. That’s the work we do, and the cost-engineering ladder below is how we do it.
What we wire from the OpenAI platform
Six capabilities, one discipline
Each one wired behind the same gateway, budget caps, and eval gold set. No special-casing, no bypass.
Chat
GPT-5 + 4o reasoning
Routed per task. Frontier where needed, 4o-mini for the rest.
Vision
Document + image understanding
OCR, charts, screenshots, scanned forms. GPT-4o multimodal.
Voice
Realtime + Whisper
Low-latency speech-to-speech for voice agents. Whisper for transcription.
Structured
Schema-validated outputs
JSON schema enforced. Valid JSON or a retry. No parsing surprises.
Tools
Function calling + Assistants
Tool-use wired to your real systems. Audit-logged. Idempotent.
RAG
Embeddings + retrieval
text-embedding-3 into your vector store. Re-ranked. Cached.
The pain. The day-1 architecture answer.
Each one decides whether your OpenAI feature ships, scales, and survives the next deprecation.
- 01
“Our OpenAI bill went from £1.4K to £14K with no warning.”
Per-call budget + anomaly alerts. Hard token and latency caps. Daily budget per tenant. Anomaly alerts to Slack. Runaway bills become impossible architecturally.
- 02
“The model returned slightly malformed JSON and our pipeline broke.”
Structured outputs + schema retry. Strict json_schema on every call. Either valid JSON or auto-retry. No regex parsing of LLM output. 100% of production calls structured.
- 03
“OpenAI deprecated our model and now we need to rewrite half the code.”
Gateway module + eval-verified swap. One module wraps OpenAI. Swap models in config. The eval suite verifies the new model on your gold set. Done four times in 2025 for clients.
- 04
“An agent looped through retries and burned £2K in one hour.”
Step budget + cycle detection. Hard step cap per agent run. Cycle detection breaks loops. Token budget per run. Over budget, it escalates instead of spending.
- 05
“Will OpenAI train on our customer data?”
Zero-retention tier + PII redaction. Zero-data-retention tier configured by default. PII redaction at the gateway. For regulated data, self-host on your VPC. DPA before week one.
- 06
“Two customers cost us £6K in LLM bills. We don’t know which two.”
Per-tenant cost attribution from week one. Every call logged with tenant ID and GBP cost. Live dashboard. Pricing decisions become data, not guesses.
- 07
“We changed a prompt and a customer started getting wrong answers.”
Versioned prompts + eval-gated CI. Prompts in your repo, versioned, changelog tracked. CI blocks any deploy that regresses more than 5% on the gold set.
- 08
“A customer says GPT gave them a weird answer. We can’t reproduce it.”
Trace ID + replay tool per call. Every call gets a trace ID. Replay it in a sandbox, tweak the prompt, compare. Never debug a hallucination in Slack again.
WHY TEAMS SIGN
OpenAI’s best, without OpenAI owning your runtime
Six things that turn a single-provider feature
into a defensible, deprecation-proof one.
OpenAI behind a gateway
Use GPT-5 where it wins. Swap to Claude or Gemini in config when you need to. No lock-in to a single provider.
Routing + caching discipline
4o-mini for the 70%. Prompt caching at 60-90% hit on RAG. Per-tenant budgets. Median 3.7× cheaper.
Per-tenant cost attribution
Every call logged with tenant + feature + GBP cost. Your CFO has answers your competitors don’t.
Structured outputs, validated
OpenAI structured outputs + JSON schema enforcement. Either valid JSON or a retry. No regex parsing.
Eval gold set, CI-gated
A gold set of 100-300 tasks built with your team. Every PR runs against it. CI blocks any regression over 5%.
Deprecation-proof provider swap
When OpenAI deprecates a model, the swap is a config change the eval verifies. We’ve done it four times in 2025.
The stack we ship every OpenAI build on.
OpenAI’s platform behind a gateway, evals on every release, AWS underneath.
T1
What we ship every OpenAI build on
gateway + evalsOpenAI GPT-5GPT-4oGPT-4o-miniRealtime APINode.js + ExpressPython (FastAPI)MongoDBTypeScriptMixpanelSentryStripeVitest
T2
For the fallback + the brief that needs it
reach when neededClaude 4.6Google GeminiOpenAI AssistantsPinecone / WeaviatePostgreSQL + pgvectorLangChaintext-embedding-3
T3
The cloud + infra
AWS-defaultAWSAzure OpenAIAWS LambdaKubernetes (EKS)DockerRedisApache KafkaS3 + CloudFrontTerraformDatadogWebSocketsMCP
What we wire
The OpenAI work we take on
Each capability wired behind the same gateway, budget caps, and eval gold set. Pick the one your roadmap needs.
3.7x
Median cost reduction after the ladder
60-90%
Prompt-cache hit rate on RAG calls
“The gap between A and B isn’t the model. It’s the engineering discipline around the model.”
001
GPT-5 + 4o reasoning, routed
Frontier models where they win, 4o-mini for the 70% of tasks that don’t need them. Routed per task, not per habit.
002
Vision + document understanding
OCR, charts, screenshots, scanned forms. GPT-4o multimodal, wired through the same gateway and budget discipline.
003
Realtime voice + Whisper
Low-latency speech-to-speech for inbound calls and voice agents. Whisper where you only need transcription.
004
Function calling + Assistants
Tool-use wired to your real systems. Idempotency keys on every call. Audit-logged, with step and token budgets per run.
005
Embeddings + retrieval (RAG)
text-embedding-3 into your vector store. Re-ranked, cached at 60-90% hit, and measured on the same gold set.
Engineer B in real life
UK compliance SaaS,
UK compliance SaaS,
£14K/month to £3.8K/month
A B2B SaaS with 280 customers ran every call on GPT-4o and burned £14K a month with no per-tenant attribution. We rebuilt with the six-rung ladder: structured outputs, 4o-mini routing for the 70% that didn’t need GPT-5, schema validation, per-tenant budgets. Same accuracy. New bill.
Cost
£3.8K
New monthly bill, down from £14K
280
Customers, now cost-attributed
Quality
97%
Accuracy, up from 96%
100%
Calls on structured outputs
Track record
30+
OpenAI integrations since 2023
<200ms
P95 first-token latency
What CTOs actually ask before signing
Pain-first, soft-second.
Yes. AWS is the default. LLM calls route through your VPC if needed, secrets in your manager, metrics in your monitoring stack. For enterprise data residency we wire Azure OpenAI through your Azure tenancy instead. Comfortable on GCP, Cloudflare, DigitalOcean, Vercel, Fly.io.
The gateway abstracts the provider. The swap is a config change. The eval suite verifies the new model on your gold set before you flip the production switch. We’ve done it four times in 18 months for clients.
Zero-retention tier configured by default. PII redaction at the gateway. For regulated data we self-host Llama or Mixtral on your infrastructure instead. Signed DPA on file before week one.
Inference plus ops typically run 0.5-3% of the headcount cost the AI replaces. We model your number in week one and re-baseline at week four on real traffic. Per-call token cap, per-tenant daily budget, anomaly alerts, and prompt caching keep it there.
30-day walk-away both ways. Milestone billing 25/25/25/25. UK VAT registered, listed on Companies House, shipping since 2019.
Every OpenAI integration has two senior engineers paired. Every decision into an ADR the same day. Mohit reviews every PR. Two handovers in seven years, both inside 48 hours.
An eval gold set of 100-300 tasks with known answers, built with your team in week one. Every PR runs against it. Pass rate is a number you both watch, and the cost dashboard sits alongside it.
NDA signed inside 30 minutes via DocuSign. Mutual NDA template ready. You can also pause the build with 14 days’ notice and no cancellation fee.
In context
The surface you hand over.
openai integration, in context — the dashboards, flows and components your team actually ships, reviews and maintains.
One paragraph. That’s it.
Name the OpenAI capability you need, the stack, and the bill or deprecation risk you’d like to de-risk. Mohit replies inside 24 hours: a clear yes, a clear no, or the one question that decides it.
What happens after the email lands
- < 24h
A personal reply.
Yes, no, or the deciding question. Straight to your inbox.
- Week 1
Gateway + cost plan.
Model routing, fallback, budget caps, eval gold set designed.
- Wk 5
Shipped + deprecation-proof.
The next OpenAI deprecation is a one-day config change.