Not Everything Needs AI: How We Decide Where AI, Deterministic Code, and Token Budgets Belong

The Honest Take: AI Is a Tool, Not a Default

After building automations across finance, medical, legal, hospitality, ecommerce and services businesses, we've learned something the loudest voices in the room won't tell you: not everything needs AI. A well-written script, a database query, a scheduled job or a plain if-this-then-that rule will beat a language model on cost, latency, and predictability almost every time — for the workflows they were built for.

AI is transformative where it's genuinely the best tool: ambiguous inputs, unstructured text, natural-language interfaces, classification with fuzzy edges, summarization, extraction. Everywhere else, reaching for an LLM is often the most expensive and least reliable way to do a job that a 20-line function already handles.

Where API Tokens Quietly Bleed You Dry

Token costs feel small — fractions of a cent per call — until they aren't. Common ways we've seen bills explode:

• Wrong model for the job. Routing every request to a frontier model when a small, cheap, fast one would nail 95% of them.
• Bloated context windows. Stuffing the entire knowledge base into every prompt instead of retrieving the right 3 paragraphs.
• No caching. The same question re-asked a thousand times a day, paying full price each time.
• Loops without ceilings. Agents that recursively call themselves without a hard budget or step cap.
• No monitoring or alerts. Nobody notices until the monthly invoice arrives.

How We Decide: AI, Deterministic, or Hybrid

Every workflow we touch during an audit gets scored on four dimensions:

1. Input shape. Is the input structured (rows, forms, IDs) or unstructured (free-text, images, voice, PDFs)? Structured almost always belongs in deterministic code. Unstructured is where AI starts to earn its cost.
2. Tolerance for variance. Does the output need to be exact and auditable (invoices, calculations, compliance, legal, medical records) or is a "close enough, human-reviewed" answer acceptable (draft replies, summaries, triage)? Exact = deterministic. Fuzzy = candidate for AI with a human in the loop.
3. Frequency and volume. A workflow that fires 100,000 times a day with a cheap deterministic path is not a place to burn tokens. A workflow that fires 20 times a day on messy input is exactly where AI pays off.
4. Failure cost. If the wrong answer costs money, reputation, or safety, deterministic wins — or AI runs as a suggestion layer behind human approval, never as the final actor.

Most real systems end up hybrid: deterministic pipelines do the heavy lifting, AI handles the ambiguous edges, and the two hand off through a clean interface.

The Abstraction Layer: Where AI Actually Plugs In

When AI does belong in a workflow, where you put it matters as much as whether you use it. We design a thin abstraction layer between your app and any model provider so that:

• You can swap models — or providers — without rewriting the app.
• Every call is logged with input, output, model, tokens, latency, and cost.
• Prompts and system instructions live in one versioned place, not scattered across the codebase.
• Retries, timeouts, fallbacks, and safety filters happen once, at the boundary.
• You can A/B a cheaper model against an expensive one on real traffic and see the actual quality delta.

Assigning the Right Model to the Right Task

Treat models like team members — different pay grades, different jobs. A rough playbook we use:

• Small, fast, cheap models (Haiku-class, Flash-class, Nano-class): classification, routing, tagging, first-pass triage, structured extraction from short text. Runs at scale for pennies.
• Mid-tier models (Sonnet-class, Pro-class): drafting, summarization, chat responses grounded in retrieved context, most user-facing tasks.
• Frontier models (Opus-class, top-tier): reasoning, code generation, complex multi-step tool use, the small percentage of calls where quality is worth the cost.
• Embeddings + retrieval for anything knowledge-heavy — so the model reads 3 relevant paragraphs instead of paying for your whole handbook every call.

Route the request to the cheapest model that actually meets the quality bar. If a small model gets it right 95% of the time, run the small model and only escalate the hard 5%.

Deterministic Layers: The Unsexy Foundation

The parts of your system that should almost never be AI:

• Math, money, quantities, quotas, and anything auditable.
• Access control, permissions, and role checks.
• Deduplication, idempotency, and "did we already send this?"
• Data validation, format conversion, and schema enforcement.
• Scheduled jobs, batching, and retries.
• Any decision a stakeholder will one day ask you to explain in writing.

These deserve code you can read line by line. Wrapping them in an LLM makes them slower, more expensive, and impossible to defend when something goes wrong.

Token Budgets: Set Them Before You Ship, Not After

Every AI feature we ship gets a budget up front:

• Per-call ceiling. Max tokens in and out, enforced at the abstraction layer.
• Per-user / per-tenant caps. One bad actor or one runaway agent can't blow up the bill for everyone.
• Daily and monthly spend alerts. Slack pings well before anything hits the credit card.
• Kill switches. Feature-flag any AI path so it can be disabled without a deploy.
• Cost dashboards. Real numbers per feature, per model, per customer — so decisions to upgrade or downgrade a model are based on data, not vibes.

What This Looks Like in an Audit

When you engage us for an Automation Audit, this is baked in. For every workflow on your list, you get:

• An AI-vs-deterministic recommendation with the reasoning — not "let's add AI" by default.
• A model assignment for the AI portions, with an estimated monthly token cost.
• The abstraction and monitoring plan so you can change your mind later without a rebuild.
• A budget and alerting design so nothing runs unmonitored in production.

The Bottom Line

Custom solutions win when they use the right tool for each part of the job. Sometimes that's a language model. Often it's a query, a script, or a well-placed rule. And when it is AI, the difference between a $200/month bill and a $20,000/month bill isn't the model — it's the architecture around it.

If you want an honest map of where AI belongs in your business (and where it very much doesn't), book an intro call or start with an Automation Audit.

Why we always start with an audit · AI automation vs. a demo that impresses