When to Say 'No' to Large Language Models
An LLM is a probabilistic engine. Before adding a model to your stack, see if a deterministic algorithm can solve it cheaper, faster, and with 100% reliability.
The Hidden Cost of AI Hype
Adding an LLM to your backend introduces latency, cost, security vulnerabilities, and non-deterministic behavior. Our engineering rule is simple: **Solve problems at the lowest effective level.**
The Decision Matrix
Before writing a prompt, we evaluate the problem against this hierarchy:
- Standard Database Queries: Can this be resolved with an indexed SQL query or text search (Postgres PGroonga / Elasticsearch)?
- Deterministic Algorithms: Can regular expressions, state machines, or simple code conditional matrices parse this input?
- Cached Embeddings: Can we pre-compute values, compute cosine similarity, and avoid invoking LLM generation altogether?
- Probabilistic LLM: Only use an LLM when the input is highly unstructured, the schema is variable, and the output requires synthesis of semantic context.
Case Study: Classification
A client was spending $4,000/month using GPT-4 to categorize support tickets into 6 departments. We replaced the LLM with a lightweight local vector search on ticket embeddings. Category matching accuracy rose from 88% to 96%, latency dropped by 950ms per ticket, and operating costs fell to under $10/month.
Looking to implement this architecture?
We partner with tech companies to resolve complexity, write robust codebases, and optimize system speed.
Work With Us