The best antidote to enterprise AI hype isn't cynicism. It's knowing the economics firsthand.

I've been building AI systems since 2016, and I've worked alongside teams shipping ML products at scale, including an AI calculus tutor and a real-time AI English tutor at Pearson. Recently I went further and trained a 1.3B-parameter Polish language model from scratch at home: my own tokenizer, weights initialized to noise, on a single RTX 4070 with 12 GB of VRAM. It took 28 days on 120GB of raw data from which I selected 10% for training.

Compute-optimal scaling suggests around 26 billion training tokens for a model this size; I got through about 1.5 billion. On one consumer GPU, that's the ceiling.

I didn't do it to compete with anyone. I did it to understand where the cost, quality and failure points actually are. Here's what it taught me, and what it means for anyone signing AI contracts.

Data quality has symptoms, not just principles

"Data quality matters" sounds like a slide. In training, it shows up as concrete symptoms in the loss curve. Firstly loss was dropping epoch to epoch very fast, but that drop slowed down and around epoch 35 it flattened. Improvement was zero. I reached a point where the model had seen the data enough times that further exposure produces diminishing returns, and I was essentially just watching the optimizer micro-adjust around a local minimum.

Feeding a model noise doesn't produce intelligence. It produces confident errors. Deduplication, domain relevance, tokenization and filtering drive downstream quality far more than raw volume, and a smaller model on clean, targeted data will often beat a larger one on messy data in business-critical workflows.

The loss plateaued around epoch 35. The last ~16 epochs, roughly a third of the compute, bought very little. More passes over the same data don't create new knowledge.

Leadership takeaway: stop asking vendors about parameter counts. Ask to inspect their data pipeline and domain curation strategy. And treat "train longer" or "add more compute" as a budget decision that needs evidence of return.

Fluency is not competence

My model writes grammatical Polish. It handles gender and case agreement well, and a native speaker would recognize its sentences as coherent. It also hallucinates facts confidently and doesn't reason. By perplexity, it looks decent. As a product, it would fail.

That gap is exactly what vendor demos and public benchmarks hide. Benchmarks like MMLU look authoritative on a slide, but they're easy to contaminate or overfit. Real validation means task-specific testing, adversarial edge cases and ongoing human review. The teams I worked with at Pearson built evaluation around what the product had to do, not around generic metrics, and I built evaluation into my run from day one for the same reason.

Leadership takeaway: never approve an AI vendor contract without validation against your company's actual production edge cases. Fluent output is the baseline, not the proof.

Inference is the real long-term cost

Training is a one-time cost. Running a model at scale is an open-ended operating expense. Memory bandwidth, KV-cache, latency and concurrency limits decide your long-term margins. Vendors tend to downplay token economics during the sales cycle, and as adoption grows, uncontrolled API costs and inference bottlenecks erode operating margin directly.

Language multiplies this. Polish has diacritics and seven grammatical cases, so one verb root produces dozens of word forms. English-centric tokenizers handle this badly: a single Polish word can cost 3–4 tokens. That's why I built my own tokenizer from my corpus. With per-token pricing, serving non-English markets can cost substantially more than a vendor's English-based estimate.

Leadership takeaway: shift vendor evaluation from initial integration cost to unit economics per task at peak concurrency, in every language you serve.

Constraints drive architecture, and cost

12 GB of VRAM decided almost everything: mixed precision, gradient checkpointing, DeepSpeed ZeRO-2, a 1024-token context window, and small batches accumulated over many steps. Every one of those is a trade between memory, speed and quality.

On the inference side, quantizing the model to 4 bits brought it to about 700 MB. That fits on a Raspberry Pi. Deployment target, not model size, is often the real design constraint.

Leadership takeaway: ask vendors what hardware and context limits their pricing assumes, and what changes when your usage doesn't match.

I built it, and then I didn't

I got the model running behind a Go inference server through llama.cpp. It worked. But for Purple, my local-first AI assistant, I use Ollama as the backend instead of my own stack. Building it once taught me what the layer involves. That's exactly why I chose not to maintain it myself.

Leadership takeaway: build to understand, buy or adopt where the layer isn't your differentiation.

Vendor due diligence

When leadership teams buy AI, they aren't just buying capability. They're taking on operational risk and a margin structure.

Questions to ask

  • "What is your data lineage and filtering methodology?" If a vendor can't define their filtering pipeline, they don't own their quality.
  • "How do you measure performance on our tasks, and how do you prevent benchmark contamination?" Push past standard test suites and ask for validation on your own data.
  • "What do latency and unit cost look like at 10× our peak concurrency, in our languages?" Make vendors prove their SLA commitments under stress.

Red flags

  • Thin API wrapper. If the core value is an interface around a third-party frontier API, you're paying a margin premium for UI and taking on their lock-in.
  • Fine-tuning as a knowledge fix. Fine-tuning steers tone, format and behavior. It doesn't reliably teach a model a domain it has no foundation in. That needs retrieval or better base data.

Build vs. buy

Hosting your own models or fine-tuning open weights can easily turn into an expensive engineering vanity project without clear criteria.

Strategy When it fits Red flag
Commercial API Standard workflows, fast time-to-market, variable demand, non-proprietary tasks Your core IP or differentiation depends entirely on a commodity API
RAG / fine-tuning Internal knowledge retrieval, strict output formats, tone alignment Using fine-tuning alone to teach new facts
Open weights, self-hosted Data sovereignty, air-gapped or edge environments, tight latency, high predictable volume No MLOps headcount or compute budget

Leaders don't need to train models. They need to know what it costs to, because once you understand the true unit economics of compute, data and inference, you stop buying hype and start buying leverage.

Technical details of the training run: My own 1.3B LLM Model