Momos · May 2022 – Sep 2025 · Abu Dhabi
Turning thousands of reviews into a weekly action list
AI Insights turned unstructured review volume into a ranked, evidence-linked list of what to fix. The LLM stack migration behind it delivered 30% better accuracy at 60% lower cost.
Role
Lead, team of 3–4
Users
Brand & ops leaders
Outcome
−90% manual hours
The stakes
Brands were paying people to read reviews and write a summary deck. It is slow, it is inconsistent between analysts, and by the time it lands the week is over. Worse, the summaries were unfalsifiable: a slide saying “customers mention slow service” gives an operator nothing to act on and no way to check whether it is true.
The bar I set was that every insight had to name a specific, countable thing, be ranked by how much it mattered, and link back to the exact reviews it came from. If an area manager could not click through to the evidence in five seconds, it was not an insight. It was a vibe.
The system, in three stages
1 · Extract
issue, sentiment and location from raw reviews, many sources and languages
2 · Map to a fixed taxonomy
induced from the brand's own corpus, then frozen
3 · Rank by impact
volume × severity × trend, every line linked to its evidence
Three decisions, and what I turned down
01
Induce the taxonomy once, then hold it still
Categories were derived from each brand's own review corpus, reviewed with the client, then fixed. A stable taxonomy is what makes week-over-week trend real. If the buckets move, every trend line is an artefact of the model.
Rejected: free-form LLM theme generation per run. Beautiful demos, uncomparable outputs. Two runs on the same data produced two different worlds.
02
Build the eval harness before the migration
Before touching the LLM stack I built a labelled evaluation set and a harness that scored extraction accuracy and cost-per-document side by side. That is what made the migration a measurement instead of a leap of faith. It is also what let me state 30% accuracy up and 60% cost down as facts.
Rejected: migrating first and spot-checking outputs. It is faster right up until a client finds the regression before you do.
03
Rank by impact, and always show the receipts
Insights are ordered by volume × severity × trend. A rare complaint that has tripled matters more than a common one that is flat. Every line links to the underlying reviews, which is the single feature that turned scepticism into adoption in client meetings.
Rejected: a narrative summary paragraph. It reads well, it is impossible to verify, and nobody can act on a paragraph.
What came of it
−90%
of manual review-analysis time removed.
+30%
extraction accuracy after the LLM migration, on a held-out labelled set.
−60%
cost per document, which made per-brand daily runs affordable.
The migration is the piece I would point an engineering leader at. Not because swapping models is hard, but because having an eval harness meant we could swap models again later in an afternoon. The harness outlived the decision it was built for.