E-Commerce Personalization Engines: How Stronger Algorithms Don't Always Mean More Money

Six months ago, a home goods curation platform came to us about a recommendation issue. They had a recommendation module below the product page labeled "You Might Also Like." Its click-through rate had plateaued at 1.8%. The engineering team wasn't idle; they'd tweaked their collaborative filtering model three times, and each time the offline metrics (recall, NDCG) improved. But real-world conversion didn't change.

The problem wasn't the algorithm. They believed that stronger algorithms automatically meant more money. Between that belief and actual revenue lies an entire implementation path.

Three generations of e-commerce personalization AI: layering, not replacement

E-commerce personalization today isn't about replacing collaborative filtering with LLMs. It's about placing each generation's technology where it actually works. Understanding what each solves and what it costs tells you where your resources should go.

Collaborative filtering is the first generation. Its core logic is simple: "people who bought A also bought B." It computes similarity from user-item interaction matrices. It's cheap, mature, and accurate with popular products. But it has two critical weaknesses: cold start (new products and new users have no interaction history) and long-tail blindness (niche items stay buried because they lack dense interactions).

The second generation is deep learning recall and ranking, including Twin Tower models, DeepFM, and similar approaches. These ingest user behavior sequences, product attributes, and contextual features together, capturing nonlinear signals like "what this person browses on Friday nights." Accuracy improves noticeably. The cost: heavy feature engineering, real-time feature pipelines, and serious demands on your engineering team.

The third generation is LLMs and semantic understanding. Its value isn't in replacing the ranking of earlier generations; it's in addressing what earlier approaches miss: product copy semantics, intent in user reviews, and cross-category common sense. Take this example: collaborative filtering doesn't know what a camping beginner needs. LLMs can parse product descriptions and reason through relationships, tent, lantern, moisture barrier, even if those items were never purchased together.

Conversion leverage lives in the moment, not the model

For that platform, we didn't jump straight to LLMs. The first thing we did was unglamorous: move the recommendation module from below the product page to a post-add-to-cart popup, and shift the logic from "similar products" to "complementary products." That alone moved CTR from 1.8% to 6.4%. The algorithm didn't change. The moment and intent did.

This is the most common mistake: strong offline metrics don't guarantee online revenue. A 10% recall improvement might add nothing, because users never reach that recommendation slot in the first place, or they reach it without purchase intent.

Then we introduced LLMs, but only for one clear use case: new and long-tail products. These are exactly what collaborative filtering fails on, and exactly where LLM semantic understanding shines. We embedded product descriptions and recent user browsing, did semantic recall, then passed candidates to the existing ranking model. LLMs didn't make the final ranking call; they surfaced the right candidates. This gave us semantic benefits while isolating LLM latency and hallucination risk to the recall stage, preventing direct user exposure.

Within three months, new product recommendation conversion had lifted approximately 2.3x. Recommendation-driven revenue share went from 7% to 15%.

What to build depends on where you're stuck

If your pain point is "popular product recommendations aren't accurate enough," don't reach for LLMs yet. Clean up your collaborative filtering data and position the module correctly; that's your highest ROI. If it's "new and long-tail products never get exposure," that's where LLM semantic recall earns its value. If you don't even have a real-time feature pipeline yet, skip the second generation. That's a maintenance black hole.

Build generations on top of each other, not around them. Each layer has to solve a specific business problem, or you've just bought an expensive system nobody uses.

When we do these implementations at Tenten, we start by asking one thing: after this model goes live, what conversion metric moves? If you can't answer, we wait. Our engineers embed themselves, watching real traffic until it's actually used and driving conversions, not handing off a report with strong offline numbers and leaving.