Don't call the model from your application

The hard part of machine learning in a marketplace is not the model. It’s that everything you’ve embedded is already out of date.

At eJobs.ro both sides of the marketplace change constantly. Employers edit job ads. Candidates add an experience, fix a title, upload a new CV. Every one of those edits invalidates whatever vector you computed for that item, and none of them announce themselves to your machine learning system.

The obvious design, and why it rots

The obvious thing is to call the model from the application. Somebody saves a job ad, the save handler calls the vectoriser, you store the result next to the row.

It works on the day you build it. Then it decays, for reasons that are all boring:

Another code path writes the same table and nobody remembers the hook. A bulk import runs and skips it entirely. The call fails, and because nobody wants saving a job ad to fail when a model is down, it fails quietly. An admin tool edits rows directly. The model gets retrained and now you need to recompute several million items, which is a completely different program from the one you built.

By the end you can’t answer the only question that matters, which is whether the vector you’re serving reflects what’s currently in the database.

Attach to the replication stream instead

We took the other route. The indexer connects to MySQL as a replica and vectorises jobs and CV experiences as the changes arrive.

The consequences are worth more than the mechanism.

Nothing in the application knows the machine learning exists. No hooks, no calls, no shared failure mode. Product teams ship without thinking about embeddings, which means they can’t forget to.

Every write is covered by construction, whatever made it. Application, bulk import, admin tool, migration, a developer fixing a row by hand at midnight. They all produce the same replication events.

Backfill and steady state are the same program. Retraining means replaying from an earlier position, not writing a separate batch job that will itself drift out of sync with the live path.

And the failure mode becomes a number you already watch. When indexing falls behind, it shows up as replication lag. Not a silent gap that surfaces months later as bad recommendations nobody can explain.

Vectorisation as a service, not a library

The second half is that the embedding itself lives behind one gRPC service that returns a vector for any text.

One place decides what a vector means. Change the model there and every consumer gets the new behaviour, rather than six services each carrying their own copy of the embedding code and quietly diverging on preprocessing. We ran a second version of that service for a while before consolidating, and the period where both existed was the period where it was hardest to reason about anything.

The general shape

Treat representation as infrastructure, not as a feature.

If your embeddings are produced by feature code, they inherit every gap in that code, and the gaps are invisible. If they’re produced by watching the source of truth change, the only way to miss something is for the database not to have it.

Which is a much easier thing to be confident about.