Nobody describes the same job the same way

An employer writes “Analist / Teste”. A candidate writes “QA engineer”. Same job. Zero words in common.

That’s the structural problem in a recruitment marketplace, and it doesn’t go away with a better search engine, because it isn’t a retrieval problem. Both sides are describing the same thing in vocabularies that were never reconciled and never will be, because nobody is going to make several million people agree on what to call things.

So you build the vocabulary in the middle.

What I said we’d do

In August 2018 I wrote that extreme multilabel classification had become affordable and that search was the thing worth pointing it at. Classifying text across five thousand-plus dimensions had been too computationally expensive to consider, and suddenly wasn’t.

That one shipped. The skill suggester built on it is still in production today, years later, and it beat the feature it replaced. I’m putting that in writing because plenty of things I’ve planned didn’t, and the ones that did are the ones worth being specific about.

A taxonomy is a product decision

Extreme multilabel classification gives you a skill vocabulary. Alongside it we run a classifier that maps job titles to departments, and we match against ISCO, the international occupation taxonomy.

Adopting an external taxonomy feels like the boring choice next to building your own, and that’s precisely the argument for it: somebody has already done the work of enumerating occupations across languages, they maintain it, and it means the vocabulary is not a thing your team has to keep alive forever. The pull toward inventing your own kind is strong and usually wrong.

Supervised means somebody has to supervise

The part that gets underestimated is that supervised learning needs labels, and labels come from people.

So the labelling interface is not internal tooling you build when you have time. It’s a product with users, and the users are your own colleagues. Ours was used for a good while by the team we called Natural Intelligence, which is the best name anyone at the company has come up with for anything.

What it actually bought us was speed on everything afterwards. It was most valuable for Skilld, our skilled candidate search, but the point is more general: once there’s a working way to get human judgement into a training set, every subsequent experiment starts from a usable position instead of from a data collection project. The tool is the thing that makes the next model cheap.

Build it early. It looks like overhead right up until the third time it saves you a month.

The order that matters

The sequence people expect is: get a model, then worry about labels, then discover you need a taxonomy.

The sequence that works is the reverse. Decide what the words mean. Build the means of labelling. Then the models are comparatively easy, because the hard problem was never the classifier, it was that both sides of your marketplace were speaking different languages and no amount of embedding fixes a disagreement about what a job is called.

LATER EDIT: We added ESCO, the European occupation taxonomy, and it’s working. The question still open is granularity: match on the most specific occupation label, or on the full category breadcrumb, which carries more context at the cost of diluting the vector.