What I'm changing at eJobs

A year into running engineering at eJobs.ro, here’s where I’m taking it.

Off the servers we install by hand

Docker first, then Kubernetes, which I settled on in February. Containerised deployments are easier to manage and harder to break, and once software is properly modularised the operational load gets impossible to carry without real orchestration.

The architecture underneath is microservices on a multi-tier split, so persistence, business logic and interface can be upgraded independently. Small pieces are more maintainable than one large monolith, which is not a controversial claim but is a lot of work to act on.

Two things make that transition survivable. The first is RPC, with a central source of truth for the interfaces each service exposes. The complexity of microservices is mostly interface complexity, and it is worth paying for that properly rather than discovering the contract by reading somebody’s code. The second is stubs. We can’t move off the old manually installed architecture in one step, so proxies keep the links to the old services alive while the new ones appear behind them.

Machine learning, now that it’s affordable

Two things are becoming practical on ordinary infrastructure, which they weren’t a few years ago.

Extreme multilabel classification, for search. Classifying text across 5,000-plus dimensions used to be far too computationally expensive to consider. It isn’t any more, and search is the thing most worth pointing it at.

Collaborative filtering, for everything else. User behaviour is feedback, and feedback read properly turns into better suggestions for the next person. That’s the whole mechanism, and marketplaces are where it pays off fastest.

Process, kept small

Kanban, because keeping things simple and as close to meeting-less as possible has beaten every technical process I’ve tried before it.

A local development cloud, so people develop in the same environment the code runs in. Most configuration effort and a good share of the error rate comes from the gap between those two, and the gap doesn’t need to exist.

Then the one I’m most interested in: multiple competing releases. The microservice architecture means we can put several versions of the same software in front of the user base at once and measure which one actually works. The winner becomes the final version and goes to master. Not a debate about which approach is better, a measurement.

Seeing what’s happening

Central logging, so errors live in one place and it’s possible to tell which ones happen most rather than which ones somebody noticed.

Monitoring and alerting on top. With this many systems, the default state is not knowing something is broken. Monitoring gives the overarching view of the KPIs that matter, alerting by email and SMS turns an incident into something that starts being handled rather than something discovered later.

The projects this sits under

A job aggregator. A CRM, which it turns out is genuinely useful to have. Search improvements, which is where the machine learning lands first. A new front end in VueJS. Automation of how content distribution money gets spent.

None of that works without the four sections above, which is why they come first.