Applying Machine Learning to reduce paperwork on the truck industry
At OnTruck we’re applying Machine Learning to solve several real-world problems, and this is one of those. Let us introduce you to Caronte and how it helps us measure and improve one of our key processes.
Disclaimer: this is not a technical article about Machine Learning; it has more to do with sharing the way we understand technology as a tool to solve problems, and if it is as cool as Machine Learning, even better!
Some context
OnTruck is transforming the road transportation industry — €600+bn in Europe. We are building the leading logistics network for B2B, on-demand road transport. We make trucking simple, transparent and on-demand.
Within OnTruck platform, every time a driver delivers a load at the drop-off place, he/she takes a picture of the proof of delivery document (POD) using our mobile Driver App. At that moment we make it available via web to the shipper who sent the load.
The problem
Some weeks ago our Operations team alerted us that many of the POD pictures our drivers were taking didn’t have enough quality. It was an important problem because it generated manual work: calls and mails between Operations and drivers to get the POD right. Manual work goes against optimization and scalability, and that’s why we decided to tackle it with a dedicated project.
As with every other project at OnTruck we followed our well-defined product process. One key step of the process is to define KPIs to measure success of each iteration. In this case we had no way of measuring how many of the pictures were good; humans can easily tell when a picture of a document is not good enough, but turning that into something automatic is not straightforward. Think of cut-off images, dark ones, blurry ones, pictures of something that is not even a POD…
Machine Learning to the rescue!
On the Tech team we had been playing with Machine Learning (ML) for a while and among other tests we had built a ML Classifier that could classify flowers by their type just by feeding the system with a few hundreds of pictures of each type (techies only: this is the codelab to build your own). Why not trying to do the same thing with good and bad POD images? Maybe the machine would be able to find its own definition of what a good POD photo is.
The experiment
We decided to give it a try. We time-boxed the experiment to a single day of one developer. That way we would not waste resources in case it didn’t work. But guess what… it did work! We fed (trained) the system with two folders, named good and bad, containing ~300 images each, classified by hand. The very first results were promising:
We got creative
Here comes the interesting part. A ML Classifier not only classifies, but also outputs how confident it is about each classification, from 0% to 100%. We had a simple idea: let’s use that confidence level as the quality score of each image. That way when the classifier told us an image was good with a confidency of 80%, we gave the image a 80% quality score. When it told us an image was bad with 90% confidence, we scored that image with a 10% quality. Easy peasy.
With that in mind we wrote a simple Python script that ran the classification over all images in a folder and output aggregated data of quality as plain text. We tested it over different set of images and results were validated by humans.
A new tool for our toolbox
We had a new tool, and it deserved a name. We tend to name our tools after mythological characters, and we decided to go with Caronte (Charon).
We also gave it some love: cleaned up its code, documented it and made it available for anyone in the company to run analysis on our PODs within a date range, isolating iOS and Android results, and enabling user based A/B testing. Its output was not plain text anymore, but a well structured CSV. We did all that within a two-week sprint.
Last but not least we shared what we had learnt with the whole team by running a ML workshop as part of our internal training sessions.
Using our new tool
We put Caronte to work and made it analyze all the POD images we had stored in our system in order to have, for the very first time, a number measuring the quality of our PODs. A ground truth. And this is what we found:
Our first reaction was 😳. Only 54% of the PODs were good? Actually the PODs were not that bad; such a low result had to do with the fact that we had trained Caronte to be really picky. Anyway, we needed a number to improve — a KPI — and now we had one!
In fact, apart from that simple number we also got the distribution of the quality scores; our mission with this project was to make that distribution move to the right (to the higher quality score ranges).
Start iterating
At that point we had everything in place to start solving the problem. Following our lean methodology, our first iteration was a simple, non-techy one: we sent our drivers some tips via chat on how to take better POD photos. Believe it or not, that bumped our good PODs KPI from 54% to 62%!
We ran some other quick iterations and already achieved a good PODs KPI value of almost 68%, and pushed the quality distribution to the higher quality score ranges 🎉.
Our last, not-yet-measured iteration introduced some heavy tech-powered features (Computer Vision) in our Driver App, but we will talk about that in detail in a different post.
The future of Caronte
We are still impressed of how much value this new Machine Learning based tool has added with very little development effort. As with any new tool, we are still figuring out where and how to apply it. One possible fit would be to integrate it into the Driver App and warn our drivers in real time when a picture they’ve just taken is not good enough, always according to Caronte. Pretty cool, huh?
Conclusion
Machine Learning is a tool that allows us solve problems in a way no other technology does. Even though it sounds risky and complicated to use, we’ve proved that it is possible to apply it with little effort and complete safety, and obtain great results. The key is to identify where and when to use it. Remember it is not a magic wand, but another useful tool for our toolbox.
We are hiring!
We are looking for Senior Android and iOS Developers among many other positions in Engineering and Product. OnTruck is leading the disruption of the logistics sector in Europe, and we have been invested by top investment funds in Europe like Atomico (UK), Idinvest (France), PointNine (Germany) and Samaipata (Spain). If you want to contribute to a better world and challenge yourself every day, let’s talk!
