Detecting damages on railroad infrastructure

abstract

our assignment aims to develop, implement, and maintain machine learning algorithms that perform visual quality control by analyzing images taken from a train.

background

who’s the customer

In Sweden, there are thousands of kilometers of railroads that need to be monitored and maintained. Traditional inspection, where people physically visit the location, is a massive undertaking that requires hundreds of people.

At the same time, delays caused by damaged equipment are a huge problem that costs many millions each year. Clearly, there are opportunities to use machine learning for more effective and automatic quality control. If successful, it’s possible to drastically reduce some of the most severe delays.

challenge 1

there’s a lot of railroads

The trains must cover 14,000km of electrified railroads. Since the cameras take a new image every 10cm, we get 140,000,000 images. However, there are currently 6 cameras, and the goal is to cover all sections between 1-6 times a year totaling around 2,500,000,000 images every year!

It’s clear that inspection from a distance by people is better than physically visiting each location, but it’s clearly not enough. Luckily, quality control is a perfect application for machine learning.

challenges 2

no labeled data & and false positives

At first, we didn’t have any real damage images, and finding them manually wasn’t an option. Our initial idea was to do anomaly detection with unsupervised learning. The goal is to teach an algorithm what normal looks like (you can do that in many different ways) and filter out everything that isn’t.

An interesting approach that often difficult in practice, at least on its own. The issue for us is that we have hundreds of anomalies that occur more frequently and look less normal than the damages we want to find. Here are two examples:

solution

creating fake damages

The key for us was to create images with fake damages. At first, we used photoshop and made a small dataset for supervised learning. As we found actual damages, we added them to the training data, which, of course, made the algorithm better.

After finding around 20-30 actual damages, we trained a cycleGAN to generate more fake damages, leading to significant improvements. Below are some examples from our GAN.

results

creating fake damages

So far, we’ve found and fixed dozens of damages before, leading to severe delays. The goal right now is to reduce the number of false positives, which is a problem simply because of the skewness of the dataset. Here are two examples of damages that has been fixed thanks to the algorit