Eagle eye

Knorr-Bremse and Israeli start-up Rail Vision are working together to produce obstacle detection systems for Autonomous Train Operation. Where can these systems be used, how do you collaborate effectively with a start-up, and how can artificial intelligence make for greater safety? We spoke with Dr. Nicolas Lange, Chairman of the Management Board of Knorr-Bremse Rail Vehicle Systems, and Shahar Hania, co-founder and CEO of Rail Vision.

Digitalization’s growing influence in the rail industry is paving the way for the progressive introduction of Autonomous Train Operation (ATO). Why is it strategically important for Knorr-Bremse to be driving forward the technology that can realize the vision of ATO?

Nicolas Lange: Autonomous Train Operation is coming, for several reasons. Even today, there are hardly enough drivers, and there is room for improvement with regard to both transportation logistics and of course safety. We want to expand our product portfolio accordingly and align it with what customers are asking for, which includes ATO. However, we are not going to become suppliers of complete ATO solutions – that role will fall mainly to the vehicle manufacturers. But Knorr-Bremse aims to supply those manufacturers with subsystems and partial solutions that support safe autonomous operation.

One of these subsystems is being developed by Rail Vision, a company in which Knorr-Bremse has acquired a 36.8% stake. Why does this investment make sense for both companies?

Nicolas Lange: Safety technology is part of Knorr-Bremse’s DNA. But our technology has a strong mechatronic focus – we are not experts in camera systems or automatic obstacle detection. That’s where the flexible use of external know-how can help to make us stronger. The way that the environment observation systems and the brakes work together will be critical for Autonomous Train Operation, making Rail Vision and Knorr-Bremse a perfect fit in terms of the expertise that they bring to the table.

Shahar Hania:.Absolutely, companies need to stay on their toes and can’t afford to let the developments of the next five to ten years pass them by. Knorr-Bremse has understood that the necessary strategic technology already exists, and recognizes the opportunities offered by obstacle detection, connectivity and the cloud. Moreover, Knorr-Bremse realizes that whoever is first to offer this technology in the rail industry stands to gain a big share of the market.

Artificial intelligence has now reached the point where it can do better than humans.

Shahar Hania – CEO and co-founder of Rail Vision

Where can ATO solutions with obstacle detection systems be deployed?

Nicolas Lange: Obstacle detection comes into its own wherever the potential hazards are greater, for instance in metro, commuter train and light rail vehicle applications. Our aim here is to enhance safety by providing drivers with support in difficult situations, for example when fog, rain or snow cause poor visibility. Or when a train emerging from a dark tunnel arrives at a brightly lit station.

Are you also planning to use this technology for freight trains?

Nicolas Lange: There is a lot of potential in this area, too. One initial application of the Rail Vision technology could be on autonomous freight trains crossing vast areas of empty desert in a country like Australia. The ability to cover long distances without a driver is particularly desirable in this type of scenario. Firstly, because there are hardly any drivers for this kind of work anymore. And secondly, because the driver has to be constantly vigilant, but nobody can maintain that level of attention for such long periods of time. Machines are much better at this kind of thing.

Shahar Hania: Train drivers are not allowed to work more than eight hours a day or so, and they have to take compulsory breaks. On long routes like these, our systems deliver huge efficiency gains by allowing trains to keep running non-stop. They also make it possible to reduce the headway between two autonomous trains, which means that trains can run more frequently. Train operators also need to factor in the contribution that our systems make to greater safety. A freight train derailment can easily cause millions of dollars’ worth of damage. Our system only needs to prevent a single accident to pay for itself.

Why has Rail Vision chosen to use artificial intelligence in its obstacle detection systems?

Shahar Hania: Until now, detection algorithms have performed much worse than humans at identifying obstacles. But artificial intelligence has now reached the point where it can do better than humans. We can now detect obstacles up to two kilometers away, something no train driver can do. But first of all, you need to train the AI. In order to learn what a bridge, a tree or a signal looks like, the system has to process huge volumes of data. Although this is a very demanding process, the results so far speak for themselves.

Knorr-Bremse aims to supply manufacturers with subsystems and partial solutions that support safe autonomous operation.

Dr. Nicolas Lange – Chairman of the Management Board of Knorr-Bremse Rail Vehicle Systems

The system is being developed mainly in Israel. How does an agile start-up like Rail Vision collaborate with a global corporation like Knorr-Bremse?

Nicolas Lange: We want to take full advantage of the agility offered by start-ups like Rail Vision. That’s why we have acquired stakes in various start-ups without taking them over in the traditional sense. We have deliberately refrained from integrating the small, agile team in Israel into our own development teams. It’s an approach that does require a change in our mindset, and some of our employees have found this challenging. But on the whole, it is working exceptionally well.

Shahar Hania: In a partnership like this, the most important things are trust, respect, and the people you work with. We have received excellent support from Knorr-Bremse – we are in touch almost every day and have a personal contact for any technical questions. We’ve been amazed at how quickly they reply. I think Knorr-Bremse has learned a lot about collaborating with start-ups, agile working and rapid decision-making. I can imagine that these things are a challenge for any large company.

The partnership with SBB Cargo in Switzerland is a sign that the investment is starting to bear fruit. Can you tell us a little about this project?

Nicolas Lange: It takes at least two people to operate a shunting locomotive: the driver, and someone outside the vehicle to assist with the shunting itself. Thanks to Rail Vision’s camera technology and the ability to send data to a tablet, the locomotive can now be controlled remotely with no one on board. As a major operator, SBB was looking for an experienced partner who could support and integrate this technology. We were able to give the project the decisive push it needed when we stepped in as the intermediary between Rail Vision and SBB in 2019.

Shahar Hania: Our system comprises at least two external cameras and a central processor. We developed and produced the entire system ourselves in a year and a half. Due to the coronavirus restrictions, we have produced a video guide to help with the installation in Switzerland. Initially, 30 locomotives will be equipped with the system, and we are hopeful that more orders will follow.

Nicolas Lange: This technology provides an outstanding solution to the shortage of personnel. That explains why there is also so much interest in other countries that rely heavily on rail transportation, such as Japan. This really is a model use case for many rail operators around the world.

Rail Vision Sensor Technology


In its surroundings detection system, Rail Vision uses a combination of high-resolution video and infrared cameras. The technology enables obstacles to be “sensed” through differences in temperature. This configuration delivers very vivid and informative images that can also be analyzed far more quickly than radar images. Thanks to this unique hardware, which it combines with cutting-edge AI software, Rail Vision is the market leader in the field of surroundings detection systems for rail applications.


With its high-resolution sensors, the system is able to detect obstacles up to two kilometers away. For applications in high-speed trains the aim going forward is for the cameras to be able to monitor the track ahead at a distance of up to four kilometers.

Detecting objects and obstacles

In the future, the aim is for the system to be able to identify not only pedestrians but also signals and other items of rail infrastructure. Artificial intelligence helps classify objects and analyze the data. Thanks to real-time surroundings detection, any damage to the tracks, switches or overhead lines can be detected before these can fail and cause disruption.