xBerry Case studies Crossing Guard

Crossing Guard

AI solution for safe passage through a rail crossing

Watch the video

At xBerry, we firmly believe that technology can benefit society. Therefore, we often engage in projects focused on improving security, especially in the transportation industry. Projects related to ensuring the safety of road users require large-scale testing and trusted solutions that are fail-proof even in emergencies. With this in mind, we created Crossing Guard – a system that enhances safety and facilites traffic control at railway crossings.

Challenge

According to a recent study, 98% of rail crossing-related accidents are caused by car drivers and pedestrians. These statistics, supported by the experience of our partner, inspired him to look for an innovative, modern solution that would improve supervision of railway crossings – and, as a result – travel safety.

Careful supervision of both cars and people is a difficult task, and yet analysis of behavioural patterns that lead to accidents is crucial in terms of prevention.

Our partner wanted to find a way to meet these requirements – that’s where we came in. In order to develop the best technological solution and facilitate the supervision process for railroad operators, we came up with the idea of an automated control system based on Artificial Intelligence.

Goals

The goal of the project was to create a system that uses AI to process video streams from several cameras around a crossing in real-time.

The image seen by the cameras needed to be divided into detection zones, so the system can assess whether a given vehicle is in the danger zone (e.g., approaching the railroad tracks while the stop light is signalling an approaching train). The system needed to recognize and classify all traffic participants and label them with unique IDs (e.g., licence plate numbers). To improve and reduce costs of supervision Crossing Guard was expected to automatically collect data in thousands of crossings across the country.

The data could be analysed later and used in many useful ways, from qualifying a crossing for additional security equipment (such as traffic signs, complex traffic light systems, and booths) to reporting all highway code violations to relevant authorities. The system should also provide information about the behaviour of traffic participants and learn behavioural patterns leading to accidents, in order to improve detection of dangerous action on railroad crossings.

Solution

We used Python as the main language. The major reason was the fact that Python provides a great development speed to performance ratio. It was fast enough to take care of non-critical functionalities but also made development very quick and easy.

We used C++ to implement functionalities that require high performance, like streams decoding/encoding and streams synchronisation
(up to 4 streams at once)
Qt with QML was used to create the front-end part of the product, which is rich in functionalities.

We relied on PyTorch which found its purpose in object detection. Its high, C++-like performance as well as ease of usage (thanks to Python bindings) made it the best choice for this task.
Because the core feature of the system is to detect objects during a real-time feed, we needed a state-of-the-art framework for widely understood image processing. OpenCV was the best choice for this task.

We used FFmpeg, which helped us enormously when it came to fetching and preprocessing RTSP feeds from surveillance cameras located at the crossing.
Due to the complexity and vast array of functionalities, we needed a mechanism that would ensure solid, accurate, and highly efficient implementation of inter-process communication. We chose ZMQ.

How does Crossing Guard work?

Results

We created a system that greatly facilitates traffic control at railway crossings and improves travel safety.

Thanks to Crossing Guard, railroad crossing supervisors can monitor accidents and dangerous events easier than ever. Thanks to the detection and identification of vehicles, Crossing Guard also helps in taking legal action when the situation so requires. Crossing Guard system is already in use at one level crossing in Poland and is proving extremely useful in tracking traffic flow and safety.

Tech Stack

ZMQ

FFmpeg

OpenCV

PyTorch

C++

Python

What are the customers saying?

They fully understood our needs, and the project scope was elastic enough to introduce changes during development. Their agile approach allowed us to avoid a lot of unnecessary costs and provided solid functionalities we didn't know we needed.

Wiktor Włodarczyk Protel