KTU researchers enhance live football broadcasts by eliminating cameramen distractions with AI

While the sports industry continually improves spectators’ viewing experience at home, some problems remain unsolved. One such issue for football fans is cameramen accidentally appearing in each other’s shots on live broadcasts. These occurrences not only detract from critical game moments but potentially lead to revenue losses for broadcasters due to viewer dissatisfaction. To address this, researchers at Kaunas University of Technology (KTU) have developed an end-to-end system to enhance the viewing experience by eliminating visual distractions caused by overlapping camera angles.

“Our new invention is an algorithm adapted to detect video operators,” says KTU professor Rytis Maskeliūnas, one of the creators of the innovation.

Serhii Postupaiev, another members of the research team, highlights that the presence of cameramen in the frame is a frequent issue in football broadcasts due to the complex nature of live sports coverage and the numerous cameras around the stadium.

Eliminating visual distractions

To address this problem and remove unwanted objects during a live broadcast, KTU scientists designed and implemented an end-to-end system.

For its functioning, the YOLOv8 model, a state-of-the-art object detection system known for its speed and accuracy, was employed. YOLOv8, which stands for “You Only Look Once”, can detect and classify objects in images in a single pass, making it ideal for real-time events such as live football broadcasts.

As the inventor explains, this process was needed to create the foundation for the actual removal of the operators. For this purpose, video inpainting technology was used.

“The term inpainting in deep learning refers to the process of reconstructing lost or deteriorated parts of images and videos. Specifically, in this case, it is used for removing cameramen from football video broadcasts,” says Postupaiev.

Artificial intelligence (AI) and computer vision-based technology analyses the video frames to detect unwanted objects such as cameramen and fills the removed areas with relevant background details. The modified frames are then streamed back to viewers, ensuring a more immersive and professional broadcast.

Shifting the focus from simply capturing the action

With this new technology, watching football matches at home will be significantly improved. One of them is a smoother viewing experience.

“The broadcast will feel more polished and professional without disruptions caused by cameramen appearing where they shouldn’t. This improvement will reduce the number of cases where important moments of the game are missed due to distracting shots,” highlights Postupaiev, who received his Master’s degree with this project.

“This could even breathe new life into old recordings of classic matches,” adds a KTU graduate.

Invention is not limited to football – it can be applied to other sports with similar broadcasting challenges. Dynamic sports like futsal and basketball, which require immersive broadcasts, can also benefit from this technology.

“This is yet another illustration of what modern AI applications can do. We often hear about medical applications, but here we have a consumer-oriented approach to editing images we don’t like. In the future, such technology will be able, for example, to remove advertisements or replace them with other ones thus constantly updating the content with a level of precision that the human eye will not notice,” says KTU Faculty of Informatics professor Maskeliūnas.

The paper Real-time camera operator segmentation with YOLOv8 in football video broadcasts can be found here.