In the innovation of football technology, researchers from Shanghai Jiao Tong University and Alibaba have taken an important step forward. Their new artificial intelligence system—MatchVision—not only watches football matches but also identifies key plays and provides commentary similar to human broadcasters.
This technology is developed based on a large dataset called "SoccerReplay-1988," which contains nearly 2,000 complete matches from the top European leagues and the UEFA Champions League between 2014 and 2024, totaling over 3,300 hours of match footage. Each match has an average of 76 commentary segments.
The table shows how MatchVision identifies key moments in the match and generates commentary for each scene. | Image: Rao, Wu, et al.
MatchVision is designed as an integrated system capable of handling multiple tasks simultaneously, including tracking match events and generating natural commentary. The system can identify 24 different types of match events, such as goals, fouls, and tactical actions. When analyzing fouls, it uses multi-angle camera shots to determine the type and severity of the foul.
Testing data shows that MatchVision achieved an accuracy rate of 84% in identifying match events, excelling not only in event recognition but also in generating commentary and judging fouls compared to existing systems. The research team plans to open-source the dataset and model on GitHub for more researchers and developers to use.
Interestingly, the research found that AI and human commentators focus on different aspects of the match. AI pays more attention to technical details and tactics, while human commentators are more focused on the emotional flow of the game and background stories.
Side-by-side examples compare how human commentators (GT) and AI (Ours) describe three key moments in the match—a controversial yellow card, a corner kick sequence, and a goal. | Image: Rao, Wu, et al.
The researchers showcased the contrasting commentary between AI and humans on specific scenes such as yellow cards, corner kicks, goals, and goalkeeper saves, highlighting the different ways they describe key moments in the match.
In the future, the application of MatchVision may extend beyond match commentary to automatically create highlight reels and even assist referees in making more accurate decisions, based on existing AI technologies like offside detection.
This research marks a new era in sports analysis and AI applications, providing football fans and professionals with a brand new viewing experience.
