Virtual Fans Want to Buy Tickets
When the general public universally accepted smartphones and 4G between 2010 and 2015, the sports industry was the first to take advantage of it in order to grow its fanbase.
Barça Innovation Hub (BIHUB) has partnered with Pixellot to develop new systems for automated recording of training sessions and games. This new alliance will allow helping improve technical and tactical analysis by the managers from the different professional sports at Ciutat Esportiva.
The company’s technology allows to film, produce, and distribute sports games automatically and efficiently for fans and the coaching staff. Through the use of AI, it will simulate a camera operator on a high-resolution panoramic image of the pitch.
The panoramic capture uses a wide-angle camera or multiple wide-angle cameras and then stitches these images together. The technology uses advanced auto-tracking algorithms to follow the flow of play within the hi-res panoramic capture. These Artificial Intelligence (AI) decisions need to be made with minimal latency to allow accurate human-like capturing. The latency buffer, in essence, is like a camera operator who looks into the future to understand the flow of the play and then makes a conscious decision about how to shoot it.
To provide the right viewing experience, the technology needs to simulate the human camera operator capture with smooth movements, preferably simulating the movement of a video tripod with a fluid head.
The fact that the same game needs to be filmed differently for the fans and for the coaching staff also allows automation to prove its efficiency. For the coaching staff, it is more important to focus on the tactical view of the game. An analyst looks at the team formation i.e. all the field players excluding the goalkeepers. For the fans watching at home, this angle might be too far to enjoy the game.
Most automatic capture technologies share the following principles. However, because each sport has its own game logic each game requires completely different algorithms.
One of the challenges in player detection is when a player is standing still for a few seconds. For example, in football, during a free kick, some of the players may not move for a period of about 30 seconds. When this occurs, the algorithm must ensure that the player will not blend into the background.
In addition, the algorithm should be able to distinguish between players who are not active (e.g. waiting in the sidelines) from the active players, even when some of the active players are far from the ball. The algorithm should also be able to identify the referee, who is not part of the gameplay.
Based on the ball and the player detection, the algorithm needs to identify the game state. The game state is the type of play currently happening. For example in football, a corner kick or a penalty kick. In basketball a free throw or a counter-attack, etc. Each game state has its own visual characteristics and by understanding the game state, the algorithm can predict certain behaviours and make smarter decisions about how to best capture the action. Every sport has a long list of different game states, making this a challenging task for an automated system.
To overcome this challenge, the game state detection may be based on Deep Learning algorithms, which can automatically learn how to identify a corner kick, based on a data set of examples. In the case of deep learning, the system trainer doesn’t have to come up with rules about how to identify the corner kick. The system will automatically generate its own rules and select its own characteristics to identify this specific game state. By taking into account all these parameters, the system can make a decision on how to capture each frame.
Today’s extraordinary advancement in computing enables the placement of extensive AI and deep learning capabilities in the field. The combination of computer vision, AI, and deep learning provides, for the first time, a viewing experience that is comparable to a human camera operator and, in some respects, supersedes the abilities of a single camera operator. These algorithms will get better and will be able to cover more types of sporting events and isolate different scenarios in the practice sessions.
The core capabilities have been already proved and tested in the pitch. With the help of professional coaches, trainers and analysts at FC Barcelona, the research and development of Pixellot’s features in close collaboration with the BIHUB will add new automation processes to streamline the current workflows, as well as catering the advanced requirements the club has.
The Barça Innovation Hub team