Overview
- Learn to Build Your Own Cricket Ball Tracking System Using Computer Vision and Python
- Understand different approaches to tracking fast-moving objects in a sports video.
- We will also discuss the various use cases of a ball tracking system..
Introduction
The Decision Review System (DRS) it's pretty ubiquitous in the sport of cricket these days. Teams are starting to rely heavily on the DRS to override strict referee rulings and that, quite often, you can change the match in your favor.
This concept of tracking the ball, part of the DRS, is now a family show for cricket fans:
This got me thinking: Could i build my own ball tracking system using my knowledge of deep learning and python?
I'm a huge cricket fan and I'm constantly looking for different use cases where I can apply machine learning or deep learning algorithms.. The idea of building a ball tracking system came to me when I was working on my previous project focused on generating insights from cricket feedback data..
Cricket teams and franchises also use this idea of ball tracking to understand weak spots in opposition players.. In what position is a particular hitter vulnerable?? Where does the bowler constantly throw in death over?
Ball tracking systems help teams analyze and understand these questions. Here's one such example from the recent cricket match:
- New Zealand bowlers followed a strategy against Virat Kohli on the previous tour from India to New Zealand. The first balls Virat faced were in his weak zone. This made him uncomfortable and he regularly lost his window early.
In this article, we will go through the various aspects of a ball tracking system and then create one in Python using the cricket example. This promises to be a unique learning experience!!
Note: If you are completely new to the world of deep learning and computer vision, I suggest you check out the following resources:
Table of Contents
- What is a ball tracking system?
- Ball Tracking System Use Cases in Sports
- Different Approaches to the Ball Tracking System
- Implementation: develop your first ball tracking system for Cricket using Python
What is a ball tracking system?
Let's quickly get acquainted with two popular terms in computer vision before a discussion of the ball tracking system.: object detection and object tracking.
Object detection is one of the most fascinating concepts in computer vision. It has a powerful role in different domains such as defense, space, sports and other fields. Here, I have listed some interesting use cases of object detection in defense and space:
- Auto target pointer
- Robot training in real word simulations to retrieve people in dangerous physical environments
- Space debris detection
But, What is object detection?
Image classification + Location = Object detection
Object detection is the task of identifying an object and its location in an image. Object detection is similar to an image classification problem, but with an additional task (also identify the location of an object), a concept known as location.
As you can see here, the location of the object is represented by a rectangular box which is popularly known as a Bounding Box.. Bounding Box represents the coordinates of the object in an image. But wait, How is object detection different from object tracking? Let's answer this question now.
Object tracking is a special case of object detection. Applies only to video data. On object tracking, the object and its location are identified in each frame of a video.
Object detection applied to every frame of a video becomes an object tracking problem.
Remember that object detection is for an image, while object tracking is for the sequence of fast moving frames. Both problems involve the same task, but the terms are used interchangeably depending on the type of data you are working with.
Then, How does this apply to ball tracking?
Ball Tracking System is one of the most interesting use cases for object detection and tracking in sports. A ball tracking system is used to find the path of the ball in a sports video. Hawk-eye is the most advanced ball tracking system used in different sports like cricket, tennis and soccer to identify the trajectory of the ball from high-performance cameras.
We can develop a similar system using the concepts of computer vision identifying the ball and its location in each frame of a video.. Here is a demo of what we will create in this article:
Impressive, truth?
Ball Tracking System Use Cases in Sports
The ball tracking system, as I am sure you will have already realized, is a powerful concept that transcends industries. In this section, i will show some popular use cases of ball tracking in sports.
Case of use 1: Critical decision making
We've talked about this before and I'm sure most of you will be familiar with hawk-eye in cricket..
The trajectory of the ball helps to make critical decisions during the game. For instance, in cricket, during the leg before the wicket (of LBW), the trajectory of the ball helps to decide if the ball has thrown in or out of the stumps. It also includes information on whether the ball hits the trunnions or not..
Similarly, in tennis, during serves or a play, the ball tracking system helps to know if the ball has been thrown in or out of the allowed lines on the court:
Case of use 2: identifying a hitter's strong and weak zones
Each team has a set of match-winning players. Choosing your grounds as early as possible is crucial for any team to win matches. With the help of ball tracking technology, we can analyze the raw videos and generate heat maps.
From these heat maps, we can easily identify the strong and weak zone of a batter. This would help the team develop a strategy against all players before a match.:
Can you think of other use cases for a ball tracking system in sports?? Let me know in the comment section below!!
Different Approaches to Ball Tracking Systems
There are different tracking algorithms, as well as previously trained models to track the object in a video. But, there are certain challenges with them when it comes to tracking a fast moving cricket ball.
These are the few challenges we must know before tracking a fast moving ball in a cricket video.
- The cricket ball is moving at a very high speed of about 130-150 km / h. Because of this, the ball goes on its way.
- Ball-like objects on the ground can be difficult to classify. For instance, the points of 30 yards on the field when viewed from above almost look like a ball.
Therefore, in this article, I will focus on 2 simple approaches to tracking a fast-moving ball in a sports video:
- Sliding window
- Color targeting
Let's discuss them in detail now..
Method 1: sliding window
One of the easiest ways could be to divide the image into smaller patches, let's say grids of 3 * 3 O 5 * 5, and then classify each patch into one of 2 lessons, whether a patch contains a ball or not. This approach is known as the sliding window approach., since we slide the window of a patch in each part of an image.
Remember that grid formation can also overlap. It all depends on how you want to formulate the problem.
Then, An example is shown showing the grids that do not overlap:
This method is really simple and effective. But it is a process that takes time, as it considers various patches of the image. Another drawback of the sliding window approach is that it is expensive, since it considers each patch of an image.
Then, then, I will discuss the alternative approach to sliding window.
Focus 2: color segmentation
Instead of considering every patch, we can reduce the patches to classify them according to the color of the ball. How we know the color of the ball, we can easily differentiate the patches that have a similar color to the ball from the rest of the patches.
This results in fewer patches to sort. This process of combining similar parts of an image through color is known as color segmentation.
Implementation: develop your first ball tracking system for Cricket in Python
Time to code! Let's develop a simple ball tracking system that tracks the ball on the field using Python. Download the necessary data files from here.
First, let's read a video file and save the frames in a folder:
Reading frames:
As our goal is to track the ball on the pitch, we need to extract the frames that contain the pitch. Here, I am using the concept of scene detection to accomplish the task:
Production:
The outlier in the graph indicates the frame number during which the scene changes. Therefore, set the threshold to get the frames before a scene change:
Now, we have obtained the frames that contain a tone. Then, we will implement a segmentation approach that we discussed earlier in this article. Let's go through all the steps of the single frame approach now.
We will read the frame and apply Gaussian blur to remove noise in an image:
Production:
How to know the color of the ball, we can easily segment white objects in an image. Here, 200 acts as a threshold. Any pixel value below this 200 will be marked as 0 and above 200 will be marked as 255.
Production:
As you can see here, white objects are segmented. White indicates white objects and black indicates all other colors. And that is! We have separated the white objects from the others.
Now, we will find the contours of segmented objects in an image:
Draw the outlines on the original image:
Production:
Then, extract patches from an image using outlines:
Time to build an image classifier to identify the patch containing the ball.
Read and prepare the data set:
Split the dataset into train and validation:
Build a baseline model to identify the patch containing the ball:
Evaluate the model on the validation data:
Repeat similar steps for each frame in a video followed by rating:
Take a look at the frames containing the ball along with the location:
Then, we will draw the bounding box around the frames containing the ball and save it back to the folder:
Let's turn the frames back into a video now:
Production:
How cool is that? Congratulations on building your own cricket ball tracking system!!
Final notes
Is all for today! This brings us to the end of the cricket ball tracking tutorial.. Note that a baseline model is created for image classification tasks. But there is still a lot of room to improve our model. And also, there are few hyperparameters in this approach, such as the size of the Gaussian filter and the threshold value to be adjusted depending on the type of video.
What are your thoughts on the system we build? Share your ideas and comments in the comment section below and let's talk..
