Ball detection and tracking: Foreground detector reinforcement through trajectory analysis

In the context of team sport events monitoring, the various phases of the game must be delimited and interpreted. In the case of a basketball game, the detection and the tracking of the ball are mandatory.

The detection of the ball in a basketball game is not easy compared to the detection of the ball in a football or a tennis game. In fact, the ball is often occluded by players. Thus we deal with the detection of the ballistic trajectory of a ball thrown between two players or toward the basket.

We address the problem of the detection and the tracking of a ball in single-view and multi-view setting.

Our method is based on a foreground detector. It is divided into two consecutive parts and follows the “detect before tracking” approaches. The first part consists in 3D ball candidates detection and the second one in ballistic trajectory filtering based on a motion model. The second part is essential because, at some time-stamp, multiple detections as well as none happen.

Concerning the first part, two approaches, based on the foreground detector, are investigated to detect the 3D ball candidates. The correlation-based (Corr) and the connected component analysis (CCA) methods fundamentally differ in the way they process the foreground masks:

  • The correlation (Corr) approach scans a relatively dense grid of 3D points, and correlates each one of the 2D views with an approximated template of the ball silhouette. Such a process is computationally realistic in a limited 3D area, e.g. around the basket or the goal in a basket-ball or soccer game for example, but becomes intractable over large scenes.
  • The connected component analysis (CCA) approach starts by picking-up 2D ball position candidates from 2D images, and then triangulates to infer 3D candidates. This method might suffer when the ball silhouette is split into several pieces of foreground mask, but it scales reasonably well with the size of the scene.

Concerning the second part, to remove false positive detection, temporal analysis is performed. Generally, the sources of false positives can be accredited to players reflect, digit on the score board, noise, etc. Fortunately, these false detections do not follow the typical ballistic trajectory of the ball in the air. We have proposed two approaches to detect this trajectory.

  • The first one is based on the RANSAC approach. In the case of a multi-view setting, the ballistic trajectory model is express in 3D (RANSAC). In the case of a single-view setting, the ballistic trajectory model is express in 2D (RANSAC 2D).
  • The second one is based on a graph (Graph).

Video samples presenting the results associated to the paper 'Consensus-based trajectory estimation for ball detection in a calibrated cameras system' (submitted to the Journal of Real-Time Image Processing):

The filter videos refer to the performance obtained by filtering out the candidates that do not fit a ballistic model, while the trajectory videos display the performances by interpolating missed detections between the inliers, i.e. the candidate positions that follow a ballistic model, in addition to rejecting outliers.

The results are computed on the APIDIS dataset composed of 7 cameras (available here).

Part of the code is available here.

  • Quantitative assessment based on a 3-minutes groundtruth

We present here under the videos associated to the tables measuring the ball detection accuracy in the paper ‘Consensus-based trajectory estimation for ball detection in a calibrated cameras system’. Red = detected candidates. Green = candidates that are validated based on trajectory analysis.

Whole field, multi-view setting (see Fig. 6-(a) in the article)

 

Trajectory

 

RANSAC

Graph

CCA

 

Filter

 

RANSAC

Graph

CCA

Basket area, multi-view setting (see Fig. 6-(b) in the article)

 

Trajectory

 

RANSAC

Graph

Corr

CCA

Basket area, single-view setting (cam 7) (see Fig. 7 in the article)

 

Trajectory

 

Corr

CCA

RANSAC 2D

  • Qualitative assessment

We present videos for which the ground truth is not available. It extends the quantitative assessment on a longer APIDIS video segment, and on video segments captured in other basketball courts.

Whole field, multi-view setting (APIDIS second quarter)

 

Trajectory

 

RANSAC

Graph

CCA

Basket area, single-view setting, Corr, RANSAC 2D

 

Trajectory

 

seq. 1

seq. 2

cam 7

 

seq. 3

seq. 4

cam 1

 

seq. 5

cam 1

References:

  • Parisot, Pascaline; De Vleeschouwer, Christophe. "Graph-based filtering of ballistic trajectory." IEEE International Conference on Multimedia and Expo (ICME). July 11-15, 2011. Barcelona, Spain. Paper
  • K.C., Amit Kumar; Parisot, Pascaline; De Vleeschouwer, Christophe. "Demo: Spatio-temporal template matching for ball detection." ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC-2011). August 22-25, 2011. Ghent, Belgium. Paper , Demo (spatial template only), Demo (both spatial and temporal templates)
  • Parisot, Pascaline; De Vleeschouwer, Christophe. "Consensus-based trajectory estimation for ball detection in a calibrated cameras system." (in submission at JRTIP (Journal of Real-Time Image Processing)) (code available here)

Categories

IntelVision AnalysisInterpretation ResearchCategoryAll

Projects

SPORTIC

ISPGroup Participant(s)




Last updated September 19, 2017, at 12:12 PM