The Computer Journal Advance Access originally published online on November 12, 2008
The Computer Journal 2009 52(7):808-823; doi:10.1093/comjnl/bxn058
| ||||||||||||||||||||||||||||||||||||||||||||||||
This article appears in the following The Computer Journal issue: Incorporating Profiling Expertise and Behaviour Special Issue [View the issue table of contents]
Semantic Analysis of Field Sports Video using a Petri-Net of Audio-Visual Concepts
1 School of Information System and Management, National University of Defense Technology, ChangSha, 410073, People's Republic of China
2 Centre for Digital Video Processing and CLARITY: Centre for Sensor Web Technologies, Dublin City University, Glasnevin, Dublin 9, Ireland
3 LERO, School of Computing, Dublin City University, Glasnevin, Dublin 9, Ireland
* Corresponding author: lbai{at}computing.dcu.ie
Received 31 October 2007; revised 15 July 2008
The most common approach to automatic summarization and highlight detection in sports video is to train an automatic classifier to detect semantic highlights based on occurrences of low-level features such as action replays, excited commentators or changes in a scoreboard. We propose an alternative approach based on the detection of perception concepts (PCs) and the construction of Petri-Nets, which can be used for both semantic description and event detection within sports videos. Low-level algorithms to detect PCs using visual, aural and motion characteristics are proposed, and a series of Petri-Nets composed of PCs is formally defined to describe video content. We call this a perception concept network–Petri-Net (PCN–PN) model. Using PCN–PNs, personalized high-level semantic descriptions of video highlights can be facilitated and queries on high-level semantics can be achieved. A particular strength of this framework is that we can easily build semantic detectors based on PCN–PNs to search within sports videos and locate interesting events. Experimental results based on recorded sports video data across three types of sports games (soccer, basketball and rugby), and each from multiple broadcasters, are used to illustrate the potential of this framework.
Key Words: Video semantic analysis Petri-Net audio-visual concepts