The Computer Journal Advance Access published online on February 1, 2007
The Computer Journal, doi:10.1093/comjnl/bxl065
| ||||||||||||||||||||||||||||||||||||||||||||||||||||
Hedging Predictions in Machine Learning
The Second Computer Journal Lecture
Computer Learning Research Centre, Royal Holloway, University of London, Egham, Surrey TW20 0EX, UK
* Corresponding author: alex{at}cs.rhul.ac.uk
Recent advances in machine learning make it possible to design efficient prediction algorithms for data sets with huge numbers of parameters. This article describes a new technique for ‘hedging’ the predictions output by many such algorithms, including support vector machines, kernel ridge regression, kernel nearest neighbours and by many other state-of-the-art methods. The hedged predictions for the labels of new objects include quantitative measures of their own accuracy and reliability. These measures are provably valid under the assumption of randomness, traditional in machine learning: the objects and their labels are assumed to be generated independently from the same probability distribution. In particular, it becomes possible to control (up to statistical fluctuations) the number of erroneous predictions by selecting a suitable confidence level. Validity being achieved automatically, the remaining goal of hedged prediction is efficiency: taking full account of the new objects' features and other available information to produce as accurate predictions as possible. This can be done successfully using the powerful machinery of modern machine learning.
Key Words: Classification • confidence • induction • learning • prediction • randomness • regression • transduction