The Set Covering Machine (SCM) was introduced by Marchand & Shawe–Taylor [6, 7] in which a minimum set cover of a class of examples was approximated to find a compact conjunction/disjunction of features for classification. Their approach was to solve the set cover problem using thegreedyalgorithm. In this paper we introduce an alternative method of solving the SCM by formulating it as a Linear Programme (LP). In this setting we can apply an LP solver to give us our set of data-dependent features and use a convex combination of these features in order to classify unseen data for both the conjunction and disjunction case. Our hope is to approximate better solutions to the set cover problem using an LP as opposed to the greedy method approach evaluated in [6, 7]. The LP formulation is motivated by the LPBoost algorithm and so we also apply boosting algorithms, LPBoost and AdaBoost, to our set of features in order to compare our results with the original SCM and Support Vector Machine(SVM) classifiers.
[1]
Yoav Freund,et al.
Experiments with a New Boosting Algorithm
,
1996,
ICML.
[2]
Catherine Blake,et al.
UCI Repository of machine learning databases
,
1998
.
[3]
Leslie G. Valiant,et al.
A theory of the learnable
,
1984,
STOC '84.
[4]
David Haussler,et al.
Quantifying Inductive Bias: AI Learning Algorithms and Valiant's Learning Framework
,
1988,
Artif. Intell..
[5]
John Shawe-Taylor,et al.
Learning with the Set Covering Machine
,
2001,
ICML.
[6]
Vasek Chvátal,et al.
A Greedy Heuristic for the Set-Covering Problem
,
1979,
Math. Oper. Res..
[7]
Gunnar Rätsch,et al.
Robust Ensemble Learning
,
2000
.
[8]
John Shawe-Taylor,et al.
The Set Covering Machine
,
2003,
J. Mach. Learn. Res..
[9]
Ayhan Demiriz,et al.
Linear Programming Boosting via Column Generation
,
2002,
Machine Learning.