MFS is a program that implements a multiple nearest neighbor classification system. It creates an ensemble of classifiers by selecting the features used for the component NN classifiers at random. The predictions from the individual NN classifiers are combined with simple voting.
The features used for the component NN classifiers are determined by randomly selecting X features from the data set. The value of X is determined by using cross-validation on the training set (i.e. try ten values from 0 to |F|). The feature sampling function is done either with or without replacement. Since the individual NN classifiers require no training, the features used are picked randomly at runtime.
As with most ensemble methods, MFS shows a large accuracy gain over using just a single nearest neighbor classifier or the k-nn classifier. In experiments with data sets available at the UCI Machine Learning Repository, MFS did extremely well even compared with boosted decision trees.
Bay, S. D. (1999). Nearest Neighbor Classification from Multiple Feature Subsets. Intelligent Data Analysis. 3(3):191-209. Abstract. Postscript. PDF. (preprint).
Bay, S. D. (1998). Combining Nearest Neighbor Classifiers Through Multiple Feature Subsets. In Proceedings of the Fifteenth International Conference on Machine Learning. Madison, WI. Abstract. Postscript. PDF.
THIS SOFTWARE IS PROVIDED "AS IS" WITH NO WARRANTIES OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO IMPLIED WARRANTIES AS TO THE PERFORMANCE, FITNESS, AND MECHANTABILITY OF THE SOFTWARE FOR A PARTICULAR PURPOSE.
THE ENTIRE RISK OF USING THE SOFTWARE IS WITH THE USER. THE SOFTWARE IS PROVIDED WITHOUT ANY SUPPORT OR OBLIGATION TO ASSIST WITH ITS USE.
THIS SOFTWARE MAY BE USED FOR EDUCATION OR NON-COMMERCIAL RESEARCH PURPOSES ONLY.
If you use MFS, please send me an email at sbay@apres.stanford.edu and let me know how you have used it.
MFS is written in C++ and has been compiled with gcc-2.8.1 on Solaris. It should work with most compilers with STL support.
The user manual has detailed instructions on running MFS. This manual is included in the program distribution.
If you have any comments about MFS or bugs to report, please email me at sbay@apres.stanford.edu. I can't guarantee that I will be able to fix the problem, but I will make an attempt.