Kamal Ali's ISLE Web page

Personal Information

• Position: Research Scientist
• Research Area: Applications of data-mining to rooftop detection in satellite images and applications for fault prevention
• Office: Room 225, Computational Learning Lab, Ventura Hall, Stanford University, 220 Panama St., Stanford, CA 94305-4115
• Phone: (650) 723 1684

My current research interests

Rooftop detection resources for the IU/APGD community

• (Use shift-click on a URL to download if you have trouble downloading it otherwise)
• Our latest report on rooftop detection (90K) including ROC curve results for a Naive Bayes rooftop-detector using 57 features at the verification level with the ROC evaluation being done at the building level rather than at the rooftop-candidate level. Several candidates may try to model one building: it is only necessary for one of them to be correct for the system to "get" the building. Furthermore, earlier work on ROC curves at the candidate level ignored buildings for which no candidates were generated giving over-optimistic results. This research is supported by DARPA.
• Paper on rooftop detection at the IU98 Monterey Workshop (256K) In this paper, we report progress on the use of machine learning to improve the process of rooftop detection in aerial images. We describe an existing system for building recognition, BUDDS, and identify its rooftop stage as a target for improvement. We then review the naive Bayesian classifier, a simple but robust approach to supervised induction, and the visual interface we developed to ease the labeling of training data. We present the results of experiments on the rooftop detection task that reveal improved recognition levels over the handcrafted BUDDS classifier, then examine the reliability and speed of the interactive labeling process itself. Finally, we consider related research and plans for future work.

Papers on data mining from real-world consulting projects

• KDD 97 paper on "Partial classification using Association Rules" (43K) Many real-life problems require a partial classification of the data. We use the term ``partial classification'' to describe the discovery of models that show characteristics of the data classes, but may not cover all classes and all examples of any given class. Complete classification may be infeasible or undesirable when (1) there are a very large number of class attributes, (2) most attributes values are missing, or (3) the class distribution is highly skewed and the user is interested in understanding the low-frequency class. In such cases, users often want insights into the data rather than a complete predictive model for each class. We show how association rules can be used for partial classification in such domains, and present two case studies: reducing telecommunications order failures and detecting redundant medical tests.

Research on committee machines, multiple models

• PHD Thesis: Learning Probabilistic Relational Concept Descriptions. (583K) This dissertation presents methods for increasing the accuracy of probabilistic classification rules learned from noisy, relational data. It addresses the problem of learning probabilistic rules in noisy, ``real-world'' data sets, the problem of ``small disjuncts'' in which rules that apply to rare subclasses have high error rates, and empirical results on multiple models research - especially showing that the degree of correlatedness of errors of models is negatively correlated with the amount of error reduction afforded by the multiple models approach.
• Error Reduction through Learning Multiple Descriptions. (149K; MLJ) This Machine Learning Journal paper (Vol. 24, No. 3) describes work on learning and combining classifications made by multiple rule sets. We show that the model-combining methods considered here are much more effective at reducing error rates in problems where there are many irrelevant attributes. We show however, that on noisy UCI datasets, the multiple models approach is not able to reduce error as much as it can on the relatively noise-free but high dimensionality problems. This leads us to consider the term "noise-limited" datasets. We also show that in the limit (although not realized in the UCI repository), large numbers of irrelevant attributes will have the same negative effect on the multiple models approach as do the noisy datasets.
• A comparison of methods for learning and combining evidence from multiple models . (112K) This technical report compares three methods for generating multiple models: Stochastic learning, k-fold partition learning and Bagging. It also compares four methods for combining evidence from such models to make a classification: Uniform Voting, Bayesian Combination, Distribution Summation and Likelihood Combination. We find that k-fold partition learning is inferior to the other methods, Bagging is better than Stochastic on noisy data sets and Stochastic is better than Bagging on small data sets.
• On the Link between Error Correlation and Error Reduction in Decision Tree Ensembles. (122K; UCI Tech. Report (122K) This technical report shows that there is a linear relationship between the amount of error reduction due to an ensemble of decision trees and the degree to which the decision trees make errors in an uncorrelated manner. It also shows that some of the greatest error reductions occur on domains which have many "gain ties". Finally, it shows that it is better to learn an ensemble that makes errors in a negatively-correlated manner rather than in an uncorrelated (statistically independent) manner.
• Classification using Bayes Averaging of Multiple, Relational Rule-based Models. (76K) This book chapter (in "Learning from Data: Artificial Intelligence and Statistics, Vol. 5.", Fisher, D. & Lenz, H. (Eds.)) presents a theoretically-sound way of combining classifications from multiple rule-set models. A rule-set model learns a rule-set for each class in the data. A rule-set for a class is a set of rules that all conclude for that class. The paper shows how to compute the posterior probability of a rule-set model and shows that using rule-set models in this way leads to higher classification accuracy on relational data base problems. We also use this approach to learn multiple, recursive concept descriptions.
• HYDRA-MM: Learning Multiple Descriptions to Improve Classification Accuracy (journal version) (105K; International Journal on Artificial Intelligence Tools, Vol. 4, 1 & 2.) This is a general TAI journal paper showing that learning multiple rule-set models can improve classification accuracy as compared to a single model learned on the same training data. There is also a comparison to the multiple rules approach showing that in some circumstances, using multiple rule-sets is better than using multiple rules. There is also the version that appeared in the Proceedings of the Tools for Artificial Intelligence Conference (104K)

Research on probabilistic relational models, noise-tolerance