Um ambiente para avaliação de algoritmos de aprendizado de máquina simbólico utilizando exemplos. / An environment to evaluate machine learning algorithms.
AUTOR(ES)
Gustavo Enrique de Almeida Prado Alves Batista
DATA DE PUBLICAÇÃO
1997
RESUMO
A learning system is a computer program that makes decisions based on the accumulative experience contained in successfully solved cases. The classification rules induced by a learning system are judged by two criteria: their classification error on an independent test set and their complexity. Practical learning systems have been developed using different paradigms including statistics, neural nets, as well as propositional and relational symbolic machine learning. Several learning methods can be applied to the same sample data and some of them may do better than others. Still, for a given application, there is no guarantee that any of these methods will work or that any single method is necessarily the best one. In other words, there is not a mathematical analysis method that can determine whether a learning system algorithm will work well. Hence, experimental studies are required. In this work we confine our attention to the learning task known as classification or prediction, where the problem concerns the construction of a classification procedure from a set of data for which the true classes are known, and is termed supervised learning. The overall objective of a classifier is to be able to predict successfully on new data. Performance is measured in terms of the error rate. Error rate estimation techniques not only provide a basis for objectively comparing the error rate of several classifiers on the same data and then estimating their future performance on new data, but they can also be a powerful tool for designing a classifier. The techniques of error rate estimation are based on statistical resampling theory. In this work, rules induced complexity of propositional and relational learning systems as well as several resampling methods to estimate the true error rate are discussed. An environment called AMPSAM has been implemented to aid in the application of resampling methods to real world data sets. AMPSAM consists of a collection of interdependent programs that can be bound together either by already defined or by new user defined scripts. The environment uses a common file format for data sets which is independent of any specific classifier scheme. It also includes facilities for splitting data sets up into test and training sets using different methods. Besides holdout, which is the most common accuracy estimation method, AMPSAM supports n-fold cross-validation --- including leaving-one-out --- and bootstrap. The confusion matrices produced in each run using those test and training sets can be input to another system called SMEC. This system calculates and graphically displays some of the most important descriptive measures related to central tendency and dispersion of those data. This work also reports the results of experiments measuring the classification error of three well known propositional and relational classifiers, using the implemented systems, on several data sets commonly used in Machine Learning research.
ASSUNTO(S)
data mining experimental evaluation machine learning avaliação experimental mineração de dados artificial intelligence inteligência artificial aprendizado de máquina
