Leandro M. Almeida

Ph.D. Project

Year 2011
Title: Building multiple classifiers systems by means of hybridization and optimization of techniques for data clustering and classification (Construção de sistemas de múltiplos classificadores por meio da hibridização e otimização de técnicas de agrupamento e classificação de dados).
Supervisor: Prof. Teresa B. Ludermir.
Keywords: Multiple Classifiers Systems, Ensembles, Artificial Neural Networks, Evolutionary Algorithms, Differential Evolution and Particle Swarm Optimization.
Funding: Brazilian Council for Scientific and Technological Development (CNPq).

Multiple Classifier Systems (also known as Ensemble of Classifiers) can use the combination or selection of hypotheses from different members to determine the hypothesis of solution for a given problem. The combination method is more popular and it is possible to find several strategies that have improved its performance since its conception. In addition, the selection method has no such advances as the combination method, although their potential has already been proved in some papers from literature. The construction of multiple classifiers system using the selection method involves the search for selection strategy, which can be through the clustering of training data and selection of classifiers specialized in the data of each cluster found.

Improvements in the classifiers selection method occur to define of the selection strategy, usually performed by a manual method. However, the best improvements of the combination method were achieved with the use of evolutionary methods (automatic) for parameter tuning. Due to the absence of hybridization with evolutionary methods for improving the selection method, to difficulties inherent in working by trial and error in search activities and to advance knowledge about the potential of selection method, it is necessary to explore the potential of the selection method by using evolutionary search methods.

This work explores the automatic construction of multiple classifiers systems using the selection method. In this way, the current thesis introduces a novel method that employs the Particle Swarm Optimization and Differential Evolution integrated to a Genetic Algorithm, used to enhance the classifiers selection strategy. The combination with evolutionary methods has the purpose to explore the potential of selection method, presenting the benefits of its hybridization with evolutionary search methods. The classifiers selection strategy adopted is composed by a phase which consists of clustering the training data and another phase for search specialized classifiers for each cluster found.

The experiments performed adopted the K-means and Self-Organizing Maps methods in clustering phase and Artificial Neural Networks, Linear and Multilayer Perceptrons in classification phase. Evolutionary Algorithms have been used (Particle Swarm Optimization with dynamic adjustment of parameters and Differential Evolution integrated to a Genetic Algorithm) in this work in order to optimize the parameters and performance of the different techniques used in clustering and classification phases. The experimental results have shown that the proposed method has better performance than manual methods and significantly outperforms the most of the methods commonly used to construct multiple classifier systems.