A novel data reduction method based on information theory and the Eclectic Genetic Algorithm

A common task in data analysis is to find the appropriate data sample whose properties allow us to infer the parameters and behavior of the data population. In data mining this task makes sense since usually the population is significantly huge, and thus it is required (for practical reasons) to obtain a subset that preserves its properties. In this regard, statistics offers some sampling techniques usually based on asymptotic results from the Central Limit Theorem. The effectiveness of such ways is bounded by several considerations as the sampling strategy (simple with or without replacement, stratified, cluster-based, etc.), the size of the population and the dimensionality of the space of the data. Due to these considerations alternative proposals are necessary. We propose a method based on a measure of information in terms of Shannon’s Entropy. Our idea is to find the optimal sample whose information is as similar as possible to the information of the population, subject to several constraints. Finding such sample represents a hard optimization problem whose feasible space disallows the use of traditional optimization techniques. To solve it, we resort to a breed of Genetic Algorithm called Eclectic Genetic Algorithm. We test our method with synthetic datasets; the results show that our method is suitable. For completeness, we used several datasets from real problems; the results confirm the effectiveness of our proposal and allow us to visualize different applications. Finally, we establish a baseline based on selection instance methods as a point reference to measure the effectiveness of our method.