Feature Selection — Part II

In the last article on Feature Selection, we discussed what feature selection is all about.

Let’s briefly discuss what some key feature selection algorithms are:

1. Filter methods

Here the features are independently categorized as important or not important without the use of any other classifier. Examples of such methods are, information gain, correlation coefficient, mean absolute difference.

2. Wrapper methods

In these methods, a subset of features is taken and evaluated for the accuracy or performance of the system. So there are two things here:

i. subset of features

ii. algorithm to compute the performance of the reduced system

The subset of features can be selected as:

(a) forward selection, here features are selected from one and are added incrementally in the set of important features, provided it increases the accuracy of the system

(b) backward selection, here all features are added to a set of important features, and the features are deleted from important features if on deletion there is no decrease in the accuracy of the system

(c ) exhaustive search, here the features are generated in an exhaustive manner, in that it creates all possible subsets and finds the one with the best accuracy to be selected as a feature subset of the entire dataset.

Other methods also exist for feature selection wherein techniques such as Genetic Algorithm based feature selection, Particle Swarm Optimization based feature selection, to mention a few, are used.

3. Embedded methods

Embedded methods combine feature selection and classification in one bundle. This means both feature selection and classification are performed together. One such famous technique is Quadratic Programming Feature Selection.

Each of these techniques would be taken independently in coming notes…