#AI #artificialintelligence #ml #machinelearning

Note: This article is present on authors medium.com account as well.

In the past few articles, you might have got an idea of what feature selection is. In short, it is used to improve accuracy, decrease noise and at the same time decrease computational complexity by reducing irrelevant features from big datasets. Further, it informs about the importance of a feature as well.

The selection of features can be considered as a combinatorial optimization problem, where we are given n features, and for each feature, a decision is to be made to select or mark the feature as important or consider it as redundant, or a noise.

In this article, you shall learn how genetic algorithms can be used to perform feature selection. Here are the key points and this is followed by working code in python.

1. The aim is to generate a population of candidates which can be evaluated for key features in the dataset.
2. In each iteration of Genetic Algorithm (GA) new features are generated as part of GA, selection, and reproduction and mutation concepts. However, we are using python package, namely, geneticalgorithm. This library hides or encapsulates the inner working of geneticalgorithms and also hides the process of generation of the next candidate.
3. This python library minimizes the objective function.
4. We have taken objective function as accuracy generated by the system when a subset of features is chosen by GA is to participate in the classification problem.
5. The final output of the code, is a subset of features that can be used in future data prediction of target class.
6. The code is in python with explanations.

Python Implementation

Install the library

pip install geneticalgorithm

Import essential libraries

import pandas as pd
import numpy as np
import sklearn
from sklearn import svm
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
from geneticalgorithm import geneticalgorithm as ga

Read the dataset

dataset = pd.read_csv('/content/colon_cancer.csv')
numRows, numCols =  np.shape(dataset)

Define the objective function to optimize by Genetic Algorithm

def f(combinational_array):
 
  print('the new ga array is: ')
  print(combinational_array)
  arrayToTest = getIndx(combinational_array)
  print(arrayToTest)


  X=dataset.iloc[0:numRows, arrayToTest].values
  Y = dataset.iloc[0:numRows, numCols-1].values
  X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=0.3)
  svmModel  = svm.SVC()
  svmModel.fit(X_train, Y_train)
  X_Predict = svmModel.predict(X_test)
  predictedValue = accuracy_score(X_Predict, Y_test)
  print ("predictedValue: ")
  print(predictedValue)  
  return  -1 * predictedValue

Auxiliary function definition

def getIndx(arrayIndx):
  lenArr = len(arrayIndx)
  arrayOutput=np.zeros(lenArr)
  j=0
  for i in range(lenArr):
    if arrayIndx[i] >= 0.5:
      arrayOutput[j] = i
      j=j+1

  if j==0:
    return ([0])

  arrayReturn=np.zeros(j)
  for i in range(j): 
      arrayReturn[i] = arrayOutput[i]

  return arrayReturn

Define parameters and call the Genetic Algorithm to perform feature selection. The dimension of colon cancer data undertaken here is more than 1500, so let’s test on a smaller length of say 10 features. When computing over full data, increase the number of features to one-fourth of the number of columns at least to start with. 10 features are taken just as an example to show how it works.

length = 10 
# length = int(numCols/4)
print("length is : ", length)
 
varbound=np.array([[0,1]]*length)
model=ga(function=f,dimension=length,variable_type='int', variable_boundaries=varbound)

model.run()

The sample output is as follows:

The sample output with more features is as follows:

The aim is to choose both the number of features and the features themselves, both using inbuild GA based implementations.

Thank You for reading.