External cluster validation: MI, AMI and NMI - Example using sklearn's 'wine' dataset

Описание: In this video, I performed external cluster validation using sklearn's wine dataset. The wine dataset has three classes: class 0, class 1 and class 2. The total number of samples is 178. This dataset contains 13 features. I used the actual classes as ground truths and performed K-Means clustering considering 3 clusters utilizing the 11 features. I dropped 2 features for carrying out this task. The labels obtained from K-Means clustering are then used as predictions. Then, I calculated mutual information (MI) and adjusted mutual information (AMI) and normalized mutual information (NMI) using sklearn library by comparing ground truths with predictions.

GitHub address: https://github.com/randomaccess2023/M...

For mathematical details:
Mutual information (MI): https://scikit-learn.org/stable/modul...
Adjusted mutual information (AMI): https://scikit-learn.org/stable/modul...
Normalized mutual information (NMI): https://scikit-learn.org/stable/modul...

Description:
01:04 --- Import the required libraries
02:35 --- Load 'wine' dataset
04:06 --- Create a dataframe
06:28 --- Drop some features
07:47 --- Perform preprocessing
08:29 --- Scaled dataframe
09:41 --- Perform K-Means clustering considering 3 clusters
10:43 --- Add two new columns to the scaled dataframe
12:42 --- Clustering comparison between ground truths and predictions
15:57 --- Calculate mutual information between two clusterings
17:03 --- Calculate adjusted mutual information between two clusterings
17:41 --- Calculate normalized mutual information between two clusterings

#data_science #jupyter_notebook #python #external_cluster_validation #sklearn #mutual_information #adjusted_mutual_information #normalized_mutual_information