Design, analyze and compare machine learning models
Introduction
During my last semester of the Bachelor's degree in Computer Science, in the Artificial Intelligence course, my partner and I designed from scratch in Python two machine learning models: a decision tree and an artificial neural network.
Data Preparation
This project was structured in several stages. The first stage consisted of data preparation. We discovered that the data contained 14 attributes and a "Class" label with 4 different classes that our models had to predict.
Model Implementation
Decision Tree
For the decision tree, we used a `DecisionNode` class representing a node and a `DecisionTree` class representing the decision tree itself. In this class, we wrote several methods, the most important of which allowed the tree to be built recursively. We trained several models with depths ranging from 3 to 6 to select the two best models for the binary decision tree.
Artificial Neural Network
Regarding the artificial neural network, we set aside 15% of the training set to constitute a validation set, which allowed us to implement early stopping with a patience of 4. The training was conducted using mini-batches of 4 instances. We trained several models with different architectures (activation functions).
Model Analysis
The final stage consisted of analyzing our models. We calculated, without using external libraries, the accuracy, precision, recall, and F1-Score for each model.