svmTrn¶

Description¶

The SVM (Support Vector Machine) model is a powerful supervised learning algorithm used primarily for classification tasks. It works by finding the hyperplane that best separates different classes in the feature space, maximizing the margin between the classes. In this implementation, we use the version provided by scikit-learn, which allows for flexible and efficient training. The inclusion of the Dimlp layer adds interpretability by enabling the extraction of decision rules via the Fidex algorithm, making the model's decisions more transparent and easier to explain. To get more details on the SVM algorithm with Dimlp, you can refer to this paper, and this one.

Arguments list¶

The svmTrn algorithm works with both required and optional arguments. Each argument has specific properties:

• Is required means whether an argument must be specified when calling the program or not.
• Type specifies the argument datatype.
• CLI argument syntax is the exact name to use if you are writing the argument along with the program call.
• JSON identifier is the exact name to use if you are writing the argument inside a JSON configuration file.
• Default value is the value that will be used by the program if the argument is not specified. If None, it could mean that the argument is not used at all during the algorithm execution or could also mean that you have to specify it yourself.

Show help¶

Display parameters and other helpful information concerning the program usage and terminate it when done.

JSON configuration file¶

File containing the configuration for the algorithm in JSON format (see more about JSON configuration files).

Root folder path¶

Default path from where all the other arguments related to file paths are going to be based. Using this allows you to work with paths relative to this location and avoid writing absolute paths or lengthy relative paths.

Train data file¶

File containing the train portion of the dataset, It can also contain training "true classes" (see Train true classes file).

Test data file¶

Path to the file containing test portion of the dataset, It can also contain testing "true classes" (see Test true classes file).

Number of attributes¶

Number of attributes in the dataset (should be equal to the number of inputs of the model). Takes values in the range [1,∞[.

Number of classes¶

Number of classes in the dataset (should be equal to the number of outputs of the model). Takes values in the range [2,∞[.

Train true classes file¶

File containing "true classes" (expected predictions), from the train portion of the dataset used to train the model.

Test true classes file¶

File containing "true classes" (expected predictions), from the test portion of the dataset used to train the model.

Train prediction ouput file¶

Path to the file where the train predictions will be stored.

Test prediction ouput file¶

Path to the file where the test predictions will be stored.

Statistics output file¶

Name of the output file that will contain all computed statistics.

Logs output file¶

Name of file containing every feedback made by the algorithm during its execution. If not specified, the feedback is displayed into the terminal.

Weights output file¶

Path to the file where the output trained weights of the model will be stored.

ROC output file¶

Path to the file where the output ROC curve will be saved.

Return ROC¶

Whether to return ROC statistics.

Positive class index¶

Index of positive class, index starts at 0. Takes values in the range [0,nb_classes-1].

Number of stairs¶

Number of stairs in the staircase activation function used in the Dimlp layer. Takes values in the range [3,∞[.

K parameter¶

Parameter to improve dynamics by normalizing input data. Takes values in the range ]0,∞[.

C parameter¶

Regularization parameter. Takes values in the range ]0,∞[.

Kernel¶

Kernel type used in the algorithm. Options are linear, poly, rbf, and sigmoid.

Polynomial degree¶

Degree of the polynomial kernel function. Takes values in the range [0,∞[.

Gamma¶

Gamma value. Kernel coefficient for rbf, poly and sigmoid. Can be float in the range [0,∞[ or String with the following options: scale, or auto.

Coef0¶

Independant term in the kernel function.

Use shrinking heuristic¶

Whether to use the shrinking heuristic.

Tolerance¶

Tolerance for the stopping criterion. Takes values in the range ]0,∞[.

Cache size¶

Kernel cache size(MB). Takes values in the range ]0,∞[.

Class weight¶

Class balance, for exemple with a dictionnary and 2 classes : {0:1.2, 1:3.5}. Can also be balanced.

Verbose¶

Enable verbose output.

Maximum number of iterations¶

Maximum number of training iterations. -1 means no limit. Takes values in the range [1,∞[.

Decision function shape¶

Decision function shape. Options are ovo(one-vs-one), and ovr (one-vs-rest).

Break ties¶

Whether to break tie decision for ovr with more than 2 classes.

Usage example¶

from trainings import svmTrn

svmTrn(
"""--train_data_file train_data.txt 
--train_class_file train_class.txt 
--test_data_file test_data.txt 
--test_class_file test_class.txt 
--weights_outfile svm/weights.wts 
--stats_file svm/stats.txt 
--train_pred_outfile svm/predTrain.out 
--test_pred_outfile svm/predTest.out 
--nb_attributes 16 
--nb_classes 2 
--root_folder dimlp/datafiles"""
)

./svmTrn --train_data_file train_data.txt --train_class_file train_class.txt --test_data_file test_data.txt --test_class_file test_class.txt --weights_outfile svm/weights.wts --stats_file svm/stats.txt --train_pred_outfile svm/predTrain.out --test_pred_outfile svm/predTest.out --nb_attributes 16 --nb_classes 2 --root_folder ../dimlp/datafiles

Output interpretation¶

Train/Test prediction file¶

This file contains the predicted probabilities for each possible class for each train (or test) sample. Each row corresponds to the prediction for a single sample, with N values 0 or 1 that the sample belongs to class 0, 1, ... or class N. The class with prediction 1 is considered the predicted class for that sample.

For example:

0 1
1 0

In the first row, the model predicts that the sample belongs to class 1 and it predicts the class 0 for the second sample.

Weights output file¶

This file contains the weights and biases of the first hidden layer of the neural network, which is the Dimlp layer.

• The first row in the file represent the bias values. There is one bias value for each neuron.
• The second row represent the values of the weight matrix between the first layer and the next one.

Statistics file¶

This file contains accuracy on the training and testing sets. It offers a clear overview of the model’s performance across different datasets, helping to evaluate how well the model has learned and generalized to unseen data.

Accuracy

Indicates the proportion of correctly classified samples in each dataset (training or testing).

Roc curve¶

This file contains a ROC (Receiver Operating Characteristic) curve, which is used to evaluate the performance of the Support Vector Machine (SVM) model during training. The ROC curve is a plot with the following components:

X-Axis (False Positive Rate)

This represents the proportion of negative samples that are incorrectly classified as positive. It measures the rate of false positives at various classification thresholds.

Y-Axis (True Positive Rate)

This represents the proportion of positive samples that are correctly classified as positive.

Curve

The curve itself shows the trade-off between the true positive rate and the false positive rate across different decision thresholds for the classifier. The curve starts at (0, 0) and moves towards (1, 1).

AUC (Area Under the Curve)

This value quantifies the overall performance of the model. It ranges from 0 to 1, with a value of 1 indicating perfect classification and a value of 0.5 indicating a model with no discriminative power. The higher the AUC, the better the model’s ability to distinguish between the positive and negative classes.

This ROC curve visually illustrates how well the SVM model is performing by showing the balance between the true positive rate and the false positive rate, with the AUC providing a summary measure of the model's classification performance.