Learning algorithm parameters#
Parameters for a learning algorithm determine the way that the model is fitted to the data. A common example is the ”C” or ”slack” parameter for soft margin SVM that defines the penalty associated with misclassifying individuals – i.e., whether the model does not allow any error within the CV1 folds and very tightly fits the decision boundary, or whether it allows errors in order to improve generalizability. Therefore, parameters such as these can greatly affect the model performance and it is common practice in machine learning to optimize the parameters for your specific problem and data.
NeuroMiner was developed in order to optimize performance across pre-defined parameter ranges within the nested cross-validation framework using a gridsearch, which protects against overfitting due to the application of the trained models to held-out data. Default parameter ranges are provided in NeuroMiner based on the literature and empirical testing, but we strongly recommend that the parameters are defined for your study and problem.
As previously stated, NeuroMiner uses a dynamic menu configuration that changes based on previous input. This is also true for the learning algorithm parameters, whereby the menu options will change based on what you have selected in the ”Classification algorithm” section.
Here we show an example for a RBF-Gaussian kernel Support Vector Machine classifier.
1 | Define Slack/Regularization parameter(s)
2 | Define RBF/Gaussian kernel parameter(s)
3 | Specify cross-parameter model selection process
1 | Define Slack/Regularization parameter(s) The Slack/Regularization parameter, also known as C parameter, is a hyperparameter specific to SVM. In NeuroMiner, a range of values can be specified here and it will be optimized throughout model training within the defined cross-validation structure.
2 | Define RBF/Gaussian kernel parameter(s) See link.
3 | Specify cross-parameter model selection process
This option controls the number of models (parameters combination) selected. One optimal model based on the criteria and regularization defined previously or an ensemble of the top performing models You are given the possibility to select between: (1) select a single optimum parameter node which returns one optimal model based on the criteria, (2) generate cross-node ensemble by aggregating base learners above a predefined percentile which results in an ensemble of the top performing models or (3) automatically determine optimal percentile for optimum cross-node ensemble performance which depends on the regularization.