[Modules]

The Modules section defines the main modules used for model selection, learning and performance evaluation. The attributes in the section are

[Parameters]

Parameters section contains the parameters supplied to RLScore. The meaning of kernel and learner parameters differs for different learning and kernel modules.

[Input]

The attributes in this section are names of RLScore variables used inside the RLScore software. The values of the attributes are filenames from which data is loaded to the variables. For example, the feature representations of the training data are loaded into a variable of name train_features. Some of the loaded [Modules] require certain valiables to be loaded. The loaded variables also have an effect on what rls_core does.

All variables have their corresponding default file formats. Detailed descriptions of the variables and their default file formats are given in RLScore variables.

[Output]

Analogously to the [Input] section, the attributes in this section are names of variables used inside RLScore. The values of the attributes are names of files into which the contents of the variable are written to. The files are written in the default format of the variable in question.

train_features

Variable containing features for training data. The default file format is the one described in Featurefile.

train_labels

Variable containing labels for training data. Necessary when training supervised learners. The default file format is the one described in Labelfile.

train_qids

Qids for the training data. The default file format is the one described in Qid file.

basis_vectors

Use reduced set approximation to speed up training and prediction. Restricts the learned hypothesis to be represented only by the training data points whose indices are in the basis vector file. The default file format is the one described in Basis vectors.

cross-validation_folds

Variable containing indices of holdout data points, one row per hold-out set. This can be used to define folds for cross-validation or, more generally, hold-out sets for repeated hold-out. The default file format is the one described in Fold file.

model

This variable contains a model learned from a data. It will be generated if user provides a learner attribute and training data for the learner. Model can be saved into a file via Python's pickle protocol. Previously learned model can be loaded from a file in order to perform predictions for unseen data.

prediction_features

Features for data one wishes to make predictions for. Prediction will be performed if a model is loaded from a file or if a predictor has been trained. The default file format is the one described in Featurefile.

test_labels

Correct labels for test data, supply these if you want to measure performance on test data. The default file format is the one described in Labelfile.

predicted_labels

Predicted labels for test data. These are generated if a model is used to perform predictions. These are also needed if one wants to measure performance on test data. The default file format is the one described in Labelfile.

prediction_qids

Qids for test data, supply these if you want to evaluate performance on test data as an average over queries. The default file format is the one described in Qid file.

predicted_clusters_for_training_data

Results of MMC clustering on the training data (see Clustering with evolutionary maximum margin clustering).

selected_features

The indices of the features selected by the GreedyRLS learner (see Feature selection with greedy RLS).

GreedyRLS_LOO_performances

The list containing the LOO performances made by GreedyRLS during the greedy forward selection process (see Feature selection with greedy RLS).

validation_features

Variable containing features for validation data. Necessary when using ValidationSetSelection for choosing the regularization parameter. The default file format is the one described in Featurefile.

validation_labels

Variable containing labels for validation data. Necessary when using ValidationSetSelection for choosing the regularization parameter. The default file format is the one described in Labelfile.

validation_qids

Qids for the validation data. Necessary when using ValidationSetSelection for choosing the regularization parameter, for query structured data with LabelRankRLS, or CGRankRLS (see the learner attribute).

Config file (classacc_all)

#Binary classification (accuracy maximizing)
#
#Accuracy maximizing binary classification is done by training a RLS regressor
#with labels +1 for positive and -1 for negative class. The algorithm
#is equivalent to the so-called 'Least-squares support vector machine',
#and is known to provide similar performance as SVMs
#
#For Area under the ROC curve (AUC) maximizing binary classification, check
#the corresponding example.
#
#this examples chooses regularization parameter, trains the method,
#makes test predictions and calculates test performance all at once

[Modules]

#RLS regressor is used as the training algorithm
learner=RLS

#Leave-one-out cross-validation can be used for parameter selection
mselection=LOOSelection

#Alternatively, 10-fold cross-validation with randomized fold partition
#could be used. It is also possible to supply your own folds.
#mselection=NfoldSelection

#Accuracy measures the fraction of correct classifications made
measure=accuracy

#Linear kernel is always a reasonable first choice, advanced examples
#show how to learn non-linear models with other kernels.
kernel=LinearKernel

[Parameters]
#search regularization parameter from grid 2^-10...2^10
reggrid=-10_10

#bias is mostly useful for linear models with low-dimensional data
bias=1

[Input]
#features of the training examples
train_features=./examples/data/class_train.features
#labels of the training examples
train_labels=./examples/data/class_train.labels
#features of the test examples
prediction_features=./examples/data/class_test.features
#true labels of the test examples
test_labels=./examples/data/class_test.labels

[Output]
#the learned model is written here
model=./examples/models/classacc.model
#the predicted labels are written here
predicted_labels=./examples/predictions/classacc.predictions

Python code (classacc_all)

from rlscore import core
from rlscore import reader
from rlscore import writer
from rlscore.measure import accuracy

kwargs = {}
kwargs['train_labels'] = reader.read_dense('./examples/data/class_train.labels')
prediction_features = reader.read_sparse('./examples/data/class_test.features')
kwargs['train_features'] = reader.read_sparse('./examples/data/class_train.features')
test_labels = reader.read_dense('./examples/data/class_test.labels')
kwargs['reggrid'] = '-10_10'
kwargs['bias'] = '1'
kwargs['kernel'] = 'LinearKernel'
kwargs['learner'] = 'RLS'
kwargs['measure'] = accuracy
kwargs['mselection'] = 'LOOSelection'
trainresults = core.trainModel(**kwargs)
model = trainresults['model']
print 'Making predictions on test data'
predicted_labels = model.predict(prediction_features)
performance = accuracy(test_labels, predicted_labels)
print 'Performance: %f %s' % (performance, accuracy.__name__)
writer.write_pickle('./examples/models/classacc.model', model)
writer.write_dense('./examples/predictions/classacc.predictions', predicted_labels)

Config file (classAUC_all)

#Binary classification (AUC maximizing)
#
#AUC maximizing binary classification is done by training the RankRLS ranker
#with labels +1 for positive and -1 for negative class. The ranker aims
#to solve the bipartite ranking task of ranking positive examples higher
#than negative ones, which corresponds to AUC-maximization.
#
#(see An efficient algorithm for learning to rank from preference
#graphs", Machine Learning, 2009 for further details on the training algorithm) 
#
#Leave-pair-out cross-validation is the recommended strategy for model
#selection, as leave-one-out estimation of AUC is known to have serious
#negative bias in some cases
#
#(see A Comparison Of AUC-Estimators In Small-Sample Studies, JMLR proceedings
#of MLSB'09. 2010)
#
#this examples trains chooses regularization parameter, trains the method,
#makes test predictions and calculates test performance all at once

[Modules]

#RankRLS is used as the training algorithm
learner=AllPairsRankRLS

#Leave-pair-out cross-validation can be used for parameter selection
mselection=LPOSelection

#Alternatively, 10-fold cross-validation with randomized fold partition
#can be used for large data sets. It is also possible to supply your own
#folds.
#mselection=NfoldSelection

#AUC measures the probability, that a randomly chosen positive example
#receives a higher score than a randomly chosen negative (which corresponds
#to the area under the ROC curve). 
measure=auc

#Linear kernel is always a reasonable first choice, advanced examples
#show how to learn non-linear models with other kernels.
kernel=LinearKernel

[Parameters]
#search regularization parameter from grid 2^-10...2^10
reggrid=-10_10

#bias is mostly useful for linear models with low-dimensional data
bias=1

[Input]
#features of the training examples
train_features=./examples/data/class_train.features
#labels of the training examples
train_labels=./examples/data/class_train.labels
#features of the test examples
prediction_features=./examples/data/class_test.features
#true labels of the test examples
test_labels=./examples/data/class_test.labels

[Output]
#the learned model is written here
model=./examples/models/classAUC.model
predicted_labels=./examples/predictions/classAUC.predictions

Python code (classAUC_all)

from rlscore import core
from rlscore import reader
from rlscore import writer
from rlscore.measure import auc

kwargs = {}
kwargs['train_labels'] = reader.read_dense('./examples/data/class_train.labels')
prediction_features = reader.read_sparse('./examples/data/class_test.features')
kwargs['train_features'] = reader.read_sparse('./examples/data/class_train.features')
test_labels = reader.read_dense('./examples/data/class_test.labels')
kwargs['reggrid'] = '-10_10'
kwargs['bias'] = '1'
kwargs['kernel'] = 'LinearKernel'
kwargs['learner'] = 'AllPairsRankRLS'
kwargs['measure'] = auc
kwargs['mselection'] = 'LPOSelection'
trainresults = core.trainModel(**kwargs)
model = trainresults['model']
print 'Making predictions on test data'
predicted_labels = model.predict(prediction_features)
performance = auc(test_labels, predicted_labels)
print 'Performance: %f %s' % (performance, auc.__name__)
writer.write_pickle('./examples/models/classAUC.model', model)
writer.write_dense('./examples/predictions/classAUC.predictions', predicted_labels)

Config file (rankqids_all)

#Ranking with query ids.
#
#In ranking the aim is to learn a function, whose predictions result in an
#accurate ranking when ordering new examples according to the predicted
#values. That is, more relevant examples should receive higher predicted
#scores than less relevant.
#
#Using qids means that instead of a total order over all examples, each
#query has it's own order, and examples from different queries should
#not be compared. For example in information retrieval, each query
#might consist of the ordering of a set of documents according to
#a query posed by a user.
#
#This example combines training, prediction and performance evaluation
#together


[Modules]

#LabelRankRLS is meant for ranking problems with qids
learner=LabelRankRLS

#For LabelRankRLS NfoldSelection performs leave-query-out cross-validation
#when choosing regularization parameter value
mselection=NfoldSelection

#Disagreement error measures the average number of pairwise mis-orderings per
#query
measure=disagreement

#Alternative: squared magnitude preserving ranking error
#measure=SqMPRankMeasure

#Linear kernel is always a reasonable first choice, advanced examples
#show how to learn non-linear models with other kernels.
kernel=LinearKernel

[Parameters]
#search regularization parameter from grid 2^-10...2^10
reggrid=-10_10

#bias is mostly useful for linear models with low-dimensional data
bias=1

[Input]
#features of the training examples
train_features=./examples/data/rank_train.features
#labels of the training examples
train_labels=./examples/data/rank_train.labels
#qids of the training examples
train_qids=./examples/data/rank_train.qids

#features of the test examples
prediction_features=./examples/data/rank_test.features
#true labels of the test examples
test_labels=./examples/data/rank_test.labels
#qids for the test examples
test_qids=./examples/data/rank_test.qids


[Output]
#the learned model is written here
model=./examples/models/rankqids.model
#the predicted labels are written here
predicted_labels=./examples/predictions/rankqids.predictions

Python code (rankqids_all)

from rlscore import core
from rlscore import reader
from rlscore import writer
from rlscore.measure import disagreement
from numpy import mean

kwargs = {}
kwargs['train_labels'] = reader.read_dense('./examples/data/rank_train.labels')
test_labels = reader.read_dense('./examples/data/rank_test.labels')
kwargs['train_qids'] = reader.read_qids('./examples/data/rank_train.qids')
prediction_features = reader.read_sparse('./examples/data/rank_test.features')
kwargs['train_features'] = reader.read_sparse('./examples/data/rank_train.features')
test_qids = reader.read_qids('./examples/data/rank_test.qids')
kwargs['reggrid'] = '-10_10'
kwargs['bias'] = '1'
kwargs['kernel'] = 'LinearKernel'
kwargs['learner'] = 'LabelRankRLS'
kwargs['measure'] = disagreement
kwargs['mselection'] = 'NfoldSelection'
trainresults = core.trainModel(**kwargs)
model = trainresults['model']
print 'Making predictions on test data'
predicted_labels = model.predict(prediction_features)
print 'calculating performance as averages over queries'
performances = []
for query in test_qids:
    performances.append(disagreement(test_labels[query], predicted_labels[query]))
performance = mean(performances)
print 'Performance: %f %s' % (performance, disagreement.__name__)
writer.write_pickle('./examples/models/rankqids.model', model)
writer.write_dense('./examples/predictions/rankqids.predictions', predicted_labels)

Config file (reg_all)

#Regression
#
#Regressor is trained by optimizing regularized least-squares loss
#
#this examples trains chooses regularization parameter, trains the method,
#makes test predictions and calculates test performance all at once

[Modules]

#RLS regressor is used as the training algorithm
learner=RLS

#Leave-one-out cross-validation can be used for parameter selection
mselection=LOOSelection

#Alternatively, 10-fold cross-validation with randomized fold partition
#could be used. It is also possible to supply your own folds.
#mselection=NfoldSelection

#Mean squared error
measure=sqerror

#Linear kernel is always a reasonable first choice, advanced examples
#show how to learn non-linear models with other kernels.
kernel=LinearKernel

[Parameters]
#search regularization parameter from grid 2^-10...2^10
reggrid=-10_10

#bias is mostly useful for linear models with low-dimensional data
bias=2

[Input]
#features of the training examples
train_features=./examples/data/reg_train.features
#labels of the training examples
train_labels=./examples/data/reg_train.labels
#features of the test examples
prediction_features=./examples/data/reg_test.features
#true labels of the test examples
test_labels=./examples/data/reg_test.labels

[Output]
#the learned model is written here
model=./examples/models/reg.model
#the predicted labels are written here
predicted_labels=./examples/predictions/reg.predictions

Python code (reg_all)

from rlscore import core
from rlscore import reader
from rlscore import writer
from rlscore.measure import sqerror

kwargs = {}
kwargs['train_labels'] = reader.read_dense('./examples/data/reg_train.labels')
prediction_features = reader.read_sparse('./examples/data/reg_test.features')
kwargs['train_features'] = reader.read_sparse('./examples/data/reg_train.features')
test_labels = reader.read_dense('./examples/data/reg_test.labels')
kwargs['reggrid'] = '-10_10'
kwargs['bias'] = '2'
kwargs['kernel'] = 'LinearKernel'
kwargs['learner'] = 'RLS'
kwargs['measure'] = sqerror
kwargs['mselection'] = 'LOOSelection'
trainresults = core.trainModel(**kwargs)
model = trainresults['model']
print 'Making predictions on test data'
predicted_labels = model.predict(prediction_features)
performance = sqerror(test_labels, predicted_labels)
print 'Performance: %f %s' % (performance, sqerror.__name__)
writer.write_pickle('./examples/models/reg.model', model)
writer.write_dense('./examples/predictions/reg.predictions', predicted_labels)

Config file (clustering)

#Performs maximum-margin clustering on the data set
#
#Details of the method can be found in
#'Fast Evolutionary Maximum Margin Clustering'
#

[Modules]
#The only clustering method currently supported
learner=MMC

[Parameters]

#number_of_clusters controls the number of clusters
number_of_clusters=2

#Currently model selection is not supported, so we fix this
#without search.
regparam=1

#bias is mostly useful for linear models with low-dimensional data
bias=1

[Input]
#features of the training examples
train_features=./examples/data/class_train.features


[Output]
#the predicted cluster memberships are written here
predicted_clusters_for_training_data=./examples/predictions/clusters.txt

Python code (clustering)

from rlscore import core
from rlscore import reader
from rlscore import writer

kwargs = {}
kwargs['train_features'] = reader.read_sparse('./examples/data/class_train.features')
kwargs['regparam'] = '1'
kwargs['bias'] = '1'
kwargs['number_of_clusters'] = '2'
kwargs['learner'] = 'MMC'
mselector = None
trainresults = core.trainModel(**kwargs)
model = trainresults['model']
writer.write_ints('./examples/predictions/clusters.txt', trainresults['predicted_clusters_for_training_data'])

Config file (fselection)

#Performs incremental forward selection, where k features
#which lead to good leave-one-out performance are chosen.
#
#Further, the method trains a sparse linear prediction model on the chosen
#features.
#
#This example is about binary classification, but by changing
#the performance measure the method is also suitable for
#regression or multiclass problems.
#
#Details of the method can be found in the forthcoming article
#'Linear Time Feature Selection for regularized least-squares'
#
#The quality of the learner model is tested on independent test data
#

[Modules]
#The only feature selecting learner currently supported
learner=GreedyRLS
#Since we are doing feature selection for a classification task,
#classification accuracy is a reasonable performance measure
measure=accuracy

[Parameters]

#subsetsize controls the number of selected features
subsetsize=3

#Currently cross-validated search for regularization parameter
#choosing is not supported for feature selection, so we fix this
#without search.
regparam=1

#bias is mostly useful for linear models with low-dimensional data
bias=1



[Input]
#features of the training examples
train_features=./examples/data/class_train.features
#labels of the training examples
train_labels=./examples/data/class_train.labels

#Calculating test performance is of course not necessary,
#but it gives some idea about the quality of selected feature
#set

#features of the test examples
prediction_features=./examples/data/class_test.features
#true labels of the test examples
test_labels=./examples/data/class_test.labels


[Output]
#the learned model that has non-zero coefficients only for
#the selected features is written here
model=./examples/models/sparse.model

#the indices of selected features are written here
selected_features=./examples/predictions/selected.findices

#The LOO performances made by GreedyRLS in each step of
#the greedy forward selection process are written here
GreedyRLS_LOO_performances=./examples/predictions/GreedyRLS_LOO.performance

Python code (fselection)

from rlscore import core
from rlscore import reader
from rlscore import writer
from rlscore.measure import accuracy

kwargs = {}
kwargs['train_labels'] = reader.read_dense('./examples/data/class_train.labels')
prediction_features = reader.read_sparse('./examples/data/class_test.features')
kwargs['train_features'] = reader.read_sparse('./examples/data/class_train.features')
test_labels = reader.read_dense('./examples/data/class_test.labels')
kwargs['regparam'] = '1'
kwargs['subsetsize'] = '3'
kwargs['bias'] = '1'
kwargs['learner'] = 'GreedyRLS'
kwargs['measure'] = accuracy
mselector = None
trainresults = core.trainModel(**kwargs)
model = trainresults['model']
print 'Making predictions on test data'
predicted_labels = model.predict(prediction_features)
performance = accuracy(test_labels, predicted_labels)
print 'Performance: %f %s' % (performance, accuracy.__name__)
writer.write_pickle('./examples/models/sparse.model', model)
writer.write_dense('./examples/predictions/GreedyRLS_LOO.performance', trainresults['GreedyRLS_LOO_performances'])
writer.write_ints('./examples/predictions/selected.findices', trainresults['selected_features'])

Config file (gaussian_kernel)

#Binary classification (accuracy maximizing)
#
#In this example we utilize the gaussian kernel
#
#this examples chooses regularization parameter, trains the method,
#makes test predictions and calculates test performance all at once

[Modules]

#RLS regressor is used as the training algorithm
learner=RLS

#Leave-one-out cross-validation can be used for parameter selection
mselection=LOOSelection

#Alternatively, 10-fold cross-validation with randomized fold partition
#could be used. It is also possible to supply your own folds.
#mselection=NfoldSelection

#Accuracy measures the fraction of correct classifications made
measure=accuracy

kernel=GaussianKernel

[Parameters]
#search regularization parameter from grid 2^-10...2^10
reggrid=-10_10

#width parameter of the gaussian kernel
gamma=0.01

[Input]
#features of the training examples
train_features=./examples/data/class_train.features
#labels of the training examples
train_labels=./examples/data/class_train.labels
#features of the test examples
prediction_features=./examples/data/class_test.features
#true labels of the test examples
test_labels=./examples/data/class_test.labels

[Output]
#the learned model is written here
model=./examples/models/classacc.model
#the predicted labels are written here
predicted_labels=./examples/predictions/classacc.predictions

Python code (gaussian_kernel)

from rlscore import core
from rlscore import reader
from rlscore import writer
from rlscore.measure import accuracy

kwargs = {}
kwargs['train_labels'] = reader.read_dense('./examples/data/class_train.labels')
prediction_features = reader.read_sparse('./examples/data/class_test.features')
kwargs['train_features'] = reader.read_sparse('./examples/data/class_train.features')
test_labels = reader.read_dense('./examples/data/class_test.labels')
kwargs['reggrid'] = '-10_10'
kwargs['gamma'] = '0.01'
kwargs['kernel'] = 'GaussianKernel'
kwargs['learner'] = 'RLS'
kwargs['measure'] = accuracy
kwargs['mselection'] = 'LOOSelection'
trainresults = core.trainModel(**kwargs)
model = trainresults['model']
print 'Making predictions on test data'
predicted_labels = model.predict(prediction_features)
performance = accuracy(test_labels, predicted_labels)
print 'Performance: %f %s' % (performance, accuracy.__name__)
writer.write_pickle('./examples/models/classacc.model', model)
writer.write_dense('./examples/predictions/classacc.predictions', predicted_labels)

Config file (polynomial_kernel)

#Binary classification (accuracy maximizing)
#
#In this example we utilize the polynomial kernel
#
#this example chooses regularization parameter, trains the method,
#makes test predictions and calculates test performance all at once

[Modules]

#RLS regressor is used as the training algorithm
learner=RLS

#Leave-one-out cross-validation can be used for parameter selection
mselection=LOOSelection

#Alternatively, 10-fold cross-validation with randomized fold partition
#could be used. It is also possible to supply your own folds.
#mselection=NfoldSelection

#Accuracy measures the fraction of correct classifications made
measure=accuracy

kernel=PolynomialKernel

[Parameters]
#The polynomial kernel is defined as
#k(xi,xj) = (gamma * <xi,xj> + coef0)**degree
#
#We use here a simple homogenous polynomial kernel of
#degree 3
gamma=1
coef0=0
degree=3

#search regularization parameter from grid 2^-10...2^10
reggrid=-10_10

[Input]
#features of the training examples
train_features=./examples/data/class_train.features
#labels of the training examples
train_labels=./examples/data/class_train.labels
#features of the test examples
prediction_features=./examples/data/class_test.features
#true labels of the test examples
test_labels=./examples/data/class_test.labels

[Output]
#the learned model is written here
model=./examples/models/classacc.model
#the predicted labels are written here
predicted_labels=./examples/predictions/classacc.predictions

Python code (polynomial_kernel)

from rlscore import core
from rlscore import reader
from rlscore import writer
from rlscore.measure import accuracy

kwargs = {}
kwargs['train_labels'] = reader.read_dense('./examples/data/class_train.labels')
prediction_features = reader.read_sparse('./examples/data/class_test.features')
kwargs['train_features'] = reader.read_sparse('./examples/data/class_train.features')
test_labels = reader.read_dense('./examples/data/class_test.labels')
kwargs['reggrid'] = '-10_10'
kwargs['coef0'] = '0'
kwargs['degree'] = '3'
kwargs['gamma'] = '1'
kwargs['kernel'] = 'PolynomialKernel'
kwargs['learner'] = 'RLS'
kwargs['measure'] = accuracy
kwargs['mselection'] = 'LOOSelection'
trainresults = core.trainModel(**kwargs)
model = trainresults['model']
print 'Making predictions on test data'
predicted_labels = model.predict(prediction_features)
performance = accuracy(test_labels, predicted_labels)
print 'Performance: %f %s' % (performance, accuracy.__name__)
writer.write_pickle('./examples/models/classacc.model', model)
writer.write_dense('./examples/predictions/classacc.predictions', predicted_labels)

Config file (cgrls_test)

#This config file runs Conjugate Gradient version of RLS
#The CGRLS is useful for very large and high-dimensional but sparse data sets,
#and can be used only with the linear kernel.
#
#
#Binary classification (accuracy maximizing)
#
#Accuracy maximizing binary classification is done by training a RLS regressor
#with labels +1 for positive and -1 for negative class. The algorithm
#is equivalent to the so-called 'Least-squares support vector machine',
#and is known to provide similar performance as SVMs
#

[Modules]

#RLS regressor is used as the training algorithm
learner=CGRLS

#Accuracy measures the fraction of correct classifications made
measure=accuracy

[Parameters]
regparam=1

#bias is mostly useful for linear models with low-dimensional data
bias=1

[Input]
#features of the training examples
train_features=./examples/data/class_train.features
#labels of the training examples
train_labels=./examples/data/class_train.labels
#features of the test examples
prediction_features=./examples/data/class_test.features
#true labels of the test examples
test_labels=./examples/data/class_test.labels

[Output]
#the learned model is written here
model=./examples/models/classacc.model
#the predicted labels are written here
predicted_labels=./examples/predictions/classacc.predictions

Python code (cgrls_test)

from rlscore import core
from rlscore import reader
from rlscore import writer
from rlscore.measure import accuracy

kwargs = {}
kwargs['train_labels'] = reader.read_dense('./examples/data/class_train.labels')
prediction_features = reader.read_sparse('./examples/data/class_test.features')
kwargs['train_features'] = reader.read_sparse('./examples/data/class_train.features')
test_labels = reader.read_dense('./examples/data/class_test.labels')
kwargs['regparam'] = '1'
kwargs['bias'] = '2'
kwargs['kernel'] = 'LinearKernel'
kwargs['learner'] = 'CGRLS'
kwargs['measure'] = accuracy
mselector = None
trainresults = core.trainModel(**kwargs)
model = trainresults['model']
print 'Making predictions on test data'
predicted_labels = model.predict(prediction_features)
performance = accuracy(test_labels, predicted_labels)
print 'Performance: %f %s' % (performance, accuracy.__name__)
writer.write_pickle('./examples/models/classacc.model', model)
writer.write_dense('./examples/predictions/classacc.predictions', predicted_labels)

Config file (cgrank_test)

#This config file runs Conjugate Gradient version of RankRLS
#The CGRankRLS is useful for very large and high-dimensional but sparse data sets,
#and can be used only with the linear kernel.
#

[Modules]

#RankRLS is used as the training algorithm
learner=CGRankRLS

#Area under ROC Curve
measure=auc

[Parameters]
regparam=1


[Input]
#features of the training examples
train_features=./examples/data/class_train.features
#labels of the training examples
train_labels=./examples/data/class_train.labels
#features of the test examples
prediction_features=./examples/data/class_test.features
#true labels of the test examples
test_labels=./examples/data/class_test.labels

[Output]
#the learned model is written here
model=./examples/models/classacc.model
#the predicted labels are written here
predicted_labels=./examples/predictions/classacc.predictions

Python code (cgrank_test)

from rlscore import core
from rlscore import reader
from rlscore import writer
from rlscore.measure import auc

kwargs = {}
kwargs['train_labels'] = reader.read_dense('./examples/data/class_train.labels')
prediction_features = reader.read_sparse('./examples/data/class_test.features')
kwargs['train_features'] = reader.read_sparse('./examples/data/class_train.features')
test_labels = reader.read_dense('./examples/data/class_test.labels')
kwargs['regparam'] = '1'
kwargs['kernel'] = 'LinearKernel'
kwargs['learner'] = 'CGRankRLS'
kwargs['measure'] = auc
mselector = None
trainresults = core.trainModel(**kwargs)
model = trainresults['model']
print 'Making predictions on test data'
predicted_labels = model.predict(prediction_features)
performance = auc(test_labels, predicted_labels)
print 'Performance: %f %s' % (performance, auc.__name__)
writer.write_pickle('./examples/models/classacc.model', model)
writer.write_dense('./examples/predictions/classacc.predictions', predicted_labels)

Config file (cgrank_qids)

#Ranking with query ids.
#Conjugate gradient RankRLS training

[Modules]

#CGRankRLS can be used for ranking problems with qids
learner=CGRankRLS

#We use the validation set for parameter selection
mselection=ValidationSetSelection

#Disagreement error measures the average number of pairwise mis-orderings per
#query
measure=disagreement

[Parameters]
#search regularization parameter
reggrid=0.001 0.1 10 1000

[Input]
#features of the training examples
train_features=./examples/data/rank_train.features
#labels of the training examples
train_labels=./examples/data/rank_train.labels
#qids of the training examples
train_qids=./examples/data/rank_train.qids

#validation set files
validation_features=./examples/data/rank_test.features
validation_labels=./examples/data/rank_test.labels
validation_qids=./examples/data/rank_test.qids

[Output]
#the learned model is written here
model=./examples/models/rankqids.model

Python code (cgrank_qids)

from rlscore import core
from rlscore import reader
from rlscore import writer
from rlscore.measure import disagreement

kwargs = {}
kwargs['train_labels'] = reader.read_dense('./examples/data/rank_train.labels')
kwargs['devel_labels'] = reader.read_dense('./examples/data/rank_test.labels')
kwargs['train_qids'] = reader.read_qids('./examples/data/rank_train.qids')
kwargs['devel_qids'] = reader.read_qids('./examples/data/rank_test.qids')
kwargs['train_features'] = reader.read_sparse('./examples/data/rank_train.features')
kwargs['devel_features'] = reader.read_sparse('./examples/data/rank_test.features')
kwargs['reggrid'] = '0.001 0.1 10 1000'
kwargs['kernel'] = 'LinearKernel'
kwargs['learner'] = 'CGRankRLS'
kwargs['measure'] = disagreement
kwargs['mselection'] = 'DevelSetSelection'
trainresults = core.trainModel(**kwargs)
model = trainresults['model']
writer.write_pickle('./examples/models/rankqids.model', model)

Config file (cgrank_test_with_preferences)

#This config file runs Conjugate Gradient version of RankRLS with a list of pairwise preferences between data points rather than labeled data points.
#The CGRankRLS is useful for very large and high-dimensional but sparse data sets,
#and can be used only with the linear kernel.
#

[Modules]

#RankRLS is used as the training algorithm
learner=CGRankRLS

#Area under ROC Curve
measure=auc

#Linear kernel is always a reasonable first choice, advanced examples
#show how to learn non-linear models with other kernels.
kernel=LinearKernel

[Parameters]
regparam=1


[Input]
#features of the training examples
train_features=./examples/data/class_train.features
#pairwise preferences between data points
train_preferences=./examples/data/rank_train.preferences
#features of the test examples
prediction_features=./examples/data/class_test.features
#true labels of the test examples
test_labels=./examples/data/class_test.labels

[Output]
#the learned model is written here
model=./examples/models/classacc.model
#the predicted labels are written here
predicted_labels=./examples/predictions/classacc.predictions

Python code (cgrank_test_with_preferences)

from rlscore import core
from rlscore import reader
from rlscore import writer
from rlscore.measure import auc

kwargs = {}
kwargs['train_preferences'] = reader.read_preferences('./examples/data/rank_train.preferences')
prediction_features = reader.read_sparse('./examples/data/class_test.features')
kwargs['train_features'] = reader.read_sparse('./examples/data/class_train.features')
test_labels = reader.read_dense('./examples/data/class_test.labels')
kwargs['regparam'] = '1'
kwargs['kernel'] = 'LinearKernel'
kwargs['learner'] = 'CGRankRLS'
kwargs['measure'] = auc
mselector = None
trainresults = core.trainModel(**kwargs)
model = trainresults['model']
print 'Making predictions on test data'
predicted_labels = model.predict(prediction_features)
performance = auc(test_labels, predicted_labels)
print 'Performance: %f %s' % (performance, auc.__name__)
writer.write_pickle('./examples/models/classacc.model', model)
writer.write_dense('./examples/predictions/classacc.predictions', predicted_labels)

Config file (reduced_set)

#Binary classification (accuracy maximizing)
#
#In this example we utilize the gaussian kernel, and use the
#reduced set approximation with 10 basis vectors given in a file
#
#this examples chooses regularization parameter, trains the method,
#makes test predictions and calculates test performance all at once

[Modules]

#RLS regressor is used as the training algorithm
learner=RLS

regparam=1.0

#Accuracy measures the fraction of correct classifications made
measure=accuracy

kernel=GaussianKernel

[Parameters]
#search regularization parameter from grid 2^-10...2^10
reggrid=-10_10

#width parameter of the gaussian kernel
gamma=0.01

[Input]
#features of the training examples
train_features=./examples/data/class_train.features
#labels of the training examples
train_labels=./examples/data/class_train.labels
#features of the test examples
prediction_features=./examples/data/class_test.features
#true labels of the test examples
test_labels=./examples/data/class_test.labels
#the ten basis vectors, the file contains indices of ten training examples
basis_vectors=./examples/data/bvectors.indices

[Output]
#the learned model is written here
model=./examples/models/classacc.model
#the predicted labels are written here
predicted_labels=./examples/predictions/classacc.predictions

Python code (reduced_set)

from rlscore import core
from rlscore import reader
from rlscore import writer
from rlscore.measure import accuracy

kwargs = {}
kwargs['train_labels'] = reader.read_dense('./examples/data/class_train.labels')
prediction_features = reader.read_sparse('./examples/data/class_test.features')
kwargs['basis_vectors'] = reader.read_bvectors('./examples/data/bvectors.indices')
kwargs['train_features'] = reader.read_sparse('./examples/data/class_train.features')
test_labels = reader.read_dense('./examples/data/class_test.labels')
kwargs['reggrid'] = '-10_10'
kwargs['gamma'] = '0.01'
kwargs['kernel'] = 'GaussianKernel'
kwargs['learner'] = 'RLS'
kwargs['measure'] = accuracy
mselector = None
trainresults = core.trainModel(**kwargs)
model = trainresults['model']
print 'Making predictions on test data'
predicted_labels = model.predict(prediction_features)
performance = accuracy(test_labels, predicted_labels)
print 'Performance: %f %s' % (performance, accuracy.__name__)
writer.write_pickle('./examples/models/classacc.model', model)
writer.write_dense('./examples/predictions/classacc.predictions', predicted_labels)

Python code (Kronecker RLS)

from rlscore.learner import KronRLS
from rlscore import reader
from rlscore import writer
from rlscore.measure import sqerror

kwargs = {}
kwargs['train_labels'] = reader.read_dense('./examples/data/Kron_train.labels')
kwargs['kmatrix1'] = reader.read_dense('./examples/data/Kron_train_1.kernelm')
kwargs['kmatrix2'] = reader.read_dense('./examples/data/Kron_train_2.kernelm')
K_test1 = reader.read_dense('./examples/data/Kron_test_1.kernelm')
K_test2 = reader.read_dense('./examples/data/Kron_test_2.kernelm')
test_labels = reader.read_dense('./examples/data/Kron_test.labels')
kwargs['regparam'] = 0.001
learner = KronRLS.createLearner(**kwargs)
learner.train()
kronmodel = learner.getModel()
kronpred = kronmodel.predictWithKernelMatrices(K_test1, K_test2)
print sqerror(test_labels, kronpred)