Classifier evaluation using the OOP interface (report generation)

We can also use the sklearn_evaluation.ClassifierEvaluator class to pack the results from our estimator. This way we can generate plots and tables without having to pass the parameters over and over again. If we are evaluating more than one model at a time this also gives us a way to keep it organized. Furthermore, the ClassifierEvaluator class offers a way to create HTML reports from our model results.

First, let’s load some data and split for training and testing.

In [1]: iris = datasets.load_iris()

In [2]: X = iris.data

In [3]: y = iris.target

In [4]: X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3)

Let’s now train a classifier and predict on the test set.

In [5]: est = RandomForestClassifier(n_estimators=5)

In [6]: est.fit(X_train, y_train)
Out[6]: RandomForestClassifier(n_estimators=5)

In [7]: y_pred = est.predict(X_test)

In [8]: y_score = est.predict_proba(X_test)

In [9]: feature_list = range(4)

In [10]: target_names = ['setosa', 'versicolor', 'virginica']

Now that we have everything we need, let’s pack our results using ClassifierEvaluator, every parameter is optional.

In [11]: ce = ClassifierEvaluator(est, y_test, y_pred, y_score,
   ....:                  feature_list, target_names,
   ....:                  estimator_name='RF')
   ....: 

We can use most of the functions in plot and table directly from the ClassifierEvaluator object, let’s see how to plot a confusion matrix.

In [12]: ce.confusion_matrix()
Out[12]: <AxesSubplot:title={'center':'Confusion matrix'}, xlabel='Predicted label', ylabel='True label'>
../_images/cm_2.png

We can also generate HTML reports from our models by using the make_report function. The first parameter is a HTML or Markdown template with jinja2 format. If a pathlib.Path object is passed, the content of the file is read. Within the template, the evaluator is passed as “e”, so you can use things like {{e.confusion_matrix()}} or any other attribute/method. If None, a default template is used

In [13]: report = ce.make_report()

The function returns a Report object, which will automatically render in a Jupyter notebook, report.save(‘/path/to/report.html’) will save the report and report.rendered will return a string with the HTML report.