ClassifierToolbox : A Matlab toolbox for classifier.

---

Authors: Hiroyuki Kasai

Collaborator: Kohei Yoshikawa

Last page update: July 10, 2017

Latest library version: 1.0.3 (see Release notes for more info)

Introduction

This package provides various tools for classification, e.g., image classification, face recogntion, and related applicaitons.

List of algorithms

• Basis
  • PCA (Principal component analysis)
    • M. Turk and A. Pentland, "Eigenfaces for recognition," J. Cognitive Neurosci," vol.3, no.1, pp.71-86, 1991.
    • See also wikipedia.
  • ICA (Independent component analysis)
    • See wikipedia.
  • LDA (Linear discriminant analysis)
    • P. N. Belhumeur, J. P. Hespanha, and D. I. Kriegman, "Eigenfaces vs. Fisherfaces: recognition using class specific linear projection," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.19, no.7, pp.711-720, 1997.
    • See also wikipedia.
  • SVM (Support vector machine)
    • See wikipedia
    • Use Matlab built-in library (svmfitcsvm and predict).
• LRC variant
  • LRC (Linear regression classification)
    • I. Nassem, M. Bennamoun, "Linear regression for face recognition," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.32, no.11, 2010.
  • LDRC (Linear discriminant regression classificatoin)
    • S.-M. Huang and J.-F. Yang, "Linear discriminant regression classification for face recognition," IEEE Signal Processing Letters, vol.20, no.1, pp.91-94, 2013.
  • LCDRC (Linear collaborative discriminant regression classificatoin)
    • X. Qu, S. Kim, R. Cui and H. J. Kim, "Linear collaborative discriminant regression classification for face recognition," J. Visual Communication Image Represetation, vol.31, pp. 312-319, 2015.
• CRC (Collaborative representation based classification)
  • L. Zhanga, M. Yanga, and X. Feng, "Sparse representation or collaborative representation: which helps face recognition?," Proceedings of the 2011 International Conference on Computer Vision (ICCV'11), pp. 471-478, 2011.
• LSR variant
  • LSR (Least squares regression)
  • DERLR (Discriminative elastic-net regularized linear regression)
    • Z. Zhang, Z. Lai, Y. Xu, L. Shao and G. S. Xie, "[Discriminative elastic-net regularized linear regression(http://ieeexplore.ieee.org/document/7814255/)," IEEE Transactions on Image Processing, vol.26, no.3, pp.1466-1481, 2017.
• Low-rank matrix factorization based
  • NMF (Non-negative matrix factorization)
    • Please refer NMFLibrary.
  • Robust PCA classifier
    • E. Candes, X. Li, Y. Ma, and J. Wright, "Robust Principal Component Analysis?," Journal of the ACM, vol.58, no.3, 2011.
    • Clasifier uses SRC.
    • Use SparseGDLibrary.
• RCM based
  • RCM+kNN (Region covariance matrix algorithm)
    • O. Tuzel, F. Porikli, and P. Meer "Region covariance: a fast descriptor for detection and classification," European Conference on Computer Vision (ECCV2006), pp.589-600, 2006.
  • GRCM+kNN (Gabor-wavelet-based region covariance matrix algorithm)
    • Y. Pang, Y. Yuan, and X. Li, "Gabor-based Region covariance matrices for face recognition," IEEE Transactions on Circuits and Systems for Video Technology vol.18, no.7, 2008.
• SRC variant
  • SRC (Sparse representation classifcation)
    • J. Wright, A. Yang, A. Ganesh, S. Sastry, and Y. Ma, "Robust face recognition via sparse representation," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.31, no.2, pp.210-227, 2009.
  • ESRC (Extended Sparse representation classifcation)
    • W. Deng, J. Hu, and J. Guo, "Extended SRC: Undersampled face recognition via intraclass variant dictionary," IEEE Transation on Pattern Analysis Machine Intelligence, vol.34, no.9, pp.1864-1870, 2012.
  • SDR-SLR (Sparse- and dense-hybrid representation and supervised low-rank)
    • X. Jiang, and J. Lai,
    • "Sparse and dense hybrid representation via dictionary decomposition for face recognition,"
    • IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.37, no.5, pp.1067-1079, 2015.
• Dictionary learning based
  • K-SVD
    • M. Aharon, M. Elad, and A.M. Bruckstein, "The K-SVD: An algorithm for designing of overcomplete dictionaries for sparse representation", IEEE Trans. On Signal Processing, Vol.54, no.11, pp.4311-4322, November 2006.
  • LC-KSVD (Label Consistent K-SVD)
    • Z. Jiang, Z. Lin, L. S. Davis, "Learning a discriminative dictionary for sparse coding via label consistent K-SVD," IEEE Conference on Computer Vision and Pattern Recognition (CVPR2011), 2011.
    • Z. Jiang, Z. Lin, L. S. Davis, "Label consistent K-SVD: learning A discriminative dictionary for recognition," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.35, no.11, pp.2651-2664, 2013.
• Geometry-aware
  • R-KSRC (Riemannian kernelized sparse representation classification)
    • M. Harandi, R. Hartley, B. Lovell and C. Sanderson, "Sparse coding on symmetric positive definite manifolds using bregman divergences," IEEE Transactions on Neural Networks and Learning Systems (TNNLS), 2016.
    • M. Harandi, C. Sanderson, R. Hartley and B. Lovell, "Sparse coding and dictionary learning for symmetric positive definite matrices: a kernel approach," European Conference on Computer Vision (ECCV), 2012.

Folders and files

./                              - Top directory.
./README.md                     - This readme file.
./run_me_first.m                - The scipt that you need to run first.
./demo.m                        - Demonstration script to check and understand this package easily. 
|algorithm/                     - Algorithms for classifcations.
|auxiliary/                     - Some auxiliary tools for this project.
|demo_examples/                 - Some demonstration files.
|lib/                           - 3rd party tools.
|dataset/                       - Folder where datasets are stored.

First to do: configure path

Run run_me_first for path configurations.

%% First run the setup script
run_me_first; 

Second to do: download datasets and external libraries

Run download for downloading datasets and external libraries.

%% Run the downloading script
download; 

• If your computer is behind a proxy server, please configure your Matlab setting. See this.

Usage example: ORL face dateset demo: 3 steps!

Now, just execute demo for demonstration of this package.

%% Execute the demonstration script
demo; 

The "demo.m" file contains below.

%% load data
load('./dataset/ORL_Face_img_cov.mat');

%% perform RCM k-NN classifier with 
% GRCM2 with eigenvalue-based distance
grcm_accuracy = rcm_knn_classifier(TrainSet, TestSet,'GRCM', '2', 'EV', 5);
% RCM4 with eigenvalue-based distance
rcm_accuracy = rcm_knn_classifier(TrainSet, TestSet, 'RCM', '4', 'EV', 5);

%% show recognition accuracy
fprintf('# GRCM2 Accuracy = %5.2f\n', grcm_accuracy);
fprintf('# RCM4 Accuracy = %5.2f\n', rcm_accuracy);

Let take a closer look at the code above bit by bit. The procedure has only 3 steps!

Step 1: Load data

First, we load datasets including train set and test set. This case uses a covariance dataset that is originally generated from ORL face dataset.

load('./dataset/ORL_Face_img_cov.mat');

Step 2: Perform solver

Now, you can perform optimization solvers, i.e., RCM-based kNN classifier, calling rcm_knn_classifier() function with appropriate paramters.

% GRCM2 with eigenvalue-based distance
grcm_accuracy = rcm_knn_classifier(TrainSet, TestSet, 'GRCM', '2', 'EV', 5);

% RCM4 with eigenvalue-based distance
rcm_accuracy = rcm_knn_classifier(TrainSet, TestSet, 'RCM', '4', 'EV', 5);

The first case performs the Gabor-wavelet-based region covariance matrix (CRCM) algorithm (type 4) with eigen-value based disctance followed by 5-NN classifier. The second cases peforms the standard region covariance matrix (RCM) algorithm (type 2) with the same setting as before. They return the final accuracy.

Step 3: Show recognition accuracy

Finally, the final recognition accuracis are shown.

fprintf('# GRCM2 Accuracy = %5.2f\n', grcm_accuracy);
fprintf('# RCM4 Accuracy = %5.2f\n', rcm_accuracy);

That's it!

License

• This toobox is free, non-commercial and open source.
• The code provided in this toobox should only be used for academic/research purposes.

Third party tools, libraries, and packages.

• Third party files are included.
  • OMPBox is used for OMP (orthogonal matching pursuit) algorithm.
  • KSVDBox is used for K-SVD algorithm.
  • SPAMS is used for various lasso problems.
  • LC-KSVD.
  • RSR.
  • Learning Discriminative Stein Kernel for SPD Matrices and Its Applications.
  • SDR-SLR.
  • DERLR.
  • JACOBI_EIGENVALUE is a MATLAB library which computes the eigenvalues and eigenvectors of a real symmetric matrix.
  • NMFLibrary is for NMF.
  • SparseGDLibrary is for Robust PCA classifier.
• Note that please see the corresponding license for each.

Problems or questions

If you have any problems or questions, please contact the author: Hiroyuki Kasai (email: kasai at is dot uec dot ac dot jp)

Release Notes

• Version 1.0.3 (July 11, 2017)
  • Add and modify SDR-SLR etc.
• Version 1.0.2 (July 07, 2017)
  • Add and modify RSR, SVM etc.
• Version 1.0.1 (July 06, 2017)
  • Add and modify many items.
• Version 1.0.0 (July 01, 2017)
  • Initial version.