Skip to content

Commit

Permalink
updated documents, added vgg-d model, about ready for the big release
Browse files Browse the repository at this point in the history
  • Loading branch information
@liuliu
liuliu committed Dec 23, 2014
1 parent 2794ed2 commit e06050e
Show file tree
Hide file tree
Showing 24 changed files with 690 additions and 1,677 deletions.
2 changes: 1 addition & 1 deletion bin/cnnclassify.c
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ static unsigned int get_current_time(void)
return tv.tv_sec * 1000 + tv.tv_usec / 1000;
}

#define BATCH_SIZE (16)
#define BATCH_SIZE (8)

int main(int argc, char** argv)
{
Expand Down
21 changes: 12 additions & 9 deletions bin/image-net.c
View file
Original file line number Diff line number Diff line change
Expand Up @@ -43,8 +43,8 @@ int main(int argc, char** argv)
char* base_dir = 0;
ccv_convnet_train_param_t train_params = {
.max_epoch = 100,
.mini_batch = 48,
.sgd_frequency = 1, // do sgd every 2 batches (96 * 2 = 192)
.mini_batch = 128,
.sgd_frequency = 1, // do sgd every sgd_frequency batches (mini_batch * device_count * sgd_frequency)
.iterations = 50000,
.device_count = 4,
.peer_access = 1,
Expand Down Expand Up @@ -129,21 +129,24 @@ int main(int argc, char** argv)
}
fclose(r1);
free(file);
ccv_convnet_t* convnet = ccv_convnet_new(1, ccv_size(257, 257), vgg_d_params, sizeof(vgg_d_params) / sizeof(ccv_convnet_layer_param_t));
#define model_params matt_params
int depth = sizeof(model_params) / sizeof(ccv_convnet_layer_param_t);
ccv_convnet_t* convnet = ccv_convnet_new(1, ccv_size(257, 257), model_params, depth);
ccv_convnet_verify(convnet, 1000);
ccv_convnet_layer_train_param_t layer_params[21];
ccv_convnet_layer_train_param_t layer_params[depth];
memset(layer_params, 0, sizeof(layer_params));
for (i = 0; i < 21; i++)
for (i = 0; i < depth; i++)
{
layer_params[i].w.decay = 0.0005;
layer_params[i].w.learn_rate = 0.00001;
layer_params[i].w.learn_rate = 0.01;
layer_params[i].w.momentum = 0.9;
layer_params[i].bias.decay = 0;
layer_params[i].bias.learn_rate = 0.00001;
layer_params[i].bias.learn_rate = 0.01;
layer_params[i].bias.momentum = 0.9;
}
layer_params[18].dor = 0.5;
layer_params[19].dor = 0.5;
// set the two full connect layers to last with dropout rate at 0.5
for (i = depth - 3; i < depth - 1; i++)
layer_params[i].dor = 0.5;
train_params.layer_params = layer_params;
ccv_set_cli_output_levels(ccv_cli_output_level_and_above(CCV_CLI_INFO));
ccv_convnet_supervised_train(convnet, categorizeds, tests, working_dir, train_params);
Expand Down
101 changes: 43 additions & 58 deletions doc/bbf.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,26 +4,25 @@ BBF: Brightness Binary Feature
Deprecated
----------

This feature is deprecated as it is no longer considered either fast or accurate
for the task its most interested in (in this case, face detection) in libccv 0.7
This feature is deprecated as it is no longer considered either fast or accurate for the task
its most interested in (in this case, face detection) in libccv 0.7

The implementation will be removed in the version after libccv 0.7 in favor of
SCD (for face detection).
The implementation will be removed in the version after libccv 0.7 in favor of SCD (for face
detection).

What's BBF?
-----------

The original paper refers to:
YEF∗ Real-Time Object Detection, Yotam Abramson and Bruno Steux
The original paper refers to: YEF∗ Real-Time Object Detection, Yotam Abramson and Bruno Steux

The improved version refers to:

High-Performance Rotation Invariant Multiview Face Detection, Chang Huang, Haizhou Ai, Yuan Li and Shihong Lao

How it works?
-------------

That's a long story, please read the paper. But at least I can show you how to
use the magic:
That's a long story, please read the paper. But at least I can show you how to use the magic:

./bbfdetect <Your Image contains Faces> ../samples/face | ./bbfdraw.rb <Your Image contains Faces> output.png

Expand All @@ -37,18 +36,17 @@ The tests are performed with MIT+CMU face detection dataset

Setup:

Download the tarball, copy out files in newtest/ test/ and test-low/ to a single
folder, let's say: all/. Since ccv doesn't support gif format, you need to do file
format conversion by your own. If you have ImageMagick, it is handy:
Download the tarball, copy out files in newtest/ test/ and test-low/ to a single folder, let's
say: all/. Since ccv doesn't support gif format, you need to do file format conversion by your
own. If you have ImageMagick, it is handy:

for i in *.gif; do convert $i `basename $i .gif`.png; done;

For the ground truth data, you can copy them out from
http://vasc.ri.cmu.edu/idb/images/face/frontal_images/list.html Only Test Set A,
B, C are needed.
http://vasc.ri.cmu.edu/idb/images/face/frontal_images/list.html Only Test Set A, B, C are needed.

bbfdetect needs a list of files, you can generate them by run the command in the
same directory of bbfdetect binary:
bbfdetect needs a list of files, you can generate them by run the command in the same directory
of bbfdetect binary:

find <the directory of converted files>/*.png > filelist.txt

Expand All @@ -60,28 +58,22 @@ run

On my computer, it reports:

real 0m9.304s
user 0m9.270s
sys 0m0.010s
real 0m9.304s user 0m9.270s sys 0m0.010s

How about OpenCV's face detector? I run OpenCV with default setting on the same
computer, and it reports:
How about OpenCV's face detector? I run OpenCV with default setting on the same computer,
and it reports:

real 0m27.977s
user 0m27.860s
sys 0m0.050s
real 0m27.977s user 0m27.860s sys 0m0.050s

You see the difference.

Accuracy-wise:

I wrote a little script called bbfvldr.rb that can check the output of bbfdetect
against ground truth, before run the script, you need to do some house-cleaning
work on the result.txt:
I wrote a little script called bbfvldr.rb that can check the output of bbfdetect against ground
truth, before run the script, you need to do some house-cleaning work on the result.txt:

Basically, the result.txt file will contain the full path to the file, for which,
we only need the filename, use your favorite editor to remove the directory
information, for me, it is:
Basically, the result.txt file will contain the full path to the file, for which, we only need
the filename, use your favorite editor to remove the directory information, for me, it is:

sed -i "s/\.\.\/test\/faces\///g" result.txt

Expand All @@ -93,54 +85,47 @@ My result for bbfdetect is:

82.97% (12)

The former one is detection rate (how many faces are detected), the later one is
the number of false alarms (how many non-face regions are detected as faces)
The former one is detection rate (how many faces are detected), the later one is the number of
false alarms (how many non-face regions are detected as faces)

The result for OpenCV default face detector is:

86.69% (15)

Well, we are a little behind, but you can train the detector yourself, just get
a better data source!
Well, we are a little behind, but you can train the detector yourself, just get a better
data source!

How to train my own detector?
-----------------------------

In this chapter, I will go over how I trained the face detector myself. To be
honest, I lost my face detector training data several years ago. Just like
everyone else, I have to download it somewhere. In the end, I settled with LFW
(http://vis-www.cs.umass.edu/lfw/). Technically, it is the dataset for face
recognition, so there are less variations. But that's the largest dataset I can
find to download. I downloaded the aligned data, cropped with random rotation,
In this chapter, I will go over how I trained the face detector myself. To be honest, I lost
my face detector training data several years ago. Just like everyone else, I have to download
it somewhere. In the end, I settled with LFW (http://vis-www.cs.umass.edu/lfw/). Technically,
it is the dataset for face recognition, so there are less variations. But that's the largest
dataset I can find to download. I downloaded the aligned data, cropped with random rotation,
translation and scale variations, got 13125 faces in 24x24 size.

The bbfcreate also requires negative images, just so happened, I have about 8000
natural scene images that contains no faces downloaded from Flickr. OK, now I
have all the data, what's next?
The bbfcreate also requires negative images, just so happened, I have about 8000 natural scene
images that contains no faces downloaded from Flickr. OK, now I have all the data, what's next?

First, you need to create a directory called data/ under the same directory of
bbfcreate. Then, you need to create two filelists of positive data and negative
images, for me, it is:
First, you need to create a directory called data/ under the same directory of bbfcreate. Then,
you need to create two filelists of positive data and negative images, for me, it is:

find ../data/faces/*.bmp > faces.dat
find ../data/negs/*.jpg > negs.dat
find ../data/faces/*.bmp > faces.dat find ../data/negs/*.jpg > negs.dat

That's all! Just find a computer powerful enough and run the following line for several
days:
That's all! Just find a computer powerful enough and run the following line for several days:

./bbfcreate --positive-list faces.dat --background-list negs.dat --negative-count 26250 --working-dir data

The --negative-count parameter denotes how many negative samples extracted for each round,
experimentally, it is something about twice of the number of your positive ones.

If you configure the makefile well, bbfcreate will use OpenMP to speed up, which will
eat up all the CPUs. My own training process ran about one week, it is a extremely
powerful desktop PC, you should expect weeks for the result on modest PC with so many
samples.
If you configure the makefile well, bbfcreate will use OpenMP to speed up, which will eat up
all the CPUs. My own training process ran about one week, it is a extremely powerful desktop PC,
you should expect weeks for the result on modest PC with so many samples.

You can stop bbfcreate at any time you want, the most recent result will be saved
in data/ directory, clean up the directory to restart.
You can stop bbfcreate at any time you want, the most recent result will be saved in data/
directory, clean up the directory to restart.

I probably will implement MPI support in near future so that you can run this with
many computers in parallel, but who nowadays have OpenMPI setup besides supercomputing
centers?
I probably will implement MPI support in near future so that you can run this with many computers
in parallel, but who nowadays have OpenMPI setup besides supercomputing centers?
49 changes: 24 additions & 25 deletions doc/cache.md
View file
Original file line number Diff line number Diff line change
@@ -1,48 +1,47 @@
Cache: We are Terrible Magicians
=================
================================

ccv uses an application-wide transparent cache to de-duplicate matrix computations.
In the following chapters, I will try to outline how that works, and expose you
to the inner-working of ccv's core functionalities.
ccv uses an application-wide transparent cache to de-duplicate matrix computations. In the
following chapters, I will try to outline how that works, and expose you to the inner-working
of ccv's core functionalities.

Initial Signature
-----------------

ccv_make_matrix_immutable computes the SHA-1 hash on matrix raw data, and will
use the first 64-bit as the signature for that matrix.
`ccv_make_matrix_immutable` computes the SHA-1 hash on matrix raw data, and will use the first
64-bit as the signature for that matrix.

Derived Signature
-----------------

Derived signature is computed from the specific operation that is going to perform.
For example, matrix A and matrix B used to generate matrix C through operation X.
C's signature is derived from A, B and X.
Derived signature is computed from the specific operation that is going to perform. For example,
matrix A and matrix B used to generate matrix C through operation X. C's signature is derived
from A, B and X.

A Radix-tree LRU Cache
----------------------

ccv uses a custom radix-tree implementation with generation information. It imposes
a hard limit on memory usage of 64 MiB, you can adjust this value if you like.
The custom radix-tree data structure is specifically designed to satisfy our 64-bit
signature design. If compile with jemalloc, it can be both fast and memory-efficient.
ccv uses a custom radix-tree implementation with generation information. It imposes a hard
limit on memory usage of 64 MiB, you can adjust this value if you like. The custom radix-tree
data structure is specifically designed to satisfy our 64-bit signature design. If compile with
jemalloc, it can be both fast and memory-efficient.

Garbage Collection
------------------

The matrix signature is important. For every matrix that is freed with ccv_matrix_free
directive, it will first check the signature. If it is a derived signature,
ccv_matrix_free won't free that matrix to OS immediately, instead, it will put
that matrix back to the application-wide cache. Sparse matrix, matrix without
signature / with initial signature will be freed immediately.
The matrix signature is important. For every matrix that is freed with ccv_matrix_free directive,
it will first check the signature. If it is a derived signature, ccv_matrix_free won't free
that matrix to OS immediately, instead, it will put that matrix back to the application-wide
cache. Sparse matrix, matrix without signature / with initial signature will be freed immediately.

Shortcut
--------

For operation X performed with matrix A and B, it will first generate the derived
signature. The signature will be searched in the application-wide cache in hope
of finding a result matrix. If such matrix C is found, the operation X will take
a shortcut and return that matrix to user. Otherwise, it will allocate such matrix,
set proper signature on it and perform the operation honestly.
For operation X performed with matrix A and B, it will first generate the derived signature. The
signature will be searched in the application-wide cache in hope of finding a result matrix. If
such matrix C is found, the operation X will take a shortcut and return that matrix to
user. Otherwise, it will allocate such matrix, set proper signature on it and perform the
operation honestly.

After finish this, I found that it may not be the most interesting bit of ccv.
But still, hope you found it otherwise :-)
After finish this, I found that it may not be the most interesting bit of ccv. But still,
hope you found it otherwise :-)
Loading

0 comments on commit e06050e

Please sign in to comment.