Skip to content

Commit

Permalink
compatible with python3
Browse files Browse the repository at this point in the history
  • Loading branch information
kuaikuaikim committed Dec 27, 2017
1 parent 07467fb commit 08efdf2
Show file tree
Hide file tree
Showing 6 changed files with 187 additions and 51 deletions.
48 changes: 28 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,11 +54,19 @@ First setting a pytorch and cv2. We suggest Anaconda to make a virtual and indep
* pytorch
* torchvision
* cv2
* matplotlib
* matplotlib

Also we provide a anaconda environment dependency list called environment.yml in the root path.

```shell
git clone https://github.com/kuaikuaikim/DFace.git
```


Also we provide a anaconda environment dependency list called environment.yml (windows please use environment-win64.yml) in the root path.
You can create your DFace environment very easily.
```shell
cd DFace

conda env create -f path/to/environment.yml
```

Expand All @@ -68,65 +76,65 @@ If you are interested in how to train a mtcnn model, you can follow next step.

#### Train mtcnn Model
MTCNN have three networks called **PNet**, **RNet** and **ONet**.So we should train it on three stage, and each stage depend on previous network which will generate train data to feed current train net, also propel the minimum loss between two networks.
Please download the train face **datasets** before your training. We use **[WIDER FACE](http://mmlab.ie.cuhk.edu.hk/projects/WIDERFace/)** and **[CelebA](http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html)**
Please download the train face **datasets** before your training. We use **[WIDER FACE](http://mmlab.ie.cuhk.edu.hk/projects/WIDERFace/)** and **[CelebA](http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html)** .WIDER FACE is used for training face classification and face bounding box, also CelebA is used for face landmarks. The original wider face annotation file is matlab format, you must transform it to text. I have put the transformed annotation text file into [anno_store/wider_origin_anno.txt](https://github.com/kuaikuaikim/DFace/blob/master/anno_store/wider_origin_anno.txt). This file is related to the following parameter called --anno_file.


* Create the DFace train data temporary folder, this folder is involved in the following parameter --dface_traindata_store

```shell
mkdir {your dface traindata folder}
```


* Generate PNet Train data and annotation file

```shell
python src/prepare_data/gen_Pnet_train_data.py --dataset_path {your dataset path} --anno_file {your dataset original annotation path}
python dface/prepare_data/gen_Pnet_train_data.py --prefix_path {annotation file image prefix path, just your local wider face images folder} --dface_traindata_store {dface train data temporary folder you made before } --anno_file {wider face original combined annotation file, default anno_store/wider_origin_anno.txt}
```
* Assemble annotation file and shuffle it

```shell
python src/prepare_data/assemble_pnet_imglist.py
python dface/prepare_data/assemble_pnet_imglist.py
```

* Train PNet model


```shell
python src/train_net/train_p_net.py
python dface/train_net/train_p_net.py
```
* Generate RNet Train data and annotation file

```shell
python src/prepare_data/gen_Rnet_train_data.py --dataset_path {your dataset path} --anno_file {your dataset original annotation path} --pmodel_file {yout PNet model file trained before}
python dface/prepare_data/gen_Rnet_train_data.py --prefix_path {annotation file image prefix path, just your local wider face images folder} --dface_traindata_store {dface train data temporary folder you made before } --anno_file {wider face original combined annotation file, default anno_store/wider_origin_anno.txt} --pmodel_file {your PNet model file trained before}
```
* Assemble annotation file and shuffle it

```shell
python src/prepare_data/assemble_rnet_imglist.py
python dface/prepare_data/assemble_rnet_imglist.py
```

* Train RNet model

```shell
python src/train_net/train_r_net.py
python dface/train_net/train_r_net.py
```

* Generate ONet Train data and annotation file

```shell
python src/prepare_data/gen_Onet_train_data.py --dataset_path {your dataset path} --anno_file {your dataset original annotation path} --pmodel_file {yout PNet model file trained before} --rmodel_file {yout RNet model file trained before}
python dface/prepare_data/gen_Onet_train_data.py --prefix_path {annotation file image prefix path, just your local wider face images folder} --dface_traindata_store {dface train data temporary folder you made before } --anno_file {wider face original combined annotation file, default anno_store/wider_origin_anno.txt} --pmodel_file {your PNet model file trained before} --rmodel_file {your RNet model file trained before}
```

* Generate ONet Train landmarks data

```shell
python src/prepare_data/gen_landmark_48.py
python dface/prepare_data/gen_landmark_48.py
```

* Assemble annotation file and shuffle it

```shell
python src/prepare_data/assemble_onet_imglist.py
python dface/prepare_data/assemble_onet_imglist.py
```

* Train ONet model

```shell
python src/train_net/train_o_net.py
python dface/train_net/train_o_net.py
```

#### Test face detection
Expand Down
102 changes: 71 additions & 31 deletions README_zh.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
<div align=center>
<img src="http://pic.dface.io/dfacelogoblue.png" width="350">
<img src="http://affluent.oss-cn-hangzhou.aliyuncs.com/html/images/dface_logo.png" width="350">
</div>

-----------------
# DFace • [![License](http://pic.dface.io/apache2.svg)](https://opensource.org/licenses/Apache-2.0) [![gitter](http://pic.dface.io/gitee.svg)](https://gitter.im/cmusatyalab/DFace)
# DFace • [![License](http://pic.dface.io/apache2.svg)](https://opensource.org/licenses/Apache-2.0)


| **`Linux CPU`** | **`Linux GPU`** | **`Mac OS CPU`** | **`Windows CPU`** |
Expand All @@ -14,22 +14,33 @@
**基于多任务卷积网络(MTCNN)和Center-Loss的多人实时人脸检测和人脸识别系统。**


[Slack 聊天组](https://dfaceio.slack.com/)



**DFace** 是个开源的深度学习人脸检测和人脸识别系统。所有功能都采用 **[pytorch](https://github.com/pytorch/pytorch)** 框架开发。pytorch是一个由facebook开发的深度学习框架,它包含了一些比较有趣的高级特性,例如自动求导,动态构图等。DFace天然的继承了这些优点,使得它的训练过程可以更加简单方便,并且实现的代码可以更加清晰易懂。
DFace可以利用CUDA来支持GPU加速模式。我们建议尝试linux GPU这种模式,它几乎可以实现实时的效果。
所有的灵感都来源于学术界最近的一些研究成果,例如 [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://arxiv.org/abs/1604.02878)[FaceNet: A Unified Embedding for Face Recognition and Clustering](https://arxiv.org/abs/1503.03832)


**MTCNN 结构**  

![mtcnn](http://pic.dface.io/mtcnn.png)
![mtcnn](http://affluent.oss-cn-hangzhou.aliyuncs.com/html/images/mtcnn_st.png)


**如果你对DFace感兴趣并且想参与到这个项目中, 请查看目录下的 CONTRIBUTING.md 文档,它会实时展示一些需要@TODO的清单。我会用 [issues](https://github.com/DFace/DFace/issues)
来跟踪和反馈所有的问题.**
** 如果你对DFace感兴趣并且想参与到这个项目中, 以下TODO是一些需要实现的功能,我定期会更新,它会实时展示一些需要开发的清单。提交你的fork request,我会用issues来跟踪和反馈所有的问题。也可以加DFace的官方Q群 681403076 也可以加本人微信 jinkuaikuai005 **

### TODO(需要开发的功能)
- 基于center loss 或者triplet loss原理开发人脸对比功能,模型采用ResNet inception v2. 该功能能够比较两张人脸图片的相似性。具体可以参考 [Paper](https://arxiv.org/abs/1503.03832)[FaceNet](https://github.com/davidsandberg/facenet)
- 反欺诈功能,根据光线,质地等人脸特性来防止照片攻击,视频攻击,回放攻击等。具体可参考LBP算法和SVM训练模型。
- 3D人脸反欺诈。
- mobile移植,根据ONNX标准把pytorch训练好的模型迁移到caffe2,一些numpy算法改用c++实现。
- Tensor RT移植,高并发。
- Docker支持,gpu版

## 安装
DFace主要有两大模块,人脸检测和人脸识别。我会提供所有模型训练和运行的详细步骤。你首先需要构建一个pytorch和cv2的python环境,我推荐使用Anaconda来设置一个独立的虚拟环境。
DFace主要有两大模块,人脸检测和人脸识别。我会提供所有模型训练和运行的详细步骤。你首先需要构建一个pytorch和cv2的python环境,我推荐使用Anaconda来设置一个独立的虚拟环境。目前作者倾向于Linux Ubuntu安装环境。感谢山东一位网友提供windows DFace安装体验,windos安装教程具体
可参考他的[博客](http://www.alearner.top/index.php/2017/12/23/dface-pytorch-win64-gpu)


### 依赖
Expand All @@ -40,98 +51,127 @@ DFace主要有两大模块,人脸检测和人脸识别。我会提供所有模
* cv2
* matplotlib

在这里我提供了一个anaconda的环境依赖文件environment.yml,它能方便你构建自己的虚拟环境。
```shell
git clone https://github.com/kuaikuaikim/DFace.git
```

在这里我提供了一个anaconda的环境依赖文件environment.yml (windows请用environment-win64.yml),它能方便你构建自己的虚拟环境。

```shell
conda env create -f path/to/environment.yml
cd dface

conda env create -f environment.yml
```

添加python搜索模块路径

```shell
export PYTHONPATH=$PYTHONPATH:{your local DFace root path}
```

### 人脸检测


### 人脸识别和检测

如果你对mtcnn模型感兴趣,以下过程可能会帮助到你。

#### 训练mtcnn模型
#### 训练mtcnn模型

MTCNN主要有三个网络,叫做**PNet**, **RNet** 和 **ONet**。因此我们的训练过程也需要分三步先后进行。为了更好的实现效果,当前被训练的网络都将依赖于上一个训练好的网络来生成数据。所有的人脸数据集都来自 **[WIDER FACE](http://mmlab.ie.cuhk.edu.hk/projects/WIDERFace/)** 和 **[CelebA](http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html)**。WIDER FACE仅提供了大量的人脸边框定位数据,而CelebA包含了人脸关键点定位数据。以下训练除了 生成ONet的人脸关键点训练数据和标注文件 该步骤使用CelebA数据集,其他一律使用WIDER FACE。如果使用wider face的 wider_face_train.mat 注解文件需要转换成txt格式的,我这里用h5py写了个 [转换脚本](https://github.com/kuaikuaikim/DFace/blob/master/dface/prepare_data/widerface_annotation_gen/transform.py). 这里我提供一个已经转换好的wider face注解文件 [anno_store/wider_origin_anno.txt](https://github.com/kuaikuaikim/DFace/blob/master/anno_store/info/wider_origin_anno.txt), 以下训练过程参数名--anno_file默认就是使用该转换好的注解文件。

MTCNN主要有三个网络,叫做**PNet**, **RNet****ONet**。因此我们的训练过程也需要分三步先后进行。为了更好的实现效果,当前被训练的网络都将依赖于上一个训练好的网络来生成数据。所有的人脸数据集都来自 **[WIDER FACE](http://mmlab.ie.cuhk.edu.hk/projects/WIDERFace/)****[CelebA](http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html)**。WIDER FACE仅提供了大量的人脸边框定位数据,而CelebA包含了人脸关键点定位数据。

* 创建 dface 训练数据临时目录,对应于以下所有的参数名 --dface_traindata_store
```shell
mkdir {your dface traindata folder}
```


* 生成PNet训练数据和标注文件

```shell
python src/prepare_data/gen_Pnet_train_data.py --dataset_path {your dataset path} --anno_file {your dataset original annotation path}
python dface/prepare_data/gen_Pnet_train_data.py --prefix_path {注解文件中图片的目录前缀,就是wider face图片所在目录} --dface_traindata_store {之前创建的dface训练数据临时目录} --anno_file {wider face 注解文件,可以不填,默认使用anno_store/wider_origin_anno.txt}
```
* 乱序合并标注文件

```shell
python src/prepare_data/assemble_pnet_imglist.py
python dface/prepare_data/assemble_pnet_imglist.py
```

* 训练PNet模型


```shell
python src/train_net/train_p_net.py
python dface/train_net/train_p_net.py
```
* 生成RNet训练数据和标注文件

```shell
python src/prepare_data/gen_Rnet_train_data.py --dataset_path {your dataset path} --anno_file {your dataset original annotation path} --pmodel_file {yout PNet model file trained before}
python dface/prepare_data/gen_Rnet_train_data.py --prefix_path {注解文件中图片的目录前缀,就是wider face图片所在目录} --dface_traindata_store {之前创建的dface训练数据临时目录} --anno_file {wider face 注解文件,可以不填,默认使用anno_store/wider_origin_anno.txt} --pmodel_file {之前训练的Pnet模型文件}
```
* 乱序合并标注文件

```shell
python src/prepare_data/assemble_rnet_imglist.py
python dface/prepare_data/assemble_rnet_imglist.py
```

* 训练RNet模型

```shell
python src/train_net/train_r_net.py
python dface/train_net/train_r_net.py
```

* 生成ONet训练数据和标注文件

```shell
python src/prepare_data/gen_Onet_train_data.py --dataset_path {your dataset path} --anno_file {your dataset original annotation path} --pmodel_file {yout PNet model file trained before} --rmodel_file {yout RNet model file trained before}
python dface/prepare_data/gen_Onet_train_data.py --prefix_path {注解文件中图片的目录前缀,就是wider face图片所在目录} --dface_traindata_store {之前创建的dface训练数据临时目录} --anno_file {wider face 注解文件,可以不填,默认使用anno_store/wider_origin_anno.txt} --pmodel_file {之前训练的Pnet模型文件} --rmodel_file {之前训练的Rnet模型文件}
```

* 生成ONet的人脸关键点训练数据和标注文件
* 生成ONet的人脸五官关键点训练数据和标注文件

```shell
python src/prepare_data/gen_landmark_48.py
python dface/prepare_data/gen_landmark_48.py
```

* 乱序合并标注文件(包括人脸关键点)
* 乱序合并标注文件(包括人脸五官关键点)

```shell
python src/prepare_data/assemble_onet_imglist.py
python dface/prepare_data/assemble_onet_imglist.py
```

* 训练ONet模型

```shell
python src/train_net/train_o_net.py
python dface/train_net/train_o_net.py
```

#### 测试人脸检测
```shell
python test_image.py
```

### 人脸识别
### 人脸对比

TODO
@TODO 根据center loss实现人脸识别

## 测试效果
## 测试效果
![mtcnn](http://affluent.oss-cn-hangzhou.aliyuncs.com/html/images/dface_demoall.PNG)

![mtcnn](http://pic.dface.io/figure_2.png)

### QQ交流群(模型获取请加群)

## License
#### 681403076

![](http://affluent.oss-cn-hangzhou.aliyuncs.com/html/images/dfaceqqsm.png)

[Apache License 2.0](LICENSE)
#### 本人微信

##### jinkuaikuai005

## Reference
![](http://affluent.oss-cn-hangzhou.aliyuncs.com/html/images/perqr.jpg)



## License

[Apache License 2.0](LICENSE)

* [Seanlinx/mtcnn](https://github.com/Seanlinx/mtcnn)
Empty file.
46 changes: 46 additions & 0 deletions dface/prepare_data/widerface_annotation_gen/transform.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
from dface.prepare_data.widerface_annotation_gen.wider_loader import WIDER
import cv2
import time

#wider face original images path
path_to_image = '/idata/data/wider_face/WIDER_train/images'

#matlab file path
file_to_label = './wider_face_train.mat'

#target file path
target_file = './anno.txt'

wider = WIDER(file_to_label, path_to_image)


line_count = 0
box_count = 0

print('start transforming....')
t = time.time()

with open(target_file, 'w+') as f:
# press ctrl-C to stop the process
for data in wider.next():
line = []
line.append(str(data.image_name))
line_count += 1
for i,box in enumerate(data.bboxes):
box_count += 1
for j,bvalue in enumerate(box):
line.append(str(bvalue))

line.append('\n')

line_str = ' '.join(line)
f.write(line_str)

st = time.time()-t
print('end transforming')

print('spend time:%ld'%st)
print('total line(images):%d'%line_count)
print('total boxes(faces):%d'%box_count)


Binary file not shown.
42 changes: 42 additions & 0 deletions dface/prepare_data/widerface_annotation_gen/wider_loader.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
import h5py
import os


class DATA(object):
def __init__(self, image_name, bboxes):
self.image_name = image_name
self.bboxes = bboxes


class WIDER(object):
def __init__(self, file_to_label, path_to_image):
self.file_to_label = file_to_label
self.path_to_image = path_to_image

self.f = h5py.File(file_to_label, 'r')
self.event_list = self.f.get('event_list')
self.file_list = self.f.get('file_list')
self.face_bbx_list = self.f.get('face_bbx_list')

def next(self):

for event_idx, event in enumerate(self.event_list.value[0]):
directory = self.f[event].value.tostring().decode('utf-16')
for im_idx, im in enumerate(
self.f[self.file_list.value[0][event_idx]].value[0]):

im_name = self.f[im].value.tostring().decode('utf-16')
face_bbx = self.f[self.f[self.face_bbx_list.value
[0][event_idx]].value[0][im_idx]].value

bboxes = []

for i in range(face_bbx.shape[1]):
xmin = int(face_bbx[0][i])
ymin = int(face_bbx[1][i])
xmax = int(face_bbx[0][i] + face_bbx[2][i])
ymax = int(face_bbx[1][i] + face_bbx[3][i])
bboxes.append((xmin, ymin, xmax, ymax))

yield DATA(os.path.join(self.path_to_image, directory,
im_name + '.jpg'), bboxes)

0 comments on commit 08efdf2

Please sign in to comment.