Skip to content

Commit d99140a

Browse files
MalekMFSnmanovic
MalekMFS
authored and
nmanovic
committed
Auto segmentation using Mask_RCNN (cvat-ai#767)
1 parent 310e208 commit d99140a

File tree

18 files changed

+586
-2
lines changed

18 files changed

+586
-2
lines changed

‎.gitignore

+1-1
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@
1212
/ssh/*
1313
!/ssh/README.md
1414
node_modules
15-
15+
/Mask_RCNN/
1616

1717
# Ignore temporary files
1818
docker-compose.override.yml

‎Dockerfile

+12-1
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,10 @@ RUN apt-get update && \
4343
unzip \
4444
unrar \
4545
p7zip-full \
46-
vim && \
46+
vim \
47+
git-core \
48+
libsm6 \
49+
libxext6 && \
4750
pip3 install -U setuptools && \
4851
ln -fs /usr/share/zoneinfo/${TZ} /etc/localtime && \
4952
dpkg-reconfigure -f noninteractive tzdata && \
@@ -78,6 +81,14 @@ RUN if [ "$TF_ANNOTATION" = "yes" ]; then \
7881
bash -i /tmp/components/tf_annotation/install.sh; \
7982
fi
8083

84+
# Auto segmentation support. by Mohammad
85+
ARG AUTO_SEGMENTATION
86+
ENV AUTO_SEGMENTATION=${AUTO_SEGMENTATION}
87+
ENV AUTO_SEGMENTATION_PATH=${HOME}/Mask_RCNN
88+
RUN if [ "$AUTO_SEGMENTATION" = "yes" ]; then \
89+
bash -i /tmp/components/auto_segmentation/install.sh; \
90+
fi
91+
8192
ARG WITH_TESTS
8293
RUN if [ "$WITH_TESTS" = "yes" ]; then \
8394
wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - && \

‎components/auto_segmentation/README.md

+38
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
## [Keras+Tensorflow Mask R-CNN Segmentation](https://github.com/matterport/Mask_RCNN)
2+
3+
### What is it?
4+
- This application allows you automatically to segment many various objects on images.
5+
- It's based on Feature Pyramid Network (FPN) and a ResNet101 backbone.
6+
7+
- It uses a pre-trained model on MS COCO dataset
8+
- It supports next classes (use them in "labels" row):
9+
```python
10+
'BG', 'person', 'bicycle', 'car', 'motorcycle', 'airplane',
11+
'bus', 'train', 'truck', 'boat', 'traffic light',
12+
'fire hydrant', 'stop sign', 'parking meter', 'bench', 'bird',
13+
'cat', 'dog', 'horse', 'sheep', 'cow', 'elephant', 'bear',
14+
'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie',
15+
'suitcase', 'frisbee', 'skis', 'snowboard', 'sports ball',
16+
'kite', 'baseball bat', 'baseball glove', 'skateboard',
17+
'surfboard', 'tennis racket', 'bottle', 'wine glass', 'cup',
18+
'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple',
19+
'sandwich', 'orange', 'broccoli', 'carrot', 'hot dog', 'pizza',
20+
'donut', 'cake', 'chair', 'couch', 'potted plant', 'bed',
21+
'dining table', 'toilet', 'tv', 'laptop', 'mouse', 'remote',
22+
'keyboard', 'cell phone', 'microwave', 'oven', 'toaster',
23+
'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors',
24+
'teddy bear', 'hair drier', 'toothbrush'.
25+
```
26+
- Component adds "Run Auto Segmentation" button into dashboard.
27+
28+
### Build docker image
29+
```bash
30+
# From project root directory
31+
docker-compose -f docker-compose.yml -f components/auto_segmentation/docker-compose.auto_segmentation.yml build
32+
```
33+
34+
### Run docker container
35+
```bash
36+
# From project root directory
37+
docker-compose -f docker-compose.yml -f components/auto_segmentation/docker-compose.auto_segmentation.yml up -d
38+
```

‎components/auto_segmentation/docker-compose.auto_segmentation.yml

+13
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
#
2+
# Copyright (C) 2018 Intel Corporation
3+
#
4+
# SPDX-License-Identifier: MIT
5+
#
6+
version: "2.3"
7+
8+
services:
9+
cvat:
10+
build:
11+
context: .
12+
args:
13+
AUTO_SEGMENTATION: "yes"

‎components/auto_segmentation/install.sh

+12
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
#!/bin/bash
2+
#
3+
4+
set -e
5+
6+
cd ${HOME} && \
7+
git clone https://github.com/matterport/Mask_RCNN.git && \
8+
wget https://github.com/matterport/Mask_RCNN/releases/download/v2.0/mask_rcnn_coco.h5 && \
9+
mv mask_rcnn_coco.h5 Mask_RCNN/mask_rcnn_coco.h5
10+
11+
# TODO remove useless files
12+
# tensorflow and Keras are installed globally

‎cvat/apps/auto_segmentation/__init__.py

+9
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
2+
# Copyright (C) 2018 Intel Corporation
3+
#
4+
# SPDX-License-Identifier: MIT
5+
6+
from cvat.settings.base import JS_3RDPARTY
7+
8+
JS_3RDPARTY['dashboard'] = JS_3RDPARTY.get('dashboard', []) + ['auto_segmentation/js/dashboardPlugin.js']
9+

‎cvat/apps/auto_segmentation/admin.py

+8
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
2+
# Copyright (C) 2018 Intel Corporation
3+
#
4+
# SPDX-License-Identifier: MIT
5+
6+
7+
# Register your models here.
8+

‎cvat/apps/auto_segmentation/apps.py

+11
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
2+
# Copyright (C) 2018 Intel Corporation
3+
#
4+
# SPDX-License-Identifier: MIT
5+
6+
from django.apps import AppConfig
7+
8+
9+
class AutoSegmentationConfig(AppConfig):
10+
name = 'auto_segmentation'
11+

‎cvat/apps/auto_segmentation/migrations/__init__.py

+5
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
2+
# Copyright (C) 2018 Intel Corporation
3+
#
4+
# SPDX-License-Identifier: MIT
5+

‎cvat/apps/auto_segmentation/models.py

+8
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
2+
# Copyright (C) 2018 Intel Corporation
3+
#
4+
# SPDX-License-Identifier: MIT
5+
6+
7+
# Create your models here.
8+

‎cvat/apps/auto_segmentation/static/auto_segmentation/js/dashboardPlugin.js

+112
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
/*
2+
* Copyright (C) 2018 Intel Corporation
3+
*
4+
* SPDX-License-Identifier: MIT
5+
*/
6+
7+
/* global
8+
userConfirm:false
9+
showMessage:false
10+
*/
11+
12+
window.addEventListener('dashboardReady', () => {
13+
function checkProcess(tid, button) {
14+
function checkCallback() {
15+
$.get(`/tensorflow/segmentation/check/task/${tid}`).done((statusData) => {
16+
if (['started', 'queued'].includes(statusData.status)) {
17+
const progress = Math.round(statusData.progress) || '0';
18+
button.text(`Cancel Auto Segmentation (${progress}%)`);
19+
setTimeout(checkCallback, 5000);
20+
} else {
21+
button.text('Run Auto Segmentation');
22+
button.removeClass('tfAnnotationProcess');
23+
button.prop('disabled', false);
24+
25+
if (statusData.status === 'failed') {
26+
const message = `Tensorflow Segmentation failed. Error: ${statusData.stderr}`;
27+
showMessage(message);
28+
} else if (statusData.status !== 'finished') {
29+
const message = `Tensorflow segmentation check request returned status "${statusData.status}"`;
30+
showMessage(message);
31+
}
32+
}
33+
}).fail((errorData) => {
34+
const message = `Can not sent tensorflow segmentation check request. Code: ${errorData.status}. `
35+
+ `Message: ${errorData.responseText || errorData.statusText}`;
36+
showMessage(message);
37+
});
38+
}
39+
40+
setTimeout(checkCallback, 5000);
41+
}
42+
43+
44+
function runProcess(tid, button) {
45+
$.get(`/tensorflow/segmentation/create/task/${tid}`).done(() => {
46+
showMessage('Process has started');
47+
button.text('Cancel Auto Segmentation (0%)');
48+
button.addClass('tfAnnotationProcess');
49+
checkProcess(tid, button);
50+
}).fail((errorData) => {
51+
const message = `Can not run Auto Segmentation. Code: ${errorData.status}. `
52+
+ `Message: ${errorData.responseText || errorData.statusText}`;
53+
showMessage(message);
54+
});
55+
}
56+
57+
58+
function cancelProcess(tid, button) {
59+
$.get(`/tensorflow/segmentation/cancel/task/${tid}`).done(() => {
60+
button.prop('disabled', true);
61+
}).fail((errorData) => {
62+
const message = `Can not cancel Auto Segmentation. Code: ${errorData.status}. `
63+
+ `Message: ${errorData.responseText || errorData.statusText}`;
64+
showMessage(message);
65+
});
66+
}
67+
68+
69+
function setupDashboardItem(item, metaData) {
70+
const tid = +item.attr('tid');
71+
const button = $('<button> Run Auto Segmentation </button>');
72+
73+
button.on('click', () => {
74+
if (button.hasClass('tfAnnotationProcess')) {
75+
userConfirm('The process will be canceled. Continue?', () => {
76+
cancelProcess(tid, button);
77+
});
78+
} else {
79+
userConfirm('The current annotation will be lost. Are you sure?', () => {
80+
runProcess(tid, button);
81+
});
82+
}
83+
});
84+
85+
button.addClass('dashboardTFAnnotationButton regular dashboardButtonUI');
86+
button.appendTo(item.find('div.dashboardButtonsUI'));
87+
88+
if ((tid in metaData) && (metaData[tid].active)) {
89+
button.text('Cancel Auto Segmentation');
90+
button.addClass('tfAnnotationProcess');
91+
checkProcess(tid, button);
92+
}
93+
}
94+
95+
const elements = $('.dashboardItem');
96+
const tids = Array.from(elements, el => +el.getAttribute('tid'));
97+
98+
$.ajax({
99+
type: 'POST',
100+
url: '/tensorflow/segmentation/meta/get',
101+
data: JSON.stringify(tids),
102+
contentType: 'application/json; charset=utf-8',
103+
}).done((metaData) => {
104+
elements.each(function setupDashboardItemWrapper() {
105+
setupDashboardItem($(this), metaData);
106+
});
107+
}).fail((errorData) => {
108+
const message = `Can not get Auto Segmentation meta info. Code: ${errorData.status}. `
109+
+ `Message: ${errorData.responseText || errorData.statusText}`;
110+
showMessage(message);
111+
});
112+
});

‎cvat/apps/auto_segmentation/tests.py

+8
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
2+
# Copyright (C) 2018 Intel Corporation
3+
#
4+
# SPDX-License-Identifier: MIT
5+
6+
7+
# Create your tests here.
8+

‎cvat/apps/auto_segmentation/urls.py

+14
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
2+
# Copyright (C) 2018 Intel Corporation
3+
#
4+
# SPDX-License-Identifier: MIT
5+
6+
from django.urls import path
7+
from . import views
8+
9+
urlpatterns = [
10+
path('create/task/<int:tid>', views.create),
11+
path('check/task/<int:tid>', views.check),
12+
path('cancel/task/<int:tid>', views.cancel),
13+
path('meta/get', views.get_meta_info),
14+
]

0 commit comments

Comments
 (0)