Deep Semantic Segmentation

This is a framework for training, validation and inference of deep semantic segmentation models, but currenty it only supports FCN. I have used it to train Fully Convolutional Network and detect road in a given scene using learnt model.

Developed using Tensorflow

Training Progress

Training Details

Parameter	Value
Optimization Algorithm	Adam
Learning Step Schedule	Exponential Decay
Regularization	Dropout = 0.5
Exploding Gradient Remedy	Clipping Gradient (Max Norm = 0.1)
Performance Metric	Intersection Over Union (IOU)
Numper Of Epochs	50
Batch Size	32

---

Epoch 5 --> 10 --> 15 on validation data

Dataset

Download the Kitti Road dataset from here.
Extract dataset in the data folder. This will create the folder data_road with all the training and test images.

---

Test Results

After training for 50 Epochs

IOU : 0.92

Total 160 Images

Usage Guide

It uses same class' object for training and inference depending on number of arguments used to create the object.

Refer train file for training code.
Refer infer file for inference code.
Detailed Explanation for using FCN class is in next section

GUI

(Incomplete, currently being developed)

Training

---

Import

from ImageSemanticSegmentor.FCN.FCN import FullyConvNet

Then set following directories

Directory	Content
vggModelDir	Pretrained VGG weights
trainDataDir	Training Images
trainLabelDir	Training Image Labels
validationDir	Validation Images
fcnModelDir	Saved Model Weights
fcnInferDir	Model Weights For Inference
testDataDir	Test Images
testResultDir	Inference Results Of Test Images

Create Object For Training

imageSegmenter = FullyConvNet(trainSession, vggModelDir, trainDataDir, trainLabelDir, 
                              validationDir, fcnModelDir, testDataDir, 
                              fcnInferDir, numOfClasses)

Then Set Optimization Parameters

Parameter	Purpose	Note
optAlgo	Optimization Algoritm	Only 3 are suppored
initLearningRate	Step Size
ImgSize	Image Dimension To Resize Train Images
maxGradNorm	Maximum Gradient Norm For Clipping Gradient	Needed To Prevent Exploding Gradient

Set optimizer

imageSegmenter.setOptimizer(optAlgo, initLearningRate, ImgSize, maxGradNorm)

Then set training parameters

Parameter	Purpose
batchSize	Number Of Images In Training Batch
keepProb	Dropout Probability
metric	Performance Metric To Use
numOfEpochs	Number Of Times To Iterate Through Whole Train Data
saveModel	Save Learnt Models ?
perfThresh	Minimum Acceptable Model Performance
showSegValImages	Display Segmented Images During Model Validation

Start training

imageSegmenter.trainFCN(batchSize, keepProb, metric, numOfEpochs, saveModel,
                        perfThresh, showSegValImages)

Then Sit Back And Wait

---

Inference

---

Set following parameters

Parameter	Purpose
inferModelDir	Directory Containing Trained Model Files
inferModelName	Model File Name
ImgSize	Image Dimension
numOfEpochs	Number Of Times To Iterate Through Whole Train Data

Create object for inference

inferSession        = tf.Session()
inferImgSegment     = FullyConvNet(inferSession, inferModelDir, inferModelName, ImgSize,
                                   numOfClasses)

Obtain segmented image

testImage           = scipy.misc.imresize(scipy.misc.imread(image_file), ImgSize)
segmentedTestImg    = inferImgSegment.segmentThisImage(testImage)

---

Future Work

• Add GUI for both training and inference, for user interaction.
• Develop a website to provide this as a web service.
• Extend/Modify this framework to support other semantic segmentation models.

References

Code to generate training batches is taken from here
Read aboout Fully Convolutional Network from here