Multi process/thread Video Decoder/Encoder/RTMP_Publisher

Introduction

Code flow

• Dispatch multiple video source (local video or live stream) to threads of child process
• Intel HW VAAPI Decoder decodes input video to YUV in RSFFMpegDecoderTask, pass the decoded frame to next task (Data Dumper)
• Dump decoded video frame to file for review, pass frame to RSFFMpegEncoderTask
• Intel HW VAAPI Encoder encodes video frame to H264, pass the encoded packet to next task (Data Dumper)
• Dump encoded video packet to file for review, pass packet to RTMP publisher
• RTMP publisher publishes raw H264 packet to RTMP server ossrs/srs

Video Decoder/Encoder

• FFMPEG 4.1.3
• Intel VAAPI HW Accel

RTMP Publisher

• through srs-librtmp

Build

1. prerequisite

   apt install build-essential ffmpeg libavcodec-dev libavformat-dev libavfilter-dev libavutil-dev mpv vlc

2. build srs RTMP server

   mkdir ~/src
   cd ~/src
   git clone https://github.com/ossrs/srs
   cd srs/trunk
   ./configure --with-librtmp --without-http-callback --without-ssl && make -j8
   # ./objs/lib/srs_librtmp.a
   # ./objs/srs -c conf/rtmp.conf

3. build RSOrionDecoder (depends on srs-librtmp)

   cd ~/src
   git clone https://github.com/z7z8th/RSOrionDecoder
   cd RSOrionDecoder
   # IMPORTANT: modify video source array `vSrcArray' to yours in demo/demo.cpp before make
   make clean
   make all

Run

1. Run srs RTMP server

   cd ~/src
   cd srs/trunk
   ./objs/srs -c conf/rtmp.conf

2. Test RTMP publish (optional)

   research/librtmp/objs/srs_h264_raw_publish path/to/encoded-test.h264 rtmp://127.0.0.1:1935/live/1 25
   

3. Run RSOrionDecoder to test Decode/Encode/RTMP_Publish

   cd ~/src/RSOrionDecoder
   ./tools/start-rtmp-server.sh
   ./bin/libOrionDecoder
   ./tools/start-rtmp-clients-mpv.sh

Test (Video file too large for git)

1. Test Environment
   • Hardware:
     • CPU: 4 Core Intel Xeon v3, 2 Logical Threads each Core
     • Memory: 24GB DDR3 2133MHz
     • Disk: Samsung SSD
   • Software
     • Debian buster/testing
     • FFMPEG 4.1.3
     • ossrs/srs 2.0rc6
2. Test Material
   • 8 1080p videos, average bit rate of them are:
     • 2 videos at ~20Mbps
     • 2 videos at ~15Mbps
     • 4 videos at ~9Mbps
3. Result
   • mpv plays smoothly with the later 6 videos, average bit rate at ~15Mbps or ~9Mbps.
   • there's some lag with the first 2 videos, average bit rate at ~20Mbps.

TODO List

TODO.md