Examples highly improved

doc/demo_overview.md

@@ -62,9 +62,9 @@ The following example runs the demo video `video.avi`, renders image frames on `
 # Best results found with 6 scales
 ./build/examples/openpose/openpose.bin --hand --hand_scale_number 6 --hand_scale_range 0.4
 # Adding tracking to Webcam (if FPS per GPU > 10 FPS) and Video
-./build/examples/openpose/openpose.bin --video examples/media/video.avi --hand --hand_tracking
+./build/examples/openpose/openpose.bin --video examples/media/video.avi --hand --hand_detector 3
 # Multi-scale + tracking is also possible
-./build/examples/openpose/openpose.bin --video examples/media/video.avi --hand --hand_scale_number 6 --hand_scale_range 0.4 --hand_tracking
+./build/examples/openpose/openpose.bin --video examples/media/video.avi --hand --hand_scale_number 6 --hand_scale_range 0.4 --hand_detector 3
 ```
 
 
@@ -181,14 +181,15 @@ Each flag is divided into flag name, default value, and description.
 
 6. OpenPose Face
 - DEFINE_bool(face,                       false,          "Enables face keypoint detection. It will share some parameters from the body pose, e.g. `model_folder`. Note that this will considerable slow down the performance and increse the required GPU memory. In addition, the greater number of people on the image, the slower OpenPose will be.");
+- DEFINE_int32(face_detector,             0,              "Kind of face rectangle detector. Select 0 (default) to select OpenPose body detector (most accurate one and fastest one if body is enabled), 1 to select OpenCV face detector (not implemented for hands), 2 to indicate that it will be provided by the user, or 3 to also apply hand tracking (only for hand). Hand tracking might improve hand keypoint detection for webcam (if the frame rate is high enough, i.e., >7 FPS per GPU) and video. This is not person ID tracking, it simply looks for hands in positions at which hands were located in previous frames, but it does not guarantee the same person ID among frames.");
 - DEFINE_string(face_net_resolution,      "368x368",      "Multiples of 16 and squared. Analogous to `net_resolution` but applied to the face keypoint detector. 320x320 usually works fine while giving a substantial speed up when multiple faces on the image.");
 
 7. OpenPose Hand
 - DEFINE_bool(hand,                       false,          "Enables hand keypoint detection. It will share some parameters from the body pose, e.g. `model_folder`. Analogously to `--face`, it will also slow down the performance, increase the required GPU memory and its speed depends on the number of people.");
+- DEFINE_int32(hand_detector,             0,              "Kind of hand rectangle detector. Analogous to `--face_detector`.");
 - DEFINE_string(hand_net_resolution,      "368x368",      "Multiples of 16 and squared. Analogous to `net_resolution` but applied to the hand keypoint detector.");
 - DEFINE_int32(hand_scale_number,         1,              "Analogous to `scale_number` but applied to the hand keypoint detector. Our best results were found with `hand_scale_number` = 6 and `hand_scale_range` = 0.4.");
 - DEFINE_double(hand_scale_range,         0.4,            "Analogous purpose than `scale_gap` but applied to the hand keypoint detector. Total range between smallest and biggest scale. The scales will be centered in ratio 1. E.g., if scaleRange = 0.4 and scalesNumber = 2, then there will be 2 scales, 0.8 and 1.2.");
-- DEFINE_bool(hand_tracking,              false,          "Adding hand tracking might improve hand keypoints detection for webcam (if the frame rate is high enough, i.e., >7 FPS per GPU) and video. This is not person ID tracking, it simply looks for hands in positions at which hands were located in previous frames, but it does not guarantee the same person ID among frames.");
 
 8. OpenPose 3-D Reconstruction
 - DEFINE_bool(3d,                         false,          "Running OpenPose 3-D reconstruction demo: 1) Reading from a stereo camera system. 2) Performing 3-D reconstruction from the multiple views. 3) Displaying 3-D reconstruction results. Note that it will only display 1 person. If multiple people is present, it will fail.");

doc/faq.md

@@ -20,6 +20,7 @@ OpenPose - Frequently Asked Question (FAQ)
     15. [3D OpenPose Returning Wrong Results: 0, NaN, Infinity, etc.](#3d-openpose-returning-wrong-results-0-nan-infinity-etc)
     16. [Protobuf Clip Param Caffe Error](#protobuf-clip-param-caffe-error)
     17. [The Human Skeleton Looks like Dotted Lines Rather than Solid Lines](#the-human-skeleton-looks-like-dotted-lines-rather-than-solid-lines)
+    18. [Huge RAM Usage](#huge-ram-usage)
 
 
 
@@ -151,6 +152,13 @@ If you wanna use your custom Caffe and it has this error: This error only happen
 
 
 ### The Human Skeleton Looks like Dotted Lines Rather than Solid Lines
-**Q:** When I use the demo to handle my images，the skeletons are dotted lines. I want to know how to make them to be solid lines.
+**Q**: When I use the demo to handle my images，the skeletons are dotted lines. I want to know how to make them to be solid lines.
 
 **A**: The reason is that your input image size is too small. You can either 1) manually rescale your images up or 2) use a bigger `--output_resolution` so OpenPose will resize them up.
+
+
+
+### Huge RAM Usage
+**Q**: During the execution of the demo, the CPU usage oscillates between 50 and 99%, but the RAM is almost at 100%.
+
+**A**: Highly reducing the `--net_resolution` and following the tips in the [Speed Up, Memory Reduction, and Benchmark](#speed-up-memory-reduction-and-benchmark) section are the only way to reduce the RAM usage. Alternatively, you can disable `USE_MKL` in CMake, which will highly reduce the RAM usage of the Caffe version, but it might also reduce the program speed.

doc/release_notes.md

@@ -267,13 +267,14 @@ OpenPose Library - Release Notes
     5. Wrapper classes simplified and renamed. Wrapper renamed as WrapperT, and created Wrapper as the non-templated class equivalent.
     6. API and examples improved:
         1. New header file `flags.hpp` that includes all OpenPose flags, removing the need to copy them repeatedly on each OpenPose example file.
-        2. `tutorial_wrapper` renamed as `tutorial_api_cpp` as well as new examples were added.
-        2. `tutorial_python` renamed as `tutorial_api_python` as well as new examples were added.
-        3. `tutorial_pose` and `tutorial_thread` renamed as `tutorial_developer`, not meant to be used by users, but rather for OpenPose developers.
-        4. Examples do not end in core dumped if an OpenPose exception occurred during initialization, but they are rather closed returning -1. However, it will still results in core dumped if the exception occurs during multi-threading execution.
-        5. Added new examples, including examples to extract face and/or hand from images.
-        6. Added `--no_display` flag for the examples that does not use OpenPose output.
-        7. Given that display can be disabled in all examples, they all have been added to the Travis build so they can be tested.
+        2. Renamed `tutorial_wrapper` as `tutorial_api_cpp` as well as new examples were added.
+        2. Renamed `tutorial_python` as `tutorial_api_python` as well as new examples were added.
+        3. Renamed `tutorial_thread` as `tutorial_api_thread`, focused in the multi-thread mechanism.
+        4. Removed `tutorial_pose`, the folder `tutorial_api_cpp` includes much cleaner and commented examples.
+        5. Examples do not end in core dumped if an OpenPose exception occurred during initialization, but they are rather closed returning -1. However, it will still results in core dumped if the exception occurs during multi-threading execution.
+        6. Added new examples, including examples to extract face and/or hand from images.
+        7. Added `--no_display` flag for the examples that does not use OpenPose output.
+        8. Given that display can be disabled in all examples, they all have been added to the Travis build so they can be tested.
     7. Added a virtual destructor to almost all clases, so they can be inherited. Exceptions (for performance reasons): Array, Point, Rectangle, CvMatToOpOutput, OpOutputToCvMat.
     8. Auxiliary classes in errorAndLog turned into namespaces (Profiler must be kept as class to allow static parameters).
     9. Added flag `--frame_step` to allow the user to select the step or gap between processed frames. E.g., `--frame_step 5` would read and process frames 0, 5, 10, etc.
@@ -312,6 +313,7 @@ OpenPose Library - Release Notes
     30. Added the flags `--prototxt_path` and `--caffemodel_path` to allow custom ProtoTxt and CaffeModel paths.
     31. Replaced the old Python wrapper for an updated Pybind11 wrapper version, that includes all the functionality of the C++ API.
     32. Function getFilesOnDirectory() can extra all basic image file types at once without requiring to manually enumerate them.
+    33. Added the flags `--face_detector` and `--hand_detector`, that enable the user to select the face/hand rectangle detector that is used for the later face/hand keypoint detection. It includes OpenCV (for face), and also allows the user to provide its own input. Flag `--hand_tracking` is removed and integrated into this flag too.
 2. Functions or parameters renamed:
     1. By default, python example `tutorial_developer/python_2_pose_from_heatmaps.py` was using 2 scales starting at -1x736, changed to 1 scale at -1x368.
     2. WrapperStructPose default parameters changed to match those of the OpenPose demo binary.
@@ -327,6 +329,7 @@ OpenPose Library - Release Notes
     12. Moved most sh scripts into the `scripts/` folder. Only models/getModels.sh and the `*.bat` files are kept under `models/` and `3rdparty/windows`.
     13. For Python compatibility and scalability increase, template `TDatums` used for `include/openpose/wrapper/wrapper.hpp` has changed from `std::vector<Datum>` to `std::vector<std::shared_ptr<Datum>>`, including the respective changes in all the worker classes. In addition, some template classes have been simplified to only take 1 template parameter for user simplicity.
     14. Renamed intRound, charRound, etc. by positiveIntRound, positiveCharRound, etc. so that people can realize it is not safe for negative numbers.
+    15. Flag `--hand_tracking` is a subcase of `--hand_detector`, so it has been removed and incorporated as `--hand_detector 3`.
 3. Main bugs fixed:
     1. CMake-GUI was forcing to Release mode, allowed Debug modes too.
     2. NMS returns in index 0 the number of found peaks. However, while the number of peaks was truncated to a maximum of 127, this index 0 was saving the real number instead of the truncated one.

examples/CMakeLists.txt

@@ -1,9 +1,9 @@
 add_subdirectory(calibration)
 add_subdirectory(openpose)
 add_subdirectory(tutorial_add_module)
-add_subdirectory(tutorial_api_python)
 add_subdirectory(tutorial_api_cpp)
-add_subdirectory(tutorial_developer)
+add_subdirectory(tutorial_api_python)
+add_subdirectory(tutorial_api_thread)
 add_subdirectory(user_code)
 if (UNIX OR APPLE)
   add_subdirectory(tests)

examples/calibration/calibration.cpp

@@ -37,7 +37,7 @@ int openPoseDemo()
     try
     {
         op::log("Starting OpenPose calibration toolbox...", op::Priority::High);
-        const auto timerBegin = std::chrono::high_resolution_clock::now();
+        const auto opTimer = op::getTimerInit();
 
         // logging_level
         op::check(0 <= FLAGS_logging_level && FLAGS_logging_level <= 255, "Wrong logging_level value.",
@@ -114,12 +114,9 @@ int openPoseDemo()
             op::error("Unknown `--mode " + std::to_string(FLAGS_mode) + "`.", __LINE__, __FUNCTION__, __FILE__);
 
         // Measuring total time
-        const auto now = std::chrono::high_resolution_clock::now();
-        const auto totalTimeSec = (double)std::chrono::duration_cast<std::chrono::nanoseconds>(now-timerBegin).count()
-                                * 1e-9;
-        const auto message = "OpenPose calibration toolbox successfully finished. Total time: "
-                           + std::to_string(totalTimeSec) + " seconds.";
-        op::log(message, op::Priority::High);
+        op::printTime(
+            opTimer, "OpenPose calibration toolbox successfully finished. Total time: ", " seconds.",
+            op::Priority::High);
 
         return 0;
     }

examples/openpose/openpose.cpp

@@ -25,6 +25,11 @@ void configureWrapper(op::Wrapper& opWrapper)
         op::Profiler::setDefaultX(FLAGS_profile_speed);
 
         // Applying user defined configuration - GFlags to program variables
+        // producerType
+        op::ProducerType producerType;
+        std::string producerString;
+        std::tie(producerType, producerString) = op::flagsToProducer(
+            FLAGS_image_dir, FLAGS_video, FLAGS_ip_camera, FLAGS_camera, FLAGS_flir_camera, FLAGS_flir_camera_index);
         // cameraSize
         const auto cameraSize = op::flagsToPoint(FLAGS_camera_resolution, "-1x-1");
         // outputSize
@@ -35,11 +40,6 @@ void configureWrapper(op::Wrapper& opWrapper)
         const auto faceNetInputSize = op::flagsToPoint(FLAGS_face_net_resolution, "368x368 (multiples of 16)");
         // handNetInputSize
         const auto handNetInputSize = op::flagsToPoint(FLAGS_hand_net_resolution, "368x368 (multiples of 16)");
-        // producerType
-        op::ProducerType producerType;
-        std::string producerString;
-        std::tie(producerType, producerString) = op::flagsToProducer(
-            FLAGS_image_dir, FLAGS_video, FLAGS_ip_camera, FLAGS_camera, FLAGS_flir_camera, FLAGS_flir_camera_index);
         // poseModel
         const auto poseModel = op::flagsToPoseModel(FLAGS_model_pose);
         // JSON saving
@@ -54,6 +54,9 @@ void configureWrapper(op::Wrapper& opWrapper)
         const auto heatMapScaleMode = op::flagsToHeatMapScaleMode(FLAGS_heatmaps_scale);
         // >1 camera view?
         const auto multipleView = (FLAGS_3d || FLAGS_3d_views > 1 || FLAGS_flir_camera);
+        // Face and hand detectors
+        const auto faceDetector = op::flagsToDetector(FLAGS_face_detector);
+        const auto handDetector = op::flagsToDetector(FLAGS_hand_detector);
         // Enabling Google Logging
         const bool enableGoogleLogging = true;
 
@@ -68,12 +71,13 @@ void configureWrapper(op::Wrapper& opWrapper)
         opWrapper.configure(wrapperStructPose);
         // Face configuration (use op::WrapperStructFace{} to disable it)
         const op::WrapperStructFace wrapperStructFace{
-            FLAGS_face, faceNetInputSize, op::flagsToRenderMode(FLAGS_face_render, multipleView, FLAGS_render_pose),
+            FLAGS_face, faceDetector, faceNetInputSize,
+            op::flagsToRenderMode(FLAGS_face_render, multipleView, FLAGS_render_pose),
             (float)FLAGS_face_alpha_pose, (float)FLAGS_face_alpha_heatmap, (float)FLAGS_face_render_threshold};
         opWrapper.configure(wrapperStructFace);
         // Hand configuration (use op::WrapperStructHand{} to disable it)
         const op::WrapperStructHand wrapperStructHand{
-            FLAGS_hand, handNetInputSize, FLAGS_hand_scale_number, (float)FLAGS_hand_scale_range, FLAGS_hand_tracking,
+            FLAGS_hand, handDetector, handNetInputSize, FLAGS_hand_scale_number, (float)FLAGS_hand_scale_range,
             op::flagsToRenderMode(FLAGS_hand_render, multipleView, FLAGS_render_pose), (float)FLAGS_hand_alpha_pose,
             (float)FLAGS_hand_alpha_heatmap, (float)FLAGS_hand_render_threshold};
         opWrapper.configure(wrapperStructHand);
@@ -114,7 +118,7 @@ int openPoseDemo()
     try
     {
         op::log("Starting OpenPose demo...", op::Priority::High);
-        const auto timerBegin = std::chrono::high_resolution_clock::now();
+        const auto opTimer = op::getTimerInit();
 
         // Configure OpenPose
         op::log("Configuring OpenPose...", op::Priority::High);
@@ -126,12 +130,7 @@ int openPoseDemo()
         opWrapper.exec();
 
         // Measuring total time
-        const auto now = std::chrono::high_resolution_clock::now();
-        const auto totalTimeSec = (double)std::chrono::duration_cast<std::chrono::nanoseconds>(now-timerBegin).count()
-                                * 1e-9;
-        const auto message = "OpenPose demo successfully finished. Total time: "
-                           + std::to_string(totalTimeSec) + " seconds.";
-        op::log(message, op::Priority::High);
+        op::printTime(opTimer, "OpenPose demo successfully finished. Total time: ", " seconds.", op::Priority::High);
 
         // Return successful message
         return 0;

examples/tests/handFromJsonTest.cpp

@@ -21,7 +21,6 @@ DEFINE_bool(hand,                       true,           "");
 DEFINE_string(hand_net_resolution,      "368x368",      "");
 DEFINE_int32(hand_scale_number,         1,              "");
 DEFINE_double(hand_scale_range,         0.4,            "");
-DEFINE_bool(hand_tracking,              false,          "");
 // Display
 DEFINE_int32(display,                   -1,             "");
 // Result Saving
@@ -51,16 +50,15 @@ int handFromJsonTest()
         op::log("Configuring OpenPose...", op::Priority::High);
         op::WrapperHandFromJsonTest<op::Datum> opWrapper;
         // Pose configuration (use WrapperStructPose{} for default and recommended configuration)
-        op::WrapperStructPose wrapperStructPose{false, op::flagsToPoint("656x368"), op::flagsToPoint("1280x720"),
-                                                op::ScaleMode::InputResolution, FLAGS_num_gpu, FLAGS_num_gpu_start,
-                                                1, 0.15f, op::RenderMode::None, op::PoseModel::BODY_25,
-                                                true, 0.f, 0.f, 0, "models/", {}, op::ScaleMode::ZeroToOne, false,
-                                                0.05f, -1, false, enableGoogleLogging};
+        op::WrapperStructPose wrapperStructPose{
+            false, op::flagsToPoint("656x368"), op::flagsToPoint("1280x720"), op::ScaleMode::InputResolution,
+            FLAGS_num_gpu, FLAGS_num_gpu_start, 1, 0.15f, op::RenderMode::None, op::PoseModel::BODY_25, true, 0.f, 0.f,
+            0, "models/", {}, op::ScaleMode::ZeroToOne, false, 0.05f, -1, false, enableGoogleLogging};
         wrapperStructPose.modelFolder = FLAGS_model_folder;
         // Hand configuration (use op::WrapperStructHand{} to disable it)
-        const op::WrapperStructHand wrapperStructHand{FLAGS_hand, handNetInputSize, FLAGS_hand_scale_number,
-                                                      (float)FLAGS_hand_scale_range, FLAGS_hand_tracking,
-                                                      op::flagsToRenderMode(1)};
+        const op::WrapperStructHand wrapperStructHand{
+            FLAGS_hand, op::Detector::Provided, handNetInputSize, FLAGS_hand_scale_number,
+            (float)FLAGS_hand_scale_range, op::flagsToRenderMode(1)};
         // Configure wrapper
         opWrapper.configure(wrapperStructPose, wrapperStructHand, producerSharedPtr, FLAGS_hand_ground_truth,
                             FLAGS_write_json, op::flagsToDisplayMode(FLAGS_display, false));
@@ -71,8 +69,8 @@ int handFromJsonTest()
 
         // Measuring total time
         const auto now = std::chrono::high_resolution_clock::now();
-        const auto totalTimeSec = (double)std::chrono::duration_cast<std::chrono::nanoseconds>(now-timerBegin).count()
-                                * 1e-9;
+        const auto totalTimeSec = double(
+            std::chrono::duration_cast<std::chrono::nanoseconds>(now-timerBegin).count()* 1e-9);
         const auto message = "OpenPose demo successfully finished. Total time: "
                            + std::to_string(totalTimeSec) + " seconds.";
         op::log(message, op::Priority::High);

examples/tests/wrapperHandFromJsonTest.hpp

@@ -202,7 +202,7 @@ namespace op
                 {
                     // Hand detector
                     // If tracking
-                    if (wrapperStructHand.tracking)
+                    if (wrapperStructHand.detector == Detector::BodyWithTracking)
                         error("Tracking not valid for hand detector from JSON files.", __LINE__, __FUNCTION__, __FILE__);
                     // If detection
                     else