Docker image & algorithm submission for Category 1 of SurgToolLoc Challenge 2022

This repository has everything you and your team need to make an algorithm submission for the SurgToolLoc Challenge Category 1.

Be sure that you have a verified account on Grand Challenge and are accepted as a participant in the SurgToolLoc challenge. You should be able to submit your Docker container/algorithm on the challenge website when the submission opens.

Here are some useful documentation links for your submission process:

• Tutorial on how to make an algorithm container on Grand Challenge
• Docker documentation
• Evalutils documentation
• Grand Challenge documentation

Prerequisites

You will need to have Docker installed on your system. We recommend using Linux with a Docker installation. If you are on Windows, please use WSL 2.0.

Prediction format

For category 1 of SurgToolLoc Challenge (surgical tool classification) the instructions to generate the Docker container are given below

Category #1 – Surgical tool classification:

The output json file needs to be a list of dictionaries, containing information of tools present (from the total of 14 possible tools) in each frame of the input video. An example is given below:

[ 
  { 
    "slice_nr": 0, 
    "needle_driver": true, 
    "monopolar_curved_scissor": true, 
    "force_bipolar": false, 
    "clip_applier": false, 
    "tip_up_fenestrated_grasper": false, 
    "cadiere_forceps": false, 
    "bipolar_forceps": false, 
    "vessel_sealer": false, 
    "suction_irrigator": false, 
    "bipolar_dissector": false, 
    "prograsp_forceps": false, 
    "stapler": false, 
    "permanent_cautery_hook_spatula": false, 
    "grasping_retractor": false 
  }, 
  { 
    "slice_nr": 1, 
    "needle_driver": true, 
    "monopolar_curved_scissor": true, 
    "force_bipolar": false, 
    "clip_applier": false, 
    "tip_up_fenestrated_grasper": false, 
    "cadiere_forceps": false, 
    "bipolar_forceps": false, 
    "vessel_sealer": false, 
    "suction_irrigator": false, 
    "bipolar_dissector": false, 
    "prograsp_forceps": false, 
    "stapler": false, 
    "permanent_cautery_hook_spatula": false, 
    "grasping_retractor": false 
  } 
] 

where slice_nr is the frame number.

Adapting the container to your algorithm

1. First, clone this repository:

git clone https://github.com/aneeqzia-isi/surgtoolloc2022-category-1.git

2. Our Dockerfile should have everything you need, but you may change it to another base image/add your algorithm requirements if your algorithm requires it:

3. Edit process.py - this is the main step for adapting this repo for your model. This script will load your model and corresponding weights, perform inference on input videos one by one along with any required pre/post-processing, and return the predictions of surgical tool classification as a dictionary. The class Surgtoolloc_det contains the predict function. You should replace the dummy code in this function with the code for your inference algorihm. Use __init__ to load your weights and/or perform any needed operation. We have added TODO on places which you would need to adapt for your model

4. Run build.sh to build the container.

5. In order to do local testing, you can edit and run test.sh. You will probably need to modify the script and parts of process.py to adapt for your local testing. The main thing that you can check is whether the output json being produced by your algorithm container at ./output/surgical-tool-presence.json is similar to the sample json present in the main folder (also named surgical-tool-presence.json). PLEASE NOTE: You will need to change the variable execute_in_docker to False while running directly locally. But will need to switch it back once you are done testing, as the paths where data is kept and outputs are saved are modified based on this boolean. Be aware that the output of running test.sh, of course, initially may not be equal to the sample predictions we put there for our testing. Feel free to modify the test.sh based on your needs.

6. Run export.sh. This script will will produce surgtoolloc_trial.tar.gz (you can change the name of your container by modifying the script). This is the file to be used when uploading the algorithm to Grand Challenge.

Uploading your container to the grand-challenge platform

1. Create a new algorithm here. Fill in the fields as specified on the form.

2. On the page of your new algorithm, go to Containers on the left menu and click Upload a Container. Now upload your .tar.gz file produced in step 5. We will not accept submissions of containers linked to a GitHub repo.

3. After the Docker container is marked as Ready, you may be temped to try out your own algorithm when clicking Try-out Algorithm on the page of your algorithm. But doing so will likely fail. WARNING: Using this container in Try-out will fail. You can still use the Try-out feature to check logs from the algorithm and ensure that processes are running but it will not pass. However, if built correctly and you see the expected logs from your algorithm, then the container should still work for the Prelim submission.

4. WE STRONGLY RECOMMEND that you make at least 1-2 Prelim submissions before August 26th to ensure that your container runs correctly. Start earlier (Aug 19th) so we can help debug issues that may arise, otherwise there will be no opportunities to debug containers during the main submission!

5. To make a submission to one of the test phases. Go to the SurgToolLoc Challenge and click Submit. Under Algorithm, choose the algorithm that you just created. Then hit Save. After the processing in the backend is done, your submission should show up on the leaderboard if there are no errors.

The figure below indicates the step-by-step of how to upload a container:

If something does not work for you, please do not hesitate to contact us or add a post in the forum.

Acknowledgments

The repository is greatly inspired and adapted from MIDOG reference algorithm, AIROGS reference algorithm and SLCN reference algorithm