created a shell script to restart download upon failure

[mvish7]

Hi
I'm downloading the dataset using the contrib/99991/download.py script. The downloading got repeatedly interrupted due to an error of "remote not responding".

Hence I created a simple shell script to restart download upon failure.

[mikeroberts3000]

Hi, thanks for submitting this PR. When you get these remote not responding errors, what happens to the file that was being downloaded at the time of the error? The reason I'm asking is because the logic in download.py doesn't account for partially downloaded files. If such files are skipped (rather than resumed) on subsequent runs, it could lead to hard-to-diagnose errors.

[mvish7]

Hi
Sorry for a super late reply and thanks for the hint.

I inspected the download.py script and currently if the connection ends abruptly then the files will be partially downloaded. I introduced a way to catch these errors + to remove the partially downloaded file and download the same file. I have added two functions for this

• download_curr_file: This function together with added logic in download_files function handles removing partially downloaded file and downloading the same file until it is successfully downloaded.
• init_webfile: This function along with the added logic in download_files function catches the "bad zip file" errors.

With these changes I could successfully all the depth_meters.hdf5 and color.hdf5 files from ai_043_004 onwards. Hence I have removed the download.sh script which was responsible for restarting the download.py after failure.