Added Duplicate File Finder

Author: vybhav72954

Duplicate-File-Finder/README.md

@@ -0,0 +1,81 @@
+# Duplicate File Finder
+
+[![forthebadge made-with-python](http://ForTheBadge.com/images/badges/made-with-python.svg)](https://www.python.org/)
+
+Many a time, we find duplicate files residing in our Directories, especially documents and downloads, there are various
+reasons: 
+
+- downloading the same file from various sources.
+- auto backup on the cloud,
+- it slipped out of our mind that we downloaded it already in the first place, etc.
+
+Manually selecting them is actually a hassle, but why do such a boring task when
+automation can do the trick. This sweet and simple script helps you to compare various files in a
+directory, find the duplicate, list them out, and then even allows you to delete them.
+
+**Sweet!!!**
+
+## Setup
+
+- Setup a `python 3.x` virtual environment.
+- `Activate` the environment
+- Install the dependencies using ```pip3 install -r requiremnts.txt```
+- You are all set and the [script](file_finder.py) is Ready to run.
+- Clearly Follow the Instructions provided in the comments.
+
+### Usage
+
+In Command Line Interface, Run the script using -
+
+`python image_finder.py <path of folder1, path  of folder2, .....>`
+
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 1. folder1 - *Parent Folder*
+
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2. folder2, folder3 .... - *Subsequent Folders*
+
+>- This acts as a reference for duplicate files, i.e. this contains the original copy, hence no file is deleted from this folder.      
+>- Comparisons are done with in the folder, and from Parent to Subsequent Folders.
+
+## Dependencies
+
+ 1. python3
+ 2. keyboard
+
+## Detailed explanation
+
+The Script works on a simple fundamental. Two files with same [`md5checksum`](https://en.wikipedia.org/wiki/MD5) will
+have similar contents. So in the script all we aim to do is determine the checksum, compare and find the duplicates.
+
+## Output
+
+- Running Script on a single folder `Stand_Alone`. In this example I pressed [n] in order to not delete anything.
+
+![Pasting the Magnet Link](https://i.imgur.com/pcABYx4.png)
+
+- Stand_Alone folder Before Deleting the files.
+
+![Pasting the Magnet Link](https://i.imgur.com/PqwxrPQ.png)
+
+- After Deleting the Files, i.e. Pressing [y] at the prompt.
+
+![Pasting the Magnet Link](https://i.imgur.com/34fR6w3.png)
+
+- `Parent`, `Duplicate`, `Duplicate_1` folder before running the script.
+
+![Pasting the Magnet Link](https://i.imgur.com/XcJ0we3.png)
+
+- Running the scripts on the Folder and deleting the duplicate files.
+
+![Pasting the Magnet Link](https://i.imgur.com/ZaEcroF.png)
+
+![Pasting the Magnet Link](https://i.imgur.com/tfo2day.png)
+
+- Final Result, Notice that all the files in `Parent` Folder remain as it is.
+
+>Also notice that similar files but wth different extensions are not deleted, cause technically they aren't same.
+
+![Pasting the Magnet Link](https://i.imgur.com/d8VXy5m.png)
+
+## Author(s)
+
+Made by [Vybhav Chaturvedi](https://www.linkedin.com/in/vybhav-chaturvedi-0ba82614a/)

Duplicate-File-Finder/file_finder.py

@@ -0,0 +1,119 @@
+# Imports
+import hashlib
+import os
+import sys
+import keyboard
+
+
+def image_finder(parent_folder):
+    # A dictionary to store Hash of Images corresponding to names
+    """
+    Sample -
+    {hash:[names]}
+    """
+    duplicate_img = {}
+    for dirName, subdirs, fileList in os.walk(parent_folder):
+        # Iterating over various Sub-Folders
+        print('Scanning %s...' % dirName)
+        for filename in fileList:
+            # Get the path to the file
+            path = os.path.join(dirName, filename)
+            # Calculate hash
+            file_hash = hash_file(path)
+            # Add or append the file path in the dictionary
+            if file_hash in duplicate_img:
+                duplicate_img[file_hash].append(path)
+            else:
+                duplicate_img[file_hash] = [path]
+    return duplicate_img
+
+
+def delete_duplicate(duplicate_img):
+    # Deleting those values whose keys are not unique
+    for key in duplicate_img:
+        file_list = duplicate_img[key]
+        while len(file_list) > 1:
+            item = file_list.pop()
+            os.remove(item)
+
+
+# Joins two dictionaries
+def join_dicts(dict1, dict2):
+    for key in dict2.keys():
+        if key in dict1:
+            dict1[key] = dict1[key] + dict2[key]
+        else:
+            dict1[key] = dict2[key]
+
+
+# For finding Hash of various Files
+# If 2 files have the same md5checksum,they most likely have the same content
+def hash_file(path, blocksize=65536):
+    img_file = open(path, 'rb')
+    hasher = hashlib.md5()
+    buf = img_file.read(blocksize)
+    while len(buf) > 0:
+        hasher.update(buf)
+        buf = img_file.read(blocksize)
+    img_file.close()
+    # Return Hex MD5
+    return hasher.hexdigest()
+
+
+def print_results(dict1):
+    results = list(filter(lambda x: len(x) > 1, dict1.values()))
+    if len(results) > 0:
+        print('Found Duplicated Images - ')
+        print('Details -')
+        print('<--------------------->')
+        for result in results:
+            # Print Path of Files
+            for subresult in result:
+                print('\t%s' % subresult)
+            print('<--------------------->')
+
+    else:
+        print('Unable to identify Similar Images')
+
+
+if __name__ == '__main__':
+    if len(sys.argv) > 1:
+        duplicate = {}
+        folders = sys.argv[1:]
+        for i in folders:
+            # Iterate the folders given
+            if os.path.exists(i):
+                # Find the duplicated files and append them to the dictionary
+                join_dicts(duplicate, image_finder(i))
+            else:
+                print('%s is not a valid path, please verify' % i)
+                sys.exit()
+        print_results(duplicate)
+        # Delete Duplicate Images
+        # Comment if not required
+        print("Do you want to delete the Duplicate Images (If Any)? Press [y] for Yes.")
+        while True:
+            if keyboard.read_key() == "y":
+                print("Deleting Duplicate Files\n")
+                delete_duplicate(duplicate)
+                print("Thank You\n")
+                break
+            else:
+                print("Nothing Deleted!!! Thank You\n")
+                break
+    else:
+        print("Use Command Line Interface")
+        print("Hint: python file_finder.py <path of folders>")
+        print("Please Read comments for greater detailing")
+        '''
+        Suggestions :------ 
+        Usage - python file_finder.py <path of folder1, path  of folder2, .....>
+        folder1 - Parent Folder
+        folder2, folder3 .... - Subsequent Folders
+        Comparisons are done with in the folder, and from Parent to Subsequent Folders. 
+
+        No Files are deleted form Parent Folder but the files which are Duplicate to the files in Subsequent Folders are
+        deleted. Make sure that the paths are correct 
+        
+        Be careful during Keyboard Input.
+        '''

Duplicate-File-Finder/requirements.txt

@@ -0,0 +1 @@
+keyboard==0.13.5