Dark Web Search and Scrape

Getting started

In the working directory of your top level python script:

mkdir darker
cd darker
git init
git add .
git remote add origin https://github.com/saadejazz/darker
git pull origin master

Install tor service on Ubuntu using

sudo apt install tor

Start the service using

sudo killall tor
tor

Make sure you have tor running before you run scripts. Also update the config file as following: Open /etc/tor/torrc and uncomment the following line

# ControlPort 9051

Install the following python packages using pip

• requests
• bs4
• stem
• lxml
• html2text
• validators
• regex

python -m pip install requests bs4 stem lxml html2text validators billiard

Dark Web Meta Searcher

Scrapes search results from the following dark web search engines:

• not Evil
• Dark Search
• Torch
• Ahmia
• Candle
• Tor66
• Visitor
• Dark Web Links
• Onion Land
• Haystack
• Deep Link
• Grams
• multiVAC
• Deep Paste

Code:

from darker.dark_search import DarkSearch

query = "guns"
results = DarkSearch().searchDarkWeb(query)
print(results)

Example Output:

{
  'title': 'Euro Guns - Number one guns dealer in onionland - Buy guns and ammo for Bitcoin.',
  'link': 'http://2kka4f23pcxgqkpv.onion/',
  'description': 'Buy guns for Bitcoin with Euroguns, best deep web arms dealer. Buy guns without a permit.',
  'score': 3
  }

The search results are sorted according to frequency of occurence (score). Python's multiprocessing is used to run requests in parallel.

Other Variants
To search using individual search engines:

from darker.dark_search import DarkSearch

query = "guns"
dark = DarkSearch()
# To view a list of correct site names
print(dark.listOfSites)

results = dark.search("tor66", query)
# or you can use the specific function
# View all functions using:
dark.sites
results = dark.visitor(query)
print(results)

To search a specific group of sites, you can either include or exclude sites from the search:

from darker.dark_search import DarkSearch

query = "guns"
results = DarkSearch().searchDarkWeb(query, include = ["not_evil", "ahmia", "dark_search"])
print(results)
# or
results = DarkSearch().searchDarkWeb(query, exclude = ["not_evil", "candle", "grams"])
print(results)

Note: Some onion sites may be down at times. In that case their results will not be fetched.

Dark Web generic scrapper

Scrapes a target web-page on the dark-web The scraper provides the following attributes as a result:

• Title
• Links
• Emails
• Images
• Text
• Bitcoin addresses

Code:

from darker.dark_scrape import DarkScrape

target = "http://4m6omb3gmrmnwzxi.onion/show.php?md5=f6c5d3bc1683338f103b18951b725551"
d = DarkScrape().scrape(target)
print(d.result) # for above-mentioned attributes
print(d.markdown) # for text in markdown format (thanks to html2text)

Output:

{
 'title': 'DeepPaste',
 'links': [],
 'emails': ['[email protected]'],
 'images': [],
 'text': '\r \r Hello Anon - Login\r DeepPaste\r Your Deep-Shit Hoster for special shit\r \r Results for f6c5d3bc1683338f103b18951b725551:hand guns and riffles Anon, March 27, 2019 - 4:32 am UTCbuy guns, full auto assault rifles, pistols, grenade launchers, etc.we ship all around america and europe.for more info or questions contact us via [email protected]: 1937 \xa0\xa0 Voting: 0 \xa0 ↑ Up \xa0 ↓ DownLogin to voteComments:-__________________________________Add a comment:Name: Anon  Captcha:    \r \r \r Last Public Pastes\r Top Last Public Pastes\r Search Pastes... \r Infos about DeepPaste\r \r For new tea:BTC: 14US287mkpaMYFszkSw2dcEZtdMi3UQak6\r Views Today: 110.432 - Views Yesterday: 337.759\r \r \r \r ',
 'bitcoin': []
 }