Skip to content

Commit

Permalink
Add Wayback Machine as a collector
Browse files Browse the repository at this point in the history
  • Loading branch information
jolle committed Sep 28, 2017
1 parent 7bc2069 commit b5e57b0
Showing 1 changed file with 22 additions and 0 deletions.
22 changes: 22 additions & 0 deletions lib/aquatone/collectors/wayback.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
require 'uri'

module Aquatone
module Collectors
class Crtsh < Aquatone::Collector
self.meta = {
:name => "Wayback Machine",
:author => "Joel (@jolle)",
:description => "Uses Wayback Machine by Internet Archive to find unique hostnames"
}

def run
response = get_request("http://web.archive.org/cdx/search/cdx?url=*.#{url_escape(domain.name)}&output=json&fl=original&collapse=urlkey")

response.parsed_response do |page|
if page[0] != "original"
add_host(URI.parse(page[0]).host)
end
end
end
end
end

0 comments on commit b5e57b0

Please sign in to comment.