TODO: New features

[shelld3v]

Hi, here are several features that we need to create in the future

• HTML report (Html Report [WIP] #827)
• SQLite report (New features, improvements, fixes #1011)
• XML report (New feature: Support XML report #525)
• Markdown report (New feature: Markdown report #553)
• CSV report (New feature: CSV report + Bugs fix #581)
• Deep recursive scan: /manager/admin.php -> add /manager/ (Several updates #804)
• Exclude response sizes (New feature: Exclude responses by sizes #523)
• Headers from file (Headers from file #507)
• Raw request (New: Raw HTTP request + default URL scheme + bug fix #724)
• No extension mode (New feature: No extension #459)
• Only entries with selected extensions (New feature: Only selected extensions #540)
• Replay proxy (New feature: Matches proxy #569)
• Option to exit whenever there is an error (New feature: Stop on any error case #503)
• Support SOCKS proxy (Support SOCKS proxy + Update CHANGELOG.md #497)
• Support CIDR (New feature: Support CIDR #558)
• Max requests rate (New feature: max request rate #801)
• Supports "dirsearch scripts", like nmap NSE
• Filter responses by redirects (New feature: Exclude responses by redirects #683)
• URLs from STDIN (New feature: URLs from stdin #701)
• Max runtime (Several updates #784)
• Wordlists based on directories (js/ then js-wordlist.txt)
• Recursion status codes (New feature: recursion status codes #790)
• Multiple authentication types (Support more auth types + update HTML report #819)
• Auto detect URI scheme, https on first priority (New feature: auto detect URI scheme #983)
• Show redirects history (New features, improvements, fixes #1011)
• Refactor batch reporting

[Phoenix1112]

hi. When we try only certain paths on collective targets, I suggest introducing a new feature to hide the banner displayed for each new target. For example, I want scan only "/api/v1/users" endpoint on 1000 target sites. Since we are using only one endpoint, a new banner is printed on the screen for each target URL address.

example targets:

https://www.google.com
https://facebook.com
https://instagram.com
https://twitter.com
https://github.com
https://gitlab.com
https://tesla.com
https://starbucks.com
https://telegram.com

example wordlist:

api/v1/users

EXample results.

Looking at the picture above, you can see that a new banner is printed for each url address and a new save file is created for each new destination. With thousands of targets, this situation will be even more complicated. Therefore, a new feature should be added for such situations. For example, while doing a normal scan, we see that the words in the wordlist change on the screen after trying them. When the next word in the wordlist is tried, we see that the words called on the screen change. something similar should be within the target url addresses. When the program is started, a single banner will be printed and then the url addresses and words will change and continue to scan on the same line. Such a feature can be introduced for a clean output.

[shelld3v]

Hi, first of all, thank you for your suggestion, I appreciate it! About your problem, I want to say that dirsearch is a tool used to brute force directories, not brute force websites, there is even a tool for this purpose: meg. And of course, there is still a solution for this situation: use the --quiet flag!

Any other question, PLEASE create a new issue! Thanks.

[Phoenix1112]

thanks for mag.. I want to make a final and new feature request.

Unfortunately, some websites return status codes such as 429 or 403 when too many requests are made. the feature that I will suggest now is a feature that will make the dirsearch program number one in the world. In such cases, it is useful to use a proxy. I know dirsearch works with the tor network. However, when you scan with a long wordlist, if the tor network has not changed your ip address again, we can be blocked by the target site again. For this, the ip change feature used with the tor network should be added to the dirsearch program.

For example, let's say the target site only allows 100 requests per minute. The user must be able to change her ip address after every 70 requests to be not blocked by the target site. The ip change limit should depend on the request range specified by the user. Such dirsearch program will continue to scan with a new ip address after every 70 requests.This may be a difficult feature for you at first, but you can achieve it with fairly simple code. To understand this easily, I want to show you a simple code below that I made with python.

from torrequest import TorRequest
import requests
import sys

tr=TorRequest()

sayı = int(sys.argv[1])

for x in range(0,50):
    if x == sayı:
        tr.reset_identity()
        tr=TorRequest()
        response= tr.get('http://ipecho.net/plain')

        print ("New Ip Address",response.text)
        sayı += 10
    response= tr.get('http://ipecho.net/plain',allow_redirects=True,timeout=20).text
    print(response)

I want my IP address to change after every 10 requests and I start my program like this.

You can add this feature with the command --proxy sock: //127.0.0.1: 9050 --change-ip 10

[shelld3v]

Hi, actually dirsearch supports TOR, by using socks proxy: socks5://127.0.0.1:9050

[Phoenix1112]

Yes I know that. so i said your program supports the tor network. That's why I wanted the feature of changing ip address to be added.

[shelld3v]

But we haven't thought about working with Tor configuration yet!

[Phoenix1112]

I understand you. even so, thank you for listening to me. I wish you continued success.🙋🙋

[nairanu9888]

Hey! can you provide me more information about "Option to save raw responses into a folder". I would like to work towards this feature.

[shelld3v]

Hey! can you provide me more information about "Option to save raw responses into a folder". I would like to work towards this feature.

Thanks for your interest, I am having 2 ideas for this feature:

• Let say dirsearch finds /apple.php and /html/index.html, I want it to save the content of /apple.php (from the server response) to /example/apple.php. With the second path, it creates a folder named html inside /example and create the file index.html with the content from the response (Note: example/ is a random folder that the user selected)
• Of course, there are some problems with the idea above, such as: what will we save if it responds with 302 redirection? So maybe we can do something simple like creating apple.php.txt with full HTTP response and request inside it, something like this:

/example/apple.php.txt

GET /apple.php HTTP/1.1
Host: exmaple.com
User-Agent: h1-username
Accept: */*
X-Forwarded-For: 127.0.0.1

HTTP/2 200
Date: Sun, 12 Dec 2021 07:21:16 GMT
Content-Type: text/html
Content-Length: 121
Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Server: cloudflare
Cf-ray: 6bc531e8a8d4195f-HKG

<h1>Apple</h1>

The reason why I came up with this idea is that some people reported to me when they visited the paths, it didn't show the same things as dirsearch showed in output, maybe because there were changes from the server, or they can't reproduce the same request as dirsearch. Right now I'm thinking about creating features for both ideas, but we can create only one (the second idea)