humble

HTTP Headers Analyzer

"A journey of a thousand miles begins with a single step. - Lao Tzu"

"And if you don't keep your feet, there's no knowing where you might be swept off to. - Bilbo Baggins"

Table of contents

Features
Screenshots
Installation & Update
Usage
Advanced Usage
 Linux: Show only the analysis summary
 Windows: In spanish. Show only the analysis summary (PowerShell >= 7 required)
 Linux: Show only the URL, date and analysis summary
 Linux: Show only the deprecated headers/protocols and insecure values
 Linux: Check for HTTP client errors (4XX)
Caveats
Checks: Missing Headers
Checks: Fingerprint Headers
Checks: Deprecated Headers and Insecure Values
Checks: Empty Values
Guidelines included
To-Do
Further Reading
Contribute
Acknowledgements
License

Features

✔️ 13 checks of missing HTTP response headers.
✔️ 749 checks of fingerprinting through HTTP response headers.
✔️ 61 checks of deprecated HTTP response headers/protocols or with values considered insecure.
✔️ Browser compatibility check for enabled security headers.
✔️ Two types of analysis: brief and detailed, along with HTTP response headers.
✔️ Export of analysis to HTML5, PDF 1.4 and TXT.
✔️ The analysis includes dozens of references, official documentation and technical articles.
✔️ i18n: analysis results in English or Spanish.
✔️ Saves each analysis, showing (at the end) the improvements or deficiencies in relation to the last one.
✔️ Shows statistics of scans performed against a URL.
✔️ Code reviewed via pycodestyle, SonarLint and Sourcery.
✔️ Tested, one by one, on thousands of URLs.
✔️ Fully tested and working on Windows (10 20H2 - 19042.985) and Linux (Kali 2021.1).
✔️ All code under one of the most permissive licenses: MIT.
✔️ Regularly updated.
✔️ Technical resource accepted in the OWASP Secure Headers Project and Kali.

Screenshots

.: Brief analysis (Windows)

.: Brief analysis and retrieved headers (Linux)

.: Detailed analysis (Linux) in Spanish.

.: Detailed analysis exported to PDF. Example.

.: Detailed analysis exported to HTML. Example.

.: Example of history file: Date, URL, Missing, Fingerprint, Deprecated/Insecure, Empty headers & Total warnings (the four previous totals).

.: Example of statistics of the analyses performed.

Installation & Update

NOTE: Python 3.9 or higher is required.

# install python3 and python3-pip if not exist
(Windows) https://www.python.org/downloads/windows/
(Linux) if not installed by default, install them via, e.g. Synaptic, apt, dnf, yum ...

# install git
(Windows) https://git-scm.com/download/win
(Linux) https://git-scm.com/download/linux

# clone the repository
$ git clone https://github.com/rfc-st/humble.git

# change the working directory to humble
$ cd humble

# install the requirements
$ pip3 install -r requirements.txt

# update humble (every week, inside humble's working directory)
$ git pull

# or download the latest release
https://github.com/rfc-st/humble/releases

Usage

(Windows) $ py humble.py
(Linux)   $ python3 humble.py

usage: humble.py [-h] [-a] [-b] [-g] [-l {es}] [-o {html,pdf,txt}] [-r] [-u URL] [-v]

humble (HTTP Headers Analyzer) - https://github.com/rfc-st/humble

options:
  -h, --help         show this help message and exit
  -a                 Show statistics of the analysis performed on the specified URL.
  -b                 Show a brief analysis; if omitted, a detailed analysis will be shown.
  -g                 Show guidelines on securing most used web servers/services.
  -l {es}            Displays the analysis in the indicated language; if omitted, English will be used.
  -o {html,pdf,txt}  Save analysis to file (URL_yyyymmdd.ext).
  -r                 Show HTTP response headers and a detailed analysis.
  -u URL             URL to analyze, with schema. E.g., https://google.com
  -v, --version      show version

Advanced Usage

Linux: Show only the analysis summary

$ python3 humble.py -u https://tesla.com | grep -A 8 "\!." | sed $'1i \n'

Windows (in Spanish): show only the analysis summary (PowerShell >= 7 required)

$ py humble.py -u https://tesla.com -l es | Select-String -Pattern '!.' -Context 1,8 -NoEmphasis

Linux: Show only the URL, date and analysis summary

$ python3 humble.py -u https://tesla.com | grep -A7 -E "0. Info|\!." | grep -v "^\[1\." | sed 's/[--]//g' | sed -e '/./b' -e :n -e 'N;s/\n$//;tn' | sed $'1i \n'

Linux: Show only the deprecated headers/protocols and insecure values

$ python3 humble.py -u https://tesla.com | sed '/3. /,/4. /!d' | sed '$d' | sed $'1i \n' 

Linux: Check for HTTP client errors (4XX)

$ python3 humble.py -u https://block.fiverr.com | grep -B5 'Note : \|Nota : ' --color=never 

Caveats

Country and suffix errors (TLDs)

These checks may generate errors in internal networks, or development environments, that do not have connectivity to https://ipapi.co.

To avoid the errors, you can replace the following code here:

sffx = tldextract.extract(URL).suffix[-2:].upper()
cnty = requests.get('https://ipapi.co/country_name/').text.strip()
if (sffx in ("UA", 'RU') and sffx not in NON_RU_TLDS) or cnty in ('Ukraine',
                                                                  'Russia'):
    ua_ru_analysis(sffx, cnty)
else:
    if not args.URL_A:
        detail = '[analysis_output]' if args.output else '[analysis]'
        print("")
        print_detail(detail)

with this code:

if not args.URL_A:
    detail = '[analysis_output]' if args.output else '[analysis]'
    print("")
    print_detail(detail)

Checks: Missing Headers

Show / Hide

Cache-Control	Clear-Site-Data	Content-Type
Content-Security-Policy	Cross-Origin-Embedder-Policy	Cross-Origin-Opener-Policy
Cross-Origin-Resource-Policy	NEL	Permissions-Policy
Referrer-Policy	Strict-Transport-Security	X-Content-Type-Options
X-Frame-Options

Checks: Fingerprint headers

Check this file.

Checks: Deprecated headers/protocols and insecure values

Check this file.

Checks: Empty values

Any HTTP response header.

Guidelines included to enable security HTTP headers

• Amazon AWS
• Apache HTTP Server
• Cloudflare
• MaxCDN
• Microsoft Internet Information Services
• Nginx

To-do

• Add more header/value checks (only security-oriented)

Further reading

https://caniuse.com/
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers
https://github.com/search?q=http+headers+analyze
https://github.com/search?q=http+headers+secure
https://github.com/search?q=http+headers+security
https://owasp.org/www-project-secure-headers/
https://securityheaders.com/
https://scotthelme.co.uk/
https://webtechsurvey.com/common-response-headers
https://www.w3.org

Contribute

• Report a Bug.
• Create a Feature request.
• Report a Security Vulnerability.
• Send me an email with your suggestions!: [email protected]

Thanks for your time!! :).

Acknowledgements

• Azathothas for reporting this bug.
• İDRİS BUDAK for reporting the need to this check.
• Eduardo, for making possible the first Demo ^^.

License

MIT © 2020-2023 Rafa 'Bluesman' Faura ([email protected])
Original Creator - Rafa 'Bluesman' Faura ([email protected])