Hakrawler in Termux – Web Crawling Commands

Hakrawler is a fast web crawling tool that helps discover URLs, endpoints, JavaScript files, and other publicly accessible resources from websites. It is commonly used by cybersecurity learners, bug bounty hunters, and penetration testers during reconnaissance and information gathering activities. You can install and use Hakrawler in Termux to crawl websites and collect useful URLs from your Android device.

Here’s what you can do with Hakrawler in Termux:

• Discover URLs from websites.
• Find hidden endpoints.
• Collect website links.
• Crawl web pages automatically.
• Gather reconnaissance information.
• Learn web crawling techniques.

Install Hakrawler in Termux

Below are the commands to install Hakrawler in Termux. Copy and run each command one by one to install the tool.

Update Termux packages.

pkg update && pkg upgrade -y

pkg update && pkg upgrade -y

Install Git and Golang.

pkg install git golang -y

pkg install git golang -y

Download and install Hakrawler.

go install github.com/hakluke/hakrawler@latest

go install github.com/hakluke/hakrawler@latest

Add the Go binary directory to PATH.

echo 'export PATH=$PATH:$HOME/go/bin' >> ~/.bashrc && source ~/.bashrc

echo 'export PATH=$PATH:$HOME/go/bin' >> ~/.bashrc && source ~/.bashrc

Check whether Hakrawler is installed successfully.

hakrawler

hakrawler

Hakrawler is now installed and ready to use.

Use Hakrawler in Termux

Hakrawler accepts target URLs through standard input. Below are some useful commands for crawling websites and collecting URLs.

Crawl a website using the default settings.

echo https://example.com | hakrawler

echo https://example.com | hakrawler

Crawl a website with a depth of 3.

echo https://example.com | hakrawler -d 3

echo https://example.com | hakrawler -d 3

Include subdomains during crawling.

echo https://example.com | hakrawler -subs

echo https://example.com | hakrawler -subs

Show only unique URLs.

echo https://example.com | hakrawler -u

echo https://example.com | hakrawler -u

Display URLs in JSON format.

echo https://example.com | hakrawler -json

echo https://example.com | hakrawler -json

Show where each URL was discovered.

echo https://example.com | hakrawler -w

echo https://example.com | hakrawler -w

Display the source type of each URL.

echo https://example.com | hakrawler -s

echo https://example.com | hakrawler -s

Set the number of crawling threads.

echo https://example.com | hakrawler -t 20

echo https://example.com | hakrawler -t 20

Set a timeout value in seconds.

echo https://example.com | hakrawler -timeout 30

echo https://example.com | hakrawler -timeout 30

Disable TLS certificate verification.

echo https://example.com | hakrawler -insecure

echo https://example.com | hakrawler -insecure

Use a proxy server.

echo https://example.com | hakrawler -proxy http://127.0.0.1:8080

echo https://example.com | hakrawler -proxy http://127.0.0.1:8080

Use custom HTTP headers.

echo https://example.com | hakrawler -h "Cookie: foo=bar"

echo https://example.com | hakrawler -h "Cookie: foo=bar"

Common Hakrawler Options

• -d: Sets the crawling depth.
• -subs: Includes subdomains during crawling.
• -u: Shows only unique URLs.
• -json: Displays output in JSON format.
• -s: Shows the source of each discovered URL.
• -w: Shows where the URL was found.
• -t: Sets the number of threads.
• -timeout: Sets the maximum crawl time.
• -proxy: Uses a proxy server.
• -insecure: Disables TLS certificate verification.

Choose the command you want and Hakrawler will automatically crawl the target website and display the discovered URLs inside the Termux terminal.

End Note

Hakrawler is a simple tool that fetches website URLs in just a few seconds using simple commands. You can install and use Hakrawler in Termux to discover website links, gather reconnaissance data, and learn web crawling from your Android device.