How To Find Hidden Web Directories Using Dirsearch

When a security analyst performing website penetration testing the initial step should be finding hidden directories of a vulnerable website.

These hidden web directories are essential because they can give useful information i.e. potential attack vectors that would not be visible on the public facing website.

One of the ways to achieve this is by attempting brute-forcing site structure that includes directories and files in websites and for that, you have to choose a powerful tool.

Although there are many tools available used to perform site brute-forcing includes Dirbuster or Dirb but these have their own limitation such as Dirbuster only offers GUI interface that is not feasible all the time and Dirb does not include multithreading feature.

The most popular choice among penetration testers for website brute-forcing is Dirsearch.

Dirsearch, written in python is a command-line website directory scanner. It has a lot of features making it the complete winner in terms of performance:

  1. It includes Multithreading, making it faster than any other site scanner tool
  2. It performs Recursive brute-forcing
  3. It has HTTP proxy support
  4. Dirsearch effectively detects invalid web pages
  5. It has User agent randomization and Batch processing
  6. Supports Request delaying

This tool can be run on any operating system (Windows, Linux, mac) making it more compatible and simpler, yet a powerful tool.

In this setup we will be using Kali Linux as an attacking machine and DVWA on Metasploitable 2 as the target.

Install Dirsearch

You need to install Dirsearch in your Kali Linux attacking machine. First update all the repositories from the command # apt-get update

After that install Dirsearch from Github by using the link: https://github.com/maurosoria/dirsearch

Type command:

# git clone https://github.com/maurosoria/dirsearch

to install Dirsearch in your Kali Linux attacking machine

After successfully installing the tool, change the root directory into Dirsearch.

# cd dirsearch/

to install Dirsearch in your Kali Linux attacking machine

Type ls to see the content under Dirsearch directoy.

to install Dirsearch in your Kali Linux attacking machine

You can see there is a configuration file and a python file named ‘dirsearch.py’ which we are going to execute.

Configuring the Tool

Next step will be configuring the Dirsearch. You can do it in three ways.

As this ‘dirsearch.py’ file needs python 3 to execute correctly, just simply run it with python.

# python3 dirsearch.py

configuring the Dirsearch

It gives a usage example, stating we need to specify a valid URL.

The second way to run Dirsearch is to run it with Bash. Type ls -la to give us the permissions of everything in this directory.

configuring the Dirsearch

As we can see from above that this tool is executable, now run it using the dot-slash.

# ./dirsearch.py

configuring the Dirsearch

The third and the easiest way to run Dirsearch is by creating a symbolic link in the /bin directory as this will permit us to run this tool not only with the directory cloned from GitHub but anywhere in the prompt.

Change into the bin directory and create a symbolic link naming it ‘dirsearch’ by using the ln -s command.

# ln -s ln -s ~/dirsearch/dirsearch.py dirsearch

configuring the Dirsearch

Site Scanning Using Dirsearch

As we have created the symbolic link, just type dirsearch in any directory to execute.

Site Scanning Using Dirsearch

Use the flag ‘-h’ with the dirsearch to get full help menu that will show a ton of options and potential configuration settings.

Site Scanning Using Dirsearch

Site Scanning Using Dirsearch

So now for searching the site it needs a valid target’s URL and a file extension to run. You can specify a valid URL with the -u flag, and a file extension to search for with the -e flag.

In our case we will give it the URL of our DVWA on Metasploitable2 machine.

# dirsearch -u http://192.168.0.102/dvwa -e php

Site Scanning Using Dirsearch

Site Scanning Using Dirsearch

This showing us a lot of information regarding  the extensions, HTTP methods in use, number of threads, and size of the current wordlist. After that it starts to dig into the directories and returns with its findings that includes the status code, size, and directory name.

If you want to exclude certain HTTP status codes use -x flag.

# dirsearch -u http://192.168.0.102/dvwa -e php -x 403

Site Scanning Using Dirsearch

Site Scanning Using Dirsearch

Here we are excluding all the 403 codes by specifying after -x flag. It leads us to get what we required, making it cleaner and simpler.

Also, you can use -w flag to use a wordlist of your choice.

# dirsearch -u http://192.168.0.102/dvwa -e php -x 403,301,302 -w /usr/share/wordlists/wfuzz/general/common.txt

Site Scanning Using Dirsearch

You can see it does not find a large pool of results with this mentioned wordlist because the size is smaller.

Now coming to its best feature i.e. performing recursive directory scanning. Use -r flag to run the recursive search.

# dirsearch -u http://192.168.0.102/dvwa -e php -x 403,301,302 -r

Site Scanning Using Dirsearch

Site Scanning Using Dirsearch

Now see there is an additional entry of ‘Recursive Level’. On completing the initial scan this will go back and starts scanning each directory it found recursively, in this case it starts config/ then docs/ and so on.

While scanning if you want to exit the scan parse ‘e’ from keyboard. To continue it from the point you have stopped, parse ‘c’. Parse ‘n’ to move to the next directory. These steps will let you have a control over the results as recursive scanning is a time-consuming process.

If you want to set the recursion level to a deeper value, you can use the -R flag and a value of how many levels deep you want to dig.

# dirsearch -u http://192.168.0.102/dvwa -e php -x 403,301,302 -r -R 3

Site Scanning Using Dirsearch

Site Scanning Using DirsearchSo, you can see this starts scanning the /administrator directory which is obviously could not be found on the top level.

Conclusion

This article demonstrates how to use an effective web directory brute-forcing tool Dirsearch to scan and search hidden web directories which may not be visible to a user. This is very a very first task in penetration tasting and Dirsearch do this job much faster than the traditional Dirbuster and Dirb.

Irfan Shakeel
Irfan Shakeel, the founder of ehacking project, he also hosts cyber security training classes at EH Academy. He has discovered many vulnerabilities in the famous platforms (like Google, Dailymotion, Harvard University & etc.). He specializes in Network hacking, VoIP pentesting & digital forensics. He is the author of the book title “Hacking from Scratch”.

Most Popular

Security Against Hacks: A Simple Game of Economics

One of the cold hard truths behind cybersecurity is that it's impossible to prevent a hack 100% of the time. It only takes one...

Certain Things That You Must Know About Microsoft MS-500 Exam and Its Practice Tests

If you want to be a Microsoft 365 security administrator, then there would be a number of responsibilities that will fall on your shoulders,...

Quick Ways to Avoid Being Watched by the NSA’s PRISM Program

Big brother is on the watch online. Today, a week hardly passes without news of government spying, whistleblowers, cell phone hacking, or even private photos...

Android Post Exploitation: Exploit ADB using Ghost Framework in Kali Linux

This article is the part of Android Hacking tutorial; it covers step by step guide to exploit Android ADB to get the persistent connection...

LOOKING FOR HACKING RECIPES FORM THE PRO?

Then sign up for FREE to the ehacking’s exclusive group. You will get the exclusive tips/tricks, tutorials, webinars & courses that I ONLY share with my fellow on this exclusive newsletter.