Down the Virtual Rabbit Hole: The Deep Web

Unlock the mystery: What is the Deep Web?

The deep web consists of those portions of the internet that are not accessible directly through standard search engines. Ironically, 96% of the actual internet content is on the deep web. So what you view in your general search results is about 4% of the information actually available.

The deep web essentially contains non-indexed contents, dynamic contents, limited access data, private data, unlinked web elements and other similar contents that cannot be detected by the crawlers of common search engines.

How does the Deep Web work?

It is hard for search engine crawlers to find data in the deep web. This data isn't always concealed on purpose. More often than not, it is difficult for the existing search engines to discover or analyze it.

To understand the workings of deep web, we first need to comprehend how normal search engines work.

Search engines generally use automated crawlers that discover domains and then follow all hyperlinks on the page to create a virtual map (index) of the internet. When you perform a search, the engine actually searches its index for the entered keyword and displays all the links where it found that word.

Now suppose there was a web page within a domain that contains relevant and crucial information; but there was no direct hyperlink to reach that page from the main domain. In that case, the webpage will not be displayed in the search result. Such pages are a part of the deep web. Other deep web components include private websites that require login passwords to access, timed-access websites, domain-specific access websites, location-specific access websites. Also, private data such as research databases, medical records, banking transactions, social media feeds etc. are all deep web data. Such data cannot be penetrated by regular crawlers.

To access the content of the deep web, special methods and technologies are used. These include deep web search engines and special browsers. The deep web search engines perform search on the indexed data as well as the background databases. They override the time, domain and location privacy of web content. They also performs deep searches to find non-indexed and unlinked web pages. These search engines can even overcome the password or payment requirements and access the private content from websites and databases.

Image Source: Deepwebtech

Exploring the Deep Web

"The biggest weakness of the Deep Web is also its greatest strength: it's really hard to find anything." - Google

Now that you know what is the deep web and how it works, lets explore it in depth.

To perform deep web searches, you first need to find an optimal deep web search engine.

These search engines are the entry gateways to a limitless well of information.

Some of the most commonly used deep web search engines that provide excellent results are:

Torch: http://www.torchtorsearch.com/
WWW Virtual Library: http://vlib.org/
DuckDuckGo: https://duckduckgo.com/
Ahmia: https://ahmia.fi/

These are the search engines that you can directly access from your regular browsers. However, if you really want to dive deep in the Deep Web, you need specialized browsers like "TOR". Some of the extensive deep web search engines and services, for eg. the ".onion" websites, only work on TOR browsers. The normal browsers either block these search domains or cannot find them due to high privacy settings. The TOR specialized Deep Web search engines include:

Onion URL Repository
TorLinks
HiddenWiki

Image Source: Quora

TOR - The Onion Router

Onion Routing was originally developed by United States Naval Research Laboratories and further advanced by DARPA.

The alpha version of the TOR Project was launched in September 2002 and its public release occurred a year later.

TOR browser is now available as a free download for the general public to surf the web anonymously.

TOR is an anonymity network that aims to hide its' users location and online activity. It allows its users to browse the internet and communicate in relative privacy without the threat of eavesdropping, snooping, network monitoring etc.

TOR firstly encrypts the data to be transferred over the network by bundling it into packets. These packets are then stripped of the header that contains source address, making the sender untraceable. TOR provides further sender and receiver anonymity by routing your network traffic through the TOR network. The TOR network is an extensive and very complicate network of servers, bridges and relays that are specifically used to conceal the path of the data packets by bouncing it through multiple random TOR servers. As there is no source header in the packet, it becomes virtually impossible to trace the originating location or path of the information.

Thus TOR helps in providing anonymity and very high level of privacy to its users. It is used by military, journalists, whistle-blowers, political and human rights activists (especially in oppressed and dictatorship regions) and anyone who wishes to protect their online privacy. However, due to its properties, it is also often used by criminals and illegal traders.

TOR basically provides a platform for safe and anonymous web browsing. It is the duty of its users to not misuse its services for their own and others' security.

Image Source: whoishostingthis

Now, it is up to you how you explore the sea of information on the internet: surf the waves of the surface web or dive deep into the hidden depths of the deep web...

Read another amazing article "The power of Voice"

Download TOR here: https://www.torproject.org/download/download-easy.html.en