The deeper layers of the Internet, better known as the Deep Web, are a huge virtual space that is beyond the reach of traditional search engines, and hide a number of activities where the main characteristic is anonymity.
For most Internet users, the term Deep Web suggests a separate world, like the dodgy neighbourhoods that appear in films, where illegal business goes on, where weapons of mass destruction are bought and all kinds of frauds and crimes committed.
With this in mind, the data provider Terbium Labs has carried out the first study using push data in order to find out what is really happening in the darkest and most remote areas of the Internet.
The study found that the volume of activity in the deep web is very similar to the volume that you can find on a normal web page. What is more, the study also says that 55% of the content registered in Deep Web is perfectly legal.
“We wanted to take a complete view of the dark web to determine its true nature and to offer readers of this report a holistic view of dark web activity — both good and bad”, said Emily Wilson, Director of Analysis at Terbium Labs.
To get all the data it needed and reach conclusions about the deep web, Terbium Labs based its study on data and statistical analysis. “The truth about the Dark Web: Separating fact from fiction” has involved a sampling of the darkest levels through the addition of billions of new records to its database to get accurate answers about what lies beyond the reach of the Internet, and that we know hardly anything about.
Big Data to get results
The results were obtained through the use of Big Data infrastructure and the skills needed to include pages in its searches that are hard to locate and hidden behind logins. This sampling has enabled Terbium Labs to understand better what is lurking in the depths of the Internet. By sampling an enormous number of pages rooted in the Deep Web it has been able to sort them and draw up a general likeness of the type of content that is present in the darkness.
Five key points
In the opinion of Terbium Labs, there are five major points in the study that help to understand a little better what is happening in this area that we know so little about:
- Anonymous does not mean criminal.
- Drugs are popular.
- There is a lot of pornography, but not all of it is illegal.
- Fraud
- There is hardly any extremism.
“Conducting research on the dark web is a difficult task because the boundaries between categories are unclear”, said Clare Gollnick, Chief Data Scientist at Terbium Labs, and added: “We put significant effort into making sure this study was based on a representative, random sample of the dark web. We believe the end result is a fair assessment of activity in this zone that few people know”.
To sum up, she says that you don’t surf the Deep Web, you dive into it. There are no search engines there, but there are reference points to start a search from, but you must be very careful about what you search for.
