A Writers Guide to The Deep Web

Mar 22
09:12

2008

Mervyn Love

Mervyn Love

  • Share this article on Facebook
  • Share this article on Twitter
  • Share this article on Linkedin

You've got a brilliant idea for an article, but your research is pulling up only scant information. What do you do? Dive into the Deep Web!

mediaimage

The Library's closed and you need some facts to finish your article and you need them NOW! It's time to search the Deep Web.

The Deep Web,A Writers Guide to The Deep Web Articles or Silent Web as it's sometimes called, is a hidden resource that every writer should know about. It could well be the answer when you want to track down some facts that Google and other search engines can't.

It is estimated that there may be as many as 500 million web pages that are indexed by Google, MSN, Yahoo and others. The Deep Web is actually not included in these and is reckoned to be many times greater in size!

The visible web is indexed by 'spiders', small robot programs that wander about from site to site gathering details on links and pages to send back to base from which the search engine's index is compiled.

Unfortunately many websites can't be accessed by spiders. The spider can call at the door, index the address, but can't get in to record the contents. The reasons for this are varied, and include the fact that website owners can deliberately ban spiders from entering their sites if they wish. Another is that if the site comprises of databases, these are anathema to spiders which can't do anything with them. And it is sites like these, made up of databases full of fascinating information on an bewildering variety of subjects, that mainly comprise the Deep Web.

There are thousands, maybe even millions, of such databases out there on the web, but you need to know where they are and what their address is in order to get into them.

The Deep Web is recognised as producing a much higher quality of information than regular search engines due to the many universities and technical and scientific institutions that contribute.

Some databases, such as SJLibrary.org (a collaboration between San Jose State University and San Jose Public Libraries) restrict access to part or all of their content to their students and faculties, but there are plenty of others that do not.

So, it's time to dip your toes into this vast ocean and see what you can find. To start you off here are some sites to try:

Direct Search is a good one (http://www.freepint.com/gary/direct.htm)

Intute - I found this an excellent site http://www.intute.ac.uk/

Complete Planet http://aip.completeplanet.com/

InfoMine http://infomine.ucr.edu/

FindArticles is a database of - you guessed it - articles http://findarticles.com/

For more information on the subject of Deep Web go to deepweb.com http://www.deepweb.com/

Categories: