What is the Invisible Web?

The Invisible Web (also called the deep web or hidden web) can be defined as Web pages that most search engines are not able to find. Components of the Invisible Web include, but are not limited to, phone numbers, E-Mail addresses, contents of library catalogs, some non-standard file formats, and most importantly the contents of Web-based databases such as ERIC or PubMed.

The Invisible Web contains some of the most authoritative information on the Internet. This information is almost always free. The problem is that the search engines can't find it.

Examples of Invisible Web Databases:

Library of Congress Catalog http://catalog.loc.gov

PubMed http://www.ncbi.nlm.nih.gov/sites/entrez

ERIC http://www.eric.ed.gov/

How Do I Find Information on the Invisible Web?

Mastering all elements of the Invisible Web would be a 60-hour per week job. The Invisible Web grows and changes as quickly as the Visible Web. We can all be thankful though, that some excellent guides to the Invisible Web exist.

About.Com: Invisible Web Directory http://websearch.about.com/od/invisibleweb/a/invisibleweb.htm

A selective directory of databases that search the invisible web, with good descriptions.

Complete Planet http://www.completeplanet.com

Tool for searching sites not usually accessed by other search engines.

Direct Search http://www.freepint.com/gary/direct.htm

Direct Search includes a large number of sites chiefly coverving academic information. A Librarian compiles Direct Search, so it is well organized. It is quite useful.

Where Can I find More Information About the Invisible Web?

University of California at Berkeley Teaching Library http://www.lib.berkeley.edu/TeachingLib/Guides/Internet/InvisibleWeb.html

This Web site does an excellent job of explaining why search engines cannot find everything on the Web, suggests strategies for exploiting the useful content of the Invisible Web, and evaluates various academic Invisible Web directories.