fbpx

The internet is an iceberg. And, as you might guess, most of us only reckon with the tip. While the pages and media found via simple searches may seem unendingly huge at times, what is submerged and largely unseen – often referred to as the invisible web or deep web – is in fact far, far bigger.

THE SURFACE WEB

What we access every day through popular search engines like Google, Yahoo or Bing is referred to as the Surface Web. These familiar search engines crawl through tens of trillions of pages of available content (Google alone is said to have indexed more than 30 trillion web pages) and bring that content to us on demand. As big as this trove of information is, however, this represents only the tip of the iceberg.

Eric Schmidt, the CEO of Google, was asked to estimate the size of the World Wide Web. He estimated that of roughly 5 million terabytes of data, Google has indexed roughly 200 terabytes, or only .004% of the total internet.

THE INVISIBLE WEB

Beneath the Surface Web is what is referred to as the Deep or Invisible Web. It is comprised of:

  • Private websites, such as VPN (Virtual Private networks) and sites that require passwords and logins
  • Limited access content sites (which limit access in a technical way, such as using Captcha, Robots Exclusion Standard or no-cache HTTP headers that prevent search engines from browsing or caching them)
  • Unlinked content, without hyperlinks to other pages, which prevents web crawlers from accessing information
  • Textual content, often encoded in image or video files or in specific file formats not handled by search engines
  • Dynamic content created for a single purpose and not part of a larger collection of items
  • Scripted content, pages only accessible using Java Script, as well as content downloaded using Flash and Ajax solutions

There are many high-value collections to be found within the invisible web. Some of the material found there that most people would recognize and, potentially, find useful include:

  • Academic studies and papers
  • Blog platforms
  • Pages created but not yet published
  • Scientific research
  • Academic and corporate databases
  • Government publications
  • Electronic books
  • Bulletin boards
  • Mailing lists
  • Online card catalogs
  • Directories
  • Many subscription journals
  • Archived videos
  • Images

But knowing all these materials are out there, buried deep within the web doesn't really help the average user. What tools can we turn to in order to make sense of the invisible web? There really is no easy answer. Sure, the means to search and sort through massive amounts of invisible web information are out there, but many of these tools have an intense learning curve. This can mean sophisticated software that requires no small amount of computer savvy; it can mean energy-sucking search tools that require souped up computers to handle the task of combing through millions of pages of data; or, it can require the searching party to be unusually persistent – something most of us, with our expectations of instantaneous Google search success, won't be accustomed to.

All that being said, we can become acquainted with the invisible web by degrees. The many tools considered below will help you access a sizable slice of the invisible web's offerings. You will find we've identified a number of subject-specific databases and engines; tools with an established filter, making their searches much more narrow.

OPEN ACCESS JOURNAL DATABASES

Open access journal databases (OAJD) are compilations of free scholarly journals maintained in a manner that facilitates access by researchers and others who are seeking specific information or knowledge. Because these databases are comprised of unlinked content, they are located in the invisible web.

The vast majority of these journals are of the highest quality, with peer reviews and extensive vetting of the content before publication. However, there has been a trend of journals that are accepting scholarship without adequate quality controls, and with arrangements designed to make money for the publishers rather than furtherance of scholarship. It is important to be careful and review the standards of the database and journals chosen. "This helpful guide" explains what to look for.

Below is a sample list of well-regarded and reputable databases.

  • "AGRIS" (International Information System for Agricultural Science and Technology) is a global, public domain database maintained in multiple languages by the Food and Agriculture Organization of the United Nations. They provide free access to agricultural research and information.
  • "BioMed Central" is the UK-based publisher of 258 peer-reviewed open access journals. Their published works span science, technology and medicine and include many well-regarded titles.
  • "Copernicus Publications" has been an open-access scientific publisher in Germany since 2001. They are strong supporters of the researchers who create these articles, providing top-level peer review and promotion for their work.
  • "DeGruyter Open" (formerly Versita Open) is one of Germany's leading publishers of open access content. Today DeGruyter Open (DGO) publishes about 400 owned and third-party scholarly journals and books across all major disciplines.
  • "Directory of Open Access Journals is focused on providing access only to those journals that employ the highest quality standards to guarantee content. They are presently a repository of 9,740 journals with more than 1.5 million articles from 133 countries.
  • "EDP Sciences" (Édition Diffusion Presse Sciences) is a France-based scientific publisher with an international mission. They publish more than 50 scientific journals, with some 60,000 published pages annually.
  • "Elsevier of Amsterdam is a world leader in advancing knowledge in the science, technology and health fields. They publish nearly 2,200 journals, including The Lancet and Cell, and over 25,000 book titles, including Gray's Anatomy and Nelson' s Pediatrics.
  • "Hindawi Publishing Corporation", based in Egypt, publishes 434 peer-reviewed, open access journals covering all areas of Science, Technology and Medicine, as well as a variety of Social Sciences.
  • "Journal Seek" (Genamics) touts itself as "the largest completely categorized database of freely available journal information available on the internet," with more than 100,000 titles currently. Categories range from Arts and Literature, through both hard- and soft-sciences, to Sports and Recreation.
  • "The Multidisciplinary Digital Publishing Institute" (MDPI), based in Switzerland, is a publisher of more than 110 peer-reviewed, open access journals covering arts, sciences, technology and medicine.
  • "Open Access Journals Search Engine" (OAJSE), based in India, is a search engine for open access journals from throughout the world, except for India. An extremely simple interface. Note: the site was last updated June 21, 2013.
  • "Open J-Gate" is an India-based e-journal database of millions of journal articles in open access domain. With a worldwide reach, Open J-Gate is updated every day with new academic, research and industry articles.
  • "Open Science Directory" contains about 13,000 scientific journals, with another 7,000 special programs titles.
  • "Springer Open" offers a roster of more than 160 peer-reviewed, open access journals, as well as their more recent addition of free access books, covering all scientific disciplines.
  • "Wiley Open Access", a subsidiary of New Jersey-based global publishers John Wiley & Sons, Inc., publishes peer reviewed open access journals specific to biological, chemical and health sciences.

INVISIBLE WEB SEARCH ENGINES

Your typical search engine's primary job is to locate the surface sites and downloads that make up much of the web as we know it. These searches are able to find an array of HTML documents, video and audio files and, essentially, any content that is heavily linked to or shared online. And often, these engines, Google chief among them, will find and organize this diversity of content every time you search.

The search engines that deliver results from the invisible web are distinctly different. Narrower in scope, these deep web engines tend to access only a single type of data. This is due to the fact that each type of data has the potential to offer up an outrageous number of results. An inexact deep web search would quickly turn into a needle in a haystack. That's why deep web searches tend to be more thoughtful in their initial query requirements.
Below is a list of popular invisible web search engines:

  • "Clusty" is a meta search engine that not only combines data from a variety of different source documents, but also creates "clustered" responses, automatically sorting by category.
  • "CompletePlanet" searches more than 70,000 databases and specialty search engines found only in the invisible web. A search engine as well-suited to casual searchers as it is to researchers.
  • "DigitalLibrarian": A Librarian's Choice of the Best of the Web is maintained by a real librarian. With an eclectic mix of some 45 broad categories, Digital Librarian offers data from categories as diverse as Activism/Non Profits and Railroads and Waterways.
  • "InfoMine" is another librarian-developed internet resource collection, this time from The Regents of the University of California.
  • "InternetArchive" has an eclectic array of categories, starting with the ‘Wayback Machine,' which allows the searcher to locate archived documents, and including an archive of Grateful Dead audience and soundboard recordings. They offer 6 million texts, 1.5 million videos, 1.9 million audio recordings and 126K live music concerts.
  • "The Internet Public Library" (ipl and ipl2) is a non-profit, student-run website at Drexel University. Students volunteer to act as librarians and respond to questions from visitors. Categories of data include those directed to Children and Teens.
  • "SurfWax" is a metasearch engine that offers "practical tools for Dynamic Search Navigation." It offers the option of grabbing results from multiple search engines at the same time, or even designing "SearchSets," which are individualized groups of sources that can be used over and over in searches.
  • "UC Santa Barbara Library" offers access to a diverse group of research databases useful to students, researchers and the casual searcher. It should be noted that many of these resources are password protected. Those that do not display a lock icon are publicly accessible.
  • "USA.gov" offers acess to a huge volume of information, including all types of forms, databases, and information sites representing most government agencies.
  • "Voice of the Shuttle" (VoS) offers access to a diverse assortment of sites, including literature, literary theory, philosophy, history and cultural studies, and includes the daily update of all things "cool."

SUBJECT -SPECIFIC DATABASES

The following lists pool together some mainstream and not so mainstream databases dedicated to particular fields and areas of interest. While only a handful of these tools are able to surface deep web materials, all of the search engines and collections we have highlighted are powerful, extensive bodies of work. Many of the resources these tools surface would likely be overlooked if the same query were made on one of the mainstream engines most users fall back on, like Bing, Yahoo and even Google.

Art & Design

  • "ArtNet" deals with pricing and sourcing work in the art market. They also keep track of the latest news and artists in the industry.
  • "The Metropolitan Museum of Art" site hosts an impressively interactive body of information on their collections, exhibitions, events and research.
  • "Musée du Louvre", the renowned museum, maintains a site filled with navigable sections covering its collections.
  • "The National Gallery of Art" premier museum of arts in our nation's capital, also maintains a site detailing the highlights, exhibitions and education efforts the institution oversees.
  • "Public Art Online" is a resource detailing sources, creators, prices, projects, legal issues, success stories, resources, education and all other aspects of the creation of public art.
  • "Smithsonian Art Inventories Catalog" is a subset of the Smithsonian Institution Research Information System (SIRIS). A browsable database of over 400,000 art inventory items held in public and private collections.
  • "Web Gallery of Art" is a searchable database of European art, containing nearly 34,000 reproductions. Additional database information includes artist biographies, period music and commentaries.

Business

  • "Better Business Bureau" (BBB) Information System Search allows consumers to locate the details of ratings, consumer experience, governmental action and more of both BBB accredited and non-accredited businesses.
  • "BPubs.com" is the business publications search engine. They offer more than 200 free subscriptions to business and trade publications.
  • "BusinessUSA" is an excellent and complete database of everything a new or experienced business owner or employer should know.
  • "EDGAR: U.S. Securities and Exchange Commission" contains a database of Securities and Exchange Commission. Posts copies of corporate filings from US businesses, press releases and public statements.
  • "Global Edge" delivers a comprehensive research tool for academics, students and businesspeople to seek out answers to international business questions.
  • "Hoover's", a subsidiary of Dun & Bradstreet, is one of the best known databases of American and International business. A complete source of company and industry information, especially useful for investors.
  • "The National Bureau of Economic Research is perhaps the leading private, non-partisan research organization dedicated to unbiased analysis of economic policy. This database maintains archives of research data, meetings, activities, working papers and publications.
  • "U.S. Department of Commerce", Bureau of Economic Analysis is the source of many of the economic statistics we hear in the news, including national income and product accounts (NIPAs), gross domestic product, consumer spending, balance of payments and much more.

Legal & Social Services

Science & Technology

  • "Environmental Protection Agency" rganizes the agency's laws and regulations, science and technology, and the many issues affecting the agency and its policies.
  • "National Science Digital Library" (NSDL) is a source for science, technology, engineering and mathematics educational data. It is funded by the National Science Foundation.
  • "Networked Computer Science Technical Reports Library (NCSTRL) was developed as a collaborative effort between NASA Langley, Virginia Tech, Old Dominion University and University of Virginia. It serves as an archive for submitted scientific abstracts and other research products.
  • "Science.gov" is a compendium of more than 60 US government scientific databases and more than 200 websites. Governed by the interagency Science.gov Alliance, this site provides access to a range of government scientific research data.
  • "Science Research" is a free, publicly available deep web search engine that purports to use a sophisticated technology that permits queries to more than 300 science and technology sites simultaneously, with the results collated, ranked and stripped of duplications.
  • "WebCASPAR" provides access to science and engineering data from a variety of US educational institutions. It incorporates a table builder, allowing a combined result from various National Science Foundation and National Center for Education Statistics data sources.
  • "WebCASPAR" World Wide Science is a global scientific gateway, comprised of US and international scientific databases. Because it is multilingual, it allows real-time search and translation of reporting from an extensive group of databases.

Healthcare

  • "Cases Database" is a searchable database of more than 32,000 peer-reviewed medical case reports from 270 journals covering a variety of medical conditions.
  • "Center for Disease Control" (CDC) WONDER's online databases permit access to the substantial public health data resources held by the CDC.
  • "HCUPnet" is an online query system for those seeking access to statistical data from the Agency for Healthcare Research and Quality.
  • "Healthy People" provides rolling 10-year national objectives and programs for improving the health of Americans. They currently operate under the Healthy People 2020 decennial agenda.
  • "National Center for Biotechnology Information" (NCBI) is an offshoot of the National Institutes of Health (NIH). This site provides access to some 65 databases from the various project categories currently being researched.
  • "OMIM" offers access to the combined research of many decades into genetics and genetic disorders. With daily updates, it represents perhaps the most complete single database of this sort of data.
  • "PubMed is a database of more than 23 million citations from the US National Library of Medicine and National Institutes of Health.
  • "TOXNET" is the access portal to the US Toxicology Data Network, an offshoot of the National Library of Medicine.
  • "U.S. National Library of Medicine" is a database of medical research, available grants, available resources. The site is maintained by the National Institutes of Health.
  • "World Health Organization" (WHO) is a comprehensive site covering the many initiatives the WHO is engaged in around the world.

[Source: This article was published in onlineuniversities.com By hilip Bump - Uploaded by the Association Member: Robert Hensonw]

Categorized in How to

If you're looking for simple ways to find what is available on the Invisible Web, curated directories like the ones listed in this article can be extremely useful tools to use. You can use any of these resources to find what is available on the Web that is not as easily searchable from a general search engine query. 

The Invisible Web is easily accessible..that is, if you know where to look. Many individuals and institutions have put together invisible Web directories, which you can use as a jumping off point to surf the Invisible Web.

Here are just a few:

  • The University of Michigan has put together OAIster, (pronounced "oyster") and encourages you to "find the pearls" on the Invisible Web. They have millions of records from more than 405 institutions as diverse as African Journals Online and the Library Network of Western Switzerland.
  • LookSmart's Find Articles.com lets you search print publications for articles; anything from popular magazines to scholarly journals. Be sure to check out their Furl tool to organize your Invisible Web search snippets.
  • The Library Spot is a collection of databases, online libraries, references, and other good info from the Invisible Web. Be sure to check out their "You Asked For It" section, where popular readers' questions are featured.
  • The US Government's official web portal is FirstGov.gov, an extremely deep (as in lots of content) site. You could spend hours here. It's interesting to note how much stuff you can get done online here as well, such as renew your driver's license, shop government auctions, and contact elected officials.
  • Search the vast holding of the UCLA Library online, including their special collections only found on the Invisible Web.
  • Check out Infoplease.com and its searchable Invisible Web databases. Results come from encyclopedias, almanacs, dictionaries, and other online resources only found on the Invisible Web.
  • The Central Intelligence Agency has the World Factbook, a searchable directory of flags of the world, reference maps, country profiles, and much, much more. Great for geography buffs or anyone who wants to learn more about their world.
  • The University of Idaho has created this Repository of Primary Sources, which contains links to manuscripts, archives, rare books, and much more. Covers not only the United States but countries all over the world.
  • Lund University Libraries maintains the Directory of Open Access Journals, a collection of searchable scientific and scholarly journals on the Invisible Web.
  • Looking for scientific information on the Invisible Web? Go to Scirus.com first. You can search either scholarly sources or Web sources or both.
  • Canada, ay? Then check out the Archival Records of Alberta. This is a web gateway to photographs, census records, and other archival records.
  • Want to find a plant that will survive overwatering, lack of sunlight, and general forgetfulness? You can probably find something in the USDA's Plants Database on the Invisible Web.
  • The Human Genome Database contains anything you would ever want to know..well, about the human genome on the Invisible Web, at least.
  • If you've got a medical question, check out The Combined Health Information Database, or CHID online. Its searchable subject directory is very user-friendly, and you can find information on pretty much anything to do with human health here.
  • Nonprofit organizations need searching tools too. The National Database of Nonprofit Organizations is an extensive site on the Invisible Web that not only provides locations and contact information for nonprofits but also gives detailed fiscal reports.
  • EEVL Xtra, a service put together by Heriot-Watt University in Edinburgh, Scotland. This excellent service has the ability to cross-search 20 engineering, mathematics and computing databases, including content from 50 publishers. Find articles, websites, and more on the Invisible Web.

Source: This article was published lifewire.com By Jerri Collins

Categorized in Deep Web

It’s not Spiderman’s latest web slinging tool but something that’s more real world.

The Invisible Web (or The Deep Web) refers to the part of the Internet that’s not indexed by the search engines. Most of us think that that search powerhouses like Google and Bing are like the Great Oracle — they see everything.

Unfortunately, they can’t because they aren’t divine at all; they are just web spiders who index pages by following one hyperlink after the other. And, there are some places where a spider cannot enter.

Take library databases which need a password for access. Or even pages that belong to private networks of organizations. Dynamically generated web pages in response to a query are often left un-indexed by search engine spiders.

Search engine technology has progressed by leaps and bounds. Today, we have real time search and the capability to index Flash based and PDF content. Even then, there remain large swathes of the web which a general search engine cannot penetrate. The term, Deep Net, Deep Web or Invisible Web lingers on. However, of a misnomer they may be.

It’s not that you can’t access the invisible web at all. It’s just that you must use the right tools to do so. Here are ten online indexes and search tools you should hit.

1. The WWW Virtual Library

This is considered to be the oldest catalog on the web and was started by started by Tim Berners-Lee, the creator of the web. So, isn’t it strange that it finds a place in the list of invisible web resources? Maybe, but the WWW Virtual Library lists quite a lot of relevant resources on quite a lot of subjects.

The WWW Virtual Library

For instance, there are 300 sub-libraries with their own categories within the main library. The History sub-library is a good example.

You can go vertically into the categories or use the search bar. The screenshot shows the alphabetical arrangement of subjects covered at the site. Even as many deep web resources have come and gone, the WWW Virtual Library keeps on going even after 26 years.

2. USA.gov

This is the official site of the U.S. Government and the portal to all the public information you need on every federal agency or state, local, and tribal government. The site has an A-Z index of all topicson the portal and it’s a better way to pinpoint the information you want.

USA.gov

Apart from the direct access, use filters like “Only USA.gov,” “Images,” or “Videos” at the top of the page for more specific results. And while you are here, don’t forget the partner sites like Kids.USA.gov and Publications.USA.gov which are other specialized information mines.

3. Science.gov

The blurb on the home page says it all. The scientific search engine taps into 60 databases and over 2,200 scientific websites that cover federal science information including the latest research and development results. Try the advanced search engine page for a deep web search across the government databases that exist in the country.

Science.gov

The federal search tool can be your first doorway for multidisciplinary research that covers everything from agricultural information to the current trends in science education for schools in the U.S. It is also a primary source for searching Federally-sponsored opportunities and programs for STEM students.

4. U.S Geological Survey

The Map Topics and the images alone could be worth the price of admission. There is none because the government site is free. The job of the organization is to broadcast real-time or near real-time data and information on current conditions and Earth observations. But it is also a goldmine for academic and even casual research.

U.S Geological Survey

Try the National Geologic Map Database catalog. Query satellite photos and Earth images with the EarthExplorer tool. Or, lean forward to search 100,000+ scientific publications, and books. And there is so much more.

5. Directory of Open Access Journals

Search for articles in Open Access journals. These are academic papers that are available to anyone “without financial, legal, or technical barriers other than those inseparable from gaining access to the Internet itself.” In short, the knowledge is free.

 

 

Directory of Open Access Journals

DOAJ maintains quality control with rigorous peer review. The current repository has 9000+ journals with almost 2.5 million articles across all subjects. This information may not be available with a Google Search though Google Scholar may be able to access some of the information. But DOAJ is a better research tool as it neatly curated with a well-designed advanced search engine.

6. Voice of The Shuttle

Studying literature? Try Voice of The Shuttle. It is a rich directory for of online resources on literature, the humanities, and cultural studies. The search directory has evolved to include topics like Sci-Tech and Culture, Cyberculture, and Technology of Writing to keep pace with the times.

Voice of The Shuttle

It started as a support tool for the English Dept. of the University of California at Santa Barbara in 1994. Today, it continues to be updated and you can browse through both primary and secondary resources.

7. RxList

Though Google does offer medical information when you search with your symptoms, you need all the help you can get. RxList is a comprehensive database of US prescription medications. The index is a prescription drug encyclopedia, pill identifier, and pharmacy locator rolled into one.

RxList

With the rise of supplements, there is also a dedicated part of the site for vitamins, herbs, and dietary supplements. Each section has its own search tool and/or an alphabetical listing.

The medical resource is a quality offshoot of the WebMD network. The data comes from sources like the FDA, Cerner Multum, and First Data Bank, Inc.

8. Infoplease

This online encyclopedia started its life as a radio quiz show in 1938. Today, Infoplease is an information portal with a host of features. Using the site, you can tap into a good number of databases, electronic journals, almanacs, electronic books, thesaurus, atlas, bulletin boards, mailing lists, online library card catalogs, articles, and directories of researchers.

Infoplease

Infoplease also has a few nice offshoots like Factmonster.com for kids and TeacherVision which are all part of the same educational network.

9. WorldCat

Think of it as the search engine for brick-and-mortar libraries across the world. The meta-catalog for 72,000 libraries in 170 countries can help you find any paper, book, thesis, videos, multimedia assets, and even museum artifacts stored somewhere.

WorldCat

The best way to use this massive database is with the advanced search tool. The information found here is also useful for creating citations for your research paper.

A search on WorldCat.org will return links to the resources in these databases. But to access these resources, you have to log in with a valid library membership. You can also use the “Ask a Librarian” feature to ask for help from librarians in charge.

10. The National Security Archive

This is a non-governmental resource for unclassified security documents. It is the largest repository of such documents outside the U.S. government. Set up to check rising government secrecy, the site uses a custom Google Search to give you access to more than 10 million pages of government documents.

National Security Archive

The papers are primary source material for journalists, security evangelists, and researchers. A growing collection of Electronic Briefing Books are a smaller part of the documents but give you a curated look at U.S. national security, foreign policy, diplomatic and military history, and intelligence policy.

More Deep Web Sites Worth a Mention

  1. Elephind
  2. MagPortal
  3. Free Lunch
  4. Clinical Trials
  5. Project Gutenberg
  6. The Library of Congress
  7. Internet Archive (Including the Way Back Machine)
  8. The National Gallery of Art
  9. Scitation
  10. PubSpace

How Do You Surf the Deep Web?

It is difficult to pin down the size of the Deep Web. But it is estimated to be several times larger than the web we know so well. What is true is that the topic focus of the invisible web makes it a ripe area to hunt for information when we by habit barely click to Page 2 of the Google Search results page.

It is also important here to make a distinction between the “invisible web” and the “Dark Net”. The invisible web is within the reach of a normal web browser while the dark net is dominated by TOR sites (and disreputable anonymous services) that need some technical wizardry to access.

Just like general web search, searching the invisible web is also about looking for the needle in the haystack. Only here, the haystack is much bigger. The invisible web is definitely not for the casual searcher. It is deep but not dark because if you know what you are searching for, information is a few keywords away.

Do you venture into the Invisible Web? Which is your preferred search tool?

Source: This article was published makeuseof.com By Saikat Basu

Categorized in Search Engine

20 Ways To Search the Invisible Web

Unlike pages on the visible Web (that is, the Web that you can access from search engines and directories), information in the Invisible Web is just not visible to the software spiders and crawlers that create search engine indexes. Since this information makes up the vast majority of available content on the Web, we are potentially missing out on some pretty amazing resources.

However, that's where Invisible Web search engines, tools, and directories come in. There are many Invisible Web search tools that you can use to dive into this wealth of information, as you'll see from the following list. We'll take a look at twenty different search engines, directories, and databases you can use to uncover amazing content.

2 Clusty

Clusty is a meta search engine, meaning it combines results from a variety of different sources, filtering out duplicates and sifting the best content that you might not have seen otherwise to the top of the search results.

More about Clusty

Find Government Information with Clusty
Ten Alternative Search Engines

3 The Internet Archive

The Internet Archive is an amazing database offering access to movies, live music, audio, and printed materials; plus, you can look at older, saved versions of nearly every site ever created on the Internet - over 55 billion at the time of this writing.

More archives

Google News Archive Search
Use the Web to Find Archives
Five online archives that are worth a second look

4 USA.gov

USA.gov is an absolutely mammoth search engine/portal that gives the searcher direct access to a wide variety of information and databases from the United States government, state governments, and local governments. This includes access to the Library of Congress, an A-Z government agency index, the Smithsonian, and much, much more.

More government resources

The Top Twenty Essential US Government Web Sites
Free Public Records Search
Google Uncle Sam

5 The WWW Virtual Library

The WWW Virtual Library gives you instant access to hundreds of different categories and databases on a wide variety of subjects, anything from Agriculture to Anthropology. More about this amazing resource: "The WWW Virtual Library (VL) is the oldest catalogue of the Web, started by Tim Berners-Lee, the creator of HTML and of the Web itself, in 1991 at CERN in Geneva. Unlike commercial catalogues, it is run by a loose confederation of volunteers, who compile pages of key links for particular areas in which they are expert; even though it isn't the biggest index of the Web, the VL pages are widely recognised as being amongst the highest-quality guides to particular sections of the Web."

6 Science.gov

Science.gov searches over 60 databases and over 2200 selected websites from 15 federal agencies, offering 200 million pages of authoritative U.S. government science information including research and development results. More about this astonishingly useful resource: "Science.gov is a gateway to government science information and research results. Currently in its fifth generation, Science.gov provides a search of over 60 scientific databases and 200 million pages of science information with just one query, and is a gateway to over 2200 scientific Websites (see Science.gov fact sheet).

Science.gov is an interagency initiative of 19 U.S. government science organizations within 15 Federal Agencies. These agencies form the voluntary Science.gov Alliance which governs Science.gov."

7 Wolfram Alpha

Wolfram Alpha is a computational search engine, which means it stores a vast amount of pure data available to you via not only search, but also a question and answer format. More about Wolfram Alpha: "We aim to collect and curate all objective data; implement every known model, method, and algorithm; and make it possible to compute whatever can be computed about anything. Our goal is to build on the achievements of science and other systematizations of knowledge to provide a single source that can be relied on by everyone for definitive answers to factual queries."

8 Alexa

Alexa gives you specific analytical information about Web properties. More about this intriguing resource: "Alexa's traffic estimates are based on data from our global traffic panel, which is a sample of millions of Internet users using one of over 25,000 different browser extensions. In addition, we gather much of our traffic data from direct sources in the form of sites that have chosen to install the Alexa script on their site and certify their metrics."

Website owners especially can benefit from the data that Alexa offers; for example, here's a list of the top 500 sites on the Web.

9 Directory of Open Access Journals

The Directory of Open Access Journals (DOAJ) indexes and provides access to quality open access, peer-reviewed journals. More about this online directory: "The Directory of Open Access Journals is a service that indexes high quality, peer reviewed Open Access research journals, periodicals and their articles' metadata. The Directory aims to be comprehensive and cover all open access scientific and scholarly journals that use an appropriate quality control system (see the section below) and is not limited to particular languages or subject areas. The Directory aims to increase the visibility and ease of use of open access scientific and scholarly journals—regardless of size and country of origin—thereby promoting their visibility, usage and impact."

More than 10,000 journals and millions of articles are searchable using the DOAJ.

10 FindLaw

FindLaw is a gigantic repository of free legal information on the Internet, and offers one of the largest online lawyer directories available online. You can use FindLaw to locate an attorney, learn more about U.S. law and legal topics, and participate in the very active FindLaw community forums.

11 The Online Books Page

The Online Books Page, a service offered by the University of Pennsylvania, gives readers access to over two million books freely accessible (and readable) on the Internet. Users will also gain access to significant directories and archives of online texts,as well as special exhibits of particularly interesting classes of online books.

12 The Louvre

The Louvre online simply begs to be discovered and cherished by art lovers all over the world. View thematic collections of art, get more information about the background of selected works, view art aligned with historical events, and much, much more.

13 The Library of Congress

One of the most vivid and interactive sites on this list of Invisible Web resources, the Library of Congress offers an incredibly rich and varied array of content. Collection highlights include Congressional records, digital preservation resources, the Veterans History project, and the World Digital Library. More about this national treasure: "The Library of Congress is the nation's oldest federal cultural institution and serves as the research arm of Congress. It is also the largest library in the world, with millions of books, recordings, photographs, maps and manuscripts in its collections."

14 Census.gov

If you're looking for data, then Census.gov is one of the first places you'll want to visit. More about this considerable resource: "The U.S. Census Bureau conducts demographic, economic, and geographic studies of other countries and strengthens statistical development around the world through technical assistance, training, and software products. For over 60 years, the Census Bureau has performed international analytical work and assisted in the collection, processing, analysis, dissemination, and use of statistics with counterpart governments in over 100 countries."

From geography to population statistics, you'll be able to find them here.

15 Copyright.gov

Copyright.gov is another U.S. government resource you can put in your Invisible Web search toolbox (for even more essential U.S. government sites, check out The Top Twenty U.S. Government Websites). Here, you can view works registered and documents recorded by the U.S. Copyright Office since January 1, 1978, as well as search records of registered books, music, art, and periodicals, and other works, including copyright ownership documents.

16 Catalog of U.S. Government Publications

The Catalog of U.S. Government Publications gives users instant access to electronic and print publications from the legislative, executive, and judicial branches of the U.S. government, with more than 500,000 records generated since July 1976.

17 Bankrate

Bankrate, an online financial resource that's been around since 1996, offers a huge library of financial information; anything from current interest rates to articles on CUSIP and much, much more.

18 FreeLunch

FreeLunch gives users the ability to quickly and easily find free economic, demographic, and financial data: "provides comprehensive and extensive historical and forecast data at the national and subnational/regional levels representing over 93% of global GDP. We cover more than 180 countries, over 150 global metro areas, all U.S. states, metro areas and counties. Our databases contain more than 200 million economic, financial, demographic and consumer credit time series, with 10 million added every year."

19 PubMed

PubMed, part of the National Center for Biotechnology Information, U.S. National Library of Medicine, is the perfect resource for anyone who's looking up medical or medical-related information. It offers more than 24 million citations for biomedical literature from MEDLINE, life science journals, and online books.

20 FAA Data and Research

The FAA Data and Research pages offer information on how their research is done, the resulting data and statistics, and information on funding and grant data. Anything from Aviation Safety to Unruly Passengers (seriously) can be found here.

Categorized in Deep Web

The Invisible Web refers to the vast amount of content and information that is not easily discoverable in a general search engine query, such as databases, private networks, or password-protected information. However, there are a wide variety of high quality Invisible Web search tools, search engines, and directories that can help you mine this fantastic resource that is considered to be at least 500 times larger than the visible Web.

The following Invisible Web resources will connect you to a virtual goldmine of knowledge, anything from medical dictionaries to moving picture archives to academically vetted articles and journals. Each of these links connects you to a resource that will help you find information that is not easily found with just a simple, rudimentary search. These tools help you do a deep dive into untapped treasure troves of information. 

The Invisible Web: A Brief Introduction

The Invisible Web: What is the Invisible Web? Is it some kind of Area 52-ish, X-Files deal that only those with stamped numbers on their foreheads can access? Well, not exactly. If you're not familiar with what the Invisible Web really is, just keep reading to get a quick overview of what the Invisible Web really is, and how you can use it to find information. 

Invisible Web Search Engines

Five Search Engines You Can Use to Mine the Invisible Web: There are many Invisible Web search tools that you can use to dive into this wealth of information, as you'll see from the following list.....keep reading to understand what these search engines can offer you.

The Top Ten Most Popular Search Engines can be used as a jumping off point for much of the Web's harder to find information. 

FactBites: Factbites retrieves results that are academically oriented, i.e., dictionaries, encyclopedias, universities, and many .org sites (typically non-profit organizations).

Invisible Web Directories and Portals

Invisible Web Directories: Many individuals and institutions have put together invisible Web directories, which you can use as a jumping off point to surf the Invisible Web. 

Medical Information on the Invisible Web: The Invisible Web has a goldmine of medical databases and specialized medical sites that just don't show up on a cursory search in the search engines. Best of all, this information is free.

Humanities and Literature Resources on the Invisible Web: There are plenty of humanities resources on the Invisible Web, such as arts, literature, and history Web sites that will give you greater insight on what you might be reading for a class or help you on a research project.

Invisible Web Research Databases and Reference Tools

47 Alternatives to Wikipedia: Wikipedia is perhaps the most popular reference site online, with millions of high quality articles available on virtually any topic. However, there are limits to what Wikipedia can offer.

How to Find Archives on the Web: Are you trying to find information about a historical event? Looking for online archives for news, music, popular culture, or movie information?

Google Scholar: From one place, you can search across many disciplines and sources: peer-reviewed papers, theses, books, abstracts and articles, from academic publishers, professional societies, preprint repositories, universities and other scholarly organizations.

Find People with the Invisible Web

Invisible Web People Search Resources: The following resources can help you delve deep into the Invisible Web, making your people searches even more rich, detailed, and authoritative.

15 Search Engines to Help You Find Someone: Finding people on the Web is getting easier and easier. Here are fifteen people search engines you can use to find the person you're looking for on the Web.

How to Find Someone on the Web, Step by Step: Need to find someone? Here's a step by step guide to finding someone on the Web using tools and services that are designed to find people online.

Use the Invisible Web to Find Public Records

The Top Twenty Essential US Government Web Sites: There are literally hundreds of thousands of US government and government-related Web sites that offer free access to a wide variety of information.

How to Find Public Records: Here are the best free public record search databases online, from obituaries to census records.

Do a Background Check on the Web: You can use the Web to do a free background check via a multitude of free sources on the visible and invisible Web.

Books and Printed Materials on the Invisible Web

How to Find and Read Full Books Online: More people than ever before in history are using the Web as a free library, and with good reason: there are literally thousands of free books online that you can download in their entirety, listen to in an audio book version, or simply read within your browser window.

How to Find Works in the Public DomainPublic domain works are works whose copyrights were issued before 1923, and have now passed into the public domain, meaning that they can be used, reproduced, or incorporated in any way without any restrictions.

How to Find PDF Files: If you are trying to find PDF (Adobe Acrobat) files on the Web, there's a number of ways that you can accomplish this. 

Author:  Wendy Boswell

Source:  https://www.lifewire.com

Categorized in Deep Web

No, it’s not Spiderman’s latest web slinging tool but something that’s more real world. Like the World Wide Web.

The Invisible Web refers to the part of the WWW that’s not indexed by the search engines. Most of us think that that search powerhouses like Google and Bing are like the Great Oracle”¦they see everything. Unfortunately, they can’t because they aren’t divine at all; they are just web spiders who index pages by following one hyperlink after the other.

But there are some places where a spider cannot enter. Take library databases which need a password for access. Or even pages that belong to private networks of organizations. Dynamically generated web pages in response to a query are often left un-indexed by search engine spiders.

Search engine technology has progressed by leaps and bounds. Today, we have real time search and the capability to index Flash based and PDF content. Even then, there remain large swathes of the web which a general search engine cannot penetrate. The term, Deep Net, Deep Web or Invisible Weblingers on.

To get a more precise idea of the nature of this ‘Dark Continent’ involving the invisible and web search engines, read what Wikipedia has to say about the Deep Web. The figures are attention grabbers – the size of the open web is 167 terabytes. The Invisible Web is estimated at 91,000terabytes. Check this out – the Library of Congress, in 1997, was figured to have close to 3,000terabytes!

How do we get to this mother load of information?

That’s what this post is all about. Let’s get to know a few resources which will be our deep diving vessel for the Invisible Web. Some of these are invisible web search engines with specifically indexed information.

Infomine

invisible web search engines

Infomine has been built by a pool of libraries in the United States. Some of them are University of California, Wake Forest University, California State University, and the University of Detroit. Infomine ‘mines’ information from databases, electronic journals, electronic books, bulletin boards, mailing lists, online library card catalogs, articles, directories of researchers, and many other resources.

You can search by subject category and further tweak your search using the search options. Infomine is not only a standalone search engine for the Deep Web but also a staging point for a lot of other reference information. Check out its Other Search Tools and General Reference links at the bottom.

The WWW Virtual Library

invisible web search engines

This is considered to be the oldest catalog on the web and was started by started by Tim Berners-Lee, the creator of the web. So, isn’t it strange that it finds a place in the list of Invisible Web resources? Maybe, but the WWW Virtual Library lists quite a lot of relevant resources on quite a lot of subjects. You can go vertically into the categories or use the search bar. The screenshot shows the alphabetical arrangement of subjects covered at the site.

Intute

invisible web search engines

Intute is UK centric, but it has some of the most esteemed universities of the region providing the resources for study and research. You can browse by subject or do a keyword search for academic topics like agriculture to veterinary medicine. The online service has subject specialists who review and index other websites that cater to the topics for study and research.

Intute also provides free of cost over 60 free online tutorials to learn effective internet research skills. Tutorials are step by step guides and are arranged around specific subjects.

Complete Planet

search invisible web

Complete Planet calls itself the ‘front door to the Deep Web’. This free and well designed directory resource makes it easy to access the mass of dynamic databases that are cloaked from a general purpose search. The databases indexed by Complete Planet number around 70,000 and range from Agriculture to Weather. Also thrown in are databases like Food & Drink and Military.

For a really effective Deep Web search, try out the Advanced Search options where among other things, you can set a date range.

Infoplease

search invisible web

Infoplease is an information portal with a host of features. Using the site, you can tap into a good number of encyclopedias, almanacs, an atlas, and biographies. Infoplease also has a few nice offshoots like Factmonster.com for kids and Biosearch, a search engine just for biographies.

DeepPeep

search invisible web

DeepPeep aims to enter the Invisible Web through forms that query databases and web services for information. Typed queries open up dynamic but short lived results which cannot be indexed by normal search engines. By indexing databases, DeepPeep hopes to track 45,000 forms across 7 domains.

The domains covered by DeepPeep (Beta) are Auto, Airfare, Biology, Book, Hotel, Job, and Rental. Being a beta service, there are occasional glitches as some results don’t load in the browser.

IncyWincy

how to use the invisible web

IncyWincy is an Invisible Web search engine and it behaves as a meta-search engine by tapping into other search engines and filtering the results. It searches the web, directory, forms, and images. With a free registration, you can track search results with alerts.

DeepWebTech

how to use the invisible web

DeepWebTech gives you five search engines (and browser plugins) for specific topics. The search engines cover science, medicine, and business. Using these topic specific search engines, you can query the underlying databases in the Deep Web.

Scirus

how to use the invisible web

Scirus has a pure scientific focus. It is a far reaching research engine that can scour journals, scientists’ homepages, courseware, pre-print server material, patents and institutional intranets.

TechXtra

TechXtra concentrates on engineering, mathematics and computing. It gives you industry news, job announcements, technical reports, technical data, full text eprints, teaching and learning resources along with articles and relevant website information.

Just like general web search, searching the Invisible Web is also about looking for the needle in the haystack. Only here, the haystack is much bigger. The Invisible Web is definitely not for the casual searcher. It is a deep but not dark because if you know what you are searching for, enlightenment is a few keywords away.

Do you venture into the Invisible Web? Which is your preferred search tool?

Author:  Saikat Basu

Source:  http://www.makeuseof.com/

Categorized in Search Engine

airs logo

Association of Internet Research Specialists is the world's leading community for the Internet Research Specialist and provide a Unified Platform that delivers, Education, Training and Certification for Online Research.

Get Exclusive Research Tips in Your Inbox

Receive Great tips via email, enter your email to Subscribe.

Follow Us on Social Media