It’s not Spiderman’s latest web slinging tool but something that’s more real world.

The Invisible Web (or The Deep Web) refers to the part of the Internet that’s not indexed by the search engines. Most of us think that that search powerhouses like Google and Bing are like the Great Oracle — they see everything.

Unfortunately, they can’t because they aren’t divine at all; they are just web spiders who index pages by following one hyperlink after the other. And, there are some places where a spider cannot enter.

Take library databases which need a password for access. Or even pages that belong to private networks of organizations. Dynamically generated web pages in response to a query are often left un-indexed by search engine spiders.

Search engine technology has progressed by leaps and bounds. Today, we have real time search and the capability to index Flash based and PDF content. Even then, there remain large swathes of the web which a general search engine cannot penetrate. The term, Deep Net, Deep Web or Invisible Web lingers on. However, of a misnomer they may be.

It’s not that you can’t access the invisible web at all. It’s just that you must use the right tools to do so. Here are ten online indexes and search tools you should hit.

1. The WWW Virtual Library

This is considered to be the oldest catalog on the web and was started by started by Tim Berners-Lee, the creator of the web. So, isn’t it strange that it finds a place in the list of invisible web resources? Maybe, but the WWW Virtual Library lists quite a lot of relevant resources on quite a lot of subjects.

The WWW Virtual Library

For instance, there are 300 sub-libraries with their own categories within the main library. The History sub-library is a good example.

You can go vertically into the categories or use the search bar. The screenshot shows the alphabetical arrangement of subjects covered at the site. Even as many deep web resources have come and gone, the WWW Virtual Library keeps on going even after 26 years.

2. USA.gov

This is the official site of the U.S. Government and the portal to all the public information you need on every federal agency or state, local, and tribal government. The site has an A-Z index of all topicson the portal and it’s a better way to pinpoint the information you want.

USA.gov

Apart from the direct access, use filters like “Only USA.gov,” “Images,” or “Videos” at the top of the page for more specific results. And while you are here, don’t forget the partner sites like Kids.USA.gov and Publications.USA.gov which are other specialized information mines.

3. Science.gov

The blurb on the home page says it all. The scientific search engine taps into 60 databases and over 2,200 scientific websites that cover federal science information including the latest research and development results. Try the advanced search engine page for a deep web search across the government databases that exist in the country.

Science.gov

The federal search tool can be your first doorway for multidisciplinary research that covers everything from agricultural information to the current trends in science education for schools in the U.S. It is also a primary source for searching Federally-sponsored opportunities and programs for STEM students.

4. U.S Geological Survey

The Map Topics and the images alone could be worth the price of admission. There is none because the government site is free. The job of the organization is to broadcast real-time or near real-time data and information on current conditions and Earth observations. But it is also a goldmine for academic and even casual research.

U.S Geological Survey

Try the National Geologic Map Database catalog. Query satellite photos and Earth images with the EarthExplorer tool. Or, lean forward to search 100,000+ scientific publications, and books. And there is so much more.

5. Directory of Open Access Journals

Search for articles in Open Access journals. These are academic papers that are available to anyone “without financial, legal, or technical barriers other than those inseparable from gaining access to the Internet itself.” In short, the knowledge is free.

 

 

Directory of Open Access Journals

DOAJ maintains quality control with rigorous peer review. The current repository has 9000+ journals with almost 2.5 million articles across all subjects. This information may not be available with a Google Search though Google Scholar may be able to access some of the information. But DOAJ is a better research tool as it neatly curated with a well-designed advanced search engine.

6. Voice of The Shuttle

Studying literature? Try Voice of The Shuttle. It is a rich directory for of online resources on literature, the humanities, and cultural studies. The search directory has evolved to include topics like Sci-Tech and Culture, Cyberculture, and Technology of Writing to keep pace with the times.

Voice of The Shuttle

It started as a support tool for the English Dept. of the University of California at Santa Barbara in 1994. Today, it continues to be updated and you can browse through both primary and secondary resources.

7. RxList

Though Google does offer medical information when you search with your symptoms, you need all the help you can get. RxList is a comprehensive database of US prescription medications. The index is a prescription drug encyclopedia, pill identifier, and pharmacy locator rolled into one.

RxList

With the rise of supplements, there is also a dedicated part of the site for vitamins, herbs, and dietary supplements. Each section has its own search tool and/or an alphabetical listing.

The medical resource is a quality offshoot of the WebMD network. The data comes from sources like the FDA, Cerner Multum, and First Data Bank, Inc.

8. Infoplease

This online encyclopedia started its life as a radio quiz show in 1938. Today, Infoplease is an information portal with a host of features. Using the site, you can tap into a good number of databases, electronic journals, almanacs, electronic books, thesaurus, atlas, bulletin boards, mailing lists, online library card catalogs, articles, and directories of researchers.

Infoplease

Infoplease also has a few nice offshoots like Factmonster.com for kids and TeacherVision which are all part of the same educational network.

9. WorldCat

Think of it as the search engine for brick-and-mortar libraries across the world. The meta-catalog for 72,000 libraries in 170 countries can help you find any paper, book, thesis, videos, multimedia assets, and even museum artifacts stored somewhere.

WorldCat

The best way to use this massive database is with the advanced search tool. The information found here is also useful for creating citations for your research paper.

A search on WorldCat.org will return links to the resources in these databases. But to access these resources, you have to log in with a valid library membership. You can also use the “Ask a Librarian” feature to ask for help from librarians in charge.

10. The National Security Archive

This is a non-governmental resource for unclassified security documents. It is the largest repository of such documents outside the U.S. government. Set up to check rising government secrecy, the site uses a custom Google Search to give you access to more than 10 million pages of government documents.

National Security Archive

The papers are primary source material for journalists, security evangelists, and researchers. A growing collection of Electronic Briefing Books are a smaller part of the documents but give you a curated look at U.S. national security, foreign policy, diplomatic and military history, and intelligence policy.

More Deep Web Sites Worth a Mention

  1. Elephind
  2. MagPortal
  3. Free Lunch
  4. Clinical Trials
  5. Project Gutenberg
  6. The Library of Congress
  7. Internet Archive (Including the Way Back Machine)
  8. The National Gallery of Art
  9. Scitation
  10. PubSpace

How Do You Surf the Deep Web?

It is difficult to pin down the size of the Deep Web. But it is estimated to be several times larger than the web we know so well. What is true is that the topic focus of the invisible web makes it a ripe area to hunt for information when we by habit barely click to Page 2 of the Google Search results page.

It is also important here to make a distinction between the “invisible web” and the “Dark Net”. The invisible web is within the reach of a normal web browser while the dark net is dominated by TOR sites (and disreputable anonymous services) that need some technical wizardry to access.

Just like general web search, searching the invisible web is also about looking for the needle in the haystack. Only here, the haystack is much bigger. The invisible web is definitely not for the casual searcher. It is deep but not dark because if you know what you are searching for, information is a few keywords away.

Do you venture into the Invisible Web? Which is your preferred search tool?

Source: This article was published makeuseof.com By Saikat Basu

Categorized in Search Engine

When you go online to search for something you either go to search engines such as Google or Bing. You probably think that if it doesn’t show up on these search engines, then it doesn’t exist, wrong! Believe it or not, there are things on the web that will never show up on your tradition search engine, no matter how hard you may try.

Why? Well, because a password is needed or the site belongs to a private network of organizations. I’m sorry to disappoint you if you thought that Google and Bing were the powerful search engines that have it all. If those tech giants had everything, neither the deep web search engines nor the hidden web would have existed.

Google and Bing follow one hyperlink after another and as a result doesn’t get everything you would want in the results. To be able to find the hidden things of the web, you have to dig a little deeper than usual, but I will show you how to do that and where to look. Hopefully, you can find what you need in the following deep web search engines.

What is the Hidden Web?

When you hear or read about the hidden or deep web, it’s anything behind a paywall, something with a password, or dynamically generated content on the fly and didn’t have a permanent URL. These are the things you are not going to find with a traditional Google search. So, where can you look? Thankfully, there are deep web search engines available on the web.

Top 10 Best Deep Web Search Engines to Explore Hidden Web

In this post, let’s find out top 10 best deep web search engines to explore hidden web.

1. TechXtra

The Best Hidden Web Search Engines - Top 10 Best Deep Web Search Engines to Explore Hidden Web

TechXtra is one of the best deep web search engines where you can search for content that has to do with Math, Engineering, and Computing. You can search for things such as technical data, industry news, classifieds, learning resources, full-text Eprints, and relevant website information. The design may not be as pretty as you might want it to be, but if you are a student who is looking for this kind of information, now you know where to look.

2. Infomine

Infomine is another great deep web search engine option for your hidden web needs. It is another site created by many online libraries of the United States. Here you can find things such as articles, books, notes, question papers, solutions, etc. The information that you find on this search engine is from universities such as Wake Forest University, University of California, University of Detroit and California State University.

How to search the Hidden web with Infomine - Top 10 Best Deep Web Search Engines to Explore Hidden Web

This hidden deep web search engine gets its information from places such as electronic books, databases, online library card catalogs, electronic journals, directories of researchers, bulletin boards, mailing lists, articles, and many other resources.

3. DeepWebTech

With DeepWebTech you can choose between five search engines. If one doesn’t work for you, you can always count on the other to help you find what you need. Just like Chrome, DeepWebTech also counts with browser plugins for you to use if you are searching for something in particular.

Deep Web Technolgies Hidden web search engine - Top 10 Best Deep Web Search Engines to Explore Hidden Web

With this deep web search engine, you can find information on a subject such as medicine, science, and business. If Google is not giving you what you are looking for, you can count on these deep web search engines getting the job done.

4. WWW The Virtual Library

WWW The Virtual Library also has a lot to offer. This hidden web search engine was created by Tim Berners-Lee and is the oldest deep web search engines out there. This dark search engine is not the most popular one out there but can say it was the first one of its kind.

The first Hidden Web search engine for Dark Web - Top 10 Best Deep Web Search Engines to Explore Hidden Web

So, isn’t it strange that it finds a place in the list of Invisible Web resources? WWW Virtual Library has quite a few useful resources on different subjects. It arranges all the categories in alphabetical order, so they are easier to find. You can choose from categories such as education, engineering, society, law, recreation, international affairs and more!

5. InfoPlease

If you are looking for an educational portal, then you should visit InfoPlease. It features all sorts of additional features for you to use. The search bar is located on the upper right-hand corner for your searching needs. You can enjoy things such as almanacs, encyclopedias, an atlas, and biographies.

InfoPlease Hidden web search engine - Top 10 Best Deep Web Search Engines to Explore Hidden Web

Infoplease is not just a one of the best deep web search engines, but it also has additional tools such as a Calculator, Spell Checker, Place, Finder, Periodic Table, Conversion Tool, Distance Calculator, Fact Monster, and a Perpetual Calendar.

6. Clutsy

Clutsy is in a class all it’s own because of all the channels it offers for your searches. Besides the traditional things such as news, images, purchases, etc. it looks for its results from a large number of places. For example, it searches in directories such as Daypop, Blogdigger, IceRocket and more.

Yippy Clutsy hidden Deep web search engine - Top 10 Best Deep Web Search Engines to Explore Hidden Web

Since things change over time, when you go to Clutsy, you will notice a name change to Yippy. I kept the original name, just in case someone remembered it by that name.

7. The Internet Archive

When you are looking for something on the Internet, one of those things are probably movies, audio or music, right? The hidden web is also full of this stuff; you just have to know where to look.

The Internet Archive for the Hidden Web - Top 10 Best Deep Web Search Engines to Explore Hidden Web

On The Internet Archive, you are going to have access to things such as movie, music, etc. that I mentioned earlier, but you can also enjoy printed materials. Do you want to see what a particular website looked like back in the day? The Internet Archive also lets you see older and saved versions of sites, I hope you have time since there are over 55 billion sites to look at.

8. Science.gov

If you are looking for something that you would only find on a site from the government, then you might want to check out Science.gov. It is a site that explores over 60 databases and more than 2200 websites from 15 federal agencies.

Science Gov Hidden Deeb Web search engine - Top 10 Best Deep Web Search Engines to Explore Hidden Web

If you need certain information for that science project, this is the place to go. The site offers 200 million pages of authoritative U.S government science information and development and research results are among that information.

9. Wolfram Alpha

With Wolfram Alpha you get a computational web search engine, in other words, you can enjoy a deep web search engine that has a significant amount of data for you to take advantage of. The site has categories such as:

  • Mathematics
  • Step-by-step solutions
  • Words % Linguistics
  • Units and Measure
  • Chemistry
  • Date & Times
  • Art & Design
  • Music
  • Astronomy
  • Engineering
  • Food & Nutrition
  • Shopping
  • Earth Sciences and more!

Worlfram Alpha Deep Web Search engine for Hidden web - Top 10 Best Deep Web Search Engines to Explore Hidden Web

Once you choose a topic, the site gives you so many options that you won´t know where to start. For example, let us say you choose Chemistry. In that category, you can either have the site give you chemical formulas, Chemical quantities, chemical solutions, functional groups, and the list keeps going.

10. FindLaw

Hopefully, you will never have to search for something that happened to you, but if you ever get into any legal trouble, this is the place to go. FindLaw is a site where you can have access to a vast repository of legal information you can use for free.

FindLaw Deep Web search engine for Hidden Web - Top 10 Best Deep Web Search Engines to Explore Hidden Web

FindLaw has one of the biggest online lawyer directories you can find on the Internet. You can either use the site to know more about U.S law, get yourself a lawyer, learn more about particular legal topics and use the Law forums. I’m sure the forums will be of big help since it’s almost certain you will come across some legal information you don’t understand.

Author: Rahul Dubey
Source: https://techreviewpro.com/deep-web-search-engines-hidden-web-13884

Categorized in Deep Web

No, it’s not Spiderman’s latest web slinging tool but something that’s more real world. Like the World Wide Web.

The Invisible Web refers to the part of the WWW that’s not indexed by the search engines. Most of us think that that search powerhouses like Google and Bing are like the Great Oracle”¦they see everything. Unfortunately, they can’t because they aren’t divine at all; they are just web spiders who index pages by following one hyperlink after the other.

But there are some places where a spider cannot enter. Take library databases which need a password for access. Or even pages that belong to private networks of organizations. Dynamically generated web pages in response to a query are often left un-indexed by search engine spiders.

Search engine technology has progressed by leaps and bounds. Today, we have real time search and the capability to index Flash based and PDF content. Even then, there remain large swathes of the web which a general search engine cannot penetrate. The term, Deep Net, Deep Web or Invisible Weblingers on.

To get a more precise idea of the nature of this ‘Dark Continent’ involving the invisible and web search engines, read what Wikipedia has to say about the Deep Web. The figures are attention grabbers – the size of the open web is 167 terabytes. The Invisible Web is estimated at 91,000terabytes. Check this out – the Library of Congress, in 1997, was figured to have close to 3,000terabytes!

How do we get to this mother load of information?

That’s what this post is all about. Let’s get to know a few resources which will be our deep diving vessel for the Invisible Web. Some of these are invisible web search engines with specifically indexed information.

Infomine

invisible web search engines

Infomine has been built by a pool of libraries in the United States. Some of them are University of California, Wake Forest University, California State University, and the University of Detroit. Infomine ‘mines’ information from databases, electronic journals, electronic books, bulletin boards, mailing lists, online library card catalogs, articles, directories of researchers, and many other resources.

You can search by subject category and further tweak your search using the search options. Infomine is not only a standalone search engine for the Deep Web but also a staging point for a lot of other reference information. Check out its Other Search Tools and General Reference links at the bottom.

The WWW Virtual Library

invisible web search engines

This is considered to be the oldest catalog on the web and was started by started by Tim Berners-Lee, the creator of the web. So, isn’t it strange that it finds a place in the list of Invisible Web resources? Maybe, but the WWW Virtual Library lists quite a lot of relevant resources on quite a lot of subjects. You can go vertically into the categories or use the search bar. The screenshot shows the alphabetical arrangement of subjects covered at the site.

Intute

invisible web search engines

Intute is UK centric, but it has some of the most esteemed universities of the region providing the resources for study and research. You can browse by subject or do a keyword search for academic topics like agriculture to veterinary medicine. The online service has subject specialists who review and index other websites that cater to the topics for study and research.

Intute also provides free of cost over 60 free online tutorials to learn effective internet research skills. Tutorials are step by step guides and are arranged around specific subjects.

Complete Planet

search invisible web

Complete Planet calls itself the ‘front door to the Deep Web’. This free and well designed directory resource makes it easy to access the mass of dynamic databases that are cloaked from a general purpose search. The databases indexed by Complete Planet number around 70,000 and range from Agriculture to Weather. Also thrown in are databases like Food & Drink and Military.

For a really effective Deep Web search, try out the Advanced Search options where among other things, you can set a date range.

Infoplease

search invisible web

Infoplease is an information portal with a host of features. Using the site, you can tap into a good number of encyclopedias, almanacs, an atlas, and biographies. Infoplease also has a few nice offshoots like Factmonster.com for kids and Biosearch, a search engine just for biographies.

DeepPeep

search invisible web

DeepPeep aims to enter the Invisible Web through forms that query databases and web services for information. Typed queries open up dynamic but short lived results which cannot be indexed by normal search engines. By indexing databases, DeepPeep hopes to track 45,000 forms across 7 domains.

The domains covered by DeepPeep (Beta) are Auto, Airfare, Biology, Book, Hotel, Job, and Rental. Being a beta service, there are occasional glitches as some results don’t load in the browser.

IncyWincy

how to use the invisible web

IncyWincy is an Invisible Web search engine and it behaves as a meta-search engine by tapping into other search engines and filtering the results. It searches the web, directory, forms, and images. With a free registration, you can track search results with alerts.

DeepWebTech

how to use the invisible web

DeepWebTech gives you five search engines (and browser plugins) for specific topics. The search engines cover science, medicine, and business. Using these topic specific search engines, you can query the underlying databases in the Deep Web.

Scirus

how to use the invisible web

Scirus has a pure scientific focus. It is a far reaching research engine that can scour journals, scientists’ homepages, courseware, pre-print server material, patents and institutional intranets.

TechXtra

TechXtra concentrates on engineering, mathematics and computing. It gives you industry news, job announcements, technical reports, technical data, full text eprints, teaching and learning resources along with articles and relevant website information.

Just like general web search, searching the Invisible Web is also about looking for the needle in the haystack. Only here, the haystack is much bigger. The Invisible Web is definitely not for the casual searcher. It is a deep but not dark because if you know what you are searching for, enlightenment is a few keywords away.

Do you venture into the Invisible Web? Which is your preferred search tool?

Author:  Saikat Basu

Source:  http://www.makeuseof.com/

Categorized in Search Engine

airs logo

Association of Internet Research Specialists is the world's leading community for the Internet Research Specialist and provide a Unified Platform that delivers, Education, Training and Certification for Online Research.

Get Exclusive Research Tips in Your Inbox

Receive Great tips via email, enter your email to Subscribe.

Follow Us on Social Media

Book Your Seat for Webinar GET FREE REGISTRATION FOR MEMBERS ONLY      Register Now