For most people if you can’t find something on Google then it’s not there on the internet. However that is not true and there are other ways to find the information you are looking for even if Google can’t find it. Now some of you might be wondering, how can something be online without Google knowing about it because don’t they index everything? Unfortunately, that is not true. According to studies there are a lot of sites out there that are not indexed by any search engine. This part of the internet is called the Deep Web. Deep web is not to be confused with Dark Net which contains sites that can’t be reached via the regular internet. Deep Web sites are accessible via the regular internet and it is a lot bigger than the visible internet. In-fact some estimates suggest that the deep web is 400 to 550 times larger than the surface web.
So how do you find something that is in the Deep web or just not indexed by Google? Well, you can always try one of the following options depending on what you are looking for.
For example, if you are making factual queries about data (e.g. facts, figures, etc) then you should take a look at Wolfram Alpha. Their Wikipedia page explains how the engine works:
Users submit queries and computation requests via a text field. Wolfram Alpha then computes answers and relevant visualizations from a knowledge base of curated, structured data that come from other sites. The curated data makes Alpha different from semantic search engines, which index a large number of answers and then try to match the question to one.
Using the Mathematica toolkit, Wolfram Alpha can respond to natural language questions and generate a human-readable answer.
Topsy maintains a comprehensive index of tweets and since Twitter is the best place for real-time sharing of thoughts/news then it is a good place to search for current events/trending topics. I just tried it out and it looks to be pretty effective and efficient.
If you are trying to identify an image, or find more information about a particular Image then you can always try Google image search. However if that doesn’t return any relevant results then you should try out specialized Image search engines like Tin Eye or yandex.ru. I use a Firefox Extension called Who Stole my Pictures that lets you search across multiple engines in one shot from your context menu. Side note: This also search on Bing but 99.99% of the time Bing doesn’t return any results no matter what you search for.
On the other hand if you are just searching for images you should try PicSearch.com which is a image search service allowing a user to search across over 3 billion pictures (as per the site).
WebForums and Discussion boards
Another great way to find answers is to search on enthusiast forums and discussion boards. These forums have a whole community of folks who are passionate about that particular topic and would love you to point you in the right direction or walk you through figuring out the solution. Just ensure that you are asking Questions The Smart Way.
BoardReader.com allows you to search across multiple discussion boards and forums available on the net. StackExchange.com has multiple sub sections on hundreds of topics, Reditt.com has subreddits that focus on thousands of topics and most of them have actual relevant information as not all of the site is dedicated to cat video’s.
IRC stands for Internet Relay Chat and is designed to facilitate group communication in discussion forums, called channels hosted on IRC servers. There are channels dedicated to pretty much any topic you can think of on some IRC server somewhere and you can get answers to questions or help with a problem in real time.
The difficult part is finding the appropriate channel to ask your question.
I have used IRC Search in the past to find channels with a good success rate. Another option is ixirc.com/.
In addition to the options listed above, you should also check out the following resources for additional information and search options/methods that you can try out when searching for data:
That pretty much covers what I wanted to talk about in this post so this is all for now. Will post more later.