background preloader

Deep Web Research 2012

Deep Web Research 2012
Bots, Blogs and News Aggregators ( is a keynote presentation that I have been delivering over the last several years, and much of my information comes from the extensive research that I have completed over the years into the "invisible" or what I like to call the "deep" web. The Deep Web covers somewhere in the vicinity of 1 trillion plus pages of information located through the world wide web in various files and formats that the current search engines on the Internet either cannot find or have difficulty accessing. The current search engines find hundreds of billions of pages at the present time of this writing. In the last several years, some of the more comprehensive search engines have written algorithms to search the deeper portions of the world wide web by attempting to find files such as .pdf, .doc, .xls, ppt, .ps. and others. This Deep Web Research 2012 report and guide is divided into the following sections: 99 Resources to Research & Mine the Invisible Web by Jessica Hupp

http://www.llrx.com/features/deepweb2012.htm

Related:  deep webDeep_WebA bit about the Deep-WebrobertofiadSearch Skills

Working Links To The Deep Web Hidden Wikis Index pages in Wiki-based format. The Hidden Wiki - The original Hidden Wiki (after Matt's), owned by ion. Deep Web Search Engines Where to start a deep web search is easy. You hit Google.com and when you brick wall it, you go to scholar.google.com which is the academic database of Google. After you brick wall there, your true deep web search begins. You need to know something about your topic in order to choose the next tool. 100 Useful Tips and Tools to Research the Deep Web By Alisa Miller Experts say that typical search engines like Yahoo! and Google only pick up about 1% of the information available on the Internet.

99 Resources to Research & Mine the Invisible Web College researchers often need more than Google and Wikipedia to get the job done. To find what you're looking for, it may be necessary to tap into the invisible web, the sites that don't get indexed by broad search engines. The following resources were designed to help you do just that, offering specialized search engines, directories, and more places to find the complex and obscure. Search Engines 11 Unknown Ways Of Using Google Search - Curious Mob Thinking what more is there to know about Google search? I mean its Google search after all, type whatever you want to search, press enter and everything in the world related to your topic is displayed in front of your eyes. But believe it or not the search engine has plenty of tricks up its sleeve. Here’s an overview of 11 Google Tricks That Will Change the Way You Search: 11. Use quotes to search for an exact phrase

100 Search Engines For Academic Research Back in 2010, we shared with you 100 awesome search engines and research resources in our post: 100 Time-Saving Search Engines for Serious Scholars. It’s been an incredible resource, but now, it’s time for an update. Some services have moved on, others have been created, and we’ve found some new discoveries, too. Many of our original 100 are still going strong, but we’ve updated where necessary and added some of our new favorites, too. Check out our new, up-to-date collection to discover the very best search engine for finding the academic results you’re looking for. List of academic databases and search engines the general list of search engines for all-purpose search engines that can be used for academic purposesbibliographic databases for information about databases giving bibliographic information about finding books and journal articles. Note that "free" or "subscription" can refer both to the availability of the database or of the journal articles included. This has been indicated as precisely as possible in the lists below.

Database search engine There are several categories of search engine software: Web search or full-text search (example: Lucene), database or structured data search (example: Dieselpoint), and mixed or enterprise search (example: Google Search Appliance). The largest web search engines such as Google and Yahoo! utilize tens or hundreds of thousands of computers to process billions of web pages and return results for thousands of searches per second. Verification Handbook for Investigative Reporting Craig Silverman is the founder of Emergent, a real-time rumor tracker and debunker. He was a fellow with the Tow Center for Digital Journalism at Columbia University, and is a leading expert on media errors, accuracy and verification. Craig is also the founder and editor of Regret the Error, a blog about media accuracy and the discipline of verification that is now a part of the Poynter Institute.

The Invisible Web: What It Is and How You Can Find It By Wendy Boswell Updated June 02, 2016. What is the Invisible Web? The term "invisible web" mainly refers to the vast repository of information that search engines and directories don't have direct access to, like databases. Unlike pages on the visible Web (that is, the Web that you can access from search engines and directories), information in databases is generally inaccessible to the software spiders and crawlers that create search engine indexes. How Big is the Invisible Web?

Cybercrime in the DeepWeb Earlier, we published a blog post talking about the recent shut down of the Silk Road marketplace. There, we promised to release a new white paper looking at cybercrime activity on the Deep Web in more detail. This paper can now be found on our site here. While the Deep Web has often been uniquely associated with The Onion Router (TOR), in this paper we introduce several other networks that guarantee anonymous and untraceable access — the most renowned darknets (i.e., TOR, I2P, and Freenet) and alternative top-level domains (TLDs), also called “rogue TLDs.” We analyzed how malicious actors use these networks to exchange goods and examined the marketplaces available in the Deep Web, along with the goods offered.

Drug Market 'Agora' Replaces the Silk Road as King of the Dark Net dzima1/Getty For two and a half years, the Dread Pirate Roberts and his Silk Road black market ruled the Dark Web. But last year’s FBI’s takedown of that narcotics smorgasbord opened the underground trade to competitors. Now those sites have a new leader, one that’s bigger than the Silk Road ever was and continues to grow explosively.

Related: