What Is Crawl Depth | Syndiket Marketing | SEO, Adwords, Web Design. One of the important things that can be noticed about how to optimize a website for PageRank is the crawler. A crawler is the tool that lets Google finds your website and gather information. This is one of the key elements in how to optimize a website for PageRank. The crawler is used to index your site. Google uses all of its tools in order to find your site and index it. In order to optimize a website for PageRank, Google is using the DeepCrawl. The crawler that is using the DeepCrawl is called the Internal Linking Index.