Crawl depth

In plain English

When search engine crawlers begin exploring a website, they almost always start at the most prominent, authoritative, and structurally central document: the root homepage. From this initial entry point, the crawler parses the page’s HTML, extracts discovered hyperlinks, and follows those links deeper into the website’s architectural tree.

Crawl depth (frequently called click depth) measures the shortest path of clicks required to travel from the homepage to any specific target URL. A page linked directly from the global navigation header has a crawl depth of 1. A page linked from a major category hub has a crawl depth of 2. If an article can only be reached by clicking through multiple date-based archives, deep category subfolders, and paginated blog archives, it might sit at a crawl depth of 5 or greater. The further a page is buried away from the homepage, the harder it is for both search engine crawlers and real human users to locate.

An example

Consider two different structural architectures for an educational resource website:

In a deep, inefficient site architecture:

Homepage (Depth 0)
└── Resources Directory (Depth 1)
    └── 2023 Archives (Depth 2)
        └── Topic: Databases (Depth 3)
            └── Subcategory: Advanced (Depth 4)
                └── Page 6 Pagination (Depth 5)
                    └── "PostgreSQL Indexing Guide" (Depth 6)

In this architecture, an automated crawler must traverse six sequential hyperlink hops to discover the guide. If the crawler exhausts its allocated crawl budget for the session, encounters a broken link along the path, or stops short after traversing three levels, the guide is never reached.

In an optimized flat site architecture:

Homepage (Depth 0)
└── Database Systems Pillar (Depth 1)
    └── "PostgreSQL Indexing Guide" (Depth 2)

By linking to key topic pillars from the main navigation and linking directly from the pillar to the core guide, the document moves from depth 6 to depth 2. When Googlebot visits the homepage, it discovers and crawls the target document in just two steps.

Why it matters

Crawl depth directly dictates crawl frequency, indexing speed, and PageRank distribution. Link equity naturally dilutes as it travels further down an internal link chain; pages situated at depth 4 or deeper receive very little authority from the homepage. Keeping important, indexable content within three clicks of the homepage ensures that new updates are discovered quickly and rank with sufficient internal authority.

Related terms

Read the full guide to crawling.