The search engines
The search engines: who actually owns an index
There are far fewer search engines than there are search interfaces. This guide sorts the market by the only distinction that matters technically — whether an engine maintains its own crawl and index, or serves results licensed from one that does.
Dozens of products call themselves search engines. Only a handful crawl and index the web themselves. The rest are interfaces over someone else’s index, which is a legitimate business but a very different technical proposition.
Engines with their own index
Google operates the largest web index and the crawler most site owners deal with first. Bing maintains its own full index and supplies results to a number of other products. Yandex and Baidu run independent indexes with strong regional coverage of Russian and Chinese content respectively. Mojeek and Right Dao are small independent crawlers, notable because building an index from scratch at any scale is genuinely difficult.
Brave Search built its own index and describes its independence from other providers as a core feature.
Interfaces over another index
DuckDuckGo combines its own crawling for some purposes with results sourced substantially from Bing. Ecosia and Startpage are privacy-focused fronts over other providers’ results. Several AI answer products retrieve through third-party search APIs rather than their own crawl.
None of this is a criticism. It does mean that optimising separately for an interface is usually pointless: if it serves Bing’s results, being indexed by Bing is the actual requirement.
Why the distinction matters
Three practical consequences:
- Crawler access. Each independent index has its own crawler with its own user-agent, and blocking one blocks that entire index. Interfaces inherit whatever their supplier indexed.
- Verification tools. Only engines with their own index publish webmaster tools worth using. Search Console and Bing Webmaster Tools are the two that matter for most sites.
- Coverage differences. A page can be indexed by one engine and not another, and the reasons are usually mundane: different crawl schedules, different duplicate clustering, different quality thresholds.
Market share is not the whole story
Market share figures move slowly and vary by source and by country, so this site does not quote a single number as fact. The stable observation is that Google holds the large majority of global web search, Bing’s index underpins a meaningful share of the remainder including several third-party interfaces, and regional engines dominate in specific markets.
Articles in this guide
The articles for this guide are being written. The guide above covers the whole topic in outline, and the article index lists everything published so far.