Apple details new 'AppleBot' web crawler used by Siri and Spotlight

article thumbnail

Apple has confirmed the existence of a long-rumored web crawling service — first noticed last November —  and provided some details of its operations in a recently-updated support document.

According to Apple, AppleBot is a crawler that primarily serves Siri and Spotlight. Both services offer web search capabilities, though until now it was thought that Apple routed queries through an existing search engine such as Google or Bing.

AppleBot operates similarly to Google's Googlebot, and Apple advises developers that it will follow Googlebot instructions if no AppleBot-specific instructions are found. Website operators are urged to contact Apple's network operations center if they have questions or concerns about the crawler.

It's not clear how long AppleBot has been operational in production. The support document first appeared in February, but was only populated with content last month.

One web developer spotted Apple's crawler last November, after seeing numerous hits from a crawler in the block. That block is assigned to Apple, one of just a handful of companies that control an entire A block.


Latest News

Latest Videos

article thumbnail

The best new features in macOS Monterey that you'll actually use

article thumbnail

Files app gets much better thanks to iPadOS 15

Latest Reviews

article thumbnail

What to expect from WWDC 2021 - and what not to