Evidence shows Apple operating a mysterious Web crawling botA developer has discovered that Apple appears to be crawling HTML websites with its own automated bot, but the exact purpose of the mysterious homegrown software remains unknown.
Evidence of Apple's Web crawler, written in the Go programming language, was first discovered by developer Jan Moesen, who has found hits from the software dating back to Oct. 15. Others have discovered significant traffic to their websites from Apple's mysterious crawler, which appears to be shut off from time to time.
The crawler was found originating from an IP address starting with "17.", which would indicate that the bot is emanating from Apple's servers.
It's unknown whether the Apple bot is an official project for the company, or if an employee built and ran the system in an unofficial capacity from a company machine.
It's possible that Apple's efforts could be related to Spotlight enhancements in both OS X Yosemite and iOS 8. Both were updated this fall with improvements that allow users to access Web search content directly, without the need to access a third-party search tool such as Google or Bing.
Though Apple doesn't compete directly with the likes of Google or Microsoft Bing, the company has been working to provide specific information directly within Siri search results. Doing so makes it unnecessary to provide potential links through the official Siri Web search partner, Bing.
On Topic: General
- Prospect of Indian Apple Stores still on the table, minister says
- Caltech sues Apple & Broadcom over alleged Wi-Fi patent infringements
- Apple's VocalIQ takeover could hint at Siri upgrades for WWDC and beyond
- Apple Stores suffering from 'cult' atmosphere, advancement barriers, says UK staffer
- This week on AI: New MacBook Pro rumors, Apple's answer to Echo, Apple Car charging & more