Applebot - Not Obeying robots.txt
"Applebot is the web crawler for Apple. Products like Siri and Spotlight Suggestions use Applebot. It respects customary robots.txt rules and robots meta tags, and
it originates in the 184.108.40.206 net block."
Applebot/0.1 does not fully obey robots.txt as it interprets allow entries for Googlebot as implied permission for Applebot, illustrated by the following example:
This scenario presents potential privacy implications for site owners who may want their content made available to Google but not Apple.
September 19, 2019 - Provided the details to Apple via email@example.com
September 19 , 2019 - Apple sent an auto acknowledgment
October 1, 2019 - Apple responded stating that they are investigating
October 29, 2019 - Asked for a status update
November 4, 2019 - Apple responded stating that they do not see any security implications
July 7, 2020 - Published an advisory to document the issue
If you do not want Apple to index your site, explicitly restrict Applebot in robots.txt using the following example: