Do AI Crawlers Train on Your Photos? 8 Image Hosts' robots.txt and ToS, Checked
Here's the short version before the data: out of 8 image hosts and photo services checked, only Catbox names GPTBot, ClaudeBot, Google-Extended, and CCBot in its robots.txt and blocks them outright. Flickr reaches roughly the same outcome through a different mechanism. Imgur's robots.txt leaves the door wide open, but its terms of service turn around and explicitly ban third parties from using platform content for AI training. ImgBB and Postimages ship an empty robots.txt — no rules at all. And lpic.cc, as of this check, has no AI-crawler-specific rule either. Everything below is something I actually fetched this round, not something reconstructed from documentation — checked in September 2026.
How this was checked
I ran curl directly against 8 domains to pull the raw robots.txt file — not a third-party tool's interpretation of it — then used a web-fetch tool to read the full text of each service's terms of service, looking specifically for language about "content license scope" or "AI / machine learning training," quoted verbatim where it exists. Anything I couldn't find, or that the terms simply never mention, is marked "not mentioned" — I'm not filling gaps from memory.
Five AI crawler user-agents, each representing a different use case:
- GPTBot — OpenAI's crawler for gathering training data
- ClaudeBot — Anthropic's training-data crawler (a different user-agent from
Claude-User/Claude-SearchBot, which are Anthropic's real-time retrieval bots, not training crawlers) - Google-Extended — Google's opt-out token specifically for training generative AI models like Gemini; separate from Googlebot, which handles search indexing
- CCBot — Common Crawl's crawler; a lot of LLM training datasets are built by processing the raw data it collects
- PerplexityBot — Perplexity's crawler
Full comparison table
Data as of September 2026. "Unlisted" means the user-agent doesn't appear by name in that site's robots.txt — what actually happens to it depends on which fallback rule it lands in.
| Service | GPTBot | ClaudeBot | Google-Extended | CCBot | PerplexityBot | ToS mentions AI training? |
|---|---|---|---|---|---|---|
| Imgur | Unlisted → falls to * rule, allowed |
Unlisted → allowed | Unlisted → allowed | Unlisted → allowed | Unlisted → allowed | Yes — explicitly bans third parties from scraping/using platform content for AI/ML purposes |
| ImgBB | Allowed (empty robots.txt) | Allowed | Allowed | Allowed | Allowed | Not mentioned |
| Postimages | Allowed (empty robots.txt) | Allowed | Allowed | Allowed | Allowed | Not mentioned |
| Catbox | Blocked (named) | Blocked (named) | Blocked (named) | Blocked (named) | Unlisted → falls to * rule, allowed |
Not mentioned |
| Flickr | Blocked (not on allowlist, falls into User-agent: * / Disallow: /) |
Blocked (same; but Claude-User and Claude-SearchBot, both non-training UAs, are separately allowed) |
Blocked (same) | Blocked (same) | Allowed (named on the allowlist) | Not mentioned |
| Google Photos | Unlisted; sharing/album paths blocked for * across the board |
Same | Same | Same | Same | Google's main ToS bans "using content from the Service to train machine learning models," but doesn't call out Photos specifically |
| iCloud shared albums | Unlisted; /sharedalbum/ and /photos/ blocked for * (Googlebot is the one named exception, allowed sitewide) |
Same | Same | Same | Same | Not mentioned; Apple's Apple Intelligence privacy page separately states "private content will not be used to train foundation models without consent" |
| lpic.cc | Allowed (Allow: /, only /admin/ and /api/ blocked) |
Allowed | Allowed | Allowed | Allowed | Not mentioned; license scope is limited to "storage, processing (e.g. compressing/converting formats), and transmission," not a blanket "any purpose" grant |
robots.txt is a gentleman's agreement — worth saying plainly
Everything in that table describes what the rules say, not whether crawlers actually obey them. robots.txt carries zero technical enforcement — a site can write Disallow: / next to any user-agent it wants, and whether that crawler actually respects it is entirely up to the crawler operator's own conduct. This round only verified the rule text itself; I did not separately measure whether these five AI companies' crawlers actually comply with the rules they claim to follow — that would take a completely different traffic-analysis methodology, not something you can answer by reading policy pages. Keep that in mind for the lpic.cc section further down — it matters for how that section should be read.
Breaking down the logic behind each rule
Catbox: the most complete blocklist, but it misses PerplexityBot
Catbox's robots.txt takes the most direct approach: a general rule for all user-agents (blocking a handful of functional paths like /contact.php and /user/), followed by an explicit named block of ClaudeBot, Claude-User, Claude-SearchBot, GPTBot, Google-Extended, CCBot, Meta-ExternalAgent, FacebookBot, Bytespider, and Applebot-Extended — ten training or aggregation-type user-agents, all hit with a sitewide Disallow: /.
But PerplexityBot isn't on that list. Since it's not named, it falls back to the general rule, which means the entire site (minus those functional paths) is open to it. I don't know whether that's an oversight in an unmaintained list or a deliberate call — Perplexity officially positions itself as a real-time retrieval assistant rather than a training-data crawler — and that's something this check couldn't determine and isn't going to guess at. What's certain is simpler: even the most thoroughly blocked host in this comparison still has one concrete, verifiable gap.
Flickr: an "allowlist" reaching the opposite of Catbox's approach
Flickr's robots.txt works in reverse: it lists a long allowlist of permitted user-agents first (Googlebot, bingbot, ChatGPT-User, Claude-SearchBot, Claude-User, OAI-SearchBot, PerplexityBot, and others), grants them access minus a few excluded paths, then closes with a catch-all User-agent: * / Disallow: /.
In other words, any user-agent not explicitly invited onto that allowlist is blocked sitewide by default. GPTBot, ClaudeBot, Google-Extended, and CCBot are all absent from the allowlist, so all four fall into that trailing Disallow: /. The net effect — search/retrieval crawlers let in, large-scale training crawlers shut out — lands in the same place as Catbox's manual blocklist, just implemented the opposite way: one is a blocklist, the other an allowlist.
Imgur: the front door is unlocked, but the legal door is bolted
None of the AI-related user-agents get any special treatment in Imgur's robots.txt — the general rule only blocks account, download, and logout-style functional pages, leaving image and gallery pages open to every crawler. But flip to its terms of service, and the third-party usage restriction gets remarkably blunt:
"...create derivative works based on our service...including for any purposes related to AI machine learning/use on AI platforms..." (part of a list of prohibited actions)
and:
"...any use of our service...for any commercial purpose, including without limitation AI machine learning, use on AI platforms, etc., you must first obtain our express signed written consent..."
That prohibition is written for third parties — it's not a self-restriction. There's no matching clause about how Imgur's parent company, MediaLab.Ai, uses content on its own platform. So the precise way to put it: Imgur's technical rule (robots.txt) lets everyone in the front door, while its legal rule (ToS) bans anyone who walks in from using what they scraped for AI training. That's a "you can crawl, but you can't use it for this" combination — a completely different strategy from Catbox and Flickr locking the door with technical means.
ImgBB and Postimages: the rulebook is blank
I verified both of these directly with curl — the HTTP response is Content-Length: 0. That's not a failed fetch; the site really does return an empty file. No Disallow rules of any kind, meaning every crawler (all five AI crawlers included) has technically unrestricted access. Neither ToS mentions AI training either, but both grant themselves an extremely broad content license — ImgBB's language literally reads "for any purpose, commercial, advertising, or otherwise." That broad grant is signed over to the platform itself, not a blank check handed to any AI company — but it also means nothing in the terms currently stops the platform from licensing that content out for AI training down the road if it chose to.
Google Photos and iCloud shared albums: maybe the wrong question entirely
Neither service's robots.txt writes a dedicated rule for any of the five AI crawlers, but the general rule blocks off /photo/, /shared, /sharing (Google Photos) and /sharedalbum/, /photos/ (iCloud) wholesale for anyone not separately allowed — none of the five AI crawlers gets an exception.
But there's a structural difference from the other six that's worth flagging clearly: Imgur, ImgBB, Postimages, Catbox, Flickr, and lpic.cc are all designed so their image pages are publicly browsable by default (several even get indexed by search engines; lpic.cc's view pages carry noindex). Google Photos albums and iCloud shared albums, by contrast, run on unguessable URLs by design — there's no public listing or on-site link pointing to them at all. Whether robots.txt blocks a crawler or not barely matters for a "the URL itself is the secret" architecture — a crawler has to discover that URL before robots.txt even becomes a relevant question. That's a fundamentally different protection logic from the other six, where pages sit in public indexes and robots.txt is the actual fence. Worth spelling out clearly here, or it's easy to misread this as "Google and Apple use the same strategy as Catbox."
What Cloudflare just did, and whether it's connected
On September 15, 2026, Cloudflare's official blog published "Have it both ways: stay discoverable in search while disallowing AI training," formally splitting crawler behavior into three categories: Search (building search indexes), Training (training or fine-tuning models), and Agent (real-time actions a user directly asked an assistant to perform, like looking something up on your behalf). The post states that, starting that day, every domain newly onboarded to Cloudflare gets a new default: ad-supported sites default to "allow search, block training, block agents on ad pages"; sites without ads default to allowing all three. This follows on from Cloudflare's Content Signals Policy, launched in September 2025, which introduced search / ai-input / ai-train signal extensions to the robots.txt syntax.
I did not check, and I'm not going to pretend I checked, whether these 8 image hosts are actually Cloudflare customers — this article doesn't verify that, so none of the above implies they've adopted Cloudflare's new defaults. Worth noting as a coincidence, not a causal claim: Flickr's existing robots.txt structure happens to match the exact logic Cloudflare is now trying to popularize (allow search/agent crawlers, block training crawlers) — but Flickr built that manually with its own hand-maintained user-agent list, unrelated to Cloudflare's new automated defaults.
lpic.cc's current policy — the honest version
lpic.cc's robots.txt currently reads like this (checked September 2026):
User-Agent: *
Allow: /
Disallow: /admin/
Disallow: /api/
Sitemap: https://lpic.cc/sitemap-index.xml
There's no dedicated rule for GPTBot, ClaudeBot, Google-Extended, CCBot, or PerplexityBot — at the robots.txt layer, all five are currently allowed. On the terms-of-service side, the license grant reads "you grant this service the right to store, process (e.g. compress or convert formats), and transmit uploaded content" — considerably narrower than Imgur's, ImgBB's, or Postimages' "any purpose" blanket grants, but the terms don't use the words "AI" or "machine learning" either. No promise, no ban.
Put those two facts together, and the honest statement is: lpic.cc's robots.txt and terms of service currently have no block against AI training crawlers, putting it in the same category as Imgur, ImgBB, Postimages, Google Photos, and iCloud — nothing here makes it better than those. (The CDN sitting in front of the site has its own separate crawler-management configuration, and that layer is outside the scope of what this article checked — this isn't vouching for it or claiming any protective effect from it.) We're not going to say "uploads to lpic.cc are less likely to get used for AI training," because no rule currently backs that claim — and even adding something like Disallow: GPTBot wouldn't change the fact, covered above, that it's still just a gentleman's agreement nobody has measured actual compliance with. Whether to follow Catbox's or Flickr's example going forward is a separate product decision from whether this article should describe the current state honestly.
Which one fits your situation
| What you care about | Look at | Why |
|---|---|---|
| A host that actively, technically blocks known training crawlers | Catbox | Named blocks on 4 of 5 target crawlers — the only one in this check doing so |
| Public sharing plus search visibility, without training-crawler scraping | Flickr | The allowlist approach happens to land on "searchable, not trainable" — though that's an existing operational choice, not a public AI-policy commitment |
| Just want to host images for free, don't care about AI issues either way | ImgBB / Postimages | No robots.txt restrictions, fastest to get started — but the terms grant an extremely broad content license, effectively defaulting to "the platform may use this for any purpose" |
| Want the platform's own license grant to be as narrow as possible (storage/processing/transmission only) | lpic.cc | The narrowest license language in this comparison, but with no matching technical block against AI crawlers |
| Want photos to never be publicly exposed or indexed in the first place | Google Photos shared albums / iCloud shared albums | Relies on unguessable URLs rather than robots.txt; a fundamentally different protection logic from the other six |
FAQ
Q: If robots.txt says Disallow, does that guarantee an AI company won't crawl it?
Not necessarily. robots.txt is a voluntary agreement with no technical enforcement behind it. This check only verified what the rules say — it didn't separately measure whether these five AI companies' crawlers actually comply with the rules they claim to follow.
Q: The terms of service don't mention AI training at all — does that mean my photos are safer? Not necessarily. Silence usually just means it wasn't a priority when the terms were written, or the platform deliberately kept its options open. Set against Imgur's, ImgBB's, and Postimages' "any purpose" style of broad license grants, an unclear clause can be a risk rather than a reassurance.
Q: Does lpic.cc specifically block AI training crawlers? Not at the robots.txt layer. As of the September 2026 check, robots.txt allows all five target crawlers (the CDN's own crawler-management layer is outside what this article checked). That's an honest description of the current state, not a promise it won't change.
Q: Does Cloudflare's new three-way split (Search / Training / Agent) directly relate to these 8 image hosts? Not directly. This is a platform-level mechanism that applies to domains onboarded to Cloudflare, and this article didn't verify whether any of these 8 are Cloudflare customers. Flickr's existing rule structure happening to resemble it is purely coincidental, not evidence Flickr designed it in response to Cloudflare's announcement.
Q: Why isn't PerplexityBot on Catbox's block list? Unknown — this check found no public statement from Catbox's operators explaining it. It could be an unmaintained list, or a deliberate call, since Perplexity positions itself as a real-time retrieval assistant rather than a pure training-data crawler. This only surfaces the gap in the rules themselves, without guessing at the motive behind it.
Q: Does the EU AI Act change any of this for European users?
Not directly for robots.txt behavior. The AI Act's transparency obligations mostly apply to AI providers (documenting training data sources, respecting opt-out signals like TDMRep or robots.txt-based signals under the EU's text-and-data-mining exception), not to the image hosts themselves. So an EU-based photo host having no AI-crawler rule in robots.txt doesn't violate the Act — the compliance burden sits mainly on whoever operates GPTBot, ClaudeBot, and the rest to honor opt-out signals when they exist.
Sources and check dates
Everything below was fetched or curled directly on 2026-09-25, not relayed from a third-party summary:
- Imgur robots.txt:
https://imgur.com/robots.txt - Imgur terms of service:
https://imgur.com/tos - ImgBB robots.txt:
https://imgbb.com/robots.txt(HTTP 200, Content-Length: 0) - ImgBB terms of service:
https://imgbb.com/tos - Postimages robots.txt:
https://postimages.org/robots.txt(HTTP 200, Content-Length: 0) - Postimages terms of service:
https://postimages.org/terms - Catbox robots.txt:
https://catbox.moe/robots.txt - Catbox terms/privacy page:
https://catbox.moe/legal.php - Flickr robots.txt:
https://www.flickr.com/robots.txt - Flickr terms of service:
https://www.flickr.com/help/terms - Flickr privacy policy:
https://www.flickr.com/help/privacy - Google Photos robots.txt:
https://photos.google.com/robots.txt - Google terms of service:
https://policies.google.com/terms - Google privacy policy:
https://policies.google.com/privacy - iCloud robots.txt:
https://www.icloud.com/robots.txt - iCloud terms of service:
https://www.apple.com/legal/internet-services/icloud/ - Apple Intelligence privacy page:
https://www.apple.com/privacy/features/ - lpic.cc robots.txt:
https://lpic.cc/robots.txt - lpic.cc terms of service: on-site
/termspage, corresponding i18n copy (termsIntellectualDesc) - Cloudflare official blog (three-category crawler policy, published 2026-09-15):
https://blog.cloudflare.com/accountable-mixed-use-ai-crawlers/ - Cloudflare Content Signals Policy (the predecessor policy, published 2025-09-24):
https://blog.cloudflare.com/content-signals-policy/