Daily Shaarli

All links of one day in a single page.

March 10, 2025

Cloudflare is user-hostile and site owners using it should beware

I occasionally encounter feeds that my feed reader cannot subscribe to because the site uses Cloudflare DDoS protection, which Cloudflare implements in a manner contrary to the ethics of the open web, in the process making it near impossible for feed readers or read-it-later services to access the content.

I have no simple work-around for such RSS feeds - they are effectively rendered useless by Cloudflare's discriminatory and user-hostile blocking implementation.

Cloudflare themselves claim doing nothing wrong, and that it is in fact the site operators that have misconfigured their Cloudflare firewall and that users should contact the site operator. Which is simply ridiculous.
If Cloudflare cared at all about the open web, this issue would not be relegated to a few obscure forum posts. They could also easily implement some form of automatic exemption from their page blocking for common feed endpoints, such as /feed, rss.xml and similar.

More awfulness: Cloudflare's Browser Integrity Check

Cloudflare's Browser Integrity Check looks for [...] HTTP headers abused most commonly [...] and denies access to your page. It also challenges visitors without a user agent or with a non-standard user agent [...]. Browser Integrity Check is enabled by default [my highlight].
https://developers.cloudflare.com/waf/tools/browser-integrity-check

I have personally experienced how RSS feeds that resolve normally in my browser still fail to fetch (403 forbidden by the Cloudflare proxy) in both TinyTinyRSS and Nextcloud News because the site owner uses Cloudflare and has not explicitly disabled "Browser Integrity Check" for their RSS endpoints. My recommendation for site owners is to avoid Cloudflare like the plague.

Discovering RSS feeds

My list of places where you can find RSS feeds. Quality may vary, buyer beware.