607 private links
I occasionally encounter feeds that my feed reader cannot subscribe to because the site uses Cloudflare DDoS protection, which Cloudflare implements in a manner contrary to the ethics of the open web, in the process making it near impossible for feed readers or read-it-later services to access the content.
I have no simple work-around for such RSS feeds - they are effectively rendered useless by Cloudflare's discriminatory and user-hostile blocking implementation.
Cloudflare themselves claim doing nothing wrong, and that it is in fact the site operators that have misconfigured their Cloudflare firewall and that users should contact the site operator. Which is simply ridiculous.
If Cloudflare cared at all about the open web, this issue would not be relegated to a few obscure forum posts. They could also easily implement some form of automatic exemption from their page blocking for common feed endpoints, such as /feed, rss.xml and similar.
- Cloudflare considered harmful, 2019-10-23, Hugo Landau
- https://git.nixnet.services/you/stop_cloudflare
- Stay away from Cloudflare, 2017-12-20, unixsheikh.com
- https://news.ycombinator.com/item?id=12646055 (thread on the subject of Cloudflare and RSS)
- https://reddit.com/r/selfhosted/comments/qars38/rss_feed_behind_cloudflare_protection/
- https://stackoverflow.com/questions/11886711/curl-cant-fetch-rss-from-website-because-of-cloudflare
- https://github.com/VeNoMouS/cloudscraper (a work-around in Python, but I don't know how to incorporate with TinyTinyRSS...)
More awfulness: Cloudflare's Browser Integrity Check
Cloudflare's Browser Integrity Check looks for [...] HTTP headers abused most commonly [...] and denies access to your page. It also challenges visitors without a user agent or with a non-standard user agent [...]. Browser Integrity Check is enabled by default [my highlight].
https://developers.cloudflare.com/waf/tools/browser-integrity-check
I have personally experienced how RSS feeds that resolve normally in my browser still fail to fetch (403 forbidden by the Cloudflare proxy) in both TinyTinyRSS and Nextcloud News because the site owner uses Cloudflare and has not explicitly disabled "Browser Integrity Check" for their RSS endpoints. My recommendation for site owners is to avoid Cloudflare like the plague.
- https://stackoverflow.com/questions/71529199/where-does-cloudflare-detect-web-and-terminal-requests-on-equal-terms - 2022-03-18
- https://news.ycombinator.com/item?id=31318456 - Cloudflare's browser integrity check feature (2022-05-09).
- https://www.ghacks.net/2022/05/05/fix-pale-moon-browser-not-passing-cloudflares-checking-your-browser-verification
- https://reddit.com/r/CloudFlare/comments/15tiaci/cloudflare_zero_traffic - 2023-08-17
- https://github.com/chromiumembedded/cef/issues/3547 - 2023-07-23