605 private links
Something I would like to be able to do: mirror all my issues that I own from across various code forges (Github, Codeberg, etc.) into a single list of my choosing, for example, as an RSS feed or a Markdown list or whatever.
Actually, as RSS feeds this already works (one feed per forge).
But what I have in mind would go further than that. Ideally, any tags associated with each issue should be visible, and full-text search should be possible. Also, the list should clearly indicate closed issues as such.
The point is, to have a single place to check to remind myself of any issues I have opened. This kind of mirroring would also insulate you from short-term service interruptions on the forge, assuming the project in questionn is still available locally (which it always is, thanks to git). But moreso, this would also help as a sort of backup in case the forge went away completely (say Github gets sold and scuppered, or whatever).
I am considering this today because I just spent the better part of the afternoon moving, manually, "tasks" from my Nextcloud instance to their proper place as issues on my Codeberg repos.
I don't know how to achieve this. I am pretty sure it needs to involve authentication to each forge. Is anyone aware of some existing work in this vein?
I occasionally encounter feeds that my feed reader cannot subscribe to because the site uses Cloudflare DDoS protection, which Cloudflare implements in a manner contrary to the ethics of the open web, in the process making it near impossible for feed readers or read-it-later services to access the content.
I have no simple work-around for such RSS feeds - they are effectively rendered useless by Cloudflare's discriminatory and user-hostile blocking implementation.
Cloudflare themselves claim doing nothing wrong, and that it is in fact the site operators that have misconfigured their Cloudflare firewall and that users should contact the site operator. Which is simply ridiculous.
If Cloudflare cared at all about the open web, this issue would not be relegated to a few obscure forum posts. They could also easily implement some form of automatic exemption from their page blocking for common feed endpoints, such as /feed
, rss.xml
and similar.
- Cloudflare considered harmful, 2019-10-23, Hugo Landau
- https://git.nixnet.services/you/stop_cloudflare
- Stay away from Cloudflare, 2017-12-20, unixsheikh.com
- https://news.ycombinator.com/item?id=12646055 (thread on the subject of Cloudflare and RSS)
- https://reddit.com/r/selfhosted/comments/qars38/rss_feed_behind_cloudflare_protection/
- https://stackoverflow.com/questions/11886711/curl-cant-fetch-rss-from-website-because-of-cloudflare
- https://github.com/VeNoMouS/cloudscraper (a work-around in Python, but I don't know how to incorporate with TinyTinyRSS...)
My list of places where you can find RSS feeds. Quality may vary, buyer beware.
- RSS Database by FeedSpot. I have no knowledge of the quality of these collections.
- https://codeberg.org/solarchemist/svenska-tidskrifter-rss - my own collection of RSS feeds to Swedish newspapers.
Good video. RSS is the way to go.
Yes, please.
RSS-Bridge
RSS-Bridge is a PHP project capable of generating RSS/Atom feeds for a multitude of sources. You self-host it on your own webserver or run it in stand-alone CLI mode.
Supports a long list of content sources (aka bridges).
- List of public RSS Bridge instances
RSSHub
RSSHub (over 17k stars on Github, wow!). It even has a browser extension and a mobile app. Docs.
Via reddit
"Kill the Newsletter!"
Kill the Newsletter!, by Leandro Facchinetti. Converts email newsletters into Atom feeds.
You can use the hosted service kindly offered by its creator, or self-host it yourself.
Via reddit.
FiveFilters
Web service that lets you create RSS feeds for any web sites that don't support it natively, among other useful content-related services. Offers a free tier, no account necessary. Paying customers also get access to the code and the ability to self-host the service (FiveFilters provide Puppet scripts).
FiveFilters maintains an open-source repo with site-specific content extraction rules that is widely used.
RSS Anything
Transform any old website with a list of links into an RSS Feed.
Uses Diffbot's Extract API to transform lists of links on websites into an RSS feed.
PolitePol
PolitePol is a web service where you can create RSS feed for any page on the internet using
interactive XPath selection. Create either XML or JSON feeds.
Service is not FOSS; pricing appears to depend on number of feeds. Unless you sign up the service is quite hobbled (feed lifetime 14 days, and feed never updates).
Feed43
Web service that lets you create RSS feeds for any web sites that don't support it natively. Offers a free plan, with or without first registering an account, but the code behind the service is proprietary.
RSS Box
RSS Box by Stefan Sundin, supports:
- Twitter (but Nitter may be better, also see this excellent guide by FiveFilters)
- Youtube (YT actually offers RSS feed per channel, also consider Invidious)
- Vimeo
- Instagram (another option is Bibliogram)
- Periscope
- SoundCloud
- Mixcloud
- Twitch
- Speedrun
- Dailymotion
- Imgur
- SVT Play
Feed me up, Scotty!
Feed me up, Scotty!, by Vincent Tunru. Hat-tip to Marie Dubremetz.
Notifier
Notifier, by Kirill Maltsev.
Can generate RSS feeds from emails and Telegram channels.
Offers up to 100 notifications per month for free. For 4 €/month you could receive up to 20,000 notifications.
Via kickscondor.
Echo (RSS cross poster)
Echo is a node
script that you can self-host to post new items from an RSS feed to services such as Mastodon.
The author also offers a hosted version at https://echofeed.app.
- In 2013 Google killed Google Reader
- Firefox removed feed discovery features in 2019 (available as plugins now, for example Awesome RSS)
- I've written a linknote on how to subscribe to Youtube channels using RSS
- About Feeds, a primer on web feeds/RSS by Matt Webb.
- You Need Feeds, quickly explains what RSS is and how to use feeds. Via @killyourfm@layer8.space.
- Vad är RSS?, bra intro till RSS av Kamratdataföreningen Konstellationen.
- Youtube channel RSS feeds
- Github RSS feeds
- https://feedcanary.com via https://kevquirk.com/dont-change-your-rss-url
- https://andregarzia.com/2024/05/feed-and-blogrolls-discovery.html
- https://defaults.rknight.me
- https://rubenerd.com/does-your-site-refresh-have-rss - Does your site refresh still have RSS?
blog posts
- p1k3, 2020-05-08
- Staying informed without big tech, 2021-07-03, The New Oil
- How to read RSS in 2020, 2020-03-24, Laura Kalbag
- A thorough comparison of three self-hosted RSS readers, 2020-05-31, Luke Singham
- Felicia Day's amazing rant about RSS, 2020-12-16
- Why I still use RSS, 2021-03-03, Marc @0066cc
- Robin Rendle writes about how RSS is the "promised land", via Roy Tang
- How to Subscribe to or Read RSS or ATOM News Feeds, Christopher Heng
- What is RSS?, 2021-09-11, Ali Reza Hayati
- What using RSS feeds feels like, 2022-02-13, Ben Werdmüller
- Styling OPML and RSS with XSLT to XHTML, 2023-06-22, Rubenerd. Something to consider for my own blog.
- RSS for Post-Twitter News and Web Monitoring, 2023-07-06, ResearchBuzz (Tara Calishain). A long, very informative article on RSS and how to use it.
- A well curated RSS feed reader is a wonderful substitute for social media
- You should be using an RSS reader, Pluralistic, 2024-10-16
- Ask HN: What RSS Reader do you use?
feed readers
- Tiny Tiny RSS, this is what I've been using for years. Android app(s) on F-Droid, and there's at least one iOS app.
- Miniflux
- Vienna RSS
- FeedReader, a desktop client.
- Fresh RSS
- Newsboat - can integrate with TTRSS, or read an OPML file. https://blog.sanctum.geek.nz/rss-with-newsboat
- https://codeberg.org/newsraft/newsraft - feed reader with ncurses UI, inspired by Newsboat.
- RSS Guard desktop RSS reader for Linux, BSD, OS/2, macOS, or Windows that supports RSS/ATOM/JSON/iCalendar/Sitemap feeds and also web-based services such as Feedly, Nextcloud News and TinyTinyRSS.
- Winds
- Stringer
- Selfoss
- feedbin
- Leed
- commafeed
- Feedbro is a feature-rich browser plugin feed reader, but it's not open source, so not recommended.
- https://www.huguesross.net/code/singularity - desktop reader, support for RSS, RDF, and Atom, and JSON-Feed.
- https://github.com/iamaziz/TermFeed Python-based, last commit 10 yrs ago
- Nom - https://github.com/guyfedwards/nom https://www.omglinux.com/command-line-rss-readers-for-linux
- https://github.com/sheepla/srss - Simple-RSS, written in Go, inspired by Newsboat. https://medevel.com/srss
- https://git.codemadness.org/sfeed - sfeed, docs have an example dmenu integration. https://codemadness.org/sfeed-simple-feed-parser.html
- tuifeed - https://github.com/veeso/tuifeed
- https://github.com/iidrees/cli-rss-feed-reader - looks simple, just a single Python script. Based on the feedparser package.
- https://codeberg.org/serghei/ozean - simple web-hosted RSS reader made with Go.
- https://github.com/Clortox/rss-cli https://old.reddit.com/r/commandline/comments/os0wom/rsscli_a_simple_program_for_parsing_rss_feeds_for
- https://github.com/gvalkov/rsstail.py - rsstail, Python, actively developed.
- https://github.com/xqtr/feedln
- https://github.com/YoloSwagTeam/feedstail
- https://github.com/mackers/theyoke - last commit 12 yrs ago, written in Perl
- https://github.com/tylerharper/wag - last commit 15 yrs ago, written in Python
Note: this list is not exhaustive. If you know of a feed reader I've missed, let me know and I'll add it.
Tools that make it easier to add sources to your feed reader
- FiveFilters: Create Feed and other services.
- https://rssgizmos.com (including feed freshness checker, OPML maker, and OPML peeler).
- Feedsearch, an API service for searching websites for their RSS, Atom or JSON feeds.
- GetRSSFeed. Shows RSS feed URL for Soundcloud or Apple "podcasts".
- FetchRSS. Works well for those Soundcloud "podcasts", but this site requires registration (which suggests that this site is creating their own feed endpoint instead of simply exposing soundcloud's already existing (but well-hidden) RSS endpoint).
- rss discovery engine, pre-alpha, by Brandon Quakkelaar (via Rubenerd)
- https://feedle.world a search engine for blogs and podcasts
- Explore RSS feeds in your own neighbourhood (interesting project) via https://mastodon.social/@paulcuth/112020399228840502
- https://siftrss.com - Nifty! It even supports regular expressions. But no source code?
Is the podcast you want to listen to hosted on Soundcloud? First of all, double-check that the podcast is not also hosted somewhere else (more often than not that is the case, and then you can simply avoid Soundcloud).
Soundcloud makes it really hard to discover the podcast's RSS feed. It is not shown anywhere on the page, nor in the HTML source.
Based on a feed URL we already possess and that channel's Soundcloud landing page, I think the following method is necessary to manually construct the hidden RSS feed endpoint for any Soundcloud "podcast":
- Open the HTML source of the Soundcloud "podcast", for example
https://soundcloud.com/user-268302561
. - In the HTML source, find the user ID property, which looks like this
soundcloud://users:819898639
(I suggest search forsoundcloud://users
). - Copy the numeric part, and construct a URL like this:
https://feeds.soundcloud.com/users/soundcloud:users:819898639/sounds.rss
.
Take a moment to reflect on the awfulness of siloed web services and do your best to avoid encouraging them in future.
RSS offers the ability to follow your favourite channels without having to be logged in to Google.
Note: you don't actually need to scour through the page's HTML code. Just paste the channel ID (it's a long alphanumeric string in the URL when looking at the channel's landing page) into the end of this URL:
https://www.youtube.com/feeds/videos.xml?channel_id=
Here's some example links to the RSS feed of some popular Youtube channels:
- https://www.youtube.com/feeds/videos.xml?channel_id=UCHnyfMqiRRG1u-2MsSQLbXA - Veritasium
- https://www.youtube.com/feeds/videos.xml?channel_id=UCEIwxahdLz7bap-VDs9h35A - Steve Mould
- https://www.youtube.com/feeds/videos.xml?channel_id=UCtESv1e7ntJaLJYKIO1FoYw - Periodic Videos
More links
- someone wrote a webapp for that (not that I think it's necessary...). Via reddit