Documentation Index
Fetch the complete documentation index at: https://docs.pipeshub.com/llms.txt
Use this file to discover all available pages before exploring further.
RSS
Subscribe to and sync content from RSS and Atom feeds into PipesHub
✅ Ready📝 Documentation Available
Overview
The RSS connector ingests items from public RSS 2.0 and Atom feeds you configure. It polls each feed URL, normalizes entries, and optionally fetches full article text from each item’s link for richer search indexing. No OAuth or credentials are required—only reachablehttp:// or https:// feed URLs.
You can add multiple feeds in one connector by listing several URLs (comma- or newline-separated). Duplicate URLs are removed automatically.
Configuration setup
Setup guide
Setup guide
Step 1: Create an RSS connector
In PipesHub, add an RSS connector instance with either personal or team scope:- Personal — Go to Workspace Settings > Your Connectors > RSS > Setup. Records are visible only to you.
- Team — Go to Workspace Settings > Connectors > RSS > Setup. Records are visible to everyone in your organization. Admins only.
Step 2: Name the connector
- Enter an Instance name (how it will appear in your workspace).
- Click Next to move to the Configure Records tab.
Step 3: Configure records
On the Configure Records tab, set the following fields:| Field | Description |
|---|---|
| Sync Strategy | Scheduled runs the connector at a fixed interval. Manual syncs only when triggered from connector settings. |
| Sync Interval | Frequency for scheduled syncs (default 1 hour). |
| Feed URLs | Required. RSS or Atom feed URLs (e.g., https://blog.example.com/rss.xml). Separate multiple feeds with commas or newlines. Invalid or non-HTTP(S) entries are skipped; duplicates are removed. |
| Max Articles Per Feed | Maximum recent articles to process per feed on each sync (1–500, default 50). |
| Fetch Full Content | When enabled (default), the connector opens each item’s link and extracts the full article text. When disabled, only the feed-provided summary is indexed — faster but less complete. |
Depending on your workspace configuration, you may also see manual sync filters and indexing toggles to control which records are indexed for search. These can be managed later from the All Records section.
Step 4: Save and start syncing
- Click Save Configuration. A confirmation dialog appears: Instance is configured and ready to sync.
- Click Start Syncing Now to trigger the initial sync, or I’ll do it later to run it from connector controls later.
Connector Workflow
How the RSS connector works
How the RSS connector works
What the connector does
- Feed fetch: Downloads each configured feed URL over HTTP(S) with standard browser-like headers and parses RSS 2.0 or Atom.
- Items: Processes recent entries up to max articles per feed.
- Content: With Fetch full content on, opens each article URL and extracts readable text where possible; otherwise uses the feed-provided description or summary.
- Permissions: Team scope grants organization-level read access to synced records. Personal scope makes records visible only to the user who created the connector; if the creator can’t be identified at sync time, the connector falls back to organization-level read access.
- Sync: Supports scheduled and manual sync; scheduled runs help keep news and blog feeds fresh without operator action.
Key limitations
- Public feeds only: Feeds that require login, cookies, or non-standard auth are not supported.
- Feed and article availability: Some publishers block automated clients, rate-limit aggressively, or serve incomplete summaries. The connector cannot guarantee access to every article page when Fetch full content is enabled.
- Not real-time: Indexing reflects the last successful sync; very high-frequency updates may still appear slightly delayed based on your schedule.
FAQ
Why are some of my feed URLs ignored?
Why are some of my feed URLs ignored?
Only URLs that start with
http:// or https:// are used after splitting on commas and newlines. Empty lines and duplicates are dropped. If every URL is invalid, setup will fail validation—fix the list and save again.Should I enable Fetch full content?
Should I enable Fetch full content?
Yes when article pages hold the real content and the feed only shows a short blurb. No when you prefer faster syncs, lower bandwidth, or the feed already contains full text in each entry.
A feed syncs but article bodies look empty or wrong. Why?
A feed syncs but article bodies look empty or wrong. Why?
The article site may block scraping, require JavaScript-only rendering, or return non-article HTML. Try disabling Fetch full content to rely on the feed summary, or choose a feed that includes fuller descriptions.
How are permissions applied?
How are permissions applied?
For team scope, synced records are readable by everyone in your organization. For personal scope, the user who created the connector is set as owner; if the creator can’t be identified at sync time, the connector falls back to organization-level read access.