Skip to content

SwipelyNewsBot

Effective date: 8 October 2026 (2026-10-08) · Last updated: 8 October 2026

SwipelyNewsBot is the crawler that SwipelyNews uses to find new articles and create short, credited news summaries that link back to the original article.

User agent:

Mozilla/5.0 (compatible; SwipelyNewsBot/1.0; +https://swipelynews.com/bot)

What it does

  • What it reads. It reads the RSS feeds and sitemaps of news outlets we've added, and only recent entries, from the last 48 hours.
  • What it fetches. It fetches the article pages and the images or videos in them, so we can write a short AI summary and show media with credit and a link to the original. See Copyright & Publisher Policy.
  • What it doesn't do.
    • It doesn't log in, and doesn't bypass paywalls, consent walls or other access controls.
    • It doesn't crawl archives: it only reads entries from the last 48 hours.
  • Load. It limits how often it visits each site: at most 500 new articles per source per day.
  • Retention. Article text is deleted after at most 7 days, and is never used to train AI models.

How to control it

  • Block it completely:
    User-agent: SwipelyNewsBot
Disallow: /
  • Block certain sections: add Disallow: rules for those paths.
  • Rules we also honour:
    • User-agent: * rules;
    • TDMRep text-and-data-mining reservations (tdm-reservation: 1 via HTTP header, HTML meta tag, or /.well-known/tdmrep.json);
    • noai / noimageai robots meta directives.
  • Timing. Changes take effect within 24 hours. To remove stories already published, email support@swipelynews.com (subject: "Publisher opt-out").

Contact

Problems with our crawler (load, errors, removal requests): support@swipelynews.com (subject: "Publisher opt-out"). Please include your domain and, if possible, sample request times.