MemoriseBot

Last updated: 2026-10-18

MemoriseBot is the fetcher Memorise uses when we check a website, for example whether AI search may read it, whether the home page has structured data and whether there is a sitemap. It only fetches public files, never anything behind a login.

How to recognise it

MemoriseBot always identifies itself with this user agent and never changes it:

MemoriseBot/1.0 (+https://memorise.se/memorisebot)

What it fetches

A check fetches robots.txt, the home page and the site's sitemap, and to see when something was last published, at most three child sitemaps. That is around ten requests per check. MemoriseBot does not follow links or crawl the site. Results are kept for ten minutes, so the same site is not fetched again every time someone looks at it.

We measure loading time with Google's own tools. Google then fetches the page with Google's user agent, not MemoriseBot.

robots.txt

MemoriseBot always reads robots.txt first. When someone asks us to check a site, we fetch on their behalf and follow rules addressed to MemoriseBot, but not general rules for all crawlers, since we only fetch a handful of files. When we check a site on our own initiative, we follow the whole robots.txt.

If you do not want MemoriseBot to fetch anything from your site, add this to robots.txt:

User-agent: MemoriseBot
Disallow: /

MemoriseBot will then read robots.txt and stop there.

Firewalls and security checks

If the site responds with a security check, MemoriseBot stops. It never tries to get past a firewall or a captcha and never pretends to be a browser.

Contact

Questions about MemoriseBot, or want us to stop fetching from your domain entirely, without even reading robots.txt? Email hello@memorise.se and we will add the domain to our block list.