Our crawler: shopfinder

If you saw the user agent shopfinder/0.1 (+https://sellmorecountries.com/crawler) in your logs, that was us.

What it does

It looks at public pages of online stores to find stores that might benefit from a translated version: the store's languages, countries, product count and the public contact address. It reads only pages anyone can open. It never logs in, never fills in forms and never tries to get around protections.

How politely

• It follows your robots.txt.
• At least two seconds between two requests to the same store, and only a handful of requests per store in total.
• It stops on errors and never retries a refusal.

How to opt out

Add this to your robots.txt:

User-agent: shopfinder
Disallow: /

Or email hello@sellmorecountries.com with your web address, and we add it to our do-not-contact list for good.