About our crawler
If you are reading this, you found optstub.com/bot in your server logs. This page explains what our crawler is and how to stop it.
Who we are
OptStub is a personal data removal service. People ask us to submit opt-out and deletion requests on their behalf to data broker and people-search sites, and we re-check afterwards to see whether the request was honoured.
What the crawler does
- Fetches publicly accessible opt-out and privacy pages to keep our removal guides accurate.
- Re-checks a specific public profile URL after a removal request, so we can tell a customer whether the page is still there.
What it does not do
- It does not solve CAPTCHAs or use any CAPTCHA-solving service.
- It does not rotate residential proxies or otherwise evade a block.
- It does not log in, create accounts, or touch anything behind authentication.
- It honours
robots.txtand rate limits, and it identifies itself on every request. - It does not collect or retain data about people who are not our customers.
How to block it
Add this to your robots.txt and we will stop:
User-agent: OptStubBot Disallow: /
We re-read robots.txt before crawling and cache it briefly. A block takes effect on the next fetch — you do not need to contact us first.
Contact
Questions, complaints, or a request to stop for reasons robots.txt cannot express: bot@optstub.com. A person reads that address and replies.