SilhouetBot The web crawler for Silhouet.

Silhouet makes software for apparel businesses. SilhouetBot reads public pages on apparel brand and retail websites, such as product pages and sitemaps. We use these pages to learn about the apparel market: what brands make, how they describe it, and how this changes over time.

We only crawl sites on a list that our team reviews. We do not crawl the whole web.

• We do not log in. The crawler has no accounts, no passwords, and no cookies. If a page asks for a login, we stop and do not try that page again for 30 days.
• We do not get around blocks. If your site shows a bot challenge or returns 401 or 403, we stop.
• We do not read private or paid content. We read only pages that anyone can see without signing in.

• We read robots.txt first. We follow RFC 9309. If we cannot read your robots.txt because of an error, we do not crawl. We only treat a missing robots.txt (404 or 410) as permission.
• We crawl slowly. Usually we send about one request each second, with no more than two at the same time. That is slower than one person using your site.
• We slow down when you tell us to. If your server returns 429 or 503, we wait and try again later.
• Each site has a daily page limit.

To block us from your whole site, add this to your robots.txt:

User-agent: SilhouetBot
Disallow: /

To block only part of your site:

User-agent: SilhouetBot
Disallow: /private-section/

We also obey these opt-out signals:

• X-Robots-Tag: noai
• The tdm-reservation header
• /.well-known/tdmrep.json (TDMRep)

We check robots.txt about once a day, so a change can take up to 24 hours to have an effect.

If your robots.txt blocks other AI crawlers by name, such as GPTBot or ClaudeBot, our team treats that as a "no" to us too and stops crawling your site, even if your rules do not name SilhouetBot.

For questions, or to ask us to stop crawling your site now, email us. Tell us your domain.