CRAWLER
The IronFountain-Leads crawler
Iron Fountain runs a research crawler that visits public websites to learn which platform they are built on and whether they have a contact form. It reads a small number of public pages per site. It does not log in, submit forms, solve CAPTCHAs or probe for vulnerabilities.
How it behaves
- It fetches websites by hostname, starting from public lists of domains. It does not sweep IP address ranges or scan ports other than the HTTP and HTTPS ports a site uses.
- It reads
robots.txtfirst and follows it, includingCrawl-delay. Disallowed paths are never fetched. - It waits at least one second between requests to the same website, and limits how many addresses it contacts in any one network or provider within a time window.
- It backs off a whole network for 30 to 60 minutes after repeated timeouts, refused connections or access denials, and stops contacting an address for 15 minutes after an HTTP 429 response.
- Requests come from
157.254.188.41.
How to block it
To stop it visiting your website, add this to your robots.txt:
User-agent: IronFountain-Leads
Disallow: /To exclude a whole network, such as a university or company range, email us the IP ranges or AS number. We add them to a list the crawler checks before every connection, and confirm by reply.
Report a problem
Email complaints@ironfountain.com with the time, your IP addresses and what you saw. We read every message and pause crawling of your network while we look into it.
Green Mango Systems, Inc. · 440 N Barranca Ave, #6264, Covina, CA 91723