Ultron Labs data collection bot. Reads public prices, availability and business details, and keeps them with a link to their source. Honors robots.txt.
What it does
Reads public pages: product listings, price lists, and the contact and about pages of companies.
Extracts facts from them: prices, stock, product attributes, company contact details.
Keeps each fact with a link to the page it came from.
Reference. Content is cited or linked back to the source.
Access
Direct. Run by Ultron Labs from its own infrastructure.
Every request is signed with Web Bot Auth over @authority and signature-agent, with the Ed25519 key published in the directory above. A request that names UltronCollector without a valid signature is not ours.
How it behaves
Every request carries our user agent and a Web Bot Auth signature (RFC 9421, Ed25519) that sites and Cloudflare can check against the bot's key directory.
Before visiting a site, the bot reads its robots.txt, keeps it for at most 24 hours and obeys the rules for its own name and for *, including Crawl-delay.
At most one request per second to a site. A 429 or 503 answer stops visits to that site for the time it asks, or for 10 minutes if it does not say.
No logging in, no paywall or challenge bypassing, no CAPTCHA solving.