SuperCrawl
A production ready web crawler built in Rust with TTL and duplicate detection.
Why it made the shelf
It is a developer tool and open source library for web crawling written in Rust.
Seen on
- First seen
- 3 months ago
- Best score
- 2 points · 1 comments
Behind the tool
Who built it. How it grew.
People, backing, growth, and ownership. Every claim links to its source.
Official sitegithub.com
Source review queued
Public trail
Where it showed up
1 reference from 1 publisher.
- First trace
- Hacker News3 months ago
- Discussion
- 1 comments
- 01Hacker NewsFirst trace
I built a production-ready web crawler in Rust with TTL and anti-dupe
