Research crawler

The crawler enforces URL normalization, public-address resolution, redirect revalidation, robots.txt, page/depth/rate limits, and bounded image collection.