myaiteam2/website-crawler

Crawl a website and get all the URLs

Public
363 runs
Playground API Examples README Versions

Works on like 75% of the websites online. Perfect to get all the URLs and then scrape data.