NaughtyBot Alpha
Aug 06, 2015
I’ve been working on a little side project for the last couple days: NaughtyBot.
This Chrome extension allows you to quickly and easily browse the contents of robots.txt ‘Disallow’ directives, and check the HTTP status of each path. This can be a fun way to find hidden site functionality, poorly designed security through obscurity, or just plain old stuff people don’t want Google finding.
It’s not ready for Chrome App Store release yet, but you can install it as an unpacked Chrome extension from the tagged release. It’s only just functional, but it’s fun to play with and bug fixes/improvements will be forthcoming in the next few days once I catch up on some sleep and work slows down a bit.
Edit: Wow. So this is a lot buggier than I thought… gimme a couple of days before you try this one (or don’t. I don’t control you.).