A robots.txt generator
By Adam Gaffin, NetworkWorld.com, 11/16/04
No, robots.txt files are not the hardest things in the world to write - given that there's not all that much you can really say in them. Still, Ian McCanerin's Robots.txt Generator might save you a couple of minutes. The page on which you fill out the information needed to generate your file also gives you basic background on what robots.txt files are.
Via Search Engine Watch.
TrackBack
Back to Compendium
Comments
Post a comment
