Robots.txt file

robots.txt

Important tiny file...

The robots.txt file is placed in your www or public_html directory. So the correct location where to put this file is in www.example.com/robots.txt. This file indicates how “deep” the spiders can crawl into your website.

It’s possible to keep a spider out of a directory that’s called “customers” or a directory like “images”. Obviously it’s a request, not a guarantee. And if you don’t want the file /secret1960 to be found by anyone, maybe it’s better not to put it on the internet anyway, and especially not in your robots.txt file. If you want to protect your information you use a password security like .htaccess or a login-script.

Key Takeaway

  • This file can make or break your website if used wrong.
  • Add the robots.txt tag in the public_html or www directory.

SEO Influence Meter

0%

The robots.txt file has a lot of
SEO influence if used wrong!

Robots.txt Example

blank Download Example File

robots.txt

Feel free to download our example file in our download area.

Affordable & Effective

We offer affordable SEO services that are fast and effective.

Need some professional
SEO help?

Contact our Customer Service representative to make an appointment and to answer all your questions!

SEO & Online Marketing blog

Budget-Friendly SEO Tips for Independent Graphic Designers

Budget-Friendly SEO Tips for Independent Graphic Designers

https://pixabay.com/vectors/seo-online-analysis-1327870/ If you are an independent graphic designer looking to draw more eyeballs to your dynamic creations online without stretching…
Services to Expect From an SEO Partner

Services to Expect From an SEO Partner

It’s high time for business leaders to accept that they can no longer manage their own company’s SEO in house.…
Mastering Digital Marketing: 5 Steps Students Need to Take

Mastering Digital Marketing: 5 Steps Students Need to Take

https://unsplash.com/photos/gxD8hCmi0IQ In the era of digitalization, a strong understanding and mastery of digital marketing is becoming a vital skill set,…