Avoiding Duplicate Content On Your Own Website
Posted on September 15, 2008 – 7:20 pm by admin
Today has been an interesting day. We have been taking a look at our websites and searching for duplicate content using Copyscape. After today’s findings, we might just go with Copyscape’s premium service.
Now, let me just tell you that duplicate content is everywhere. Actually, someone has probably written this sentence a million times. What we were searching for today was blatant and far reaching content theft. We found a few instances of one of our homepages and general website idea taken for someone else’s use as well as many instances of interior pages taken. Needless to say, we made screen copies of these cases and sent them to our attorney’s office. These are serious and can’t be ignored.
I would like to talk about two things you can do to help out a more subtle form of duplicate content, on your own website.
The first form of duplicate content on your own website is in the form of www vs. non-www. If you go to your website and type in “www.mysite.com” and then type in “mysite.com,” you may see the same page appear. In the search engine’s eyes, these are two copies of the same page. How do you fix this? It’s easy. Just open up your .htaccess file and type in the following code:
RewriteEngine On
RewriteCond %{HTTP_HOST} !^www\.mysite\.com
RewriteRule ^(.*)$ http://www.mysite.com/$1 [R=permanent,L]
When someone types in “mysite.com” to visit your website, they will automatically be forwarded to “www.mysite.com.” The search engines will be forwarded as well.
Another form of duplicate content on your own website comes in the form of “www.mysite.com/” vs. “www.mysite.com/index.html.” The search engines see this same page as two different ones. What to do? That’s easy too. Just open up your .htaccess again and type in the following code:
RewriteEngine On
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /index\.html\ HTTP/
RewriteRule ^index\.html$ http://www.mysite.com/ [R=301,L]
When someone either types in “www.mysite.com/index.html” or follows a link like that to your website, they will be automatically be forwarded to “www.mysite.com.”
Now, here is the disclaimer. I used this on my server setup and it worked. Please check with your own hosting company to see if something similar will work for your too.
