Avoiding URL Canonicalisation With mod_rewrite And Apache

22nd February 2008 - 3 minutes read time

URL canonicalisation is where you have a website with different URLs outputting the same content. When search engine spiders see all this content that is the same they can get confused as to what page to display in search engine result pages. The following URLs, although they are different, actually produce the same content.

http://www.example.com
http://example.com
http://www.example.com/
http://www.example.com/index.html

The way to solve this issue is to redirect any requests to a single page using mod_rewrite. Add a .htaccess file to your root directory and include the following line to turn on the engine.

RewriteEngine On

The following rule will redirect the www page to the non-www page.

#Redirecting non-www to www.domain.com:
RewriteCond %{HTTP_HOST} ^domain\.com$ [NC]
RewriteRule ^(.*)$ http://www.domain.com/$1 [R=301,L]

Use the following rule to redirect from the index.html page to the directory name.

#Redirecting /index.html to /:
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /index\.html
RewriteRule ^index\.html$ http://www.domain.com/ [R=301,L]

If you want to detect for the existence of mod_rewrite you can include all of the previous lines in an if statement like this.

<IfModule mod_rewrite.c>
RewriteEngine On

#Redirecting non-www to www.domain.com:
RewriteCond %{HTTP_HOST} ^domain\.com$ [NC]
RewriteRule ^(.*)$ http://www.domain.com/$1 [R=301,L]

#Redirecting /index.html to /:
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /index\.html
RewriteRule ^index\.html$ http://www.domain.com/ [R=301,L]
</IfModule>

 

canonicalisation
mod_rewrite
Apache

Add new comment

The content of this field is kept private and will not be shown publicly.
CAPTCHA
9 + 3 =
Solve this simple math problem and enter the result. E.g. for 1+3, enter 4.
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.

Related Content

Speeding Up Apache And Drupal With Varnish

8th June 2014

Varnish is a web application accelerator that provides an easy speed increase to most web applications and Drupal is no exception. It works by creating a reverse proxy service that sits in front of your web server and caches traffic that comes through it.

Read more

Adding iptables Rules With Ansible

16th February 2014

Many systems and applications require certain access to certain ports and protocols. When installing these systems using Ansible it is necessary to also open up the needed ports so that the systems can function correctly. As there is no iptables module in Ansible the shell command is needed to add the iptables rules.

Read more

Turning Off Apache Basic Authentication For A Single Directory

11th September 2013

When setting up staging sites or similar I often add a simple Apache authentication check in order to stop everyone from viewing the site. This is also useful in stopping search engine spiders from accessing a site with testing content on it, which generally causes trouble. It isn't amazingly secure, but it keeps almost everyone out.

Read more

Using Phing To Create Apache Virtual Hosts

6th April 2012

Phing is an awesome tool for automating things and I use it more and more for automating all kinds of different tasks. One of the tasks that I don't tend to do all that much is setting up a new local virtual host for Apache on my development machines.

Read more

IE8 Downloading docx Files As zip On Apache

23rd January 2012

I recently built a Drupal site on an Apache server setup and everything seemed fine until someone running IE8 tried to download a .docx file. For some reason IE8 insisted that this file was a .zip file and would open it as such, causing a bit of confusion as to what the problem was.

Read more