.htaccess File: A Small, But Effective Tool for Your SEO

1. What is a .htaccess file?
2. How does the .htaccess file work?
3. Where is the .htaccess file located?
4. Why is the correct syntax in a .htaccess file so important?
5. 6 uses of the .htaccess file in search engine optimization
6. Why do you get an error message after updating the .htaccess file?

What is a .htaccess file?

The .htaccess file (short for Hypertext Access) is a web file used on Apache web servers. It configures the server per directory to enable or disable website features. This means that the changes made in the file apply to both the directory in question and its subdirectories.

The most common applications of a .htaccess file include:

• Redirects
• Password protection
• Access restriction for certain IP addresses
• Extension of your server resources

How does the .htaccess file work?

The .htaccess file is placed in a directory on the web server. The commands in the file are executed by the Apache web server when accessing this specific directory. For this reason, it is also common to place the .htaccess file in the root directory of a website. This overrides the default settings and configures the server for the entire website. Like any other Apache configuration file, the .htaccess file is read from top to bottom. This means that the commands at the top are executed before the syntaxes at the end of the file.

Where is the .htaccess file located?

Since the .htaccess file configures the server per directory, you can find it in almost all folders of your web directory. If you have a single web directory with multiple subdirectories, you can place the file in both the root directory and each subdirectory.

If your website is powered by WordPress, you can find the .htaccess file in the WordPress installation directory. Here is the default configuration of the file for most WordPress websites:

# BEGIN WordPress

RewriteEngine On

RewriteBase /

RewriteRule ^index.php$ - [L]

RewriteCond %{REQUEST_FILENAME} !-f

RewriteCond %{REQUEST_FILENAME} !-d

RewriteRule . /index.php [L]

# END WordPress

With different hosting platforms that use cPanel, you can find the .htaccess file via the file manager. Simply navigate to the root directory and there activate 'Show Hidden Files'. If you still can't see the file, you may need to manually create it and upload it to your server. To do this, follow these steps:

1. First, open Notepad.
2. Enter the configuration there or use the standard syntax given above.
3. Save the file in ASCII format with the filename '.htaccess'.
4. Make sure the file is not saved in .txt format.
5. Use an FTP application like Filezilla to upload the .htaccess file to your web directory.

Why is the correct syntax in a .htaccess file so important?

The .htaccess file is a highly sensitive configuration file. Any error in the syntax can cause significant damage to your website. For this reason, the file is always hidden by web servers. If you have no experience with .htaccess files, you should best entrust this task to a web developer. If you still want to experiment with it, you should first create a backup copy of the original file. This way, you can easily revert to the old version of your website in case there are errors in the syntaxes of the .htaccess file.

6 uses of the .htaccess file in search engine optimization

The .htaccess file is a powerful tool for technical SEO. Here are some ways you can use the file to increase your website performance:

1. Creating user-friendly URLs with the .htaccess file

According to Matt Cutts (a Google engineer), URLs play a crucial role in search engine rankings. Your URL should not only contain keywords to attract more visitors. It should also be short and memorable.

To create clear and concise SEO-friendly URLs, you can optimize your .htaccess file. For example, you can convert abc.com/seo-page.html to abc.com/seo-page by using this code:

RewriteCond %{REQUEST_FILENAME} !-d

RewriteCond %{REQUEST_FILENAME}\.html -f

RewriteCond %{REQUEST_URI} ^(.+)\.html$

RewriteRule (.*)\.html$ /$1 [R=301,L]

You can do the same with abc.com/seo-page.php by changing .html to .php in the above code.

2. Allow or deny access to your website

You can also block IP addresses from accessing your website using a .htaccess file. For example, if you want your website to be accessible only for a single IP address (e.g., IP 1.2.3.4), you can enter the following code:

# Order Allow, Deny

Deny from All

Allow from 1.2.3.4

The word 'Order' is used to define the execution process. In the example above, the Allow command is processed first, before the Deny command is executed.

Similarly, you can use the following code to deny access to the same IP address:

# Order Allow, Deny

Deny from 1.2.3.4

Deny from 1.2.3.5

Allow from All

If you want to block access to hidden files or directories with .htaccess, you can also use this syntax:

RewriteCond %{SCRIPT_FILENAME} -d [OR]

RewriteCond %{SCRIPT_FILENAME} -f

RewriteRule "(^|/)\." – [F]

3. Password protect directories

The .htaccess file can also help you password protect important pages, files, folders or directories of your website. For this you need the following syntaxes:

AuthType Basic

AuthName "Password protected area"

AuthUserFile /pathToFile/.htpasswd

Require valid-user

This command tells the Apache server that your 'Password Protected Area' should be secured. The first command sets the basic authentication, while the third line indicates the password storage location. The last command then prompts users to enter their login details.

To store the passwords and usernames of the users, you also need to create a .htpasswd file. Here is an example:

# .htpasswd file for user names and passwords

tom_muller:oCF9Pam/MXJg2

tina_schmidt:vfavgfkndjaZGH

juergen_meyer:bhsgcanjFG5vdh

Note: Check if the server you are working on requires that the password file be in the same directory where the .htaccess file is also located.

4. Improvement of indexing and crawling

For your website to appear in search engines, it must be indexed. To achieve this, bots must be able to crawl your website effectively and regularly. Here too, the .htaccess file can help.

The .htaccess file offers you two ways to improve the crawlability and indexability of your website:

By guiding the Googlebot to the important parts of your website

Normally, the robots.txt file is used to tell bots which pages, files, folders, and subdomains should be crawled. However, the problem with this is that it only contains crawler directives, but no indexer directives. Crawler directives such as Allow, Disallow, Sitemap and User-agent tell search engine bots where they can go.

Indexer directives, on the other hand, tell bots what to index. Unlike the crawler directives, which usually reside in the robots.txt file, the indexer directives are in the HTML head of a particular page.

However, the problem here is that PDF files or images do not have HTML heads. This means that once they are indexed, they will appear in search results. Once they appear on the SERP, visitors may click directly on them instead of on your website – which negatively affects your conversion rate. To prevent these files from being indexed and to minimize outbound traffic, you need to create a custom header called 'X-robots-tag'. The setup is the same as for any other custom HTTP header.

For a single file, you can insert this code into your .htaccess file:

<Files white-paper.pdf>

    Header add X-robots-tag "noindex, noarchive, nosnippet"

</Files>

To set an X-robots-tag for all .docx and .pdf files, you can instead use the following code:

<FilesMatch ".(docx|pdf)$">

    Header add X-robots-tag "noindex, noarchive, nosnippet"

</FilesMatch>
By setting rel="canonical" tags on non-HTML resources

Creating rel="canonical" tags for PDFs and images can also prevent bots from indexing them. While you can't insert these tags into the HTML head, you can use them to create a custom HTTP header.

The following code sets a canonical tag for the file white-paper.pdf that points to the desired HTML page:

<Files white-paper.pdf>

    Header add Link '<http://www.example.com/white-paper-download.html>; rel="canonical"'

</Files>

5. Creating Redirects

URL redirects are important for several reasons. From the user's perspective, they improve the UX (User Experience). Instead of showing users a 404 error page, for example, you can redirect them to a new URL with the content they are looking for. This reduces the bounce rate and increases the time they spend on your website.

From the perspective of crawlers, redirects can help them navigate a website more easily. They can then crawl and index the important pages of your website, which positively affects your SEO and your ranking.

Creating redirects with the .htaccess file is pretty easy. For the error pages 400 (bad request), 401 (unauthorized), 403 (forbidden) and 500 (internal error) you only need to use the following syntax:

ErrorDocument 402 /temp/page-unavailable-temporarily402

The above code works for any error page. All you have to do is edit the error code and adjust the associated redirect page.

To redirect individual pages via 301 redirect to a new URL, you can use the syntax below:

Redirect 301 /old-page.html new-page.html

You can also redirect a complete website to a new URL by using .htaccess with the following code:

Redirect 301 / http://example.com

To guide visitors and crawlers to the latest version of your website, this syntax is helpful:

RewriteCond %{HTTP_Host} ^example.com$ [NC]

RewriteCond ^(.*)$ http://www.example.com/$1[R+301,L]

RewriteCond %{THE_REQUEST} ^.*/index

Rewriter ^(.*)index$ http://www.site.com/$1 [R=301, L]

6. Faster Page Speed

Page Speed measures how long it takes a page to load. It is one of Google's direct ranking factors and can significantly influence your search engine optimization. As a rule of thumb: the longer it takes for your page to load, the more likely visitors are to leave your page. This leads to a higher bounce rate, less time spent and of course fewer conversions.

There are several ways you can improve a page's speed using the .htaccess file:

Browser Caching

Browser caching stores resources of your website on the visitor's computer. This then retrieves the cached copy of the image or any other type of file from the user's local drive - which in turn saves bandwidth and server load when visitors return to your website later.

To activate browser caching via the .htaccess file, you can use this syntax example:

# turns cache on for 1 month

<IfModule mod_expires.c>

ExpiresActive On

ExpiresByType text/css “access plus 1 month“

ExpiresByType text/javascript “access plus 1 month“

ExpiresByType text/html “access plus 1 month“

ExpiresByType application/javascript “access plus 1 month“

ExpiresByType image/gif “access plus 1 month“

ExpiresByType image/jpeg “access plus 1 month“

ExpiresByType image/png “access plus 1 month“

ExpiresByType image/x-icon “access plus 1 month“

</IfModule>

The timeframe tells the browser of the visitors how long it can use the locally cached copy of the resource. You can set it to the time you prefer. The most common timeframes are 'Access plus 1 Day', 'Access plus 1 Week', 'Access plus 1 Month' and 'Access plus 1 Year'.

Cache-Control

The second method is to create a Cache-Control HTTP header for your website files. This sets how and where a resource is cached and how long it may be stored at most before it expires. To set a Cache-Control HTTP header with the .htaccess file, you can use this syntax:

##### 1 Month for most static resources

<filesMatch ".(css|jpg|jpeg|png|gif|js|ico)$">

    Header set Cache-Control "max-age=2592000, public"

</filesMatch>

Compressing your website

Another method to speed up your website is to compress it. For this, you can reduce the image size, the file size and the number of files. You can also compress the files you send to the client browser. This way, your website can transfer data faster.

To compress your website with the .htaccess file, you have to activate gzip (file format and software application for file compression) with this code:

##### Enable gzip compression for resources

<ifModule mod_gzip.c>

    mod_gzip_on Yes

    mod_gzip_dechunk Yes

    mod_gzip_item_include file .(html?|txt|css|js|php)$

    mod_gzip_item_include handler ^cgi-script$

    mod_gzip_item_include mime ^text/.*

    mod_gzip_item_include mime ^application/x-javascript.*

    mod_gzip_item_exclude mime ^image/.*

    mod_gzip_item_exclude rspheader ^Content-Encoding:.*gzip.*

</ifModule>

Since not all servers support gzip, you can alternatively use the Deflate algorithm with this code:

##### Compress resources

AddOutputFilterByType DEFLATE text/plain

AddOutputFilterByType DEFLATE text/html

AddOutputFilterByType DEFLATE text/xml

AddOutputFilterByType DEFLATE text/css

AddOutputFilterByType DEFLATE application/xml

AddOutputFilterByType DEFLATE application/xhtml+xml

AddOutputFilterByType DEFLATE application/rss+xml

AddOutputFilterByType DEFLATE application/javascript

AddOutputFilterByType DEFLATE application/x-javascript

Why do you get an error message after updating the .htaccess file?

There are some common problems that marketers and web developers encounter when updating the .htaccess file. Here is a list of common .htaccess problems:

1. Missing AllowOverride: For the .htaccess file to work, you first need to enable the 'AllowOverride' option. If this option is set to 'None', all configurations set in the .htaccess file are disabled.
2. Misspelled filename: Since the .htaccess file is a Unix-based configuration file that is stored in ASCII format, any misspelling in the filename leads to an error. This happens, for example, when the filename does not start with a '.' or when it is uploaded in a different format like .txt. In this case, the Apache server ignores the .htaccess file and the configurations you have made.
3. Hierarchy of .htaccess: .htaccess files are executed based on their hierarchy. This means that the commands mentioned at the top can override the syntaxes in the later part of the file.
4. Multiple .htaccess files: Since .htaccess files can be used for individual directories, there may be several .htaccess files on your website. In such cases, one file may contradict the configuration settings of another, leading to errors.
5. Syntax errors: The .htaccess files work entirely based on the syntax you used when configuring your website. Thus, a syntax error or a typing error within the commands can cause problems or even take your website offline.