« WebMail, and other Mailbox Options | Home | Peel Away Ads – Highly Effective Advertising on Your Web Site »

The robots.txt and favicon.ico files

By admin | January 8, 2008

This post is intended for everyone.

These are 2 little known files that the huge majority of webmasters forget to include in their web sites. I think the reason why most people forget about them is that they need to be created manually … none of the HTML editors that I know of create them automatically.

ROBOTS.TXT

The robots.txt is a small text file that you load into the root folder of your website ie. the same folder where your homepage is stored. Its purpose is to tell a search engine spider what it can and cannot index within your site.

All the good search engines will check for the presence of the robots.txt file … and then will only index the parts of your site that the robots.txt file allows. I would not rely on this religiously of course, as I am sure that there are plenty of “rogue” spiders out there that wont give a damn about the robots.txt file.

So, if you have content on your site that you do not want indexed by the search engines then it is far better to restrict access to those files/folders using a password (this can be done via your cPanel).

Below is the contents of the robots.txt file from one of my own sites. It instructs all search engine spiders to ignore 2 specific folders when it indexes the site ….

# /robots.txt file
# mail ppcisme at gmail dot com with problems
User-agent: *
Disallow: /protected/
Disallow: /forum/configuration

The robots.txt file must be created using a plain text editor, like notepad. You should not use a word processor as they often insert a lot of weird characters into the file.

For more detailed information on the use of the robots.txt file, including additional directives/commands that can be used, please visit http://www.robotstxt.org/

FAVICON.ICO

Another handy little file to have on your website is favicon.ico

This is the little graphic that displays in the browser address bar immediately to the left of the website address. Have a look at your browsers address bar now … you will see a small image … it looks like a capital U with a tail (its the greek letter mu). That is my favicon.ico file.

Now, you should be wondering “… of what possible use is this little file, and why should I go to the effort of making one?”.

When a visitor bookmarks your website (so they can maybe return later) their browser saves your favicon.ico file. Click on the “Bookmarks” menu item at the top of your browser now. This should drop a menu down with all your favourites/bookmarks.

Can you see now why you want a favicon.ico file ?

The favicon.ico file for each site will display next to its individual bookmark.

All of the sites that did not bother to make a favicon.ico file will just have the boring old default browser icon.

Its just another small way that can help your site stand out from the rest of the crowd in your visitors list of bookmarks. You should be doing everything possible to attract visitors :-)

An .ico file is a windows “icon” file format. You usually need a special piece of software that will create an .ico file from another graphic format (like .jpg and .gif). However, if you navigate to http://www.chami.com/html-kit/services/favicon/ you will find an online tool that can make them for you. All you need is the graphic that you want to convert.

Tip. The physical display size of these images is small … so the best type of graphic to use is a very simple and small one. A highly detailed image will not look nice when its compressed tiny.


Why not invest a quick 15 minutes in your web site right now? Thats all it will take for you to make these 2 files for yourself, and to upload them to your sites root folder.

If you have any questions about robots.txt or favicon.ico then please don’t hesitate to post them as comments below.

Thank you for reading.

Topics: Site Tips & Tricks | No Comments »

Comments

You must be logged in to post a comment.