Make Money from Blogs » Search

Search Engine Optimization Tips and Tricks – Learn how to increase visibility in Search Engines – Part 23 – Understanding the robots.txt file

ashish — Sun, 28 Mar 2010 17:44:55 +0000

In the previous post (All about Crawler), there were a lot of details about what a search engine crawler is, what it looks for, and why you must welcome this crawler to your site. Well, let us move further in this regard, and understand what a robots.txt file is, and what it has to do with search engines and the crawler.

What is a robots.txt file ?

Well, a robots.txt file is actually not about those robots you read in science fiction, instead it deals with those script robots / bots that you read about in the previous post. As per the search engine crawler protocol, a robots.txt file is a file that you put on the directories of your site with instructions on how you want your site to be read by the bots (and not all the search engine crawlers obey it, but the major ones do). It is a very simple text file that you can create in Notepad (or any other text editor, so if you are a vi fanatic, then use that by all means). You can use this text file to even ask the bots to prevent certain sections of your site from being copied, which means that those sections of the site will not be visible in search engines (and why would you want this to happen ? Well, one example is when you use this set of instructions to prevent duplicate reading of your site, so for example, the you want all the archive section of your site to be read, but not other paths to the same parts of your site (to prevent the duplicate content penalty), then this is a good way to do this (and the All-In-One SEO Plugin does indeed do that)).

Some scenarios in which you want to use the robots.txt file

- It is a matter of good protocol to have a robots.txt file since search engines expect these files to be there (but this is not a necessary condition)
- If you are in the habit of building different doorway pages of a site, with customized versions of the doorway page for each search engine or platform (a different doorway page for the iPhone, for example), then you would want to customize the search engines entry into these doorways (for example, if the Google search bot found its way to all off these doorway pages, they might consider you to be trying to hoodwink users). Alert: Doorway pages are now under alerts by search engines and you should consider your policies and intent in this area carefully.
- if your page has many sections that are still under construction, then you would not want the search engines to crawl and show these not-ready sections of the site. In such cases, you can prevent the spiders from reading the site and showing the result in search engines
- There are some sections of your site that are meant for individual persons, and you are not really interested in making them available for wider reading. Other people can access these if they go through the whole structure of your site, but there is no reason to show the results in a search engine result.
- You have created a print friendly version of your articles, but really don’t want search engines to go there (to avoid the duplicate content penalty and also to prevent users from directly accessing the print friendly versions). So you can enforce this in the robots.txt file.
- Keep the robots away from such content as has no value for the bots, such as directories with flash content, with JavaScript, and so on.

Search Engine Optimization Tips and Tricks – Learn how to increase visibility in Search Engines – Part 7

ashish — Sat, 27 Feb 2010 10:07:39 +0000

The previous post (optimizing sections inside your post structure) talked about some assorted steps you can to optimize your post (such as permalinks, bread crumbs (example, the Breadcrumb NavXT plugin), speeding up your site, and so on).
This post looks at optimizing the images on your site for better SEO. I can just hear some of you saying, Huh ! Why do we need to do something for images. Well, consider that Google Image Search drives a large number of people (on one of my blogs, it drives more than 70% of visitors, and this is not a photo blog), and Google, when displaying search results, many times shows images as a part of the search results, right near the top. If you were able to get your blog in there through some image optimization, would it not be worth it ? Well, if you are convinced, here goes:
- Number one item, you should ensure that all your images have an ‘Alt’ tag filled out properly. A search engine cannot read the image, and can only find out what the image is by the alt tag, and by the context in which the image appears. And of course, you earn credit from some visitors who are visually challenged and use these tags to figure out what the image wants.
- If you are not able to add ‘Alt’ tag properties / text to your images manually, make sure that they are not blank. Use a plugin such as the ‘SEO Friendly Images’ (link – automatically updates all images with proper ALT and TITLE attributes).
- This one is more speculative, but it is believed that if you host your image on a popular image sharing platform such as Picasa Web or Flickr, then you get a slight boost to your SEO rankings for the image.
- If possible, ensure that your images are in a separate folder that is accessible by search engines
- Don’t put too many images per page, so if you load 100 or so images, then it is taken negatively
- Make sure that you change the image file names that come from the camera and make it something distinctive. Don’t use something like P121765.jpg, instead make it something ‘Photo of Sydney Opera House from water level’. This helps in search engine ranking and gives a boost to the image ranking
- Now, something that is different. Look at the words surrounding a photo / picture, and make sure that the photo or picture sits in a section of text that describes it some degree and has some keywords in it. This also boost the ability of the Search engines image search crawler to rank the images higher.
- If somebody requests you for being able to use your photos or a thumbnail of your photos, then consider whether the photo is exclusive enough. If you feel that it is fine to do so, ensure that there is a link to your original blog post under the photo in the other site, since this helps in giving your link popularity a boost.
- When you are giving a link to your photos somewhere, don’t use the words ‘link’ or ‘click here’, instead use descriptive text as the anchor since doing this naming also helps in SEO
- Ensure that your photos have a standard aspect ratio, since search engines show thumbnails and these thumbnails are easier to show when they are of a standard size.

Ad: Click Here for SEO Elite, a SEO software to propel your site to new rankings!

What to do if your site is hacked and Google penalizes your site

ashish — Fri, 12 Feb 2010 16:32:13 +0000

Sites getting hacked is not something that is so rare nowadays. You have a good site that is getting more viewers, but have not kept your site (maybe built in WordPress) totally safe in terms of frequent updates, security patches, and so on, and you find that your site behaves a bit strange. However, once this happens, a lot of bad karma happens to your site
- It could have bad software loaded that infects viewers computers, and now browsers are quick to catch that and warn users, and there go your page views down to 0
- You have some small changes in the page that are not fully visible, but these are picked up by the Google crawler (some porn words, or links) and these in turn reduce your Adsense earnings and reduce your site viewers; in addition, some sites may flag your page as ‘adult’, thus ensuring that your viewers go down
What do you do in these cases ? Well, Google (and other search engines) would have a list of things you can do after something like this has happened. In the case of Google, here is the relevant page (link to article)

1. Getting your site off-line:
Take your site off-line temporarily, at least until you know you’ve fixed things.
If you can’t take it off-line, return a 503 status code to prevent it from being crawled.
In the Webmaster Tools, use the URL removal tool to remove any hacked pages or URLs from search results that may have been added. This will prevent the hacked pages from being served to users
2. Damage Assessment
3. Recovery
4. Restoring your online presence

Read the full article if you are in the position of having had your blog hacked.

Why your site may not be appearing high in Google Search – some tips from the Google blog

ashish — Wed, 10 Feb 2010 16:10:00 +0000

A site needs to appear high in Google Search in order to generate a high number of visitors. In fact, a number of onliness businesses consider visitors from Search Engines (particularly Google) as a significant source of their visitors, and consequently, of the revenue that they will make. Getting your site high on Search Engine Ranking is a big quest, and needs considerable work and effort. Paying some expert on Search Engine Optimization is something that you should really consider on priority, and at the same time, also try to figure out whether you will benefit by reading any of the guidelines on this Google Help page (link to article). This site talks about problems that the Google crawler may be having with your site:

Is your server treating Googlebot like a normal visitor?
Does your website have systematically broken links somewhere?
Is your website redirecting some users elsewhere?
Is your server unreachable when we try to crawl?
Has your website been hacked?

For example, if your website has been hacked, then the recommendation is to make sure that you clean your site, and plenty more information. You also learn some tips about how to detect if your site has indeed been hacked, and this can greatly help if you this is indeed the case.

Using Photos in your blog posts

ashish — Sat, 02 Jan 2010 19:49:06 +0000

I believe I have mentioned this in past blog posts, but more in the nature of a tip rather than an actual focused post. For some time now, I have been reviewing the sources of page visits on the Gadgets blog that I maintain, called Ultra Cool Gadgets (link). Sitemeter provides a simple set of statistics for how the users come to the blog, and if you look at the following Sitemeter site (link), you will get an idea of where a large section of users come to my site from.
Surprisingly, a number of users to my site come from URL’s starting with http://images.google.com …; these are people coming from Google’s Image search. How do I manage to get people to come to my site from searching for images ? This was not intentional when I started, but I had made it a point to ensure that with every gadget that I posted, I also posted an image. But it was not just an image, I also made sure that I did the following:
1. Made sure that the image contained an alt description. Most tools that allow you to upload an image to your Blogger blog do not provide a default to add the alt tag (although I believe posting the URL for an image in WordPress gives you an option for entering the description of the image). You need to modify the code for the image to ensure that the alt keyword has a description added to it.
2. This tip is slightly lower in importance from the previous tip, but try and make sure that the name of the image is also relevant. This is also starting to matter in terms of the discoverability of your images via Google Image search.
Once you follow these simple rules and ensure that images are posted along with the blog post, you increase the chances of making your images being found by more people, leading to a higher traffic for your blog. However, do not stuff images that are not relevant, and do not add description in the alt tag that is false, since that might lead to your images getting a penalty associated with them.
Over a period of time, Google automatically finds these images and adds them to its Images search database.