Wednesday, March 30, 2011

List of about 20,000 web-directories for SEO

List of about 20,000 web-directories are given below,

Please check the link: www.web-directories.ws/

Monday, March 21, 2011

How Does Google Search Works?


How Google Works.
Infographic by the Pay Per Click Blog

Thursday, March 17, 2011

List Of Top 100 Forums (SEO Forum Submission Sites)


http://www.addthis.com/forum PR9
http://chronicle.com/forums/ PR8
http://forums.myspace.com/ PR8
http://forum.joomla.org PR8
http://forums.cpanel.net PR8
http://forum.siteground.com PR8
http://c7y-bb.phparchitect.com PR8
http://forum.meebo.com PR8
http://forums.mysql.com/ PR7
http://forums.cnet.com/ PR7
http://forums.microsoft.com/ PR7
http://www.bookforum.com PR7
http://mathforum.org/ PR7
http://forums.photobucket.com/ PR7
http://www.gardenweb.com/ PR7
http://forums.wsj.com/ PR7
http://forums.cnet.com/ PR7
http://forum.videolan.org PR7
http://www.wrensoft.com/forum PR7
http://pkp.sfu.ca/support/forum PR7
http://audacityteam.org/forum PR7
http://arch.designcommunity.com PR7
http://forum.filezilla-project.org PR7
http://www.awasu.com/forums PR7
http://forums.mozilla.or.kr PR7
http://www.claroline.net/forum PR7
http://message.snopes.com PR7
http://www.softcatala.org/forum PR7
http://forum.claroline.net PR7
http://forum.wordreference.com PR7
http://forums.mozillazine.org PR7
http://forums.spry.com PR7
http://www.fckeditor.net/forums PR7
http://www.000webhost.com/forum PR7
http://flashpanoramas.com/forum PR7
http://www.hostmonsterforum.com PR7
http://forum.visagesoft.com PR7
http://forum.maxthon.com PR7
http://gallery.menalto.com/forum PR7
http://forum.filezilla-project.org PR7
http://forum.videolan.org PR7
http://forum.wordreference.com PR7
http://www.accessifyforum.com/ PR7
http://forum.developers.facebook.com/ PR7
http://www.sitepoint.com/forums/ PR6
http://w3seo.org/ PR6
http://forums.feedburner.com/ PR6
http://forums.delphiforums.com PR6
http://forums.amd.com/ PR6
http://forums.oracle.com/forums PR6
http://forums.gentoo.org/ PR6
http://forum.skype.com PR6
http://bbpress.org/forums/ PR6
http://forums.metacafe.com/ PR6
http://guru-forum.net PR6
http://kinder.univie.ac.at/forum PR6
http://forum.whatismyip.com PR6
http://forums.foruminternet.org PR6
http://forums.phpbb-fr.com PR6
http://forums.searchenginewatch.com PR6
http://www.careerbuilderforums.com PR6
http://www.iggsoftware.com/forums PR6
http://forums.yourkit.com PR6
http://antionline.com PR6
http://forum.wordpress-deutschland.org PR6
http://sb.xrea.com PR6
http://forums.omnigroup.com PR6
http://www.antionline.com PR6
http://forums.allaboutjazz.com PR6
http://sends.hqu.edu.cn/bbs PR6
http://www.portugal-info.net/forum PR6
http://boards.core77.com PR6
http://www.city-data.com/forum PR6
http://www.ozzu.com PR6
http://www.forum.uni-hannover.de PR6
http://www.chinalanguage.com/forums PR6
http://www.textkit.com/greek-latin-forum PR6
http://forums.naseej.com PR6
http://ubuntuforums.org PR6
http://www.freewebsitetemplates.com/forum PR6
http://www.ddth.com PR6
http://www.alice.org/community PR6
http://forum.boxee.tv PR6
http://forum.springframework.org PR6
http://forums.foxitsoftware.com PR6
http://www.baltimoresun2.com/talk PR6
http://www.idpf.org/forums PR6
http://www.cmsimpleforum.com PR6
http://www.htmlvalidator.com/CSEForum PR6
http://forum.parallels.com PR6
http://fhq.forumer.com PR6
http://www.forum.onlineconversion.com PR6
http://www.svetovalnica.com/klepetalnica PR6
http://forums.futura-sciences.com PR6
http://www.nhse.org PR6
http://forum.ertonline.gr/deytero PR6
http://forums.macosxhints.com PR6
http://forum.isilo.com PR6
http://www.kirupa.com/forum PR6
http://icl.cs.utk.edu/lapack-forum PR6

Friday, March 11, 2011

Google's SEO Report Card And Google Beginners SEO Guide

How many of Google's web pages use a descriptive title tag? Do we use description meta tags? Heading tags? While we always try to focus on the user, could our products use an SEO tune up? These are just some of the questions we set out to answer with Google's SEO Report Card.

Google's SEO Report Card is an effort to provide Google's product teams with ideas on how they can improve their products' pages using simple and accepted optimizations. These optimizations are intended to not only help search engines understand the content of our pages better, but also to improve our users' experience when visiting our sites. Simple steps such as fixing 404s and broken links, simplifying URL choice, and providing easier-to-understand titles and snippets for our pages can benefit both users and search engines. From the start of the project we also wanted to release the report card publicly so other companies and webmasters could learn from the report, which is filled with dozens of examples taken straight from our products' pages.

The project looked at the main pages of 100 different Google products, measuring them across a dozen common optimization categories. Future iterations of the project might look at deeper Google product web pages as well as international ones. We released the report card within Google last month and since then a good number of teams have taken action on it or plan to.

We hope you find our SEO Report Card useful and we'd love to hear your feedback in the comments below or in the Webmaster Central Help Forum. And if you'd like to do your own SEO tune up, a good place to start is by reading our free SEO Beginner's Guide.

How To Build An In House SEO Team ?


Once you have executive support for the company-wide SEO effort, establish a core team. This is a small group ( maybe 10 people) that should include:
  • a couple SEO subject matter experts (SMEs): provide SEO knowledge base;
  • a systems architect: helps connect the dots;
  • a technical developer lead: helps translate business requirements to the engineers;
  • a couple of people from design/content programming/product groups: help you navigate the waters as to how to best work with these teams overall;
  • a program manager: keeps the overall project plan together; and
  • a couple project managers: to track the individual tasks and roll up into one über-plan.
The core team should be responsible for defining issues that affect all or most content channels/product pages across the company (like URL structure, site hierarchy, or page structure). This team then lays out the plan of action to fix these large-scale problems, including defining the tactics to fix the problems/optimize the assets, that is scalable across the company.

Thursday, March 10, 2011

Top 12 Google Search Tips


  • Explicit Phrase:
    Lets say you are looking for content about internet marketing.  Instead of just typing internet marketing into the Google search box, you will likely be better off searching explicitly for the phrase.  To do this, simply enclose the search phrase within double quotes.
    Example: "internet marketing"

  • Exclude Words:
    Lets say you want to search for content about internet marketing, but you want to exclude any results that contain the term advertising.  To do this, simply use the "-" sign in front of the word you want to exclude.
    Example Search: internet marketing -advertising

  • Site Specific Search:
    Often, you want to search a specific website for content that matches a certain phrase.  Even if the site doesn’t support a built-in search feature, you can use Google to search the site for your term. Simply use the "site:somesite.com" modifier.
    Example: "internet marketing" site:www.smallbusinesshub.com

  • Similar Words and Synonyms:
    Let’s say you are want to include a word in your search, but want to include results that contain similar words or synonyms.  To do this, use the "~" in front of the word.
    Example: "internet marketing" ~professional

  • Specific Document Types:
    If you’re looking to find results that are of a specific type, you can use the modifier "filetype:".  For example, you might want to find only PowerPoint presentations related to internet marketing.
    Example: "internet marketing" filetype:ppt

  • This OR That:
    By default, when you do a search, Google will include all the terms specified in the search.  If you are looking for any one of one or more terms to match, then you can use the OR operator.  (Note:  The OR has to be capitalized).
    Example: internet marketing OR advertising

  • Phone Listing:
    Let’s say someone calls you on your mobile number and you don’t know how it is.  If all you have is a phone number, you can look it up on Google using the phonebook feature.
    Example: phonebook:617-555-1212 (note:  the provided number does not work – you’ll have to use a real number to get any results).

  • Area Code Lookup:
    If all you need to do is to look-up the area code for a phone number, just enter the 3-digit area code and Google will tell you where it’s from.
    Example: 617

  • Numeric Ranges:
    This is a rarely used, but highly useful tip.  Let’s say you want to find results that contain any of a range of numbers.  You can do this by using the X..Y modifier (in case this is hard to read, what’s between the X and Y are two periods.  This type of search is useful for years (as shown below), prices or anywhere where you want to provide a series of numbers.
    Example: president 1940..1950

  • Stock (Ticker Symbol):
    Just enter a valid ticker symbol as your search term and Google will give you the current financials and a quick thumb-nail chart for the stock.
    Example: GOOG

  • Calculator:
    The next time you need to do a quick calculation, instead of bringing up the Calculator applet, you can just type your expression in to Google.
    Example: 48512 * 1.02

  • Word Definitions:
    If you need to quickly look up the definition of a word or phrase, simply use the "define:" command.
    Example: define:plethora

  • Tuesday, March 8, 2011

    Top 10 SEO And Internet Marketing Blogs

     Search Engine Land – news on the search industry and landscape
     SEOmoz – daily blog on SEO & web marketing techniques + how-to
     SEO Book – Smart analysis of the search field and SEO
     Dave Naylor – One of the more brilliant individuals in the search field
     Distilled Blog – Top notch SEO consultants sharing their findings
     Blogstorm – Patrick Altoft’s infrequently updated but always insightful analysis
     David Mihm – Among the best resources for local/places search analysis
    • Seer Interactive Blog – very talented and well-written Philadelphia SEO team
    Conversation Marketing – Ian Lurie’s advice, rants & analysis of search
     Matt Cutts  Matt Cutts: Gadgets, Google, and SEO

    Sunday, March 6, 2011

    Robots.txt Explained ( What Is Robots.txt File ?)



    "Robots.txt" is a regular text file that through its name, has special meaning to the majority of "honorable" robots on the web. By defining a few rules in this text file, you can instruct robots to not crawl and index certain files, directories within your site, or at all. For example, you may not want Google to crawl the /images directory of your site, as it's both meaningless to you and a waste of your site's bandwidth. "Robots.txt" lets you tell Google just that.

     Creating your "robots.txt" file

    So lets get moving. Create a regular text file called "robots.txt", and make sure it's named exactly that. This file must be uploaded to the root accessible directory of your site, not a subdirectory (ie: http://www.mysite.com but NOThttp://www.mysite.com/stuff/). It is only by following the above two rules will search engines interpret the instructions contained in the file. Deviate from this, and "robots.txt" becomes nothing more than a regular text file, like Cinderella after midnight.
    Now that you know what to name your text file and where to upload it, you need to learn what to actually put in it to send commands off to search engines that follow this protocol (formally the "Robots Exclusion Protocol"). The format is simple enough for most intents and purposes: a USERAGENT line to identify the crawler in question followed by one or more DISALLOW: lines to disallow it from crawling certain parts of your site.

    1) Here's a basic "robots.txt":
    User-agent: *
    Disallow: /
    With the above declared, all robots (indicated by "*") are instructed to not index any of your pages (indicated by "/"). Most likely not what you want, but you get the idea.

    2) Lets get a little more discriminatory now. While every webmaster loves Google, you may not want Google's Image bot crawling your site's images and making them searchable online, if just to save bandwidth. The below declaration will do the trick,

    User-agent: Googlebot-Image
    Disallow: /
    3) The following disallows all search engines and robots from crawling select directories and pages:

    User-agent: *
    Disallow: /cgi-bin/
    Disallow: /privatedir/
    Disallow: /tutorials/blank.htm
    4) You can conditionally target multiple robots in "robots.txt." Take a look at the below:
    User-agent: *
    Disallow: /
    User-agent: Googlebot
    Disallow: /cgi-bin/
    Disallow: /privatedir/
    This is interesting- here we declare that crawlers in general should not crawl any parts of our site, EXCEPT for Google, which is allowed to crawl the entire site apart from /cgi-bin/ and /privatedir/. So the rules of specificity apply, not inheritance.

    5) There is a way to use Disallow: to essentially turn it into "Allow all", and that is by not entering a value after the semicolon(:):
    User-agent: *
    Disallow: /
    User-agent: ia_archiver
    Disallow:
    Here I'm saying all crawlers should be prohibited from crawling our site, except for Alexa, which is allowed.

    6) Finally, some crawlers now support an additional field called "Allow:", most notably, Google. As its name implies, "Allow:" lets you explicitly dictate what files/folders can be crawled. However, this field is currently not part of the "robots.txt" protocol, so my recommendation is to use it only if absolutely needed, as it might confuse some less intelligent crawlers.
    Per Google's FAQs for webmasters, the below is the preferred way to disallow all crawlers from your site EXCEPT Google:

    User-agent: *
    Disallow: /
    User-agent: Googlebot
    Allow: /

    What is "Robots" Meta Tag ?


    The "robots" meta tag

    If your web host prohibits you from uploading "robots.txt" to the root directory, or you simply wish to restrict crawlers from a few select pages on your site, an alternative to "robots.txt" is to use the robots meta tag.

     Creating your "robots" meta tag

    The "robots" meta tag looks similar to any meta tag, and should be added between the HEAD section of your page(s) in question:
    <meta name="robots" content="noindex,nofollow" />
    Here's a list of the values you can specify within the "contents" attribute of this tag:

    ValueDescription
    (no)indexDetermines whether crawler should index this page. Possible values: "noindex" or "index"
    (no)followDetermines whether crawler should follow links on this page and crawl them. Possible values: "nofollow" and "follow."

    Here are a few examples:

    1) This disallows both indexing and following of links by a crawler on that specific page:
    <meta name="robots" content="noindex,nofollow" />
    2) This disallows indexing of the page, but lets the crawler go on and follow/crawl links contained within it.
    <meta name="robots" content="noindex,follow" />
    3) This allows indexing of the page, but instructs the crawler to not crawl links contained within it:
    <meta name="robots" content="index,nofollow" />
    4) Finally, there is a shorthand way of declaring 1) above (don't index nor follow links on page):
    <meta name="robots" content="none">

    How To Build An SEO Team ? (Structure Of An SEO Team)

    There’s no definitive SEO team structure out there, and ultimately the roles you choose need to fit with the needs of your organisational framework. That said, it pays to really consider your team design, especially when you’re communicating your resourcing requirements to HR and the Board.
    Here’s an example of a team structure:


    SEO Team Structure

    SEO typically breaks down into 4 main tasks
    - Technical SEO
    - Content Creation and social media
    - Link building, baiting and realignment
    - Metrics, analytics and reporting

    Wednesday, March 2, 2011

    Google Technology - The Technology Behind Google's Search Engine Results


    As a Google user, you're familiar with the speed and accuracy of a Google search. How exactly does Google manage to find the right results for every query as quickly as it does? The heart of Google's search technology is PigeonRank™, a system for ranking web pages developed by Google founders Larry Page and Sergey Brin at Stanford University.

    PigeonRank System
    Building upon the breakthrough work of B. F. Skinner, Page and Brin reasoned that low cost pigeon clusters (PCs) could be used to compute the relative value of web pages faster than human editors or machine-based algorithms. And while Google has dozens of engineers working to improve every aspect of our service on a daily basis, PigeonRank continues to provide the basis for all of our web search tools.


    Why Google's patented PigeonRank™ works so well
    PigeonRank's success relies primarily on the superior trainability of the domestic pigeon (Columba livia) and its unique capacity to recognize objects regardless of spatial orientation. The common gray pigeon can easily distinguish among items displaying only the minutest differences, an ability that enables it to select relevant web sites from among thousands of similar pages.
    By collecting flocks of pigeons in dense clusters, Google is able to process search queries at speeds superior to traditional search engines, which typically rely on birds of prey, brooding hens or slow-moving waterfowl to do their relevance rankings.
    diagramWhen a search query is submitted to Google, it is routed to a data coop where monitors flash result pages at blazing speeds. When a relevant result is observed by one of the pigeons in the cluster, it strikes a rubber-coated steel bar with its beak, which assigns the page a PigeonRank value of one. For each peck, the PigeonRank increases. Those pages receiving the most pecks, are returned at the top of the user's results page with the other results displayed in pecking order.


    Integrity
    Google's pigeon-driven methods make tampering with our results extremely difficult. While some unscrupulous websites have tried to boost their ranking by including images on their pages of bread crumbs, bird seed and parrots posing seductively in resplendent plumage, Google's PigeonRank technology cannot be deceived by these techniques. A Google search is an easy, honest and objective way to find high-quality websites with information relevant to your search.

    Data









    PigeonRank Frequently Asked Questions


    How was PigeonRank developed?
    The ease of training pigeons was documented early in the annals of science and fully explored by noted psychologist B.F. Skinner, who demonstrated that with only minor incentives, pigeons could be trained to execute complex tasks such as playing ping pong, piloting bombs or revising the Abatements, Credits and Refunds section of the national tax code.
    Brin and Page were the first to recognize that this adaptability could be harnessed through massively parallel pecking to solve complex problems, such as ordering large datasets or ordering pizza for large groups of engineers. Page and Brin experimented with numerous avian motivators before settling on a combination of linseed and flax (lin/ax) that not only offered superior performance, but could be gathered at no cost from nearby open space preserves. This open space lin/ax powers Google's operations to this day, and a visit to the data coop reveals pigeons happily pecking away at lin/ax kernels and seeds.

    What are the challenges of operating so many pigeon clusters (PCs)?
    Pigeons naturally operate in dense populations, as anyone holding a pack of peanuts in an urban plaza is aware. This compactability enables Google to pack enormous numbers of processors into small spaces, with rack after rack stacked up in our data coops. While this is optimal from the standpoint of space conservation and pigeon contentment, it does create issues during molting season, when large fans must be brought in to blow feathers out of the data coop. Removal of other pigeon byproducts was a greater challenge, until Page and Brin developed groundbreaking technology for converting poop to pixels, the tiny dots that make up a monitor's display. The clean white background of Google's home page is powered by this renewable process
    .
    Aren't pigeons really stupid? How do they do this?
    While no pigeon has actually been confirmed for a seat on the Supreme Court, pigeons are surprisingly adept at making instant judgments when confronted with difficult choices. This makes them suitable for any job requiring accurate and authoritative decision-making under pressure. Among the positions in which pigeons have served capably are replacement air traffic controllers, butterfly ballot counters and pro football referees during the "no-instant replay" years.

    Where does Google get its pigeons? Some special breeding lab?
    Google uses only low-cost, off-the-street pigeons for its clusters. Gathered from city parks and plazas by Google's pack of more than 50 Phds (Pigeon-harvesting dogs), the pigeons are given a quick orientation on web site relevance and assigned to an appropriate data coop.

    Isn't it cruel to keep pigeons penned up in tiny data coops?
    Google exceeds all international standards for the ethical treatment of its pigeon personnel. Not only are they given free range of the coop and its window ledges, special break rooms have been set up for their convenience. These rooms are stocked with an assortment of delectable seeds and grains and feature the finest in European statuary for roosting.

    What's the future of pigeon computing?
    Google continues to explore new applications for PigeonRank and affiliated technologies. One of the most promising projects in development involves harnessing millions of pigeons worldwide to work on complex scientific challenges.

    What is Competitor Analysis?


     A competitor analysis is one very effective method of deconstructing their online marketing strategy to discover how they are doing so well.

    What Exactly Can a Competitor Analysis Reveal?
    This is a very common question because many site owners don't know the lengths that a competitor may have gone to obtain top rankings. The following examples are some of the discoveries I have uncovered in a typical competitor analysis:
    • By examining a competitor's link structure I have found that many of the links with the most credibility came from websites the competitor actually owned. (Determining the ownership of the domain names required some sleuthing because the whois information was 'private' but ultimately the info became available.) In a couple of cases several of these domains had legitimate websites and this prompted some great ideas for my client to attain more traffic.
    • While researching a competitor I noticed that although the competitor's website was very similar to my client's, there was one major difference; the competitor's website structure was far better optimized. By outlining the structure the competitor used and improving on it with my own expertise our client had the information he needed to apply changes to his own site.
    • In another instance I provided a client the list of all the pay per click keywords and organic keywords that each competitor was currently using. The client was flabbergasted when she realized just how many keywords she had missed promoting for her own comparable services.
    The Basics of Conducting Your Own Competitor Analysis
    Now that you have seen some examples of what can be gleaned from a competitor analysis you might want to conduct one of your own. For the purpose of this tutorial I am assuming that you are fairly new to SEO so I created a basic plan that works for most users; but even this will require a little preparative reading. The following is a list of essential reading material:
    Many more free SEO tutorials are available if you find yourself needing more information. The following is an outline of the most revealing steps with the least amount of technical expertise required. Please keep in mind that the objective of this competitor analysis is to compare what you find to your own website later on. What you find may not seem earth shattering (or it might) but this analysis is meant to show you what you might be missing:

    Competitor Walkthrough
    Grab a piece of paper and a pen and while you walk through your competitor's website look for any particularly obvious search engine optimization techniques. Here are some elements you should check:
    • Does the title tag appear well written and if so is there a common syntax used throughout the website?
    • Look at the source code of the home page and search for "H1", "H2" or "H3". Do any of these tags show up? If so that means the competitor is using heading tags within the page. Now try identifying the text they used in the heading. Likely you will find the competitor's Keyphrase is found within the tag.
    • Check if the navigation is search engine friendly. Sometimes the navigation is a drop-down menu; make sure it is a type that is search engine friendly. If not, check the footer of the page and see if a text menu is placed there.
    • Keep an eye out for a pattern of keywords being used in text links. Certain words are likely to appear more often and these are likely some of the target phrases your competitor has decided to focus on.
    • Look for nofollow tags. No follow tags are often used to channel Page Rank efficiently throughout a website. This is called a themed structure and it can have incredible ranking benefits. If you see a pattern of nofollow tag use then you can be relatively certain your competitor has/had a well-informed SEO firm on hire.
    • While you roam through the site look for pages that have particularly high Google PageRank and try to identify why. In most cases these pages have information that visitors decided to link to. Perhaps this will give you some ideas for creating similar quality content for your website.
    • Check the site for the presence of an XML sitemap. Usually it will reside at the root of the website so try typing in the basic URL of the competitor's website and add (minus the quotes) "\sitemap.xml" on the end. The details within the sitemap might be a little confusing to you but just acknowledging that the competitor has one is noteworthy.
    • Have you found any incidences of spam throughout the site? Take note, I have lost count how many competitors succeeded using shady tactics. This doesn't mean you copy them, however, but it may at least give you yet another indication of what helped the competitor attain rankings. Believe me, in most cases these sites will get caught with their hands in the cookie jar at which point you won't want to be associated with the same tactics.
    I can't possibly list everything you need to keep an eye out for when walking through a competitor's website; at least not in an article format. Just keep an eye out for anything that looks particularly purposeful in the site and linking structure as well as the content of the website. If you find something you can't be sure is worth noting, then try researching it online; chances are someone has written about the topic/concept or can provide you advice in a forum.

    Backlink Analysis
    This portion of the analysis will require that you use one of the following link analysis tools: OptiLink (not free but my first choice) or Backlink Analyzer from SEO Book (free). In each case these tools have excellent help files that I suggest reading in order to get the best results from the data they generate.
    In this particular stage you are going to use your new tool to analyze the first 1000 backlinks of your competitor's domain.
    Program Setup Note: Be certain to set up the software to acquire Google Rank and Alexa Rank information for each backlink and filter out any rel=nofollow links. The setting is easily found on the front of both applications with the exception of the rel=nofollow which is an option in Optilink but automatically checked in Backlink Analyzer.
    When the report is completed sort the backlinks by both PageRank and then Alexa Rank; examine each sorting separately.

    Why Are Both PageRank and Alexa Rank Used?
    The reason both are used is because they each have notable disadvantages and advantages. PageRank is notoriously unreliable especially lately since Google now penalizes the PageRank of any site with any relation to link buying. As a result, sites with low PR could be missed as a quality site. Furthermore Alexa Rank is a decent indicator of a site's popularity but I can't rely on it since it is not an established indicator of how well a site is regarded in Google. Between the two stats, however, we can glean a good indication of the sites that have the best reputation for link building.

    Creating a List of Authority Competitor Backlinks
    Using Excel or another spreadsheet application copy and paste the data you received from OptiLink or Backlink Analyzer into a worksheet. Then create a copy of the sheet so that you have an exact copy of all the data on a single sheet. Now follow these steps:
    1. On the first worksheet sort the data by Google PageRank (PR) from highest numbers to lowest. Now remove all of the pages that had less than a PageRank of 4 so you are left with the best sites according to this data. OR just separate the lower PageRanked sites so they don't get in the way.
    2. On the second worksheet sort the data first by Alexa Ranking (sort lowest to highest numbers) and then do a secondary sort by the Google PageRank (highest to lowest numbers). Delete or remove all sites that have a negative Alexa Ranking ("nm" is how it shows in OptiLink) or otherwise partition them from your other more valuable data.
    Now you have two excellent worksheets that provide lists of authority pages that have links pointing to your competitor.

    How to Use the Backlink Data
    Take some time now to filter the links by domain and you will see just how many links per domain each competitor has. If you see a website that appears to be linking to a website a lot it is usually because either the competitor owns the website or has purchased a link on the website. To find out if your competitor owns the website try running a Whois on the domain.
    Also check the content of the link data for how many pages listed are from the competitors own website. If you see a great deal from their own website then you can be relatively assured they have good content which is important to note; perhaps you need to focus on better content on your own website OR how to get others to notice your good content.
    Now the most logical step is to figure out which links are worth getting for yourself. Chances are a decent number of the links you found are from pages that would be willing to link to you as well.

    Don't Lose Focus on Your Own Website
    So now you have a few tools to conduct a cursory competitor analysis. You will likely find some very useful data that you can act on but is this all you need to do? Is a competitor analysis going to be the golden key to increased profits? No. I have a great deal of faith in competitor analysis because I know determining what a competitor is doing successfully can improve a marketing plan dramatically. That said, you also have to pay close attention to your own website and the quality information that can be gained from using free tools like Google Analytics or handy paid tools like ClickTracks Professional.
    Using a quality analytics program will allow you to get as granular as monitoring the success of each page in your website with details such as: where did visitors come from (somewhere in your site or from another?), how long on average visitors stayed at a particular page, what keywords led visitors to the page (if any), and much more.
    With proper analytics you can actually compare and contrast the effects of minor edits to a page's content; this is called multivariate testing. For example you can run tests to see if you can improve the retention of visitors by adding a better image or a better tag line because you noticed that many visitors were entering at a page deep within your site that was not originally designed as an entry page.
    Truly, the sky is the limit with analytics and it would be irresponsible for me to state that competitor analysis is more important than making your own website run smoothly. Do yourself a favour, if you haven't already got an analytics program running on your site, get it done now or learn how to use the one you have; it will pay off in the long run. Especially when you want to monitor the success of the tactics you applied to your site from your competitor analysis findings.

    Competitive Analysis- Tools & Techniques


    Understanding your competitors -- their strategy, their tactics, their level of success, etc. -- is crucial to the success of your SEO initiatives. I'm not just talking about your traditional competitors, I'm referring to the other sites occupying spots in the SERPs (search engine results pages) for keywords that you are targeting.
    Many free competitive analysis tools are out there, but you have to know where to look for them. One of my favorite SEO blogs (Stuntdubl) offers a veritable Home Depot of such tools, at Mr. Ploppy's Monday Tool List.
    It's a bit like walking into a DIY store and being faced with an overwhelming array of options. What is the right tool for the job?
    Here's a sampling of some of the SEO tools that I use for competitive analysis and what I specifically use them for:

    How To Make An RSS Feed?


    RSS is a method of distributing links to content in your web site that you'd like others to use. In other words, it's a mechanism to "syndicate" your content.
    To understand syndication, consider the "real world" situation where artist Scott Adams draws a daily Dilbert cartoon. The cartoon is made available to any newspaper that cares to run it, in exchange for a fee -- and 2,000 papers in 65 countries do so.
    Unlike Scott Adams, syndication of web content via RSS is unlikely to make you rich. However, it can be an easy way to draw attention to your material, bringing you some traffic and perhaps a little net fame, depending on how good your information is.

    What Is RSS?

    How does RSS syndication work? Say you publish a new web page about a particular topic. You want others interested in that topic to know about it. By listing the page as an "item" in your RSS file, you can have the page appear in front of those who read information using RSS readers or "news aggregators" (explained more in my sidebar article, RSS: Your Gateway To News & Blog Content). RSS also allows people to easily add links to your content within their own web pages. Bloggers are a huge core audience that especially does this.
    What does RSS stand for? There's a can of worms. RSS as introduced by Netscape in 1999 then later abandoned in 2001 stood for "Rich Site Summary." Another version of RSS pioneered by UserLand Software stands for "Really Simple Syndication." In yet another version, RSS stands for "RDF Site Summary."
    History buffs might be interested that there's been some rivalry over who invented RSS. This is why we have both different names and indeed different "flavors" or versions of RSS. Mark Pilgrim's written an excellent article, What Is RSS, that charts the different versions with recommendations on which to use. I'll also revisit the version choice you'll need to make. But first, let's look more closely at some basics of what goes into any RSS file.

    How Easy Is RSS?

    I've been exploring RSS because it was time that Search Engine Watch offered its own stories in this manner. I've read several tutorials about making a feed, and they generally suggest that it is easy. They often offer code that you can "cut-and-paste" and link over to specifications that I actually don't find that clear.
    For example, the RSS 2.0 specification has an "cloud" element that's optional but which a lay person might still wonder if they should use it. Meanwhile, heaven help the person who stumbles into the RSS 1.0 specification and its complicated RDF syntax.
    Sure, making an RSS file IS easy for many. If you understand HTML, you'll probably understand enough to do a cut-and-paste from someone else's RSS file to make your own file. Don't know HTML? Start a blog, because several blogging tools automatically generates RSS files.
    As for those non-technical people using WYSIWYG page building tools or personal home page building systems, have faith. Even you can build an RSS file from scratch, as long as you dispense with some of the extra features you probably don't need. We'll go through how to do this below. Later, I'll also mention some tools that will even do some or all of the work for you.

    The RSS File

    At the heart of an RSS file are "items." No matter what version of RSS you settle on, your file will have to include at least one item. Items are generally web pages that you'd like others to link to. For example, let's say you just created a web page reviewing a new cell phone that's being released. Information about that page would form an item.
    To enter your item into the RSS file, you'll need three bits of information:
    • Title
    • Description
    • Link
    The title and description of your item need not match exactly the HTML title tag of the web page that the item refers to, nor the meta description tag, assuming you use these (don't know what they are? See my How To Use HTML Tags article). You can write any title and description that you think will describe the page. However, using your page's title and meta description tag certainly makes it easy to copy and paste to build your RSS feed.
    In the case of our example page, let's say this is the information we settle on to define it as an item:
    Nokia 3650 Combines Phone And PhotosI've been playing with the new Nokia 3650. Finally, someone has got the combination of a cell phone with digital camera capabilities right!
    http://allgadgetsreviewed.com/nokia3650.html
    Now we have to surround that information with XML tags. These are similar to HTML tags, with the exception that unlike with HTML, there's no set definition of XML tags. Anyone can make up a particular XML tag. Whether it is useful depends on the program that reads the resulting XML file. In the case of RSS feeds, they have their own unique set of XML tags that are defined. Use these correctly, and then anything that reads RSS will understand your information.
    Did that make your head spin? If so, don't reread -- just carry on to see how simple it is. First, open a text editor like Notepad. We're going to build our RSS file using it.
    For your title, you need to start it with the <title> tag, then follow this with the text of the title, then end with the </title> tag. It looks like this:
    <title>Nokia 3650 Combines Phone And Photos</title>
    For your description, you do the same, starting out with the opening <description> tag, then following with the actual description, then "closing" with the </description> tag. Now you have this:
    <title>Nokia 3650 Combines Phone And Photos</title>
    <description>I've been playing with the new Nokia 3650. Finally, someone has got the combination of a cell phone with digital camera capabilities right!</description>

    Next, we add the link information, beginning with <link>, following with the actual hyperlink, then closing with </link>. That gives us this:
    <title>Nokia 3650 Combines Phone And Photos</title>
    <description>I've been playing with the new Nokia 3650. Finally, someone has got the combination of a cell phone with digital camera capabilities right!</description>
    <link>http://allgadgetsreviewed.com/nokia3650.html</link>

    Now there's one more thing we need to do. We actually have to define all this information as forming a particular "item," which we do using a special item tag.
    You place the opening item tag, <item> at the top or start of all the information we've listed. You then place the closing item tag, </item>, at the bottom or "end" of the item information. The finished product looks like this:
    <item>
    <title>Nokia 3650 Combines Phone And Photos</title>
    <description>I've been playing with the new Nokia 3650. Finally, someone has got the combination of a cell phone with digital camera capabilities right!</description>
    <link>http://allgadgetsreviewed.com/nokia3650.html</link>
    </item>

    Congratulations! You've now made your first item. There's a bit more to do to finish our RSS file. First, what if we have other items we want to syndicate? Then we simply add more item elements, just as we did above. You can have up to 15 items. New items tend to be inserted at the top, with old items removed from the bottom, to make room for new stuff.
    With our example, let's see how things look if we add two more items:
    <item>
    <title>Nokia 3650 Combines Phone And Photos</title>
    <description>I've been playing with the new Nokia 3650. Finally, someone has got the combination of a cell phone with digital camera capabilities right!</description>
    <link>http://allgadgetsreviewed.com/nokia3650.html</link>
    </item>

    <item>
    <title>Sanyo Tablet PC Amazes!</title>
    <description>I was dubious about the new Tablet PCs, but then I saw the latest from Sanyo. Wow, cool looks and it works!</description>
    <link>http://allgadgetsreviewed.com/sanyotablet.html</link>
    </item>

    <item>
    <title>Canon MegaTiny Digital Camera Too Small</title>
    <description>OK, there is a limit to just how small is too small. Canon's MetaTiny, no larger than a quarter, simply is too little to use properly</description>
    <link>http://allgadgetsreviewed.com/metatiny.html</link>
    </item>

    Having defined items we want to distribute, we now have to define our site as a "channel." You'll use the same tags as with the items: title, description and link. However, this time the information will be about your entire site, rather than a particular page. That means our channel information would look like this:
    <title>All Gadgets Reviewed</title>
    <description>If it's a gadget, we review it. Learn what gadgets are hot and what's not!</description>
    <link>http://allgadgetsreviewed.com</link>

    Now, how does something reading our RSS file know that the information above is for our "channel" when it looks just like item information? Simple. As long as we don't surround this information with an opening and closing <item> tags, it won't be seen as item information but rather as channel information. That gives us this:
    <title>All Gadgets Reviewed</title>
    <description>If it's a gadget, we review it. Learn what gadgets are hot and what's not!</description>
    <link>http://allgadgetsreviewed.com</link>

    <item>
    <title>Nokia 3650 Combines Phone And Photos</title>
    <description>I've been playing with the new Nokia 3650. Finally, someone has got the combination of a cell phone with digital camera capabilities right!</description>
    <link>http://allgadgetsreviewed.com/nokia3650.html</link>
    </item>

    <item>
    <title>Sanyo Tablet PC Amazes!</title>
    <description>I was dubious about the new Tablet PCs, but then I saw the latest from Sanyo. Wow, cool looks and it works!</description>
    <link>http://allgadgetsreviewed.com/sanyotablet.html</link>
    </item>

    <item>
    <title>Canon MegaTiny Digital Camera Too Small</title>
    <description>OK, there is a limit to just how small is too small. Canon's MetaTiny, no larger than a quarter, simply is too little to use properly</description>
    <link>http://allgadgetsreviewed.com/metatiny.html</link>
    </item>

    There are a few last things we need to do. First, we need to add a tag at the very top of the file saying that this is written according to the XML 1.0 specifications. Right under this, we also have to say what RSS version we are using.
    So far, everything we've done is compatible with UserLand's popular RSS 0.91 version. However, it also matches UserLand's latest RSS 2.0 version, as well, so we'll define the file as meeting that specification. This will allow us to add other neat features in the future, if we want.
    Finally, after the RSS tag, we need to add an opening "channel" tag. That gives us this at the top of the file:
    <?xml version="1.0"?>
    <rss version="2.0">
    <channel>

    At the bottom of the file, after all the items we want to syndicate, we have to insert a closing channel and RSS tag, in that order. Those look like this:
    </channel>
    </rss>

    This means our complete file looks like this:
    <?xml version="1.0" ?>
    <rss version="2.0">

    <channel>
    <title>All Gadgets Reviewed</title>
    <description>If it's a gadget, we review it. Learn what gadgets are hot and what's not!</description>
    <link>http://allgadgetsreviewed.com</link>

    <item>
    <title>Nokia 3650 Combines Phone And Photos</title>
    <description>I've been playing with the new Nokia 3650. Finally, someone has got the combination of a cell phone with digital camera capabilities right!</description>
    <link>http://allgadgetsreviewed.com/nokia3650.html</link>
    </item>

    <item>
    <title>Sanyo Tablet PC Amazes!</title>
    <description>I was dubious about the new Tablet PCs, but then I saw the latest from Sanyo. Wow, cool looks and it works!</description>
    <link>http://allgadgetsreviewed.com/sanyotablet.html</link>
    </item>

    <item>
    <title>Canon MegaTiny Digital Camera Too Small</title>
    <description>OK, there is a limit to just how small is too small. Canon's MetaTiny, no larger than a quarter, simply is too little to use properly</description>
    <link>http://allgadgetsreviewed.com/metatiny.html</link>
    </item>

    </channel>
    </rss>

    Mind Blowing Options

    Everything shown above is the bare basics you need to create a file and start syndicating content from your web site. However, there are additional things you could do.
    For example, rather than your entire web site being a "channel," you could actually have different content from within the web site be defined into separate channels. That's something I'm not going to explore in this article, but some of the resources below will guide you through this, when you feel more comfortable.
    As hinted at, RSS 2.0 allows you to insert all types of additional information into your feed. It can make your head spin to look at these and decide what to use. The easy answer is, don't bother with anything you don't know. Not every aggregator supports all the options offered. As long as you provide the minimum information suggested above, you should be fine.

    Did I Choose The Right RSS Version?

    Earlier, I'd mentioned there are different versions of RSS. Even though we entered the bare minimum of information, it turned out that we were able to have a file that was easily considered to be RSS 2.0, the latest version promoted by UserLand and which is widely used.
    Indeed, the Syndic8 site reports that the most popular format of RSS by far is RSS 0.91 -- and though we've used RSS 2.0, our file as shown is entirely compatible with RSS 0.91. In short, we're in safe company.
    What about that RSS 1.0 version that I said was complicated. Well, it is complicated. However, some people might want to make use of some of the special capabilities that it offers. If you are interested in it, then check out the official specification.

    Saving The File

    Now that we're done adding to the file, we need to save it. But what name shall we give it? I've looked and not seen any guidance on this. I imagine that's because as long as the file is valid (more below), it probably doesn't matter what it's called.
    To make my own decision for Search Engine Watch, I decided to imitate what I saw out at UserLand, which promotes the RSS 2.0 standard that we used. UserLand's example feeds all ended .xml, so let's do the same. As for the first part, that really can be whatever you like. For our example, let's say we just call it feed.xml.
    Now that our file is saved, we can place it anywhere we want on our web server. Let's say we put it in the root or home directory. Then the address to our RSS file would be:
    http://allgadgetsreviewed.com/feed.xml

    Validating The File

    Now our RSS file is done, but did we do it right? To find out, we need to validate it. Use the aptly named Feed Validator service. Simply enter the address to your RSS file, and you'll be told if everything is OK -- or if there's something wrong you need to fix.
    How about a quick preview of how your new feed actually looks? Then pay a visit to Wytheville Community College News Center. Again, enter your feed URL, and you'll see the clickable headlines and story descriptions nicely assembled inside a box.
    The service will also generate a JavaScript code that you can post on your site. Anyone copying the JavaScript can automatically have your feed syndicated into their pages -- pretty neat!

    Get Syndicated!

    Now that your file is validated, you want the world to know about it! To make this happen, visit the RSS directories and search engines listed in the RSS: Your Gateway To News & Blog Content article. They generally offer submission pages, where you can inform them of your feed.
    You also want to make sure people who come to your web site see that you provide a feed. It's very common to post a link to the feed somewhere on the home page of a web site. If you have a dedicated news page, you may want to put it there, as well.
    You can link to your feed with an ordinary HTML link. However, many sites use a small orange XML icon to link to the feed. I've also seen some sites use blue RSS icon. I could find no standard about using these. So, to be safe, I did all three with Search Engine Watch. Look on the home page, and you'll see how it's done (and help yourself to the icons, if you need them).
    Finally, it's good to "ping" one of the major services that track when web logs and RSS content changes. By doing this, you ensure that other sites that monitor these know to check back at your site for more content.
    Weblogs.com is one of these major sites. Enter your site's name and the URL of your feed into the manual Ping-Site Form, and it will know you've updated your feed. The Specs page explains how to set up automatic notification.
    blo.gs is another major change monitoring site. It is supposed to receive any changes that come from Weblogs.com, so you shouldn't need to notify it separately. However, if you want to be on the safe side, it's easily done. Use the ping form, which also explains how to set up automatic pinging, as well.

    Tutorials & Resources

    RSS Headline Creator allows you to skip coding and manually choose the number of headlines you'd like to include in your file, up to the 15 maximum allowed. Then a form with boxes that you fill out will be made. Enter the right information, then push the "Make The Code" button. Your RSS file's code will be generated, for you to copy and paste into a text editor and save.
    How To Get Your Web Site Content Syndicated is a Dec. 2002 tutorial by Kalena Jordan and Dan Thies from which I drew inspiration to get my own feed going.
    Syndic8's How To Section lists a variety of tutorials that discuss how to build RSS files.
    Content Syndication with RSS is a book by Ben Hammersley that was just released in March 2003. I haven't read it, but everything I've heard is that it should be excellent.
    RSS Resources Directory from UserLand categorizes helpful information related to RSS, in different categories.

    What Is RSS Feeds?


    What is RSS?
    RSS stands for "Really Simple Syndication". It is a way to easily distribute a list of headlines, update notices, and sometimes content to a wide number of people. It is used by computer programs that organize those headlines and notices for easy reading.

    What problem does RSS solve?
    Most people are interested in many websites whose content changes on an unpredictable schedule. Examples of such websites are news sites, community and religious organization information pages, product information pages, medical websites, and weblogs. Repeatedly checking each website to see if there is any new content can be very tedious.
    Email notification of changes was an early solution to this problem. Unfortunately, when you receive email notifications from multiple websites they are usually disorganized and can get overwhelming, and are often mistaken for spam.
    RSS is a better way to be notified of new and changed content. Notifications of changes to multiple websites are handled easily, and the results are presented to you well organized and distinct from email.

    How does RSS work?
    RSS works by having the website author maintain a list of notifications on their website in a standard way. This list of notifications is called an "RSS Feed". People who are interested in finding out the latest headlines or changes can check this list. Special computer programs called "RSS aggregators" have been developed that automatically access the RSS feeds of websites you care about on your behalf and organize the results for you. (RSS feeds and aggregators are also sometimes called "RSS Channels" and "RSS Readers".)
    Producing an RSS feed is very simple and hundreds of thousands of websites now provide this feature, including major news organizations like the New York Times, the BBC, and Reuters, as well as many weblogs.

    What information does RSS provide?
    RSS provides very basic information to do its notification. It is made up of a list of items presented in order from newest to oldest. Each item usually consists of a simple title describing the item along with a more complete description and a link to a web page with the actual information being described. Sometimes this description is the full information you want to read (such as the content of a weblog post) and sometimes it is just a summary.
    For example, the RSS information for headlines on a local news website could contain the following information:
    Item 1: 
      Title:Sidewalk contract awarded
      Description:The city awarded the sidewalk contract to Smith Associates. This hotly contested deal is worth $1.2 million.
      Link:http://www.gardencitynews.com/contractawards/sidewalk.htm
    Item 2: 
      Title:Governor to visit
      Description:The governor is scheduled to visit the city on July 1st. This is the first visit since the election two years ago. The mayor is planning a big reception.
      Link:http://www.gardencitynews.com/news/2004/06/gov-visit.htm
    The RSS information is placed into a single file on a website in a manner similar to normal web pages. However, the information is coded in the XML computer language for use by a program (the RSS aggregator) and not by a person like a normal web page.

    RSS aggregator programs
    Think of an RSS aggregator as just a web browser for RSS content. RSS aggregators automatically check a series of RSS feeds for new items on an ongoing basis, making it is possible to keep track of changes to multiple websites without needing to tediously read and re-read each of the websites yourself. They detect the additions and present them all together to you in a compact and useful manner. If the title and description of an item are of interest, the link can be used to quickly bring the related web page up for reading.
    Here is a screen shot of an RSS aggregator in action. On the left is a list of the RSS feeds being monitored, along with an indication of the number of unread items in each feed in parenthesis. On the right are the details of the most recent items in a selected RSS feed (in this case, the New York Times).
    Boston Globe (10), CNET News.com, etc.
    There are many RSS aggregators available. Some are accessed through a browser, some are integrated into email programs, and some run as a standalone application on your personal computer.
    How do I find out if a website has an RSS feed?
    It is getting more and more common for websites to have RSS feeds. They usually indicate the existence of the feed on the home page or main news page with a link to "RSS", or sometimes by displaying an orange button with the letters "XML" or "RSS". RSS feeds are also often found via a "Syndicate This" link. Text "RSS" links sometimes (there are lots of variations) point to a web page explaining the nature of the RSS feeds provided and how to find them. The buttons are often linked directly to the RSS feed file itself.
    Once you know the URL of an RSS feed, you can provide that address to an RSS aggregator program and have the aggregator monitor the feed for you. Many RSS aggregators come preconfigured with a list to choose from of RSS feed URLs for popular news websites.

    How is the RSS feed file produced?
    Unless you are maintaining a website or want to create your own RSS feed for some other purpose, how the RSS feed is produced should not be of concern and you may skip this section.
    The special XML-format file that makes up an RSS feed is usually created in one of a variety of ways.
    Most large news websites and most weblogs are maintained using special "content management" programs. Authors add their stories and postings to the website by interacting with those programs and then use the program's "publish" facility to create the HTML files that make up the website. Those programs often also can update the RSS feed XML file at the same time, adding an item referring to the new story or post, and removing less recent items. Blog creation tools like Blogger, LiveJournal, Movable Type, and Radio automatically create feeds.

    Websites that are produced in a more custom manner, such as with Macromedia Dreamweaver or a simple text editor, usually do not automatically create RSS feeds. Authors of such websites either maintain the XML files by hand, just as they do the website itself, or use a tool such as Software Garden, Inc.'s ListGarden program to maintain it. There are also services that periodically read requested websites themselves and try to automatically determine changes (this is most reliable for websites with a somewhat regular news-like format), or that let you create RSS feed XML files that are hosted by that service provider.
    Tying it all together

    Here is a diagram showing how the websites, the RSS feed XML files, and your personal computer are connected:
    Two web servers each with an RSS file being checked by an aggregator
    The diagram shows a web browser being used to read first Web Site 1 over the Internet and then Web Site 2. It also shows the RSS feed XML files for both websites being monitored simultaneously by an RSS Feed Aggregator.
    Other uses
    In addition to notifying you about news headlines and changes to websites, RSS can be used for many other purposes. There does not even have to be a web page associated with the items listed -- sometimes all the information you need may be in the titles and descriptions themselves.
    Some commonly mentioned uses are:
    • Notification of the arrival of new products in a store
    • Listing and notifying you of newsletter issues, including email newsletters
    • Weather and other alerts of changing conditions
    • Notification of additions of new items to a database, or new members to a group
    One RSS aggregator is all that you need to read all of the RSS feeds, be they headlines, alerts, changes, or other notifications. RSS is shaping up to be a very popular and useful means for communicating.  

    Free Article Submission Directory List (More Than 4000 Directories)

    Get some good quality topic-related links back to your site by writing an article and submitting it to the sites listed in our article directory list. Submitting articles can not only get you links, but could get you some serious traffic if your article is picked up by a big website or ezine.

    Please check the link for FreeDirectory List:

    http://www.directorycritic.com/article-directory-list.html

    Top 35 SEO Techniques/ Strategies

     1.Website analysis
    2. Keyword Research
    3. Google Webmaster Tools
    4. Canonicalization
    5. Competition Analysis
    6. CSS Validation
    7. HTML Code Clean Up & Optimization
    8. Image Optimization
    9. Link Validation
    10. Meta Description Tag Optimization
    11. Meta Keywords Tags Optimization
    12. Navigation & Design Optimization
    13. Robots.txt Optimization
    14. Title Tag Optimization
    15. URL Rewrite
    16. W3C Validation
    17. Link Development.
    18. Check Broken Links
    19. Directory Submissions
    20. Internal Link Structuring
    21. Manual Link Request to Related Sites
    22. Local Search Engine Optimization
    23. Optimization for Multiple Browsers
    24. Article Submission
    25. Blog Designing for the website
    26. Social Network Marketing
    27. RSS Feeds
    28. Social Bookmarking
    29. Video Submission
    30. Content Writing & Optimization
    31. Blog Writing
    32. Content Modification
    33. Website Spell Check
    34. XML Site Map Creation & Submission
            35. Slide sharing

    How To Remove An Entire Directory Or Site ?

    In order for a directory or site-wide removal to be successful, the directory or site must bedisallowed in the site's robots.txt file. For example, in order to remove the http://www.example.com/secret/ directory, your robots.txt file would need to include:

       User-agent: *
       Disallow: /secret/


    It isn't enough for the root of the directory to return a 404 status code, because it's possible for a directory to return a 404 but still serve out files underneath it. Using robots.txt to block a directory (or an entire site) ensures that all the URLs under that directory (or site) are blocked as well. You can test whether a directory has been blocked correctly using either the Fetch as Googlebot orTest robots.txt features in Webmaster Tools.



    Only verified owners of a site can request removal of an entire site or directory in Webmaster Tools. To request removal of a directory or site, click on the site in question, then go to Site configuration > Crawler access > Remove URL. If you enter the root of your site as the URL you want to remove, you'll be asked to confirm that you want to remove the entire site. If you enter a subdirectory, select the "Remove directory" option from the drop-down menu.

    How To Block Access To your Site Using Meta Tags


    The noindex meta standard is described at http://www.robotstxt.org/meta.html. This method is useful if you don't have root access to your server, as it allows you to control access to your site on a page-by-page basis.
    To prevent all robots from indexing a page on your site, place the following meta tag into the <head> section of your page:

    <meta name="robots" content="noindex">
    
    To allow other robots to index the page on your site, preventing only Google's robots from indexing the page:
    <meta name="googlebot" content="noindex">
    When we see the noindex meta tag on a page, Google will completely drop the page from our search results, even if other pages link to it. Other search engines, however, may interpret this directive differently. As a result, a link to the page can still appear in their search results.

    Note that because we have to crawl your page in order to see the noindex meta tag, there's a small chance that Googlebot won't see and respect the noindex meta tag. If your page is still appearing in results, it's probably because we haven't crawled your site since you added the tag. (Also, if you've used your robots.txt file to block this page, we won't be able to see the tag either.)

    If the content is currently in our index, we will remove it after the next time we crawl it. To expedite removal, use the URL removal request tool in Google Webmaster Tools.

    How To Block or remove pages using a robots.txt file ?


    robots.txt file restricts access to your site by search engine robots that crawl the web. These bots are automated, and before they access pages of a site, they check to see if a robots.txt file exists that prevents them from accessing certain pages. (All respectable robots will respect the directives in a robots.txt file, although some may interpret them differently. However, a robots.txt is not enforceable, and some spammers and other troublemakers may ignore it. For this reason, we recommend password protecting confidential information.)

    You need a robots.txt file only if your site includes content that you don't want search engines to index. If you want search engines to index everything in your site, you don't need a robots.txt file (not even an empty one).
    While Google won't crawl or index the content of pages blocked by robots.txt, we may still index the URLs if we find them on other pages on the web. As a result, the URL of the page and, potentially, other publicly available information such as anchor text in links to the site, or the title from the Open Directory Project (www.dmoz.org), can appear in Google search results.

    In order to use a robots.txt file, you'll need to have access to the root of your domain (if you're not sure, check with your web hoster). If you don't have access to the root of a domain, you can restrict access using the robots meta tag.

    What are HTTP status codes ?


    When a request is made to your server for a page on your site (for instance, when a user accesses your page in a browser or when Googlebot crawls the page), your server returns an HTTP status code in response to the request.
    This status code provides information about the status of the request. This status code gives Googlebot information about your site and the requested page.
    Some common status codes are:
    • 200 - the server successfully returned the page
    • 404 - the requested page doesn't exist
    • 503 - the server is temporarily unavailable
    A complete list of HTTP status codes is below. You can also visit the W3C page on HTTP status codes for more information.
    1xx (Provisional response) 
    Status codes that indicate a provisional response and require the requestor to take action to continue.
    CodeDescription
    100 (Continue)The requestor should continue with the request. The server returns this code to indicate that it has received the first part of a request and is waiting for the rest.
    101 (Switching protocols)The requestor has asked the server to switch protocols and the server is acknowledging that it will do so.
    2xx (Successful)
    Status codes that indicate that the server successfully processed the request.
    CodeDescription
    200 (Successful)The server successfully processed the request. Generally, this means that the server provided the requested page. If you see this status for your robots.txt file, it means that Googlebot retrieved it successfully.
    201 (Created)The request was successful and the server created a new resource.
    202 (Accepted)The server has accepted the request, but hasn't yet processed it.
    203 (Non-authoritative information)The server successfully processed the request, but is returning information that may be from another source.
    204 (No content)The server successfully processed the request, but isn't returning any content.
    205 (Reset content)The server successfully proccessed the request, but isn't returning any content. Unlike a 204 response, this response requires that the requestor reset the document view (for instance, clear a form for new input).
    206 (Partial content)The server successfully processed a partial GET request.
    3xx (Redirected)
    Further action is needed to fulfill the request. Often, these status codes are used for redirection. Google recommends that you use fewer than five redirects for each request. You can use Webmaster Tools to see if Googlebot is having trouble crawling your redirected pages. The Crawl errors page underDiagnostics lists URLs that Googlebot was unable to crawl due to redirect errors.
    CodeDescription
    300 (Multiple choices)The server has several actions available based on the request. The server may choose an action based on the requestor (user agent) or the server may present a list so the requestor can choose an action.
    301 (Moved permanently)The requested page has been permanently moved to a new location. When the server returns this response (as a response to a GET or HEAD request), it automatically forwards the requestor to the new location. You should use this code to let Googlebot know that a page or site has permanently moved to a new location.
    302 (Moved temporarily)The server is currently responding to the request with a page from a different location, but the requestor should continue to use the original location for future requests. This code is similar to a 301 in that for a GET or HEAD request, it automatically forwards the requestor to a different location, but you shouldn't use it to tell the Googlebot that a page or site has moved because Googlebot will continue to crawl and index the original location.
    303 (See other location)The server returns this code when the requestor should make a separate GET request to a different location to retrieve the response. For all requests other than a HEAD request, the server automatically forwards to the other location.
    304 (Not modified)The requested page hasn't been modified since the last request. When the server returns this response, it doesn't return the contents of the page.
    You should configure your server to return this response (called the If-Modified-Since HTTP header) when a page hasn't changed since the last time the requestor asked for it. This saves you bandwidth and overhead because your server can tell Googlebot that a page hasn't changed since the last time it was crawled
    .
    305 (Use proxy)The requestor can only access the requested page using a proxy. When the server returns this response, it also indicates the proxy that the requestor should use.
    307 (Temporary redirect)The server is currently responding to the request with a page from a different location, but the requestor should continue to use the original location for future requests. This code is similar to a 301 in that for a GET or HEAD request, it automatically forwards the requestor to a different location, but you shouldn't use it to tell the Googlebot that a page or site has moved because Googlebot will continue to crawl and index the original location.
    4xx (Request error)
    These status codes indicate that there was likely an error in the request which prevented the server from being able to process it.
    CodeDescription
    400 (Bad request)The server didn't understand the syntax of the request.
    401 (Not authorized)The request requires authentication. The server might return this response for a page behind a login.
    403 (Forbidden)The server is refusing the request. If you see that Googlebot received this status code when trying to crawl valid pages of your site (you can see this on the Web crawl page underDiagnostics in Google Webmaster Tools), it's possible that your server or host is blocking Googlebot's access.
    404 (Not found)The server can't find the requested page. For instance, the server often returns this code if the request is for a page that doesn't exist on the server.
    If you don't have a robots.txt file on your site and see this status on the robots.txt page of the Diagnostic tab in Google Webmaster Tools, this is the correct status. However, if you do have a robots.txt file and you see this status, then your robots.txt file may be named incorrectly or in the wrong location. (It should be at the top-level of the domain and named robots.txt.)
    If you see this status for URLs that Googlebot tried to crawl (on the HTTP errors page of the Diagnostic tab), then Googlebot likely followed an invalid link from another page (either an old link or a mistyped one).
    405 (Method not allowed)The method specified in the request is not allowed.
    406 (Not acceptable)The requested page can't respond with the content characteristics requested.
    407 (Proxy authentication required)This status code is similar 401 (Not authorized); but specifies that the requestor has to authenticate using a proxy. When the server returns this response, it also indicates the proxy that the requestor should use.
    408 (Request timeout)The server timed out waiting for the request.
    409 (Conflict)The server encountered a conflict fulfilling the request. The server must include information about the conflict in the response. The server might return this code in response to a PUT request that conflicts with an earlier request, along with a list of differences between the requests.
    410 (Gone)The server returns this response when the requested resource has been permanently removed. It is similar to a 404 (Not found) code, but is sometimes used in the place of a 404 for resources that used to exist but no longer do. If the resource has permanently moved, you should use a 301 to specify the resource's new location.
    411 (Length required)The server won't accept the request without a valid Content-Length header field.
    412 (Precondition failed)The server doesn't meet one of the preconditions that the requestor put on the request.
    413 (Request entity too large)The server can't process the request because it is too large for the server to handle.
    414 (Requested URI is too long)The requested URI (typically, a URL) is too long for the server to process.
    415 (Unsupported media type)The request is in a format not support by the requested page.
    416 (Requested range not satisfiable)The server returns this status code if the request is for a range not available for the page.
    417 (Expectation failed)The server can't meet the requirements of the Expect request-header field.
    5xx (Server error) 
    These status codes indicate that the server had an internal error when trying to process the request. These errors tend to be with the server itself, not with the request.
    CodeDescription
    500 (Internal server error)The server encountered an error and can't fulfill the request.
    501 (Not implemented)The server doesn't have the functionality to fulfill the request. For instance, the server might return this code when it doesn't recognize the request method.
    502 (Bad gateway)The server was acting as a gateway or proxy and received an invalid response from the upstream server.
    503 (Service unavailable)The server is currently unavailable (because it is overloaded or down for maintenance). Generally, this is a temporary state.
    504 (Gateway timeout)The server was acting as a gateway or proxy and didn't receive a timely request from the upstream server.
    505 (HTTP version not supported)The server doesn't support the HTTP protocol version used in the request.