Use Webmaster Tools to Clean Up Organic Search URLs

 

If a search engine is showing listings for your pages where the URL contains a campaign parameters such as WT.mc_id or even utm_campaign, you have a reporting accuracy problem.  Think about it.  Anybody who clicks on one of those organic listings will show up in your reporting as having come from the campaign AND having come from organic search, at the same time.  The latter is what you want, the former is going to inflate your campaign numbers because the visit isn’t a campaign visit at all.

Yes, Google, Bing, and Yahoo frequently do have, in their indexes, URLs with campaign parameters.  See our Canonical URLs post for a description of several ways those superfluous parameters sneak into the index.

To see if your own site has any, take a break right now and run a search on this phrase and scan all the results:

site:yoursitenamehere.com (yes, include “site:”)

If you found more campaign-identified links than you wanted to see, there are things you can do.

One is to use Canonical URLs in your page code.  However, be warned that only Google actively honors this – Yahoo and Bing still are not complying with the Canonical URL tag after two years!

So … forget the Canonical URL tag.  Use Google and Bing/Yahoo Webmaster Tools to keep superfluous URL parameters out of the indexes.

(These instructions assume you already have Webmaster Tools accounts on both Google and Bing.  If not, get that to happen!)

Below are the steps we use.

  1. Go first to Google Webmaster Tools (GWT).  Do not go to Bing/Yahoo first.
  2. In Google Webmaster Tools, go to Site Configurations >> Settings >>Parameter Handling tab
  3. GWT will show you a list of parameters that it has found during its crawls.
  4. For those parameters you want Google to omit from the URLs in its index, change the Action to “Ignore.”
  5. Print the list to paper – you’ll need it for the next step.
  6. Save and close GWT
  7. Go to Bing Webmaster Tools
  8. In Bing Webmaster Tools, go to Crawl >> Crawl Settings
  9. Using the list you printed, enter all the parameters you want suppressed

The tip embedded in the above is to use Google Webmaster Tools to get a pretty complete list of all possible parameters. In fact, you’ll probably see parameters you aren’t aware of or have forgotten about.  Then, with your sure-to-be-complete list of parameters, it’s easy to fill up the Bing/Yahoo list, which is a blank slate with no starter list like Google has.

What if the Google list shows parameters that you don’t understand or are not familiar with?

Here’s a second tip that we discovered by accident.  You can coax information from the Google Webmaster Tools parameter list that will help you figure out what some of the parameters are all about.  It won’t be a complete answer, but it will help.

You have to be using Internet Explorer or Chrome, probably Firefox.  Opera, our favorite ultra-fast browser for home use, doesn’t work for this.

The GWT list of parameters looks something like this:

GWTparams

In the screen shot, note that the second column, Action, is all drop-down lists.  If any of your rows are NOT dropdown lists on your screen, click on the “Edit” or “Reset” link at the far right.  The second column item should turn into a dropdown list.

With your mouse, click on the heading or the first parameter, drag, and copy it to the clipboard.

GWTparams3

Paste it into Word.  Not Paste Special, but Paste.  It should paste as messy HTML, with extra stuff, like this:

GWTparams4

Note the red arrow above.  The MS Word pasted copy shows TWO dropdown menus per row, not one!  The second dropdown is live.  Click on it and you’ll see the known values of the parameter, as below:

GWTparams5

This second dropdown has been there all along, but was coded to not display in a browser window.  Copying and pasting it to Word just happens to make it visible.  Cool eh?  You can also use a debugger such as Fiddler to break the invisibility in the browser window, but using Word is much faster.

Now you have more information on what the mystery parameters are all about.  Some will have only one value and might be typos in the code.  Others, like “denomination” above, is revealed to have values of 10, 20, 50, 100 and so on … which we immediately recognized as denominations for gift card purchases.  Not a necessary parameter.

So, with the new information, you can set even more parameters to “ignore” and clean up your organic listings further.

While you’re in Webmaster Tools, especially the Google one, look around.  There is some very useful stuff in there.

Cool custom report: What on-site search terms led to an exit?

Suppose somebody on your site searched for something (using on-site search) and then left the site directly from the results page. Wouldn’t you want to know what the search term was? This is a semi-easy custom report that will give you just that — a list of search terms for site search results that were followed by an exit.

 

Suppose somebody on your site searches for something (using on-site search) and then leaves the site from the search results page.  Wouldn’t you want to know what the search term was?  Apparently the search was unsatisfactory to the visitor.

This is a semi-easy custom report that will give you just that — a list of search terms for site search results that were followed by an exit.

To do this report, you have to know the URL of the search results page and the name of the parameter that contains the search term.  If you set up your site according to the WebTrends book you’ll have WT.oss as the name for the search term parameter, but the parameter could also be “keywords” or “q” or anything else.  (You’re not collecting the on-site search term?  Give it some thought; it’s worth its weight in gold.)

You need to create:

1.  A custom visit filter that includes only visits where the Exit Page was the search results page.  Note that I said visit filter, not hit filter.

2.  A custom dimension based on “Query Parameter,” naming the on-site search term parameter that your site uses or WT.oss if it’s collected by <meta>.  But this dimension has a twist — the “When To Collect Data” setting must be “Last Occurrence in Visit.”  This is key,  otherwise you’ll get all the search terms in the visit when you want only the one that “caused” the visitor to leave the site.  Handy anti-confusion tip: put “last occurrence” into the name of the dimension to prevent mixups with the ordinary version of this dimension.

That’s all.   Put together the report as usual.

A really good (I would even say essential) follow-up analysis is, for each term, the ratio of “exit instances” to the total number of searches using the term.  The latter is of course available from another custom report, set up without a filter and an “Collect Data On All Hits” version of the dimension.

Related Post:  What on-site search term led to this page?

Postscript:

A story: using on-site search analysis, we found a major blunder by the merchandising people — they thought everybody used the industry term “swimwear” and that’s the only word they used. They were one of the few sites that sold a wide selection of swimwear year ’round so it was important to the winter vacation population. The on-site search showed additional terms that actually got no search results — “swim suits” and a couple other regional terms. They corrected the on-site thesaurus and also widened the vocabulary in the visible text and product names. In addition to getting the on-site searchers to the right part of the site, the copy changes influenced their external search rankings for those extra terms. So, business for these products went up two ways. And the people looking for swimwear also found tropical clothes in the middle of winter, so average ticket size increased.