Adding new search engines to WT’s definitions

The new search engine Cuil was announced today. If you want to add it to your WebTrends setup, it’s very easy provided you can edit the WebTrends installation files. The file you have to change is called keywords.ini. Following is a how-to.

Applies to: software

(Note – in November 2008 WebTrends started making updated copies of browsers.ini and keywords.ini available to software users from their web site!  We’ll post more about this soon once we have checked it out.) 

http://www.webtrends.com/support/browser-and-keywords-updater.aspx

Back to the original July post:

The new search engine Cuil was announced today. If you want to add it to your WebTrends setup, it’s very easy provided you can edit the WebTrends installation files.

The file you have to change is called keywords.ini.

There are usually two or three copies of keywords.ini on the typical WebTrends installation. Typical locations are:

/WebTrends/modules/analysis/engine/8.0d (8.1, 8.5, etc)
/WebTrends/storage/config/component/lookupdata/
/WebTrends/storage/config/engine/8.0d (8.1, 8.5, etc)

Open the first instance of keywords.ini with a proper text editor. By “proper” we mean something like TextPad rather than Notepad, because Notepad doesn’t play well with the system when the file is in use. With TextPad, you can [usually] take the risk of changing the file while WebTrends is running.

Step 1 – Change the Engine list. Go to the end of the long Engine list. Find the last numbered entry. It’ll look something like this:

Engine337=Looksmart (if your last numbered entry is 337)

Add a line using the next number, like this:

Engine338=Cuil

Step 2 – Add the specifications.  Go to the end of the specification list (groups of three or four lines).  The last one might look like this (if your last one is Looksmart)

[LookSmart]
ID1=looksmart.com
KeywordIndicator1=key=

Add a blank line and then this:

[Cuil]
ID1=cuil.com
KeywordIndicator1=q=

Step 3.  Save and close the file.

Step 4.  Make the same changes to other copies of keywords.ini in your installation.

 That’s all it takes.  If you’re wondering where we got the specification for the KeywordIndicator line, we just went to cuil.com, did a search, and looked at the name of the query parameter containing the search keyword.  It’s “q” just like Google.

No, The WebTrends Outsider is NOT in Cuil’s index.  Obviously Cuil is not very hip.

NOTE for the future:  Keep a copy of the new and old keywords.ini somewhere safe.  Any WebTrends upgrades will overwrite your modified keywords.ini with the one that comes with the install, so you will have to do some additional manual stuff after those upgrades or complete reinstalls.

One last thing. Here is a Cool Custom Report that will show you any gaps in your keywords.ini file.

  • Dimension: Referring Site
  • Filter:  Exclude Search Engines

The referring sites in the resulting report are those that are not being counted as search engines.  Enjoy.

Cool custom report: How first visits are different

Comparing the behaviors of first-time visitors to veteran visitors can be a real eye-opener. Unless you take a close look at first-time and veteran visitors separately, you won’t know if your site works exquisitely for people who are already familiar with it but is an overwhelming unhelpful mess for newbies. Or the other way around. Happily, WebTrends makes it easy (if you know how) to separate first-timers from experienced visitors.

Comparing the behaviors of first-time visitors to veteran visitors can be a real eye-opener. 

Unless you take a close look at first-time and veteran visitors separately, you won’t know if your site works exquisitely for people who are already familiar with it but is an overwhelming unhelpful mess for newbies.  Or the other way around. 

Happily, WebTrends makes it easy (if you know how) to separate first-timers from experienced visitors. 

The first-time-versus-veteran-visitor report is so fruitful that we return to it every couple of months.  Here are some things we’ve found in past analyses that would not have been evident from an undifferentiated look:

  • Pattern uncovered:  First-timers linger on the home page and then apparently give up on the navigation they see there and use on-site search instead.  Veterans on the other hand use the navigation because they’ve apparently become familiar with the site structure or labels.  (alternative interpretation:  first-timers who use on-site search just don’t come back!)  Lesson:  Do user studies of the home page and find that the labels are baffling first-timers.  Do simple text changes that produce big changes for first-timers who enter at the home page.
  • Pattern uncovered:  A variation of the above.  First-timers plunge into the navigation and after flailing around for several clicks they resort to on-site search.  Veterans meanwhile go immediately to on-site search.  Lesson:  Do user studies with veterans and find out why the navigation still doesn’t work for them after they know the site fairly well.   Find that the site’s huge numbers of products are sorted into product groups using a categorization approach that doesn’t fit with how the public thinks.  Allow products to be in more than one silo and change categorization to fit with card sorting exercises done in the user studies.
  • Pattern uncovered:  A big chunk of first-timers use search but revert to using navigation because the search results are useless.  Veterans know about the typical irrelevance of search results and hardly ever use search.  Lesson:  Identify some common search terms, run the searches, show results page to management and get immediate funding for fixing search. 
  • Pattern uncovered:  The main distinguisher between first-timers and veterans’ page views is the Advisor feature.  Returning visitors have found that the Advisor is a great way to narrow down their choices intelligently.  First-timers don’t know what “advisor” is and use it late in the visit if at all.  Lesson:  Make Advisor more prominent and explain it better.  Watch for better KPIs and return visits among first-timers.
  • Pattern uncovered:  Visitors’ first and second visits resemble each other a lot.  But somewhere around the third visit, they diverge.  The later visits include more product detail, specifications, downloads, comparisons, and warranty views.  Lesson:  you’ve discovered some of the timing of the decision cycle and a major dropout point in the multi-visit decision process.  Create an email campaign just for two-visit people.  Also, change one of the home page teaser panels to show content typical of late visits so visitors know it’s available.  Increase the amount and depth of the advanced information since it seems to be an enabler for eventual purchase decisions.
  • Pattern uncovered:  First-timers are brought to the site by generic search terms.  Veterans almost always arrive by brand-specific search terms, if they use search engines at all.  Lesson:  make sure paid search terms that are generic go to landing pages that sell the visitor on your company as a whole and provide other first-time-critical info.  Monitor conversion and retention rates for the revised landing pages, concentrating on effects on first-timers.
  • Pattern uncovered:  A large proportion of first-timers go directly to the “clearance” part of the site and then leave without looking at regular-priced offerings.  Apparently they are being brought to the site in large numbers by bargain-hunting sites.   Lesson:  make sure the clearance pages have lead-ins to related non-clearance content, with persuasive text about your company’s value propositions.
  • Pattern uncovered:  Veteran visitors to your travel site use the “my dates are flexible” button almost half the time, while first-timers hardly ever use it.  Lesson:  make the button more prominent and add a “what’s this?” explanation that showcases the value of this feature.  After the change, monitor first-timer behavior especially with respect to use of that feature, conversion, and return visits.

Convinced?  Want to try it?  WebTrends makes it easy because the count of past visits by each visitor is a special parameter attached to the first hit of every visit after the visitor’s first visit. 

  • You must be sessionizing with a persistent cookie to get at this information.
  • You must have visitor history turned on
  • You must have enough past history to be confident that a first-time visitor is, in fact, probably new to your site (as opposed to simply being new to the WebTrends cookie)
  • You must have enough visitors to make the subgroups large enough to give statistically reliable results.  I wouldn’t base any decisions on fewer than 5,000.

It’s important to realize that the best cutoff point between newbie visitor and veteran may vary from site to site.  A common starting point for the definition of “veteran” is the fourth visit.  It depends on your site’s audience.  Explore.

Here’s the generic way to create custom reports on Pages (or content groups or scenario analyses or whatever you want as a dimension) for visitors with different amounts of site experience. 

To make a filter that includes only first-time visitors: 

  • Create a custom visit filter based on New versus Return.  The screen will refresh giving you radio buttons for New or Return.  Choose New.

To make a filter that includes only visitors with at least three visits:

  • Create a custom filter of the Visit type.  Filter on Entry Page, matching values equal to *.  (The asterisk  means it doesn’t matter what page the visit started on. )   Do not check the Regular Expression box.  Specify a URL parameter named WT.vr.vc with a value of >3 (if you want 3 to be your cutoff point).  Make sure you also indicate that this is a numeric parameter, or else the “greater than” sign won’t mean anything.

That’s it.  Apply each parameter to its own custom report with a dimension of Pages or Content Groups or Scenario or whatever you like.   Compare using your favorite comparison method.

If you’re curious about how we know about WT.vr.vc, look in the “Visitor History Parameters” section of the Administration User’s Guide, available in the Product Documentation section of the Customer Center.  There are nine types of visitor history parameters!

The Incredible Editable SDC Tag

If you’ve had the opportunity to take a look under the hood of your SDC page tag (and you understand javascript) then you should be pleasantly surprised at how readable the SDC tag is. Take a look at almost any other tool and you’ll find an obfuscated mess. There is not a chance that you could modify how those work. But the SDC tag? Go ahead and hack it up to your heart’s content!

Applies to Software, On Demand

If you’ve had the opportunity to take a look under the hood of your SDC page tag (and you understand javascript) then you should be pleasantly surprised at how readable the SDC tag is. Take a look at almost any other tool and you’ll find an obfuscated mess. There is not a chance that you could modify how those work. But the SDC tag? Go ahead and hack it up to your heart’s content! (Sidenote: This points to a fundamental difference between WebTrends and most other tools: WebTrends wants you to customize, to make it work just how you want it to work. The price is that you can screw things up monumentally, but if you don’t have a cookie cutter site then why would you want a cookie cutter analytics tool?)

There are probably some WebTrends’ Insiders who are breaking out in a cold sweat after reading that. In the past I have been discouraged by WT from messing with the code. Talk of upgrades and difficulty of support were bandied around, and ignored by me. Just try to segregate your additions and comment your modifications and you’ll be fine. The code is pretty mature and it doesn’t tend to change much from version to version so additions and modifications are usually fairly easy to port to a new version of the tag.

That said, it must be noted that WT just rolled out http://tagbuilder.webtrends.com. This is a nifty new tool that will create customized code that contains only what you need it to contain. The code it puts out has also been completely re-written from the previous standard WT code. It is still readable, but now it’s all object oriented. Very fancy, and it requires slightly more sophisticated javascript knowledge if you want to customize it without making a mess of the whole thing.

So anyway, what are some common customizations? Here are a few simple ones:

  • Translate some sort of information on your page into a standard WT variable. There are a bunch of ways to do this that don’t involve modifying the Javascript, but sometimes it’s easiest to just say WT.xyz = myVariable in the code. That way you can add all sorts of If..Then’s and such to get it just right.
  • Modify the page title. Many sites have very useless or repetitive title tags. Writing a bit of Javascript to pull actual useful information into the WT.ti tag can make your reports much more readable. Examples include stripping out your company name if it appears in every single title tag, replacing the title with a main header from the page body, or parsing the URL and extracting a useful page title from that.
  • Pull in information from 3rd party iframes on your page. These are starting to become pretty common as sites contract with other companies to provide services or content. Examples of these 3rd party services include product reviews, FAQs, maps and news. Typically information in these iframes isn’t available to the WT tag because they often load after the tag fires. What to do? Write some javascript that delays the tag firing until the iframes are loaded and you’ve got the information you need from them. (This can be tricky because you don’t want to miss short page views that are over before the delay is up, but a smart javascripter can make sure the tag fires almost all the time.)

How have you modified the WT tag?

 

The deadly WebTrends “Re-Analyze” button

When you click on Re-Analyze, you get the warning “Any previously analyzed data will be lost.”

Let’s make that little warning really clear. It means: Should you go ahead with a re-analyze, ALL previous processing will be erased, along with all WebTrends-controlled backups. Your configurations stay, but WebTrends deletes all statistics and all backups and starts analysis over with the earliest raw data it can find.

Applies to:  Software

“Danger!” 
“Don’t remind me of old heartbreak.” 
“Aargh.” 

These are a few of the reactions by Outsiders to mentions of the Re-Analyze function in WebTrends.  

To its credit, WebTrends a while ago renamed it to “Clear Analysis Data” and shows a warning when asking you to confirm this action:  Any previously analyzed data will be lost.”   

However, in 8.1 WebTrends renamed it again, back to “Re-Analyze.”  Confusion and danger return.

Let’s make that little warning really clear.  It means:  Should you go ahead with a re-analyze, ALL previous processing will be erased, along with all WebTrends-controlled backups.  Your configurations stay, but WebTrends deletes all statistics and all backups and starts analysis over with the earliest raw data it can find. 

The old data is really gone.  There is no “restore” unless you have a tape backup. 

It can ruin your day.

Experienced WebTrends users instead use this simple alternative:  create a clone (copy) of the profile and analyze it.  When it’s done and you’ve checked it, you can delete the original.   

And never touch “Re-Analyze.”

 

 

 

Ask or Suggest

This is the place where you can suggest topics or questions. In the comments, we mean. Go for it.

This is the place where you can suggest topics or questions. In the comments, we mean. Go for it.