Tips, tricks, and pokes, just WebTrends Analytics

Applies to:  Software

We haven’t been posting much because we have no time and no life.  The day job is 110% overwhelming us. 

But one thing that has saved a lot of time in the past few weeks has been being able to see all the configuration settings for our many custom reports.  In one big mad spreadsheet.  Since we have 1,297 custom reports at the moment, it helps.

Just to give you a glimpse of our daily life expressed as a spreadsheet, here’s our listing, in three columns and shrunk to about a jillionth of legible size.  No you can’t magnify it.  But read The Outsider for what we consider to be the best and brightest.

You can have one of these too, floor-to-ceiling if you wish.  Attached to this post is a quick and dirty (actually, somewhat slow and dirty) cscript that parses through all your WebTrends configuration files, or .ini files, and creates a tab-delimited text file with all the settings for all your custom reports.  It shows the name of each custom report, its GUID, and it looks up all the filter and dimension GUIDs and displays their names.  It was written by Mister Peabody who plans to polish it up some day, unless WebTrends Inc provides us with something slicker and better.

The tab-delimited file the script produces can be opened with Excel, producing a spreadsheet with these columns:

• Ini Section:  this column shows the section heading within the original config file, should you want to open the config file and find the report’s raw configurations quickly.  You’ll probably never use the information in this column.
• Report Name:  the name or title of the report.
• Report GUID:  the GUID or ID number assigned to the report by WebTrends and used internally by WebTrends.  It’s listed here because if you open a profile .wlp file, you’ll see only GUIDs for custom reports..
• Profile Category:  if you assigned the report to a WebTrends custom category, the category name appears in this column.
• Dimension 1:  the name of the report’s first or primary dimension, with the dimension’s GUID in case you ever want to look it up in the original ini file.  Also, the dimension information is annotated as to whether it’s a hit dimension or visit dimension, according to WebTrends, but only for custom dimensions you’ve created.  Out-of-the-box dimensions are identified as such (”OOTB”) but the config files provide no information about visits-vs-hit basis, so the output doesn’t show it.
• Dimension 2:  Same as above, but for the second or secondary dimension, if there is one.   
• Exclude Filters:  names of all Exclude filters applied to the report.  They will be marked ”hit” or “visit” depending on their basis.
• Include Filters: names of all Exclude filters applied to the report.  They will be marked ”hit” or “visit” depending on their basis.

Rightclick here and Save To Disk to get the “Dump of WebTrends Custom Report Configurations” script.

Here’s how to use this little script once you have it.   

1. Remove the “txt” filename extension
2. Edit the line that defines the location of the configuration files.  If your configuration files are in the default location and your program is in the default location, all you have to do is replace “X:” with the drive letter
3. Put the file somewhere accessible, like the root of the drive where WebTrends is installed
4. Browse to it with Windows Explorer and double click on it
5. Wait – it can take a while.

Details

• If a cell contains more than one configuration setting (for example, if the report has multiple filters), you’ll see all of them together in one cell, but each will be separated by a pipe character (vertical line |).  You can neaten up these cells by turning the pipes into within-cell line breaks, i.e. turn them into a list inside the cell.  Here’s how to do this for all of them at once:  do a global search and replace where “|” is replaced by Alt-0010 (that’s “hold down the Alt key and press – ON THE NUMERIC KEYPAD NOT ON THE ALPHA KEYPAD – zero zero one zero”).  Onscreen it will look like you’re going to replace the pipe with nothing, but Alt-0010 is an invisible “within-cell-line-break” character.
• Excel 2007 can handle cells with a lot of text, i.e. a lot of filter names.  Excel 2003 … not so much.
• If you look at the script code you’ll see that it’s fairly repetitive and adaptable.  That means we’ll use it for other similar functions in the future.  Watch this space.   But, remember, we’re just tossing it out there.  We don’t support or warranty it.
• Meanwhile, if you can improve it or replace it with something better, PLEASE DO. 
• If you have more than 1,297 different custom reports, let’s hear about it!

Tags

Posts that WordPress seems to think are related :)

Bing is [almost] here.

WebTrends is making it a lot easier to update your keywords.ini and browsers.ini fields.  Go to http://www.webtrends.com/support/browser-and-keywords-updater.aspx for the latest, which for keywords.ini is version 6.  (You’ll get a new keywords.ini and browsers.ini together.)

The referrer is “bing.com” and the query term is in the parameter “q.”  There’s another parameter, “form=” that you might want to consider reporting on also.  One value of  form= is QBLH, an interesting choice.  Look it up. 

If you’ve already customized your keywords.ini and you don’t want it to be overwritten, then you don’t want to use the updated file from WebTrends. So, here are instructions for modification of keywords.ini.

Step 1 – Find the file

There are usually two or three copies of keywords.ini in the typical WebTrends installation. Typical locations are:

/WebTrends/modules/analysis/engine/8.0d (8.1, 8.5, etc)
/WebTrends/storage/config/component/lookupdata/
/WebTrends/storage/config/engine/8.0d (8.1, 8.5, etc)

Open the first instance of keywords.ini with a proper text editor. By “proper” we mean something like TextPad rather than Notepad, because Notepad doesn’t play well with the system when the file is in use. With TextPad, you can [usually] take the risk of changing the file while WebTrends is running.

(Make a backup!)

Step 2 – Change the Engine list. Go to the end of the long Engine list. Find the last numbered entry. It’ll look something like this:

Engine367=Looksmart (if your last numbered entry is 367)

Add a line using the next number, like this:

Engine368=Bing

Step 2 – Add the specifications.  Go to the end of the specification list (groups of three or four lines).  The last one might look like this (if your last one is Looksmart)

[LookSmart]
ID1=looksmart.com
KeywordIndicator1=key=

Add a blank line and then this:

[Bing]
ID1=bing.com
KeywordIndicator1=q=


Step 3.  Save and close the file.

Step 4.  Make the same changes to other copies of keywords.ini in your installation.

 That’s all it takes. 

Postscript

We’ve done the manual change a little differently than WebTrends does.  We’ve chosen to use “bing.com” as the identifier, while WebTrends just uses “bing.”  It’ because we expect to see a bingaroo and a bingbang and who knows what else before long.

We’d like to note, being perfectionists as we are, that the Bing site has no information about itself that’s written for webmasters, analysts, etc.  Its “about” information is entirely a sell to potential searchers.

And, we noticed that The WebTrends Outsider is not properly indexed in Bing.  No big deal.  The same thing happened with Cuil, and look what happened to THAT.

Tags

Posts that WordPress seems to think are related :)

You probably know that Google announced the following last month:  when somebody clicks on an organic search result item on a Google search results page, the referrer string for the landing page will now contain the rank of that original result item as it appeared on the Google results page, to that visitor.  So — if theyclick on a #6 position item, the referrer field of the landing page will contain a parameter “cd=6″ in addition to the normal “q=<search+term>” and other Google info.  Google’s plan is to have this happening in all of its search results, eventually.  Right now, it’s fairly rare.  But we’re on top of it anyway.

If you want to turn that “cd=<rank>” string into a custom dimension, this post tells you how.

With this dimension, you can make a custom report on keywords where each keyword has a list below it of ranks (1…2…3…), showing the number of visits that came from a listing at that rank.  The “list below it” is, in WebTrends-speak, called a secondary custom report dimension.  The list of keywords would be of course the primary custom report dimension for the report.

It’s pretty easy and actually uses the techniques we described in a recent post about using a subpart of a URL.  (In fact, it was our playing around with the new Google referrer field that led to that other post in the first place.)  In that post, we talked about using a subpart of a URL.  In this post, we’re going to use part of the referrer field.

Set up this dimension as follows:

1.  Create it and give it a name

2. On the next screen, base it on “Referring Page (initial in visit)”.  Using this basis will turn it into a visit-based dimension.  That’s how the WebTrends built-in search phrase dimension works too.  If you want, you can instead base it on “Referring Page (any in visit)” and it will be a hit-based dimension that will act in a different way.  See our post on visit versus hit dimensions for a little more information.

3. Click on the Advanced button to open the subpart dialog.

4.  Activate the Regular Expression button and type in this regular expression: 

cd=([0-9]*)

(what this does is … locates “cd=” then extracts any immediately-following numeral(s), and stops extracting when it hits something that is not a numeral.)

That’s it.  Now it’s up to you to make a new custom report with “Search Phrase” as the primary dimension and this as the secondary dimension.  We suggest NOT checking the box that says “Exclude activity without dimension data” for this new dimension, because it helps to see how incredibly sparse these new referrers are.  We also suggest that you filter out PPC traffic and filter in google.com referrers.

PostScript:

• As said above, Google hasn’t rolled this out completely; we are still in the early stages.  You may not see results at first.  Some people estimate this is appearing in 10% of search results links on Google.  My experience is that it’s less than that.
• Google’s initial writeup of this hardly mentioned the rank parameter, but here’s the link.  http://analytics.blogspot.com/2009/04/upcoming-change-to-googlecom-search.html
• Google is doing this only for organic listings, not PPC.  If you use PPC ads, it would be a good idea to remove PPC traffic from this report.  We talked about identifying PPC traffic in another post with a bit more in yet another post on PPC reporting.
• We’d love it if you’d bother WebTrends with a feature request about adding the ability to base custom measures on the referrer.  At the very least, they will find out if it’s easy or hard for them to do, should they decide to do it.  We suspect they are already looking into it – give them encouragement.

Tags

Posts that WordPress seems to think are related :)

This post is about the “Advanced” button that you see (and probably ignore) whenever you define a custom dimension or measure.   The how-tos we describe will seem a little daunting, but … you owe it to yourself to at least read the first part of this post.  You need to know what is possible.

The Advanced button appears in the “Based On” part of the configuration.  It leads to a very powerful way to simplify complicated elements.  You can set up the dimension so that it uses a subpart or section of an element (a URL, query parameter, referrer, etc). 

The Advanced button functionality can save a whole lot of potential hassle compared to the alternative, which is to change code on a page so that the logs or SDC tags collect extra information.  Yay, WebTrends, for allowing us to do this from the UI!   Our programmers are grateful because of the work they don’t have to do.  And our end users are grateful because we can do more, faster (although they don’t KNOW how grateful they really are).

IMPORTANT NOTE:  We’re keeping this post simple from this point forward by describing only what it can do with a custom dimension based on URL.  BUT …

• It works with any kind of custom dimension, not just ones based on URL.  Query parameters, referrer, user agent strings, cookies …. the possibilities are huge. 
• It works for custom measures too.  For example, you can use it to extract an individual numerical value from a long cookie.
• It’s not available when setting up custom filters.  After our first surprise about this, we realized that it’s actually not needed for custom filters — the existing filtering logic can do everything this functionality would achieve.

End of IMPORTANT NOTE.

Getting back to the “Based On” screen … click on that Advanced button, and the simple “Based On” screen expands to show a bunch of fields and choices, shown below.  

The expanded screen offers three different operations (WebTrends usability people, please observe that the screen design doesn’t do a good job of making clear that there are three different operatioins). 

In this post, we’re only discussing the three radio buttons in the middle.  Below is the same screen shot as above, with the portion we’re discussing shown in heavy red outline.  (FYI the other two functions, not discussed in this post, are “Override Default String” and the ultra-valuable Translation (Lookup) file function.  Lookup files are mentioned in this other post of ours.  What, you’re not already using lookup files?  Hmm, we’ll have to write a whole post on it.)

Okay, let’s go back to the part of the screen we care about, the center part that allows us to select a section or subpart of the URL.  Remember, that’s our original topic?

When you first get to this expanded screen, the first radio button of the center (outlined in heavy red) section is turned on by default; it is ”Use Full String.”  The other two radio buttons are two ways to force your dimension to use only a subpart or section of the URL.   

• The first of the two additional radio buttons allows you to dissect the URL using a “pattern match” – i.e. the characters before and after the chunk you want to isolate. 
• The other radio button does the same thing using regular expressions which, in the hands of a master, are able to do just about anything..

What this “base the dimension on a subpart” operation does is best conveyed by examples.  After you’ve read the examples, those of you who are daunted can leave.  But at least get this far!

Example 1:  Your URLs are like this:

/discount/Pineapple_Puppy/coupon/ 105135RED01EP001.html
/discount/Catamaran/coupon/ 8299937-fda.html
/discount/Star_Bar/coupon/ 29348-203849-fda.html
/discount/Cigartown/coupon/ 29348-203849-fda.html

and you want your dimension to use only the values that appear between /discount/ and /coupon/, i.e.:

Pineapple_Puppy
Catamaran
Star_Bar
Cigartown

Example 2:  Your faceted search produces, for every search menu choice made by the visitor, a URL that looks like this:

/facetedsearch/search/|F_1_25|F_2_8|F_19_3|
/facetedsearch/search/|F_2_6|F_4_8|F_7_21|F_9_1|

and you want your dimension to show only the numbers that appear after the F_2 combination, in other words:

8
6

Example 3:  Your URLs are like this:

/shc/s/ s_10153_12605_Clothing_Juniors_Shorts +Capris
/shc/s/ s_10358_12988_Tools_Juniors_Shorts +Capris

and you want your dimension to show only the text in the fourth underscore-delimited field in /shc/s/ hits, thereby reducing the above to just:

Clothing
Tools

How to isolate what you’re after

This is the daunting how-to part and you may leave if you really think you don’t need to know any more.  But, realize that those who keep forging on will know that it’s really not so bad, sleep better, and will feel sorry for those who left. 

Example 1.  In the URL “/discount/Catamaran/coupon/ 8299937-fda.html”, you want to use only the value that appears between “/discount/” and “/coupon/”, when “/discount/” and “/coupon/” are in the URL immediately before and after.  In this example, in other words, you want the value “Catamaran”.

Sample fixed pattern method:  “/discount/%val%/coupon/”

Sample regex method:  “/discount/([^/]*)/coupon/”

We say “sample method” because there are a lot of ways to do it.

Example 2.  in the URL “/facetedsearch/search/|F_1_25|F_2_8|F_19_3|”, you want to use only the numeral after “|F_2_”.  In this case, “8″ is what you’re after.

Fixed pattern method:  can’t use Fixed Pattern because the “|F_2_” does not always occupy the first spot in the string.  The fixed pattern method starts at the beginning of the string, see.  (WebTrends, could you see if you could change this?)

Sample regex method:  “\|F_2_([0-9]*)”

Example 3. in the URL “/shc/s/ s_10153_12605_Clothing_Juniors_Shorts +Capris”, you want to use only the text that’s in the fourth position of the longish string following “/shc/s/”.  The longish string’s fields are separated by underscores _, you’ll notice, and the method will take advantage of that fact.  In this example, you want to isolate “Clothing”.

Fixed pattern method:  can’t use Fixed Pattern because the characters preceding and following the desired section are unpredictable.

Sample regex method:   ”/shc/s/[^_]*_[^_]*_[^_]_([^_]*)”

PostScript

Whoa.  You can see that the regex method can be a little sophisticated-looking.  We highly recommend the method we described in our Frabjous Regular Expressions post, to wit:  “Find somebody who knows regular expressions, and ask them to do it.” 

But don’t let that prevent you from thinking about the potential, in your analytics, of this very powerful little corner of the WebTrends custom reporting functionality.  Find that regex maven.  Or learn regex yourself (it’s more fun than Sudoku).  Either way, go to the next level.

Tags

Posts that WordPress seems to think are related :)

Applies to:  Software

Do you have any custom reports that you’d like to NOT process any longer — but you want to keep the already-processed results available?  An example would be a custom report for a completed campaign.

You can suspend or discontinue an individual custom report without actually deleting it.  With this method, the report stays in the template, its old data can be viewed, but it will no longer process new data.

Add a line to a configuration file that tells WebTrends which custom report(s) should NOT be processed any more.

Step 1.  Know the GUID or ProfileID of the custom report

There are several ways to obtain a custom report’s 11-character GUID, but the easiest is:  From the custom report editing interface, go to the Summary tab and get it from the line “Internal ID.”

Step 2.  Add a line to the appropriate config file

For an individual profile — edit the profile’s *.wlp file, find the [profile] section, and add this line:

reportonlycustomtableprofiles = <GUID>

Globally, i.e. for all profiles at once — Edit the global settings file wtm_wtx.ini, find the [defaults] section, and add this line: 

reportonlycustomtableprofiles = <GUID>

That’s all it takes! 

PostScript and Supporting Details

• If you are suspending more than one report, put a space between GUIDs.
• You might see the GUID elsewhere in the *.wlp file.  Leave it where it is!  Don’t remove anything, just add the line we talked about.
• If you ever re-analyze old data, the suspended custom reports won’t analyze anything.  If you’re going to re-analyze, un-suspend the reports.
• You can find *.wlp files here in your installation directory:  ../storage/config/wtm_wtx/datfiles/
• You can find wtm_wtx.ini here in your installation directory:  ../storage/config/wtm_wtx/
• We agree with that you’re doubtless thinking – it’s odd that the line should say “report only these custom reports” when it means, in fact, “report everything except these custom reports”
• We agree with the other thing you’re thinking — that this should be in the UI!  There are several payoffs for doing this.
  • For end users, it’s a lot less trouble than finding and editing configs
  • For WebTrends, they will be spared the ridicule associated with users seeing a config line that makes no sense at all in the English language
  • For WebTrends, On Demand users who suspend custom reports will be reducing the processing load on the ever-reliable On Demand servers

Tags

Posts that WordPress seems to think are related :)