How to display any KPI as a measure column – Pt 2 – Server Logs

More on the WebTrends work-around for getting KPIs into measure columns, but this time doing it with server logs, not SDC logs. Plus, a rave about “unity”.

Applies to: Server Log analysis

Well.  Last week’s post on this same topic certainly created a nice fuss.  Thanks everybody. 

We received a question about how to do it if your raw data is server logs instead of SDC logs.  This post has an answer.

The whole thing, you see, depends on creating a custom measure.  In WebTrends, custom measures can be based on two things, either a URL query parameter or a cookie. 

If you’re using SDC, every single log file line will, without exception, have oodles of query parameters created by SDC, even if the original URL had no parameters at all (for example /default.html).  So when talking about SDC data it was easy for us to pick one at random to build the measure on in our previous post.

In ordinary server logs, on the other hand, there are usually quite a few URLs with no parameters such as the previously mentioned /default.html.  There are also a lot of log lines with no cookie.  So, in order to build a custom measure the same way as our previous post, we have to get a parameter of some kind to happen in every server log file line. 

This can happen by editing the logs (ugh) or modifying your page code (um, no) or … much better … by using CDT Script! 

What is CDT Script?  It’s a scripting language similar to javascript that was invented by WebTrends a few years ago.  Their support for it was discontinued in 2006, but it still seems to work just fine.   (And, we suspect it’s still used in-house because it’s simply a very handy thing.  These days, WT is touting ActiveX for accomplishing the same purposes but writing ActiveX requires a compiler.  CDT script is just interpreted, i.e. nothing is hidden.)  Anyway, if you put a pointer (to a CDT script) in one of your profiles, WebTrends’ analytics engine is programmed to execute it as part of the analysis for that profile.  In the case of our particular CDT script, the script simply adds a URL parameter to the URL before it analyzes each log file line.  The log files themselves are never touched or changed.  It happens during processing and the parameter is there only during the act of analysis. 

Got that?  If not, no worries, just follow the instructions below.

Step 1.  Create the CDT script and save it in the CDT Script Storage Place on your WebTrends server. 

This happens outside the WT interface, on your WebTrends server.  Use a text editor like Notepad or Textpad to create a new file that consists of these lines

[cdt-script]
        name= WTuno
        author= The_WebTrends_Outsider
        version= 1.0
        description= Adds parameter WT.uno with value that is always 1
[app/set] var= oneparam:WT.uno value= <1> 

Save this new file under the name WTuno.cdt in this location (in the WebTrends program installation directory):   ./storage/config/wtm_wtx/cdt/

There are already some CDT files there.  Don’t worry about them.
 
 

Step 2.  Activate this script in the profile where you want it to happen

This, also, is done outside the WT interface and involves editing an existing file on the WebTrends server.

Decide which profile you want to use this in and make a note of its GUID (the gobbledegook string from the first screen when you edit a profile).  On the WebTrends server, go to the corresponding *.wlp file (profile configuration file) and open it with a text editor like NotePad or TextPad.  The *.wlp files are located here:  ./storage/config/wtm_wtx/datfiles/profiles/ 

With a text editor, open the *.wlp file you want to change. Find the section labeled [cdt] and add this line to the [cdt] section:

hit-script = wtm_wtx\cdt\WTuno.cdt

Save and close the *.wlp file. 

 

That’s it.  Go to the other post for the details on making your favorite KPIs into measures, and analyze or re-analyze the profile.

 

Postscript 1.   FYI, “WT.uno” is a name we made up, wanting something short that would also refer to one-ness.  

Postscript 2.   Yes, you do have to do it profile by profile.  As far as we can tell, you cannot just add it to the global wtm_wtx.ini file that contains so many other global settings. 

Postscript 3.  You might want to change the name or description of the profile you added the CDT script to, so that the name will remind you that the WTuno script is turned on.  Or you could add it to the profile description. 

Postscript 4.  We actually like the CDT approach more than our previously posted method which relies on a text parameter such as the page title parameter, WT.ti.  There just is something so kludgey about taking a text parameter and tricking it into turning quantitative.  The WT.uno parameter on the other hand is numerical and is a pure clean number 1 every time, and it feels more cosmically appropriate as the basis for a quantitative measure.  

  • In math, the number 1 is the Multiplicative Identity, after all! 
  • And 1 appears TWICE in the beautiful beautiful Fibonacci Sequence, and it’s a Lucas Number and a Catalan number too, which is not too shabby in the aesthetics department. 
  • Sadly, it is NOT a prime number, but at the same time it is the only positive integer that is neither prime or composite. 
  • And don’t forget that it is half of the entire binary system to which most of us owe our livelihoods. 
  • It is its own factorial, and factorials are good because they have exclamation points and are therefore exciting, yes-we-did kinds of things.   
  • It is a square, which is unhip, but it is the only square-free square. 
  • It’s absolutely, definitely an excellent Starting Point, and these days we need as many starting points as we can get because not enough good stuff gets started at all. 
  • It is the number you get if you raise any number to the power of 0, which is almost impossible to wrap your mind around, which makes it good for the soul. 
  • It is NOT a Google number!  Who knew?   
  • In the math geek world, it is classified as a Happy, Lucky, Narcissistic, Powerful, and Practical number. 
  • It is an Odd number but we’re cool with that and you should be too.   
  • Curiously, it’s also classified as an Odious number (coincidental to the fact that in some circles it is the symbol for a certain bodily function) but at least it doesn’t go all the way to being an Evil number.    I am not making this up.
  • In the end, it’s also The Loneliest Number, so isn’t it great that we’re giving it so much company in a log file? 

 Yes, we like WT.uno=1!

(Thank you to Number Gossip for the Unity Number factoids.  Jes’ having fun)

How to show counts of a KPI page as a report measure

Got KPI pages or KPI page groups? Get counts of visits or views of any important pages to show up as their own measure columns in your WebTrends custom reports.

You know how orders and revenue show up in their own columns in WebTrends’ e-commerce reports?

You can get the same thing for any page or group of pages, not just orders.

Such as, a form.  All forms (as a group).    Each separate page in the checkout process.  The acknowledgement page that happens after a form is submitted.  A particular download.  A promotionally-oriented page.  The FAQ.  The site map page.  A particular folder or section of your site.  In other words, anything you think is important and that you want to quantify for different visit segments – you can display it as a measure.

Which means you can use these measures for report where the rows are:

  • a list of campaigns
  • a list of search terms
  • a list of visits coming from affiliate sites
  • visits coming from search terms that are brand words
  • first time and/or returning visitors
  • 1st visits, 2nd visits, 3rd visits, 10th visits, whatever

And, by using the calculated measures feature, you can display conversion rates from form view to form submission, or percentage of all visits that had a certain KPI page.  Since Webtrends allows 20 measure columns per custom report, so you can cram in a lot of great stuff.

The report segment below shows where we used this trick to show campaign results for six campaigns (six rows), showing campaign results in terms of:  visits that contained an application form, visits that contained the acknowledgement page after the application is submitted, and a couple of calculated conversion rates.

 

 

Here’s how to do this core, very central, very useful semi-trick.

Create a custom measure to show VISITS to the KPI page/group

Decide what page(s) (downloads, etc) you want to act as your KPI event(s).  As an example, it could be the page /apply/thankyou.aspx, which is the page seen after an application is submitted.

Open the WT admin screen and go to:

Report Configuration >> Custom Reports >> Measures >> New Measure

  • Give the measure a name that includes the word “visits” because this is going to be the visits version of the measure.  Same for the “column name.”  (Example:  “Visits with Submitted Applications”)
  • “Value to Base On” is Query Parameter and “Parameter Name” is WT.ti.  (it could be ANY parameter with a text value; we chose WT.ti.  This is what we’re calling “hijacking.”)
  • “When to Measure” is “Hits that Match Specified URL.”  It’s the last choice in the dropdown menu that has “All Hits” as the top choice.
  • For “Do you want to sum this measure across the visit” choose “YES.”
  • In the “URL Expression” window, fill in your KPI URL or regex string, following the same process you’d use if you were defining a Content Group.  For our example, this is where you’d enter “/apply/thankyou.aspx”.  Check the Regular Expression box if appropropriate.
  • In the currency window, choose “No Currency” but set the decimal places to 0.

Create a custom measure to show VIEWS of the KPI page/group

This is the same process as above with two differences, indicated by boldface.  The name is of course different, and “Sum Across The Visit” becomes “No.”  You can create a new measure from scratch or just clone the one you made in the previous step and make the needed changes.

Decide what page(s) (downloads, etc) you want to act as your KPI event(s).  As an example, it could be the page /apply/thankyou.aspx.

Open the WT admin screen and go to

Report Configuration >> Custom Reports >> Measures >> New Measure

  • Give the measure a name that includes the word “views” because this is going to be the views version of the measure.  Same for the “column name.”  (Example:  “Application-Thankyou Views)
  • “Value to Base On” is Query Parameter and “Parameter Name” is WT.ti.  (it could be ANY parameter with a text value; we chose WT.ti)
  • “When to Measure” is “Hits that Match Specified URL.”  It’s the last choice in the dropdown menu that has “All Hits” as the top choice.
  • For “Do you want to sum this measure across the visit” choose “NO.”
  • In the “URL Expression” window, fill in your KPI URL or regex string, following the same process you’d use if you were defining a Content Group.  For our example, this is where you’d enter “/apply/thankyou.aspx”.  Check the Regular Expression box if appropropriate.
  • In the currency window, choose “No Currency” but set the decimal places to 0.

 Use the measure in a custom report

Follow the usual steps for adding a measure to a custom report.  You’ll see your new measures in the dropdown list of available measures.

In the Measures screen, after you’ve chosen the measure from the dropdown list, you’ll see a little box where you choose “Method” (usually says “SUM”).   Set it to COUNT.  This is important.

 

Attach it to a profile and analyze.

 

 

Postscript 1:  If you use this in the “visit” version, use it with a dimension that does not change during a visit.   Your stats, if in the “visit” form, get a little crazy if you use a dimension that changes constantly during a visit.

Postscript 2:  This doesn’t get into Visitor History, at least not in its present form.  We’re working on that.

 

Adding the Google Android browser and platform to your reports

For WebTrends software users: Getting WebTrends to report on Google Android in the Browsers and Platforms reports.

Applies to:  software

(Note – in November 2008 WebTrends started making updated copies of browsers.ini and keywords.ini available to software users from their web site!  We’ll post more about this soon once we have checked it out.) 

http://www.webtrends.com/support/browser-and-keywords-updater.aspx

Back to the original post:

Android (the platform for the Google phone) is out.  Here’s how to add it to your WebTrends software configuration so it will show up in browser and platform reports.  It’s a little different from most of the new stuff we have seen lately because Android is, technically, both a platform and a browser.  To be reported as both, it has to be added to the configuration in the platforms area AND the browsers area in the configuration file.  Feel free make your own choice about whether to do one or two.

For you software users, the file you have to change is called browsers.ini.  On Demand users already are seeing Android, though I’m not sure whether it’s as a platform, a browser, or both.

There are usually three or four copies of the file browsers.ini on the typical WebTrends installation, and there could be extra copies on other servers if you use distributed architecture. You’ll need to change all of them.  Typical locations are:

/WebTrends/modules/analysis/engine/8.0d (8.1, 8.5, etc)
/WebTrends/storage/config/component/lookupdata/
/WebTrends/storage/config/engine/8.0d (8.1, 8.5, etc)
/WebTrends/

Open one instance of browsers.ini with a proper text editor. By “proper” we mean something like TextPad as opposed to Notepad, because Notepad doesn’t play well with the system when the file is in use. With TextPad, you can [usually] take the risk of changing the file while WebTrends is running.

Step 1.a – Change the Browser list. Go to the end of the long [Browsers] list. Find the last numbered entry. It’ll look something like this:

Browser29=GECKO 

(you may be at 35, or any number.  Just find the last one.)

Because Android’s User Agent string contains the name “Safari” and “Gecko,” you should put Android above both Gecko and Safari in the list, because WebTrends will use the first one it finds.  So work back up the last, above Gecko, above Safari, and insert an Android line.   I’ve chosen “Android_browser” as the name.  It’ll look something like this. 

Browser26=ANDROID_browser

Re-number all the entries that appear below this one!  Make sure there are no dupes or gaps!

Step 1.b – Add the [Browsers] section specifications.  Go to the end of the [Browsers] specification list (groups of three-four lines).  Add a blank line and then this:

[ANDROID_browser]
log=ANDROID
text=Google Android

That’s all, for getting Android to show in your Browsers report. 

Now for the Platforms reporting.

Step 2.a – Change the Platforms list. Go to the end of the [Platforms] list. Find the last numbered entry. It’ll look something like this:

Platform52=IPHONE

Because Android’s User Agent string contains the name “Linux,” you should put Android above Linux in the list.  So insert a line above the Linux line.  It’ll look something like this (the name Android_Platform is my arbitrary choice):

Platform37=ANDROID_Platform

Re-number the entries!  Make sure there are no dupes or gaps!

Step 2.b – Add the [Platforms] section specifications.  Go to the top of the [Platforms] specification list (groups of three-four lines).  (I have no idea why Platforms is in reverse order from Browsers and the most recent ones are at the top instead of the bottom.  Maybe order doesn’t matter as long as the main list is numbered.)  Anyway, add a blank line and then this:

[ANDROID_Platform]
log=ANDROID
text=Android OS (Linux)

(I chose to add a reference to (Linux) in the text line so that, in the report, users would be reminded that Android is a form of Linux.  It might matter to somebody.)

That’s all for the Platforms part.

Step 3.  Save and close the browsers.ini file.

Step 4.  Copy the file to all locations where there’s a browsers.ini.

Step 5.    Restart your Scheduler Service

There’s a decent chance that this restart won’t be necessary but it sometimes is needed, so do this step if you can (or if you notice that the browsers.ini change didn’t have an effect in your reports).   How do you do this?  You have to log on to the WebTrends machine as a local administrator, go to Services (Control Panel > Administrative Tools > Services), select “WebTrends – Scheduler Agent” and click on Restart.  Or, you ask somebody with admin access to do it.

 

Done!  Your Browsers, Browser Versions, and Platforms reports will now show Android. 

Postscript:  For your reference, here’s how the User String is showing up these days:

Mozilla/5.0 (Linux; U; Android 0.5; en-us) AppleWebKit/522+ (KHTML, like Gecko) Safari/419.3