Making your first Custom Reports!

This is for WebTrends newbies who are ready to try a custom report. We think, we hope, that WebTrends users who have hesitated to tackle this ultra-valuable feature will find it far easier than they thought. Often, the hesitation is simply due to terminology issues! We’ll go slow.

This is for WebTrends newbies who are ready to try a custom report.  We think, we hope, that WebTrends users who have hesitated to tackle this ultra-valuable feature will find it far easier than they thought.  Often, the hesitation is simply due to terminology issues!  We’ll go slow.

A “report” is simply a table just like you see everywhere in WebTrends’ results .  It’s just rows and columns.  The rows have labels and are a list of things, like a list of page URLs or referrers.  The columns have labels and contain numbers that quantify the things in the rows, like number of visits or number of page views … per “thing” on the “list”.

Don’t read further until you have the above nicely fixed in your mental concept map.  List of things …  numbers for each thing on the list … the list of things goes down the side … the numbers for each thing go across.

Ready?

  • The “list of things” is called a “Dimension” in WebTrends.  WebTrends has a lot of ready-made dimensions, plus you can easily make additional ones.  

Examples of out-of-the-box dimensions:  Page URLs and titles.  Content Groups.  Referring sites.  Campaign names found in the WT.mc_id parameter.  Visitor’s cookie value.  Day of the week. New visitors and Return visitors. On-site search terms that appear in the parameter called WT.oss.

Examples of dimensions you can create:  Product names as found in your site’s “productID=” parameter.  Campaign names found in a parameter that has a name other than WT.mc_id.  Product colors as found in your site’s “color=” parameter.  On-site search keywords as found in a parameter called “searchterm” or something other than WT.oss.

  • The columns containing numbers are called “Measures.”  Again, WebTrends has a lot of them already made.  In addition to the out-of-the-box ones, you can of course make additional ones..

Examples of out-of-the-box measures:  Number of visits.  Percent of total site visits.  Number of views.  Viewing time.  Number of orders.

Examples of measures you can create:  Number of instances of the parameter “color” having the value “purple.”  Number of instances that contained the parameter “promocode=yes”.

Big point:  The two bullet points above, Dimensions and Measures, are in fact a basic custom report!   Making a custom report in WebTrends goes something like this, once you have opened the Custom Reports >> Reports >> New Custom Report screen:

  1. You choose a dimension.
  2. You choose at least one measure.
  3. You give the report a name and save it into the custom report pool.
  4. You attach it to a profile.
  5. You make sure the template will allow the report to be displayed.
  6. You analyze some data.
  7. You look at the data.
  8. If you don’t like the custom report you modify it or you can un-attach it from the profile and delete it from the pool of custom reports.

That’s the basic structure, but it’s of course not the whole story.  Here are the two other essential things:

  • Use filters to make a custom report that shows data only for a subgroup of your overall data.   For example, you may want the custom report to display data only for first-time visitors, or visits from Google, or visits that included a purchase.

Examples of out-of-the-box filters:  Day of the week is Sunday.  Entry page is URL “xxxx.”   Visitors are Returning.  Campaign ID (from WT.mc_id) is “zzzzz.”  Visits that did NOT arrive from a search engine.

Examples of filters you can make:  Product page views where the product has the color parameter “purple” or “blue.”  Visits that contained at least one product page view where the product has the color parameter “purple” or “blue”.  Pages classified as error pages.  Visits that arrived through search terms that contained your company’s name.  On-site search terms that returned no results, i.e. that had a value of zero for the parameter than shows number of search results returned.

  • If you want, you can nest one dimension inside another, in a so-called 2-dimension Custom Report.  For example, you can nest the “Page URLs Viewed” dimension inside the “New vs Return Visitor” dimension.   The result would be a list of all the Page URLs Viewed by New visitors, followed by another list of Page URLs Viewed, this time by Return visitors.  All in the same report.  The “outside” dimension (New vs Return in this example) is called the Primary dimension and the inner nested dimension is called the Secondary dimension and the whole thing is a Two-Dimension Report.  By the way, when you’re ready, The WebTrends Outsider has a post with more details about the ins and outs of 2D custom reports.

Finally, there are some smaller details that you don’t have to worry about until you’re fairly comfortable making custom reports:

  • If you want your report to show a trend graph (over time) for a particular measure you have to tell WebTrends to do so, by checking the “use interval data” box.  Otherwise WebTrends will conserve database space by not storing the day-by-day info necessary for a trend graph.
  • If you have a trend graph, the first measure will be the one graphed in the default view.  Keep this in mind as you decide on your measures.
  • Check the box “Exclude activity without dimension data” if you don’t want a “None” row in your data for hits/visits that don’t fit the dimension.  We recommend not checking this box while you test your report, because the “None” row can help with troubleshooting.
  • If you use both Include and Exclude filters, remember that Exclude filters trump Include ones.

Having covered the basic concepts and structure of a custom report and hoping you’ll just want to jump in and feel your way through the setup of one, we want to add this:

The hard part of custom reports is deciding what should be the dimension and filtering. Really.  It is not always easy to translate some vague “I wanna know …” question into specifics of dimensions and filters.  If this stumps you, don’t be discouraged.  You will get better at it as your mind wraps itself around this way of thinking.

To get examples of some custom reports that have been explicitly described here in the Outsider, go to the Cool Custom Reports category.  A few of them are a little high-level but you’ll see custom report logic in action.

 

Cool custom report: How visitors refine their on-site searches

The report we describe here is a good one for understanding visitors’ interactions with your on-site search.  It focuses on people who do an on-site search and then, for some reason, immediately search again with a different search term.   In the on-site search biz, the second term is called the “refined” term. 

If somebody makes a second search immediately, they may have thought the first results were too broad (too many results), or perhaps they didn’t get appropriate-looking (or any) results the first time. 

This two dimension report shows the first term they used, paired with the second (refined) term.  

In the analyzed report you’ll find a lot of misspellings (and people’s re-attempts to get it right).  More importantly, you’ll get a sense of visitors’ persistence as they try to find the elusive correct vocabulary for what they want … where “correct vocabulary” all too often means what your site’s copy writers used. 

Overall, if you pay attention to how they refine their first term, you’ll get a better idea of what they wanted on their first search.  This report can help you improve your site’s vocabulary so it more closely matches the vocabulary your visitors use on their first search attempt. 

Report Construction

This report requires that your search results page’s URL contains the search term used in the search.  In other words, it must be something like “/searchresults.aspx?keyword=wild+haggis” or “/searchresults.aspx?WT.oss=wild+haggis”.

As said above, this report has two nested dimensions.  The whole thing depends on a hit filter that allows only search results pages that were immediately preceded by another search results page.  The primary dimension is the search term displayed in the URL of the previous hit (the referring page).  The secondary dimension is the search term displayed in the URL of the current hit.   Got that?

We’ve described the primary dimension previously, in this post on on-site search terms.   It’s basically the search term for the immediately preceding search.  There’s a cool trick to extracting the on-site search term for the previous hit, detailed in the other post.

The secondary dimension is the on-site search term parameter for the current hit (“WT.oss=” or “keyword=” or something else) .

The report filter is –

  • a hit filter
  • “must match ALL criteria” 
  • first criterion:  URL is a search results page (searchresults.aspx or whatever)
  • Second criterion:  Referring Page (per hit) is a search results page (searchresults.aspx or whatever)

That’s it.  Ask if you have questions.  One desirable thing that you can’t get out of this report is detail when three or more searches are strung together.  This report only deals with pairs.  If somebody searched three times in a row, their first pair (search #1 and #2)  will be in the report and so will their second pair (#2 and #3), but there will be no way to see #1 and #3 together.

Beyond WT.srch — The Better Way To Track PPC

If you set up your pay-per-click program so landing pages contain the parameter WT.srch=1, WebTrends Analytics has some out-of-the-box reports that will use WT.srch to generate info on paid search engines and terms.

However, in most cases you will want more useful PPC reporting than what WT.srch allows you. This post is about additional tracking parameters for PPC.

If you set up your pay-per-click program so landing pages contain the parameter WT.srch=1, WebTrends Analytics has some out-of-the-box reports that will use WT.srch to generate info on paid search engines and terms.  WebTrends also uses WT.srch as a flag for adding information to the Visitor History table for things like the initial paid search engine, ever, for that visitor.

You can also use WT.srch=1 yourself to create filters for pay-per-click traffic, for example to look at geography or to look at natural search by filtering PPC out.

If you want to go all-out, you can bump up your WebTrends arrangement to include the services of Dynamic Search, a separate WebTrends offering that does a sophisticated job of tracking and, especially, automatically managing bids and placements.  For extensive programs, the payoffs for automated bid management tend to be spectacular compared to even skilled human management.

However, if you don’t have Dynamic Search you will still want much more PPC reporting than what WT.srch gives you.

Why?  The problem with the canned WebTrends Analytics paid search reports is that they are designed to report only on search-related information that’s in the referrer field of the landing page.  In these days of broad matching, that’s pretty much useless to your PPC management efforts.

  • The referrer field contains only the actual search terms used and the actual engine where the ad appeared.  The referrer field does not reveal the terms you paid for, much less the match type, so there’s no way to connect a visit to your program’s operating details – the terms and match type you paid for, the keyword subgroup, how much you paid, and so on.  And, possibly worst of all, you can’t even match the actual search engine (AOL, myway, etc) to its formal affiliated PPC network – Google, Yahoo, or MSN.
  • Furthermore, you may not realize that a substantial number of PPC visits have no recorded referrer at all ! In a large site’s records that we examined recently, Google’s PPC visits had 19% “no referrer” visits; Yahoo topped 12%, and MSN had around 5%.  (Go here for possible reasons for direct traffic visits)

All of this points to one solution that involves some work on your part >>>  get extra tracking parameters into the destination (landing) page URL.

If you put tracking parameters in the destination landing page, those parameters will be there even if the referrer field is empty.  And they will allow you to track the business value of visits in ways that influence the management of your PPC spending.

The tracking parameters you get into the destination URL should consist of some or all of the following:

  • The mother program for the PPC ad (Google Adwords, Yahoo Sponsored Search, Microsoft search advertising, etc.
  • The actual term paid for
  • The match type, which varies from program to program.  Its values might be “broad,” “exact,” “phrase,” “advanced,” “standard,” or possibly “content match.”
  • The site on which the ad was displayed when the user clicked on it
  • A campaign or group ID that corresponds to how you have the term grouped in the PPC admin program.
  • A subcampaign or subgroup ID that corresponds to the optional subgroup in the PPC admin program
  • Whether the PPC ad is text or image
  • The ad copy variation (the “creative”) that the visitor clicked on  (Google’s ad optimization program, for example, rotates ad copy for you.  Knowing the text that the visitor actually saw is essential for doing your own evaluation of alternatives)
  • On Google AdWords, the Network used (Google itself, versus the Google Search Network which includes AOL and many others)
  • And finally, as a sort of obvious overkill, the fact that the visit comes from paid search (the aforementioned WT.srch, value of 1)

If you don’t track and report on the quality of visits for at least some of the above additional PPC factors, in my humble opinion you shouldn’t be bothering to use WT.srch=1 at all.

The custom reports and filters related to these parameters are not hard to set up.

Getting these parameters to happen in the first place is harder.  It’s a different process for Google and Bing.

And here’s a design for a custom report that shows which actual search terms were matched (by the PPC program) to the terms you actually paid for.

Finally, we have to note with regret that this way of tracking sponsored search information, while much superior to what WebTrends does out of the box, is NOT recorded in the Visitor History table.  In other words, the Visitor History reports such as Most Recent Paid Search Term will still show the crummy information from the referrer field.  There are workaround for this involving the versatile and valuable WT.seg parameters, and we’ll go into that in another post.

Miscellaneous “candy jar” post #1

This post covers a lot of short WebTrends related questions that jump out at us when we look at our on-site and off-site search reports. The topics range from critical ones like DCS Multitrack and increasing the length of reports to little guys like seeing # in URLs.

This post covers a lot of short WebTrends related questions that jump out at us when we look at our on-site and off-site search reports.  The topics range from critical ones like DCS Multitrack and increasing the length of reports to little guys like seeing a # sign in URLs.  We’ve also noted some big topics that we’ve put into our backlog of let’s-do-a-whole-post-on-this.

If you did one of these searches and think we misinterpreted what you are after, let us know!

  1. Chantilly, Virginia,  The number one miscellaneous search!  — It’s the home base for IANA (Internet Assigned Numbers Authority) and thus is the geographic location for many IPs that don’t have listings in DNS or GeoTrends.  Same for Marina del Rey, California.  The state of Virginia is also the home base for AOL’s North American traffic, but the cities vary – Dulles, Manassas, etc.   For US sites, Chantilly will almost always be in the top 5 cities and Virginia will likely be the top state. 
  2. Anything having to do with the WebTrends parameter WT.oss — Lots of searches for this one.  Check out our on-site search category.  We’ll try to do more in this general area. 
  3. Does the parameter have to be WT.oss in order to report on on-site search?  — One thing people may not understand is that many on-site search engines already produce a satisfactory equivalent for WT.oss on their results pages (query=, words=, term=, q=) that allows you to adapt any of WT’s prebuilt custom on-site search reports to use the different parameter name.
  4. What about deleting old log files — The quick answer (in our opinion) is “never.”  You can zip them to a tenth their size and store them.  If they are server logs, you can first edit out image, css, and js hits then zip them, and their final size will be about one percent of the original.  You never know if you’ll need them again.
  5. In the Pages report, you are seeing pages that do not exist on your site — We can think of two possible answers.  If you’re using log files, then somebody could be retyping your URLs or purposely asking for non-existent pages, sometimes for nefarious reasons.  If you’re using SDC logs, somebody could have copied your page code to their site without realizing the SDC tag would send signals back to you, with their URLs (and domain names) attached.  For the latter, if you don’t have a report on “multi-homed domains” to look at from time to time, make one.  Once in a while this becomes a very exciting report!
  6. Does WebTrends keep track of dynamic pages that have query parameters?  — Absolutely yes.  You have a lot of ways to get that information.  The quickest but possibly messiest is to go to Page File Types and turn OFF the “truncate parameters” setting for your page file type.  (To clean up the mess somewhat, investigate the URL Rebuilding function which will allow only your favorite parameters in the reports.)  The more roundabout way is to make reports on the values of just those query parameters, one by one.  The URL Parameter Analysis settings are a good place to start.  If you know Custom Reports, you can make ones for the parameters of interest.
  7. What’s the difference between a URL Parameter Analysis report and a Custom Report on a parameter?  — The subtext for this search is probably “why bother with custom reports on a parameter?  URL Parameter Analysis does the same thing and is easier.”  There are three main reasons:  1)  A custom report can be set up to have more and different measures, 2) A custom report can be placed somewhere logical in your template, close to related reports, while a URL Parameter Analysis report is always stuck in the same section of the template as all other URL Parameter Analysis reports, and 3) URL Parameter Analysis does not work with the ODBC driver and therefore can’t be automatically pulled in to Excel.
  8. Where is the screen resolution report in WebTrends?  — It’s in Site Design >> Browsers and Systems >> Client Details, in the out-of-the-box “Complete View” template.
  9. What is a “page view” in WebTrends licensing?  — Licensing is based on page view limits.  The answer is somewhat complicated and may change so you should ask your account manager for the details and for confirmation of what’s said here.  Usually people on this want to know three things:  1) What file types don’t count as pages views?  (answer:  Images and css files are two main ones.  There are others.)  2) Can I cut down on page views by filtering out certain ones?  (answer:  No, not if WebTrends is doing the filtering.  The act of filtering is a page view action if the file is a page type.  You can however remove lines from the logs before WebTrends sees them.)  3)  If I re-analyze logs, or analyze the same logs in more than one profile, does my license get charged a second time?  (answer:  No.  You can re-analyze the same logs as often as you want without it counting against your license.  But don’t alter or move a log because then WebTrends may not be able to tell that it has already analyzed that file.)
  10. Does WebTrends SDC track off-site links (links that go to other sites)?  — Yes, you have to use the Tag Builder’s advanced tag, and set up the tag so it knows what domains to consider “on-site” versus “off-site.”  The tag watches for links going to those domains and records clicks on those particular links.
  11. Does WebTrends SDC track downloads of files that cannot be tagged, such as pdf, doc, txt, wmv, mp3?  — Yes. You have to use the Tag Builder’s advanced tag, and set up the tag so it knows what untaggable file extensions  it should record as downloads.  The tag watches for links going to those kinds of files and records clicks on those particular links. 
  12. Does WebTrends SDC track form button events (submittals of forms)?  — Yes, the Tag Builder’s advanced tag has code to detect this if your form button uses one of four common submittal actions.  This is probably worth a whole post unless WebTrends already has the documentation for it.  We haven’t checked.
  13. Why should I make the advanced tag track form button events (submittals of forms) if I have acknowledgement pages for every form?  — Good point.  You probably should turn off this part of the tag.  The acknowledgement page is usually all you need.
  14. Does WebTrends SDC track javascript links and events?  — Yes, the Tag Builder’s advanced tag has code to detect this but you have to be careful about the names and labels in the javascript in order to not end up with gibberish.  This is probably worth a whole item unless WebTrends already has the documentation for it.
  15. Does WebTrends SDC track clicks on links that go from one part of a web page to another (anchor tags, bookmark anchors)?  — Yes, the Tag Builder’s advanced tag has code to detect this.
  16. What does it mean when the Pages report shows some URLs with # (pound sign) in them?  — They are clicks on an anchor or bookmark link in your page code; see the preceding item.
  17. How can I remove the # from the URLs in reports?  — You can use WebTrends’ URL Search & Replace function to remove the entire anchor name including the #.  Or you can change the SDC tag to turn off recording of these.
  18. Why can’t I find documentation for DCS Multitrack?  — Embarrassingly, it actually disappeared from documention some time after May 2006 as far as we can tell.  Who knows why.   DCS Multitrack is the key to all of the SDC Tag Builder’s advanced tag event tracking, and it’s useful on its own even without the advanced tag, so it really, well REALLY, needs to be documented.
  19. I am using v. 8.0 and can’t find a link to the Tag Builder.   Will the Tag Builder tag work with 8.0?  Do I have to upgrade?  How do I find the Tag Builder?  — All the tag builder does is grab extra analyzable information and puts it into the SDC logs.  Pretty much any version of WebTrends can work with those parameters, though not all versions have the right pre-configured reports.   You definitely don’t need to upgrade to 8.5 software, though it’s not a bad idea, if you are a software user.   You should however (if you host your own SDC) consider upgrading SDC to a recent version, which is 8.0d right now.  If you are an On Demand user you already have 8.5 of Analytics and 8.0d of SDC.  The general address is http://tagbuilder.webtrends.com.    See also this Outsider post
  20. What is “the same visit“?   — If WebTrends doesn’t see any activity from a visitor’s cookie in 30 minutes, any additional activity won’t be “the same visit.”  (30 minutes is the out-of-the-box inactivity time-out length.  You can change it to something else and there are good reasons for having it longer.) 
  21. How does WebTrends SDC handle https pages?  — The SDC server has to have an SSL certificate and the .gif request in the tag has to be changed from http:// to https://. 
  22. Why doesn’t WebTrends report on time-to-serve?  — Probably because your server logs are Apache, which does not record this.  IIS does.  If you have IIS logs, the time-to-serve stat can be found as the right-most column in the Pages report.
  23. How can I make the Search Phrases and Keywords Reports show more search terms?  — For standard (non-custom) reports you can change the number of displayed rows in any report by going to Web Analysis >> Options >> Analysis >> Table Sizes then changing the “Report Limit.”  Note that this will change the limit for all your profiles.  For custom reports you can do the same by editing the individual profile and going to Analysis >> Table Sizes then changing the “Report Limit.”  There are disk space and performance implications for increasing report table sizes so don’t get carried away.

Those are all the easy-to-answer queries we’ve found in on-site and external search reports for June and July.  Phew.  That’s a lot of search terms to read through.  If we get a good response to this item we’ll try to do it every month.  Thanks for all the traffic!