Making a Test Log

Applies to:  On-Premises (Software) version of WebTrends

Every On-Premises user should know how to make a test log.  It’s essential if you want to

• be absolutely sure your customizations are right
• just want a bunch of data to analyze, for your own WebTrends practicing

It usually takes only a few minutes to make a test log and feed it into a test WebTrends report profile.  In this post I describe one way to do it.

There are a lot of steps below, but if you understand how WebTrends and log files work together in a general way, you shouldn’t have any trouble and you can probably skim most of it.

Steps 6 and 7 in the second section are the value-adds of this post, the details that most people don’t think about.

You’ll need:

• Access to existing logs (ones you already feed to WebTrends)
• Admin access to WebTrends (the ability to create a new profile, Data Source, custom report, etc)
• A plain text editor, preferably one that will handle large files, for example TextPad.  NotePad will work okay if your logs aren’t very big.
• Knowledge of what you want to test, i.e. what parameters, URLs, whatever.
  • If you already know what you want in the log, you can build a short hypothetical visit in a fresh log by making copies of just one log file line, then editing the lines so they work as one visit.
  • Or go to your site and click around, making your own test visit that you can isolate into a test log and analyze by itself.  Often, just a few lines will be all you need to test.
  • If you mainly want bulk data as a WebTrends sandbox, do a lot of clicking.  Change browsers and erase cookies as you go in order to make yourself look like more than one visitor.  Use this as a core and copy, paste, jumble up to make the file bulkier.  You’ll straighten it out in Step 5.
  • For bulk data, you can also install a crawler and tell it to crawl your site, then use the resulting log.   HTTP Track is one crawler; see Wikipedia for others.  (contribute a little $$ to Wikipedia while you’re at it!)
  • Also using a crawler, you can feed it a list of the important URLs, then use the resulting log.

Steps to create a test log:

1. Find out where the current logs are located.  A basic way to find out is to edit a profile, identify the profile’s Data Source, then edit that Data Source  to see the file location info.
2. With the text editor, create a new empty file.
3. With the text editor, open one of the existing log files.  Open the one that contains your test visit or your crawler’s visit, depending on your basic approach.  If you want to build a hypothetical visit, you can copy just one line from any log.
4. Copy one or more data lines and paste it/them into a new blank file.
   1. Short hypothetical visit – Open any log file.  Copy one existing line and paste it several times into the new log, then edit the lines so they contain whatever URLs (etc) you are trying to check.  With this method, all the lines will have the same cookie and therefore they will all look like the same visitor, same visit.  Your resulting WebTrends report will be small, but it will be focused on exactly what you wanted to test.
   2. Test visit - Locate the lines from your test visit, if you did one, and copy  and paste them into a new file.  (After analysis, you’ll check to see if WebTrends shows exactly what you did in your test visit, no more and no less.)
   3. Bulk data – Grab the log containing your many-click visit or your crawler activity.  Copy and paste all the data lines.  To get more bulk, paste over and over again.  You probably should change the cookie values here and there to show a bigger number of unique visitors.
5. Change the time/date stamps on all the data lines in your file so they are strictly consecutive, although the time spacing between them doesn’t matter.  Check to make sure not a single line is out of order!  (I usually do this editing manually.) (Oh, and it goes without saying that none of your dates/time should be in the future.)
6. Copy and paste the #Fields line from an existing log into the first position of the new log.

   #Fields: date time c-ip cs-username cs-host cs-method cs-uri-stem cs-uri-query sc-status sc-bytes cs-version cs(User-Agent) cs(Cookie) cs(Referer) dcs-id

7. Copy the very last line of your new log and paste it into the very last position (you’ll have a pair of identical lines at first) then change its date to be 1 day later.WebTrends will not analyze this line but it MUST be there, with a time stamp at least a half hour later than everything else in the log.  I use a whole day later to be easier to see.
8. Save this new file.
   • The name of the file and the extension don’t matter to WebTrends; you’ll be telling the Data Source the exact path and name.
   • Save it in same place as the other logs or create a new folder, it doesn’t matter.  The important thing is that WebTrends can reach the file as well as not mistake it for a regular log and process it into your regular reports!

Steps to analyze the log:

1. Go into the WebTrends admin console
2. Go to Administration >> Data Sources and find the data source that corresponds to the one you’ve been messing with.  Create and edit a copy of it.
3. Edit the file location information to point to the file you just created.
4. Save and close the new Data source.
5. Go to the list of profiles and clone one or create a new one.  Name it appropriately.
6. Edit the new profile’s Data Source to be the new Data Source you just created.
7. Turn off all hit and visit filters (such as spider filters) especially if you used a crawler.
8. Save the profile, analyze it, and, most importantly, see if the result is what you expected.

Tags

Somewhat Related Posts