Applies to: Software
It’s about time we wrote this post. Overdue in fact.
You, WebTrends software user, need to know that this issue exists and you need to monitor it. Starting now.
What is it? After processing your logs, WebTrends stores data in a database. The database is made up of linked tables. Those tables have size limits. It’s a fact of life, and table size limits are a good thing – they allow the program to work faster.
Why should you care? You’re probably not aware that **IF** you fill up one of these tables the related reports can be wrong. Most times you won’t notice or care. Sometimes you will notice and you will care a LOT, as in the ugly examples below.
Ugly example #1: Your end users contact you wanting to know about traffic to their expensive new microsite. You know you’ve been collecting the data correctly because you triple-checked the tagging before and after launch. So you open the Pages report and WebTrends tells you those pages don’t exist. Those expensive pages got no traffic at all, apparently. Knowing how the CEO’s been obsessed with the new microsite, you call in sick indefinitely.
Ugly example #2: Your marketeers contact you wanting to know about the effects of their latest paid search campaigns … were the visits from those keywords producing leads and purchases? You know you have the data because you spent a long time checking all the destination URLs that the marketeers were about to upload to the SEs and you even clicked on a few of the ads after the campaign went live, to check the marker parameters. Yet WebTrends can’t find a trace of most of those keywords, with or without marker parameters. The whole campaign failed completely, you report. But the marketeers have Google AdWords reports that show many clicks. You hear the sounds of shouting in the distance and see glimmers of torchlight …
What has happened? The WebTrends Pages table (ugly example 1) or Search Phrases table (ugly example 2) filled up at some point and aren’t accepting new entries any more. Your new pages and fresh keywords aren’t being recorded. They’re still in your raw data, but you’ve missed the chance to get them into reports.
Let me try saying it another way. If the Page report’s Analysis table is filled up with URLs, new URLs won’t be recorded any more. Yes, they’ll be counted anonymously in page view counts. But their identity, the URL, gets discarded. You won’t see the new URLs in any list of URLs. Same for search terms, referring domains, whatever.
What do you need to do to avoid all this?
- Check for filled-up tables in your profiles.
- Determine which ones need to be watched, based on the size the tables are at already.
- Set up a schedule for checking them every week or month.
- Watch for tables that are getting close to filling up.
- Expand the table sizes for those that are close, or take other actions such as starting a new profile.
How do you check a profile’s table size situation?
- Edit the profile
- Open the Table Sizes tab – you’ll now see a biggish table that looks like this in 8.0 (looks slightly different in 8.1 and up).
Look at the Analysis columns, identifying reports where:
There’s a colored ball-icon under Analysis Status — this means you’ve filled up a table
The “Count” is close to the “Limit” in the Analysis Limit and Analysis Count columns. For example, the Limit may be 100,000 and the Count may be 91,000. Since the Count hasn’t reached 100,000 you haven’t yet filled up a table, but you’re pretty close.
What about the “Report” columns?
Those have to do with the displayed tables, the ones WebTrends shows you. We’ll talk about that more in the next post.
How do you increase table sizes?
There’s an “edit” icon over at the right for some of the tables. Not all of them. We’ll talk about it more in another post.
This seems cumbersome. Why doesn’t the program take care of all this?
Goooood question, so glad you asked.
WebTrends can’t have unlimited tables. It just can’t. The limits are meant to keep processing and retrieval speedy, so they’re a good idea. Part of your job, as the human expert, is to evaluate which tables are important enough that they need to be made bigger.
Having said that from WebTrends’ perspective, there are still a few things to rant about, oh definitely.
WebTrends utterly fails to tell the user how important table limits are. It’s not in the documentation (we can’t find it) and it’s not in the UI. I’m not sure it’s in the training either.
This means that every user will find out the hard way, except maybe those who use consultants or who read The WebTrends Outsider. Having to be burned first is an irresponsible way for WebTrends to manage this.
It’s a credibility killer in the eyes of the end user.
- A search in the forum on “table limits” produces five pages of mentions going back to 2004 and none of those mentions are happy. It’s not like WebTrends doesn’t know it’s a problem.
- At the very least, WebTrends should have the Alerts function cover this. That’s what computers are for. And the Alerts framework is already in place, hey!
- Two or three years ago WebTrends removed the warnings that appeared in the headings of reports when table limits had been exceeded for that report. WebTrends replaced that very useful warning with … nothing. Rumor has it that the warning was removed because the tech support people were tired of dealing with calls about it.
End of rant.