Page titles in reports – where do they come from?

by rocky

(Applies to: server log data sources, SDC)

If you are using server log files, have turned on “Retrieve HTML Page Titles, and if WebTrends doesn’t already know the title, WebTrends actually visits your site to collect the title. Well, it visits what you’ve told it is the site. It goes to the domain that you entered in the “Web Site URL” field on the “Home” tab of the profile’s setup. Once on the domain, it looks for the exact URL displayed in the report. WebTrends does NOT use the domain it finds in your logfiles. It uses the domain you specified in the setup.

So, if you put a non-existent domain name in the “Web Site URL” field, WebTrends will not find your web site and will not collect any titles.

I mentioned “… if WebTrends doesn’t already know the title.” Here’s the deal. For every profile, WebTrends creates a cache file of all the URLs and their titles, as it finds them. WT checks in that file first, and then visits the site only if it doesn’t find the URL in the title cache file for the profile.

The file’s name is [profileGUID].wdb and it’s here: …/WebTrends/storage/config/wtm_wtx/datfiles/titles/

The advantage of having this file around, for server log analysis, is that WebTrends isn’t constantly visiting your site to get titles. (Note: when you do a fresh server log analysis of a profile and you’ve turned on HTML Title Retrieval, WebTrends will hit your server a LOT – once for each unique page. To avoid frenzy on the part of your hosting people, you might want to run that first analysis at night or just turn off HTML Title Retrieval. Or you can create a fake title cache file using the fresh profile’s GUID and fill it with titles from a different profile for the same site.)

Another nice thing about the file is that you can go into it and change the titles. And if you’ve got a title cache file that you’re happy with, as I mentioned above, you can copy the entire contents into the title cache file belonging to a different profile.

A disadvantage of this file is that once WT has a title/URL combination in that title cache file, it won’t know about any later title changes you make. That is, until the individual line in the *.wdb cache expires. The default expiration time is 14 days and it can be set globally (for all profiles) here: Web Analysis >> Options >> General >> HTML Titles. You can also force a fresh title-collecting effort by emptying that file.

For advanced people: A quirk of Page Titles in reports is that if you’ve got parameter truncation turned on for a file type (i.e. your Pages report does not display parameters in the URL) and if your page titles vary according to the content of those parameters, WebTrends doesn’t really know which of the page title variations to use for a truncated file name. Get it? Suppose you have /product.asp?productID=123 that uses title “Product 123’s Page” and /product.asp?productID=456 that uses title “Product 456’s Page”. If WebTrends is set up to display only /product.asp in the Pages report, it won’t know which title variation to display. The answer: WT is set to display and store (in the title cache file) the very first instance of the page title that it comes across in the logs. And forever after, /product.asp will be displayed with that page title. In that situation, it’s a really good idea to edit the title cache file so a non-confusing title is shown.

One final thing while we’re talking about page titles. Suppose you want to suppress page titles completely in your reports. If you use server logs, clear the titles file for that profile and turn off Retrieve HTML Page Titles in the UI.

A postscript about SDC:

If you are using SDC for data collection, WebTrends looks for the title in the title cache file, as described above. The title cache file was filled using data from the values of the WT.ti parameter in the SDC log. If WebTrends doesn’t find the URL in the title cache file, it gets the title from the current SDC log file line, WT.ti, in turn, obtained the title from the <title> field of the page the tag is on. Or, if there’s a WT.ti <meta> tag in the head (put there by you, because you want to override the <title>), SDC will collect the <meta> WT.ti information instead of the page’s title. The <meta> tag trumps the <title> tag.

If you use SDC and want to suppress titles in reports, clear the titles file for that profile and modify the *.wlp file by adding a section called [autoconfig] plus this line:

HtmlTitleAutoConfig=WT.nonexistent

WT.nonexistent can be any WT. parameter name as long as it doesn’t exist.

Share: