Article Index
Tutorial
Part 1: Tutorial Setup
Part 2: Install Beet
Part 3: Examine the data
Part 4: JMX Administration
Part 4 Continued: Performance Statistics
Part 5: Log analysis with beet-util
Following Up
All Pages

Part 5:  Log analysis with beet-utils

The beet-utils command line utilities are primarily for the default compressed binary log format. Thus far in the tutorial we've been working with plain text logging, so first we'll use the JMX console to switch back to the default behavior.

For this tutorial we assume you have the command line utility zcat installed on your system.  For Windows users this means Cygwin or a similar *nix-like utilities package.  If you don't have zcat, you can still try the tutorial, but skip the part in Step 2 that tells you to enable Compression.

Step 1:  Verify beet-utils setup

Open your favorite command interpreter (e.g. Cygwin or cmd on Windows, xterm on *nix) and change into your beet installation directory.  Test the installation by running "java -jar beet-utils.jar".  You should see a terse usage message:

beet utils usage message

Leave the terminal window open.

Step 2:  Use jconsole to change logging format

Connect to Tomcat with jconsole and examine the beet-hello/com.mtgi.analytics/defaultTrackingManager/BeetLog mbean as described in Part 4 of the tutorial.  Select the Attributes node under BeetLog and change both Binary and Compress attributes to true:

beet log settings in jconsole

Since we're changing the log format, these changes only take effect after the next log rotation.  By default this happens every night at midnight.  If you are like me and doing this tutorial late at night, you can wait.  Otherwise, select the Operations node under BeetLog:

beet log operations in jconsole

Click the rotateLog button; you should see a window indicating the name of the newly archived log file.

beet rotateLog result

Use a filesystem browser to examine the [TOMCAT]\logs directory.  You should see both the newly archived log file (named like beet-hello-perf.xml.TIMESTAMP), and the new current log file, beet-hello-perf.bxml.gz.  The file extension has been changed to .bxml.gz to reflect the new format.

Step 3:  Click around beet-hello

Once again point your browser at http://localhost:8080/beet-hello. Click around on create, delete, update, clear buttons.  Let the app know that you are there.

Step 4:  Use jconsole to flush any in-memory events

Back in jconsole, select the Operations node under beet-hello/com.mtgi.analytics/defaultTrackingManager/BeetManager.  Click the flush button to force any accumulated events to be written to the log. 

BeetManager operations in jconsole

The result window indicates the number of events flushed to the log.  Because early in the tutorial we configured our flush interval very low (30 seconds), this will likely be 0, unless your window-changing and mouse-clicking skills are unusually well-honed.

Again use the BeetLog.rotateLog operation to rotate our log.  This should create a new log named beet-hello-perf.bxml.gz.[TIMESTAMP] to use in the following steps. 

Step 5:  Convert the current log to other formats

Back in your command prompt window, you can convert the log to various formats using the beet-utils jar.  Try the following commands:

> zcat [TOMCAT]/logs/beet-hello-perf.bxml.gz.[TIMESTAMP] | java -jar beet-utils.jar -tool xml

> zcat [TOMCAT]/logs/beet-hello-perf.bxml.gz.[TIMESTAMP] | java -jar beet-utils.jar -tool csv

> zcat [TOMCAT]/logs/beet-hello-perf.bxml.gz.[TIMESTAMP] | java -jar beet-utils.jar -tool xslt -format text -split event -xsl sql/etl/insert-events.xsl

Where of course you should replace [TOMCAT] with your Tomcat install directory, [TIMESTAMP] with the actual file extension, and use '\' instead of '/' if you are a Windows user.  The last example, which applies a custom XSLT stylesheet to the event data, is a powerful way to customize event log processing.  The provided XSL produces the same output as the '-tool csv' variant, but you can obviously customize its behavior.