Unsupported != Doesn’t Work
by July 11, 2013

Filed under: Performance Monitoring

AppNeta no longer blogs on DevOps topics like this one.

Feel free to enjoy it, and check out what we can do for monitoring end user experience of the apps you use to drive your business at www.appneta.com.

For every growing tech company there comes a day when the dev team is simply too busy cranking out cool new features to act as front line support when users hit snags. For AppNeta the time to build a dedicated TraceView support team was a couple months ago; enter myself, one of your friendly neighbourhood TraceView support engineers.

Recently a request came along that stood out from the average “getting started” type questions and set my nerd senses tingling.

“Is it possible to install tracelytics on Gentoo at this time?”

Officially supported? No. But possible?

UnSupported Broken Application Code

It turns out it is possible, but getting there takes a little hackery finesse.

First off I had to find a Gentoo box to play with. In my typical let’s-try-it-out fashion I downloaded and booted the “installer” in a VM before actually looking at the installation instructions. Whoa! I hadn’t seen an installation process that involved since the great battle of of Slackware back in 98’ (many Bothans died to get that sweet K6-2 500 booting!). No time for that now, fortunately some very worthy constituents offer a curated collection of VirtualBox images for situations like this so I grabbed a pre-rolled Sabayon image and got to work.

How It Works

The journey of a thousand traces begins with getting the Tracelyzer daemon and liboboe installed. These are the heart and soul of our instrumentation, collecting events on your app server and securely communicating them back to us for analysis. On supported platforms our install script has these up and running in about twenty seconds, but if he doesn’t whiff the familiar smell of a Debian or Redhat based OS he’ll take his ball and go home.

Fortunately we keep a super secret (sshhh!) stash of files for just such a situation, so let’s start with unpacking the latest version of our base packages (2013-04-03 at the moment).

$ wget https://files.appneta.com/appneta-traceview-base-20130403.tgz
$ tar zxvf appneta-traceview-base-20130403.tgz
$ sudo mv appneta /opt/appneta

Since it’s good practice to run system daemons with as few privileges as needed we’ll get that ready too.

$ useradd appneta
$ chown -R appneta:daemon /opt/appneta/var
$ chown -R appneta:daemon /opt/appneta/etc

Our daemon uses a couple environment variables for configuration, and as Gentoo supports quite a few different init systems we’ll just put together a simple shell script to launch the daemons and let you decide how you want to control their start up.

$EDITOR ~/tracelyzer-start.sh
#!/bin/sh
export APPNETA_HOME=/opt/appneta
export APPNETA_USER=appneta

Unsupported Broken Application Code 2

(I did point out that this is unsupported right? Good.)

Okay now we need to show our binaries where to find the libraries they depend on. The Gentoo libraries have a slightly different name than the ones we link against so we’ll need a couple symlinks. The specific versions below won’t necessarily be the same as yours so take a look at where your base symlinks (/usr/lib/somelib.so) point and adjust accordingly.

$ ln -s libcurl.so.4.2.0 /usr/lib/libcurl.so.3
$ ln -s libcrypto.so.1.0.0 /usr/lib/libcrypto.so.6

We should be able to run the Tracelyzer config tool now. First run our shell script to set your environment variables.

$ sudo -i
$ source ~/tracelyzer-start.sh
$ /opt/appneta/sbin/tracelytics-config

Once that’s finished we’ll need to make some configuration changes, namely adding your TraceView client key which can be found at http://youraccount.tv.appneta.com/getstarted. These two files will need your key added to them.

/opt/appneta/etc/tracelytics.conf
/opt/appneta/etc/tracelyzer/tracelyzer.conf

We’re almost there, just need to create a log directory.

$ mkdir /var/log/tracelyzer
$ chown appneta:daemon /var/log/tracelyzer

And finally add these two obscenely long lines to the startup script to launch the daemons.

$EDITOR ~/tracelyzer-start.sh
#!/bin/sh
export APPNETA_HOME=/opt/appneta
export APPNETA_USER=appneta
/opt/appneta/bin/tracelyzer-conn -M 0  -NnC -i /opt/appneta/etc/tracelyzer/auth_client -L 7831:127.0.0.1:1463 -o ExitOnForwardFailure=yes -o UserKnownHostsFile=/opt/appneta/etc/tracelyzer/auth_server.pub -o StrictHostKeyChecking=yes -o CheckHostIP=no -o ServerAliveInterval=30 -o ServerAliveCountMax=2 -p 2222 customer@incoming.tracelytics.com &
/opt/appneta/bin/tracelyzer -c /opt/appneta/etc/tracelyzer/tracelyzer.conf -l /var/log/tracelyzer/tracelyzer.log &

Executing that script should put you on the TraceView map and start sending host metrics like CPU and memory usage along to your dashboard. The real value of TraceView isn’t collecting host data though, it’s in monitoring your app stack of course. Tune in next time when we’ll walk through manually installing and configuring PHP instrumentation.