Go Long with Golang
by June 8, 2016

Filed under: Company News, Industry Insights

AppNeta no longer blogs on DevOps topics like this one.

Feel free to enjoy it, and check out what we can do for monitoring end user experience of the apps you use to drive your business at www.appneta.com.

We are excited to announce beta availability of Go support for TraceView.

godoc-reference build-passing coverage-100 codecov-100

Are you migrating services to Go? If so, you’ll want performance visibility in production to assess your new components.

At AppNeta, we were in the same boat as we moved part of our data processing pipeline from Python to Go (read our blog post Better Data at Lower Cost to learn more about the key drivers behind this move). Our data collectors process and store well over 100,000 events per second, which come from AppNeta agents installed around the globe. Using Go helped us scale while keeping our operating costs low.

Aside from the cost savings, Go was the programming language of choice for us because of several key considerations:

  • Go is super fast and performs well at scale
  • Go’s fast compilation into static binaries means an easier deployment
  • Built-in concurrency and garbage collection support means our developers do not have to worry about memory management

To learn more, check out a talk our Chief Architect Chris Erway gave at AWS re:Invent on Python, Go and the Cost of Concurrency in the Cloud

As with any company dealing with scale, we cannot afford the loss of visibility into our production services. Over the past several months we have been dogfooding instrumentation to monitor our new Go-based services. We are excited to share it with the community!

Here is an example of what a part of our queue processing pipeline written in Go looks like.

Queue Processing Pipeline

Queue Processing Pipeline

This snapshot represents a single transaction in which a batch of messages was moved from SQS and stored into S3 and DynamoDB. At the top level we can see how batch_s3_worker pulls messages out of SQS, which are decoded by the decodeSQS() function. Once decoded, the AddReq() function takes those messages and adds them to a batch. The BatchWait layer indicates how much time was spent waiting. At the end of the wait time, the batch is flushed to S3 and DynamoDB. Once that’s done, the DeleteMessage() action of the SQS controller deletes those messages from SQS. In this trace details page, it is easy to see what component of the pipeline is actually contributing to the latency, and if there are performance issues, what do I need to fix?

Last week, we utilized the host metrics that we built in to the Go beta instrumentation to fix a memory leak issue. The memory leak could not hide from us for long. As you can see in the snapshot below, our engineering team plotted the host metrics for our Go applications next to the heatmap, saw the growth over time, applied a fix and everything was back to normal!

Go Application Heatmap

Go Application Heatmap

How to Get Go-ing?

To get started, please create a free AppNeta account and follow the install instructions for Go instrumentation here. As with any beta program, the GA version may differ from the instrumentation support available today, but you will play a vital role in the evolution of AppNeta’s Go monitoring by joining in.

We love hearing from our customers and look forward to your questions and feedback as you take the beta instrumentation for a test drive. We would also love to hear more about your plans to incorporate Go in your production environment and the different frameworks and libraries you are planning to use. Email us with your thoughts, comments or suggestions.