Filed under: Performance Monitoring
“Can you give me a use case where you created some value for a customer?”
As a systems engineer at Apparent Networks, we get this question a lot. A few months ago, I was working with a managed service provider (MSP) attempting to provide virtualization as a service for a housing association with 6 locations. As expected, we were using applications from Citrix & VMWare. High latency and data jitter, low capacity, packet loss and other various network impairments were serious challenges for the organization.. They had VPN tunnels established between offices using D-Link DIR-130s. Comcast was their ISP connecting all 6 locations. Immediately, after configuring their network, they started receiving major complaints- frequent disconnects, congestion and overall slowness.
The MSP deployed one microAppliance at the main office and one microAppliance at a secondary office using a PathView Cloud Assessment Kit. Diagnostic paths were set up in parallel to measure end-to-end performance through the VPN tunnel and again outside of the tunnel across the WAN, as well as to monitor impairments of the LAN. With PathView Cloud, we were easily able to monitor the unmanaged WAN between the two sites, through third party devices and through the housing association’s VPN tunnel.
We continued monitoring the network for a week to get a better sense of the situation during peak and off-peak times. The results were scary. There was a severe amount of packet loss inside of the VPN tunnel; conversely, loss outside the tunnel was marginal, often below 1%. We uncovered that targets within the local area networks at both ends had a mixture of duplex mismatches as well as poor Wi-Fi performance and media errors. These LAN issues however, were not the primary cause for the performance loss of the interoffice VPNs.
Using the new PathView Path Comparison reporting capability, we were able to easily trend, compare and analyze paths inside and outside of the VPN tunnel. Static IP addresses were added to the microAppliances, which were then plugged into the office sites- in a few minutes we were good to go. Yes, it really is that easy! After gathering data for just a day we were able to find a high degree and frequency of packet loss as shown in the above graph. Next, we set up dual-ended testing, with each microAppliance testing one direction of traffic (upload/download) to determine exactly where the loss was occurring. There were often spikes of up to 40% loss on the download side of the tunnel as shown in the graph below. It’s no wonder there were so many complaints about timeouts and slowness! With this much loss, the housing association was lucky that the VPN tunnel was even operating, let alone maintaining a connection. One other question we had was – What did traffic look like outside of the VPN tunnel? Perhaps Comcast was to blame for packet loss outside of the tunnel that resulted in an unstable VPN connection? To find out, we ran paths outside of the VPN, to and from the same locations. On the graph below, the blue line is the path setup between the remote office and one of Apparent Networks’ hosted appliances. This test gave us insight into performance outside of the VPN tunnel from the remote office to our appliance in NY. There is zero loss during the selected time range. Similar paths were setup at the main office and both the results and graphs were the same.
Using PathView’s path comparison, we pinpointed the problem to the routers creating the VPN tunnel themselves. We determined there was data loss occurring and the network infrastructure was not to blame, leaving us with the two devices on each end of the VPN. Our final suggestion was to trade in the D-Link VPN routers for more industry grade equipment to establish a reliable tunnel for use with the virtualized applications. Needless to say the problem was resolved between these two sites. The proof is real, and the content was easy to print out as a deliverable for the end customer. Not only did the housing association begin to use PathView Cloud for continual monitoring of their VPNs, they began to use it to resolve the rest of their connectivity and performance issues within the LANs.