The last thing any IT guy wants to hear from a group of sales reps is “Salesforce is down! There’s something wrong with Salesforce!!!”
Unfortunately, yours truly was in that position on Monday.The AppNeta sales team is an inside team based outside of Boston, Massachusetts. On Halloween, at roughly 4pm EST, I heard the groans from the team that Salesforce was crawling. Those groans put me into action with our PathView Cloud network performance management service in tow.
Considering Salesforce is utilized by almost all AppNeta employees, we monitor the performance of the CRM Website from multiple microAppliances located in AppNeta offices and data centers around North America. That is roughly ten different perspectives. At any rate, we have some interesting and diverse data.
With a hosted service such as Salesforce.com and other cloud based services, every office is a remote office. For Salesforce.com we are continuously monitoring the pure network performance from our locations to Salesforce.com and also using AppView Web to monitor actual HTTP/HTTPS performance any public or private web service.
PathView Cloud provides a distinctly unique perspective of application performance from the point of view of end users and remote sites. So I could easily login to PathView Cloud and see the web paths running out to Salesforce.com. What I saw was fascinating! There were sporadic outages for about 15 minutes visible from a variety of locations. Once the connections were reestablished, the performance was still extremely slow, taking roughly 10 seconds to establish a connection to the web server.
See the full report here: SalesForceOutageFromBoston
Here is where things get interesting: the PathView network monitoring was still gathering network performance statistics from Salesforce.com even though the website was not accessible.
Now that I figured out we had data from the network, not from the web, I knew the problem. It is clear that there was a hiccup on their web application infrastructure and the issue was not network -related. The public interfaces for na2.salesforce.com never went offline, connectivity was always available between the PathView microAppliances and the IP of the service. However, the SalesForce.com web application was not responding, clearly indicating that the problem was in the web service infrastructure and not the network connectivity.
There you have it, an application problem which was not caused by the network!
Network Guys: 1 Application: 0