Rate Limiting Detection: Bandwidth and Latency
Ever wonder if your ISP is giving you the bandwidth you are paying for? Do you think your download speeds are different at different times of day? If so, you might want to detect and measure the rate limiting on your network.
What is Rate Limiting?
Network rate limiting is used to limit the amount of traffic on a network. There are two main methods of rate limiting: traffic policing and traffic shaping. Traffic policing measures the rate of incoming traffic and drops packets that exceed the maximum allowed rate. Traffic shaping, on the other hand, queues packets and controls the rate at which they are output. Traffic shaping will increase packet latency by introducing delays, but will only drop packets if the queue overflows.
Rate limiting can be used to control congestion on a network, or to give certain types of traffic lower priority and reserve bandwidth for higher priority traffic. For example, an ISP might limit the rates of BitTorrents or YouTube videos. Some people might think this is a great use of rate limiting because it leaves bandwidth available so they can still access other web sites without experiencing delays. However, there are those that don’t think an ISP should be deciding what traffic is high priority, especially when it is done in a manner that is not transparent.
Detecting Rate Limiting
So how can rate limiting be detected? In most cases, small bursts of packets can be sent without experiencing packet loss or increased delay. It takes a certain amount of sustained traffic to trigger traffic policing or traffic shaping policies to take effect. One method to measure rate limiting is packet flooding. A tool that uses packet flooding will send packets as fast as possible, for long enough to trigger rate limiting, and then measure the rate of traffic received. Latency and packet loss can also be measured. Because of the amount of traffic generated by flooding, tests need to be short, for example 5 or 10 seconds, so that they are not interfering with other network traffic.
Example 1: Packet flooding and incoming packet measurement
In the example below, packet flooding was used and the rate of incoming packets was measured. Initially the rate fluctuated between 30 and 35 Mbps, but after about 4 seconds the rate dropped to a steady 15 Mbps. Packet loss increased from 8% during the first 4 seconds to 57% for the rest of the test duration.
Example 2: Comparing packet flooding on different networks
In the second example, similar methods were used on a different network. In this case the rate started out at 30 Mbps and dropped to 25 Mbps after about 2 seconds. Every so often the rate is lower than average and then higher than average before it settles again to the average 25 Mbps. Looking more closely at packet delays when this occurs, there is a large delay of 40 to 50 ms between two packets, causing the rate to drop, and then packets arrive in a burst, causing the rate to increase briefly. As in the previous example, packet loss also increased when the rate dropped. During the first two seconds there was only 1% packet loss, but it increased to 16% packet loss for the rest of the test.
How Protocols Affect Rate Limiting Detection
The IP protocol used for testing will affect where you can test and will also impact the results. ICMP can be used to test to any node that responds to ping, but it only gives you measurements from the point of view of the sender. By sending large bursts of ICMP packets from your local network to a location on the internet, you will get an indication of traffic rates. However, if your download speed is larger than your upload speed, as is usually the case, you are limited by the upload speed and that is the rate that will be measured.
TCP and UDP protocols can be used to measure two-way results, but require a target that will respond to packets. Speed testing tools generally provide internet servers in various locations for users to target. TCP performance drops significantly if there is packet loss and the round-trip time between the source and target is large (e.g., coast to coast). Therefore, to get meaningful results you need a server that is physically close to your location. UDP does not retransmit packets when there is packet loss, so round-trip time and packet loss do not impact the measurements.
If you are wondering if rate limiting is in use on your network, find a speed testing tool to measure your upload and download speeds. Doing so may shed some light on what is happening to packets on your network. You can also get a ton of overall visibility using AppNeta performance monitoring.