Verizon 4G LTE: Two Datacards and a WiFi Hotspot Massively Reviewed

4G LTE Performance Comprehensively Explored

So I’ve managed to do an absolute ton of drive testing with virtually every device. I’ve recorded 542 data points in total, and run countless more that I didn’t record just for subjective measure. My testing was done primarily in Phoenix, AZ over a number of weeks, though the dates are somewhat different for each device. In Phoenix, I drove a complete loop around the 202 while running continuous tests, and also tested downtown in central Phoenix, in Scottsdale, Mesa, Gilbert, Chandler, Tempe, and inside every major mall. I also tested the Pantech UML290 in Houston in the George Bush Intercontinental Airport, in Newark Liberty International Airport, and in Washington Dulles International Airport. Remember that even though in some markets Verizon has more 700MHz spectrum, right now Verizon’s LTE network only uses 10MHz channels, so performance really is comparable everywhere right now. 

This first set of tests were taken using the Speedtest.net flash test to the Phoenix, AZ server. I tested across Windows 7 and XP on a Latitude XT, and later on a 2010 and 2011 MacBook Pro. I didn’t notice any throughput differences or other big differences between Windows or OS X beyond the software stability issues mentioned earlier. I’ve divided things among devices, though I haven’t collected as much data with the USB551L as I have the Pantech UML290. That said, given the same chipset you’d expect similar performance, and I’ve tested with both at the same locations and found them virtually identical. 

First up is the all-important downstream measure. I’ve created histograms here, to show the complete performance distribution. 

There’s been a lot of discussion centering around whether the speeds of these launch devices are representative of what loaded cells will look like. It’s really impossible for us to say for certain, however network operators have been known in the past to shape traffic so as to not create unrealistic expectations when cells start being loaded. I think there’s likely some of that going on, in addition to some genuine slowdown. I started the UML290 testing immediately after launch (1/22/11), and the data for it represents effectively the earliest set of samples I have. Testing on the Thunderbolt and Samsung SCH-LC11 were done later (mid April) and show a bit of a drop in throughput with a nice gaussian distribution starting to form around 10Mbps. With even more testing, I fully expect we'd see that shape even clearer. It’s obvious to me at least that some loading has happened, but speeds are still impressive. Keep in mind that Verizon’s advertised down speeds for their 4G LTE network are between 5 and 12Mbps. 

My highest measured downstream throughput from speedtest.net on 4G LTE was actually on the Thunderbolt, at 32.96Mbps. I did see a peak of over 39Mbps briefly at one point on the Pantech UML290, as shown below.

Next is upstream, of which again I’ve created histograms. Note that the Thunderbolt is a 2x1 device while the others are 2x2, which explains some of the upstream throughput distribution difference. Verizon advertises upstream speeds of between 2 and 5Mbps, which seems justified if a bit less conservative than their downstream throughput advertising.

Finally is latency, which speedtest.net doesn’t do a super perfect job of characterizing, but does provide representative measures.

Sub-100ms latency most of the time is impressive, and makes LTE more than suitable for online gaming. I’ve put together a short video showing that playing CS:S over LTE is more than bearable.

I have found that often longer than expected latency is due to the fact that the PDN Gateway I connected to is located in California. This is very familiar to the Home Agent behavior in EVDO where cellular data is anchored to the internet somewhere very different from your actual physical location. 

On the Pantech UML290 I also collected RSSI and SINR alongside each test as it started. This was somewhat laborious but I collected 218 tests alongside signal characteristics to illustrate how performance varies at cell center and cell edge. Note that again these numbers are from the VZAccess Manager, and I believe that SINR is being reported incorrectly, as higher SINR should correlate with better throughput. What I've plotted is data straight from the VZAccess Manager. If you'd like, I've zipped the data from the PantechUML 290 with RSSI, SINR, downstream, upstream, and latency up in CSV format here for anyone wanting to play around with it. 

Throughput

Latency

Speedtest.net is great but I also ran a number of tests that were much longer just to prove performance isn’t limited to a rather small and quick speed test. I’m a fan of downloading huge things over both FTP and well seeded legal torrents. I have a 100Mbps synchronus box seeding Ubuntu torrents, which is generally one of the most well seeded torrents around and easily saturates 300+Mbps connections. I’ve seen speeds between 10 and 28Mbps downloading Ubuntu 10.10 over that connection on 4G LTE.

I’ve put together a short video showing me downloading some Ubuntu torrents, running speedtests, and browsing online while connected using the Pantech UML290. 

As I mentioned before, the other part of the picture is how connectivity and handovers work between EVDO and LTE. I’ve timed the Thunderbolt taking 3-5 minutes to hand up to LTE from EVDO when driving into the Phoenix market with no activity, which is a bit long. I’ve also seen very fast handovers happen in the rare circumstance that signal is lost. 

Sometimes there’s a brief pause in connectivity, but all in all behavior is pretty good and will improve in time. The screenshot above illustrates how sometimes throughput pauses as the context is transferred over; in this case I was streaming the YouTube performance test video at maximum resolution. The data cards I’ve tested are sometimes a bit finnicky about handing back up if you get stuck on EVDO; to do so they really want to go into EVDO dormancy state first instead of while a data transaction is ongoing. This is exactly how I’ve seen 1x to EVDO handovers happen in the past, and isn’t anything new.