So, tonight, I decided to benchmark my mobile phone connections.
From all the media and advertising, it would appear that both Airtel Malawi and TNM Malawi claim to have the fastest mobile Internet in Malawi.
It’s obvious that at least one of these claims must be false, and I just wanted to find out the truth.
To conduct this benchmark, I used two smartphones. One Sony Xperia Z (C6603 with: a modem capable of HSPA+ (3G or “H+”) with a speed of 42.2/5.76 Mbps or LTE (4G) Cat3 100/50 Mbps) and one Sony Xperia Z5 Compact (E5823 with: a modem capable of HSPA+ (3G or “H+”) 42.2/5.76 Mbps and LTE (4G) Cat6 300/50 Mbps).
Before I continue, I should state a few important things:
- Both of these devices have the same 3G modem, so in as far as 3G internet is concerned, it’s pretty much the same device.
- Technically speaking, these tests were conducted on a 3.9G network as both Airtel and TNM offer HSPA+ speeds. TNM just launched a 4G (LTE) network, but despite multiple attempts and buying a new SIM card, I was unable to get it working on either one of my devices. My devices worked with Access Communications’ 4G (LTE) network, so someone should fix something at TNM.
- If you want to know the difference between 3G, 3.5G, 3.75G, 3.9G and 4G, just click here.
Ok, so testing conditions:
I figured the best time to run these benchmarks would be late in the evening on a weekday. I ran both tests at approximately 21:30hrs on Friday, 1st July 2016 (CAT).
At the time of benchmarking, I made sure to have good signal strength as I’m fortunate enough to live very close to a cell tower (-70dbm / 12asu on average. This is equivalent to “5 bars of network”).
Ookla Vanilla SpeedTest
I ran three tests on each provider using the Ookla Speedtest App to get the first result. It is worth noting that both Airtel and TNM have Ookla Speedtest servers in Lilongwe so I believe it was a pretty fair match. I tested each provider on their own Ookla server. Below, the average results:
Practically speaking, Airtel and TNM were about the same in terms of download speed, but Airtel’s upload speed is absolutely terrible. It’s completely unacceptable that we should be getting less than 1mpbs of upload speed on a SpeedTest to a server that they’re running.
Seriously, do better guys.
TNM’s download speed was better by about 2Mbps so that’s another win for them, and their ping response time was much lower than Airtel’s.
TNM won on all 3 metrics in this test.
Ookla’s fancy SpeedTest app is all well and good, but what does it mean for real world usage? I figured the “laboratory style” speed test may not represent the reality of what it’s like to actually use either one of these providers, so I ran some real world tests as well.
YouTube SpeedTest
A quick and easy way to test internet speed is to measure how long it takes to load a YouTube video. I didn’t feel like spending the data to load a whole video, but I measured how long it took to start streaming a 720p video (this one, in case you’re curious) and that’s when things got a little interesting.
Airtel was by far the better performer here, starting a butter-smooth HD video stream after only 4.02 seconds of waiting time. TNM on the other hand, didn’t even start the stream at all. Maybe I bought the wrong bundle or something? (I bought a 100MB Smart Bundle I think — The lady at the shop said it was the general purpose one, but couldn’t get video to load).
TNM may need to simplify their data/bundle products, because I have a friggin’ BSc in Information Systems Tech (with a concentration in telecommunications and networks, which I passed with honours by the way) and I still don’t understand what’s going on with these bundles, man.
Airtel won on this metric.
File Download SpeedTest
Perplexed at my mixed results, I decided to conduct one last test. A good ol’ file download speed test. Again, wanting to save on precious data, I used Think Broadband’s handy 5MB ZIP download (again, available here, for the curious).
To iron out 3rd party traffic interference, I had only Google Chrome version 51.0.2704.103 (64-bit) running on a fully-updated 2015 Apple MacBook Pro (OSX El Capitan, 2.9Ghz Core i5, 8GB 1867MHz LPDDR3 RAM, 500GB Solid State Flash Drive). I further used Little Snitch to disable all Internet traffic except from Google Chrome and this is what I found:
Again, Airtel just blowing TNM out of the water here with an incredible 7.5 seconds vs TNM’s 28.9 seconds to download the same 5MB file from the same server. Airtel’s speed effectively translates to 682.66 kB/s while TNM’s effective speed was 177.16 kB/s (Look who can do math :).
Airtel Malawi is the clear winner here as it was in the previous test.
At this point in time, I got too lazy to run more tests and figured that Airtel’s Internet is currently better than TNM’s, in as far as 3G is concerned. Though TNM’s Internet is pretty great too. Can’t really complain about either, though, it would have been lovely to test TNM’s 4G, but it just doesn’t seem to agree with either one of my phones.
Anyway, these results should be taken with a HUGE GRAIN OF SALT because there are many, many, many, many, many factors that can affect the speed of the connection between you and your favourite Internet service at any given time, especially on mobile. I could run this test again tomorrow and get wildly different results because of things like network congestion, cached content, noise and attenuation, packet loss (nothing to do with crisps unfortunately) and a whole other list of fancy words I learned in network school. The point is, these results aren’t definitive. But if someone has a few hundred thousand Kwacha to spare, I could easily take a day or two to run laboratory standard network tests on both providers to answer the age-old question that plagues thousands of Malawians everyday:
Who’s got the fastest mobile Internet?
(*Patiently awaits defamation lawsuit*)