CDN performance in China
We recently had a project that required CDN-hosted files and, in order to prove a point, I ran a series of traceroute/ping tests on popular CDNs. Seeing as I’ve already formatted the data and made the graphs I thought I’d post it here in case it’s of use to anyone.
The tests were run with mtr for 10 minutes each over the course of an evening on a typical China Unicom home ADSL connection in Xi’an.
The common CDNs don’t have edge nodes in mainland China, and suffer as a result. You can see an obvious “step” in the graphs when the route goes international.
Microsoft Ajax CDN
I get routed to a node in Hong Kong so the response is good but intermittent delays and a little packet loss and the average suffers.
HOST: deck Loss% Snt Last Avg Best Wrst StDev
1.|-- 111.194.0.1 0.0% 60 6.5 18.4 5.7 155.7 27.6
2.|-- 61.148.149.241 0.0% 60 10.6 9.8 3.2 70.4 11.0
3.|-- 61.148.155.5 1.7% 60 11.3 15.0 8.1 110.8 18.3
4.|-- 123.126.0.85 0.0% 60 11.4 20.1 8.5 177.8 26.9
5.|-- 219.158.101.230 0.0% 60 43.5 49.1 43.1 123.7 14.0
6.|-- 219.158.3.62 0.0% 60 112.2 115.4 48.6 273.4 32.5
7.|-- 219.158.96.198 1.7% 60 128.1 117.1 49.2 241.0 30.3
8.|-- 219.158.102.122 0.0% 60 125.6 122.1 53.0 303.2 35.3
9.|-- 219.158.39.26 3.3% 60 121.1 125.0 52.2 291.2 42.8
10.|-- 191.234.80.198 6.7% 60 117.5 118.5 52.4 214.1 23.1
11.|-- xe-4-0-0-0.hkn-96cbe-1a.n 0.0% 60 140.2 123.0 49.4 297.4 40.8
12.|-- ??? 100.0 60 0.0 0.0 0.0 0.0 0.0
13.|-- cds49.hkn9.msecn.net 3.3% 60 112.4 115.9 48.4 307.4 33.5
Google Hosted Libraries
Good latency for a US route (Mountain View), and not a single dropped packet which is rare for Chinese ADSL.
HOST: deck Loss% Snt Last Avg Best Wrst StDev
1.|-- 111.194.0.1 0.0% 10 6.6 7.6 5.6 12.4 2.5
2.|-- 61.148.149.237 0.0% 10 30.8 22.3 3.1 61.0 19.4
3.|-- 61.148.6.117 0.0% 10 11.3 12.9 7.0 25.5 6.1
4.|-- 123.126.0.253 0.0% 10 11.0 10.3 6.9 13.1 1.8
5.|-- 219.158.101.122 0.0% 10 49.2 50.0 46.9 58.6 3.1
6.|-- 219.158.11.170 0.0% 10 48.3 47.2 46.1 49.4 0.9
7.|-- 219.158.97.6 0.0% 10 48.6 48.9 46.9 53.7 1.9
8.|-- 219.158.3.238 0.0% 10 48.2 50.6 47.9 57.7 2.7
9.|-- 72.14.215.130 0.0% 10 68.3 61.7 50.9 113.6 19.0
10.|-- 209.85.248.62 0.0% 10 70.9 67.5 46.8 80.5 14.1
11.|-- 209.85.253.71 0.0% 10 49.9 55.8 45.8 122.6 23.7
12.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
13.|-- hg-in-f95.1e100.net 0.0% 10 48.0 49.4 47.4 54.5 2.4
Amazon CloudFront
Amazon route me via Japan but the latency is higher that I was expecting.
HOST: deck Loss% Snt Last Avg Best Wrst StDev
1.|-- 111.194.0.1 0.0% 10 44.5 35.3 7.2 62.1 20.9
2.|-- 61.148.149.241 0.0% 10 5.9 12.2 5.6 57.3 15.9
3.|-- 124.65.59.1 0.0% 10 7.6 8.6 6.9 11.8 1.3
4.|-- 123.126.0.97 0.0% 10 8.8 11.6 7.7 20.6 3.6
5.|-- 219.158.100.166 0.0% 10 59.8 58.1 55.8 62.1 1.9
6.|-- 219.158.11.50 0.0% 10 153.3 128.8 118.4 155.6 13.6
7.|-- 219.158.97.6 0.0% 10 125.1 125.2 121.7 135.9 4.1
8.|-- 219.158.33.22 0.0% 10 185.5 184.0 181.3 186.8 2.0
9.|-- otejbb205.int-gw.kddi.ne. 0.0% 10 179.6 181.4 179.2 189.7 3.0
10.|-- obpjbb205.int-gw.kddi.ne. 0.0% 10 238.8 213.7 206.8 238.8 10.2
11.|-- jc-osa302.kddnet.ad.jp 0.0% 10 218.8 211.3 203.5 237.4 10.1
12.|-- 118.155.203.62 10.0% 10 199.0 200.2 191.0 237.7 14.9
13.|-- 27.0.0.191 10.0% 10 313.4 310.2 300.2 319.8 6.8
14.|-- 27.0.0.150 10.0% 10 316.3 312.7 299.8 328.3 9.4
15.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
16.|-- server-54-230-114-37.nrt5 10.0% 10 317.2 312.7 305.7 320.1 5.8
MaxCDN
I included this as it’s the new jQuery official CDN, so should see widespread use. Not bad overall, but still serving out of the US.
HOST: deck Loss% Snt Last Avg Best Wrst StDev
1.|-- 111.194.0.1 0.0% 60 5.7 12.9 5.7 67.9 12.3
2.|-- 61.148.149.237 0.0% 60 6.3 11.4 5.4 72.2 11.9
3.|-- 123.126.8.161 0.0% 60 7.3 12.1 6.2 70.6 11.3
4.|-- 123.126.0.185 0.0% 60 9.5 15.0 5.9 143.1 22.3
5.|-- 219.158.97.190 0.0% 60 9.7 16.0 7.5 104.8 15.5
6.|-- 219.158.97.178 0.0% 60 13.0 17.8 7.4 102.5 14.4
7.|-- 219.158.102.154 1.7% 60 190.1 209.4 179.9 336.6 36.5
8.|-- xe-6-0-3.ar1.lax2.us.nlay 3.3% 60 253.1 271.0 203.5 441.0 45.2
9.|-- 108.161.181.2 23.3% 60 190.5 207.6 180.4 343.1 30.4
10.|-- 108.161.188.209 1.7% 60 256.5 271.5 197.1 446.2 45.2
CloudFlare
Another popular choice, but their performance in China is just dire. Mediocre at best and awful at worst, plus more packet loss than anyone else I tested.
HOST: deck Loss% Snt Last Avg Best Wrst StDev
1.|-- 111.194.0.1 0.0% 60 7.2 39.5 5.5 938.3 120.8
2.|-- 61.148.149.241 0.0% 60 6.1 26.4 3.1 871.4 111.8
3.|-- bt-228-181.bta.net.cn 0.0% 60 11.7 28.4 5.9 808.0 104.0
4.|-- 124.65.194.21 0.0% 60 9.8 31.6 6.5 741.1 96.7
5.|-- 219.158.100.170 2.1% 60 55.0 74.6 53.2 720.6 87.9
6.|-- 219.158.11.22 0.0% 60 118.6 154.0 61.2 1719. 221.8
7.|-- 219.158.96.214 0.0% 60 124.5 159.7 58.0 1652. 211.8
8.|-- 219.158.102.158 1.8% 60 346.0 379.8 285.7 1807. 199.9
9.|-- 144.223.54.221 6.7% 60 333.4 364.9 277.5 1753. 199.4
10.|-- 144.232.0.194 6.7% 60 337.8 367.6 279.3 1667. 186.7
11.|-- ae-0.gtt.lsanca03.us.bb.g 16.7% 60 414.3 418.4 288.4 1661. 186.0
12.|-- ae1-80g.cr1.lax1.us.nlaye 5.0% 60 392.9 411.9 306.4 1592. 165.0
13.|-- ae2-50g.ar1.lax1.us.nlaye 5.0% 60 401.0 418.3 294.4 1530. 155.2
14.|-- as13335.xe-9-0-7.ar1.lax1 6.7% 60 410.6 413.5 300.6 1463. 146.4
15.|-- 198.41.212.162 11.7% 60 398.2 412.9 307.2 1394. 141.7
Akamai
Not a bad showing for Akamai out of Hong Kong, but MS do better.
HOST: deck Loss% Snt Last Avg Best Wrst StDev
1.|-- 111.194.0.1 0.0% 60 19.0 14.6 5.1 104.0 18.2
2.|-- 61.148.149.237 0.0% 60 34.7 13.6 5.2 97.7 19.9
3.|-- 123.126.8.161 0.0% 60 7.3 14.4 6.3 101.3 16.1
4.|-- 123.126.0.169 0.0% 60 13.0 16.4 5.9 104.5 20.0
5.|-- 219.158.4.142 0.0% 60 8.1 14.2 7.1 109.2 17.4
6.|-- 219.158.98.2 0.0% 60 110.5 64.6 42.7 159.7 26.1
7.|-- 219.158.32.238 10.0% 60 80.7 95.6 80.4 174.5 23.8
8.|-- as-1.r20.tkokhk01.hk.bb.g 91.7% 60 317.6 235.0 139.5 317.6 86.8
9.|-- ae-1.r03.chwahk02.hk.bb.g 0.0% 60 120.0 140.1 118.7 299.7 45.6
10.|-- 203.131.246.250 0.0% 60 119.4 133.2 118.3 337.3 34.6
11.|-- a23-2-16-40.deploy.static 1.7% 60 129.5 143.6 119.7 296.8 44.9
Conclusion
These are obviously flawed and essentially anecdotal results, but they are in line with my experiences over the last two years on domestic ADSL. If you are serious about serving your content in China don’t assume your CDN will “just work” - it almost certainly won’t.
Update: January 2014
The selection of decent China-targetting CDNs is much improved, as are the
reporting tools. If you’re shopping around make sure you check out the Cedexis
report for China. To demonstrate I ran mtr with the same
settings from my original notes on an image on the ChinaCache CDN:
HOST: ophelia Loss% Snt Last Avg Best Wrst StDev
1.|-- 111.194.0.1 0.0% 60 6.6 29.4 5.3 119.5 28.8
2.|-- 61.148.149.237 0.0% 60 6.2 13.0 5.4 98.1 16.3
3.|-- 123.126.8.201 0.0% 60 10.4 17.0 7.1 127.1 18.4
4.|-- ??? 100.0 60 0.0 0.0 0.0 0.0 0.0
5.|-- 219.158.96.106 0.0% 60 15.5 20.1 10.9 86.3 14.1
6.|-- www.he.cninfo.net 0.0% 60 14.3 17.8 10.7 103.4 17.4
7.|-- 61.182.174.178 0.0% 60 18.6 27.9 18.3 185.2 25.2
8.|-- 221.192.129.185 0.0% 60 20.5 27.8 19.7 103.3 17.8
9.|-- 211.90.1.74 0.0% 60 20.1 27.2 18.7 86.7 13.7
10.|-- 221.192.149.107 0.0% 60 18.9 25.3 17.7 106.6 16.7
I also took the opportunity to redo my graphs as I recently did some work generating SVG graphs from mtr reports. Read those notes if you would like to script gnuplot to process mtr reports as I’ve done here.