Copper is faster than fibre!

Latest News AND EVENTS

Stay up to date and find all the latest news and latest events from Metamako right here.

Copper is faster than fibre!

Posted on August 19, 2016 by David Snowdon 19 August 2016

 

While fibre can move more data over longer distances, we've used MetaWatch to show that direct attach copper cables have the edge over both SMF and MMF fibre (which were essentially equivalent to each other). 

Our testing setup looks a little bit like this: 

Cable Test Setup

We connected two machines to a MetaMux 32 device running MetaWatch using 10G Ethernet. We then created a loopback through two ports on the device's front panel, first using direct-attach copper cables, and then using fibre cables. For each cable that we tested, we passed around 1,000,000 "ping" packets through the device, pinging backward and forward between the two test servers.  By measuring the time that each packet passed through the MetaWatch device, and then correlating those measurements and differencing them, we were able to calculate the time taken for each packet to pass through the device.  Each result is pretty darn accurate: +/- 2 ns, but we averaged the million results we had to get even more accurate numbers.

We used mellanox direct-attach copper cables, and Finisar SFPs for the purpose. You can see the part numbers for these in the table below. We used generic MMF and SMF fibres -- it's possible that some will be faster than others (including hollow-core fibres, which should be *much* faster). 

Note also that these are passive direct-attach cables, not Cat 5 1GBase-T or 10GBase-T cables with the corresponding SFPs -- we've measured these types of SFPs to have a very high latency of round 300 ns round-trip, which dwarfs any potential savings from the propagation delay of the copper. 

Results

Here are the raw results:

Cable Type SFPs (1 and 2) Cable Distance (m) Latency (ns) Description
DAC N/A MC3309130-001 1.008

7.8345

1.008 m direct-attach copper
DAC N/A MC3309130-002 2.02 12.3344 2.02 m direct-attach copper
DAC N/A MC3309130-003 2.998 16.7289 2.998 m direct-attach copper
DAC N/A MC3309124-007 6.992 34.9989 6.992 m direct-attach copper
MMF FTLX8571D3BCV 1 m MMF OM1 fibre 1.094

9.4755

1.094 m MMF fibre
MMF FTLX8571D3BCV 2 m MMF OM1 fibre 2.190 14.9769 2.190 m MMF fibre
MMF FTLX8571D3BCV 3 m MMF OM1 fibre 3.286 20.4918 3.286 m MMF fibre
MMF FTLX8571D3BCV 4 m MMF OM1 fibre 4.381 25.7473 4.381 m MMF fibre
MMF FTLX8571D3BCV 5 m MMF OM1 fibre 5.477 31.2473 5.477 m MMF fibre
MMF FTLX8571D3BCV 6 m MMF OM1 fibre 6.572 36.7727 6.572 m MMF fibre
MMF FTLX8571D3BCV 7 m MMF OM1 fibre 7.667 42.2407 7.667 m MMF fibre
SMF FTLX1471D3BCV 2 m SMF Fibre 2.100 14.4991 2.1 m SMF fibre
SMF FTLX1471D3BCV 4m SMF Fibre 4.200 24.7130 4.2 m SMF fibre
SMF FTLX1471D3BCV 6m SMF Fibre 6.300 35.2940 6.3 m SMF fibre

For each test we took 1,000,000 samples. To check that our test setup was accurate, we looked at the distribution of the timestamps captured by MetaWatch and see that it is well formed, with a spread of +/- 4 ns. 

 

Metawatch Delta Accuracy1516171819202122015304560Measured Latency (ns)Percentage (%)
Measured Latency (ns) Sample %
15 0
16 0
17 0.233
18 44.938
19 47.041
20 5.048
21 2.74
22 0
 

So, if we plot those results and fit a trendline, we get slopes and offsets:

 

Cable LatenciesSMF Latency (ns)y = 4.951 * x + 4.04r^2 = 1MMF Latency (ns)y = 4.977 * x + 4.048r^2 = 1Twinax Latency (ns)y = 4.595 * x + 3.113r^2 = 1012345678010203040Cable Length (m)Latency (ns)
Cable Length (m) SMF Latency (ns) MMF Latency (ns) Twinax Latency (ns)
1.008     7.835
1.094   9.47  
2.002     12.334
2.1 14.499    
2.19   14.98  
2.998     16.729
3.286   20.49  
4.2 24.713    
4.381   25.75  
5.477   31.25  
6.3 35.294    
6.572   36.77  
6.992     35.294
7.667   42.24  
 

Analysis

The trendlines above show that single-mode and multi-mode fibre have near identical latencies of 4.96 ns per metre. This is close to the oft-quoted 5 ns per metre for fibre. The latency for the twinax copper cables shown is 4.60 ns per metre -- faster by about 400 ps per metre. What's also interesting is the zero-offset. When we extrapolate down to a hypothetical 0 m cable, the copper cables have a lower fixed offset compared with the fibre. We consider the copper direct-attach cable to have zero latnecy in the SFPs, since the twinax cable is soldered directly to the pins on the SFP module itself. The difference between the offsets is about 1 ns (copper shows an offset of 3 ns, which we believe is a mis-calibration in the MetaWatch device). This is due to the latency in the SFPs themselves. Note that the latency is nearly identical for both the FTLX8571 and FTLX1471 SFPs: around 500 ps. 

It's important to use the right medium for the job. Direct-attach copper cables have a maximum reach of around 7-10 m, depending on the devices being used, compared with a 300 m range for MMF fibre used above, and 10 km range for the SMF -- three orders of magnitude difference. This really only applies within a small footprint. 

Conclusion

Direct attach copper cables have the edge over both SMF and MMF fibre when it comes to latency, and this is even more prominent for short cable runs because of the latency introduced by the SFP modules which drive the fibre. 

If you really care about latency, (twinax) copper is faster than fibre.