Issue 1, December 2008
FOCUS OPINION
COPPER AND FIBER DECISIONS FOR
A 10GB/S DATA CENTER An end to end approach is needed for fiber and copper to co-exist says Carrie Higbie
In the “old days” a few short years ago, cabling, both copper and fiber, was rather benign to data center design. Likewise, power was a necessary evil that was considered to be a cost of doing business. With green initiatives swaying decisions, power with resultant cooling needs, and upgraded cabling infrastructures are bringing facilities back into the discussions for networking gear. With varied capabilities of copper and fiber to support the higher bandwidth demands in the data center, an end to end approach to these decisions is the most cost effective approach for data center planning.
There are several factors that are driving data centers to revise, remediate, relocate or consolidate current equipment. Power and cooling are the two largest factors. Many of the older air handling units operated at roughly 80% efficiency. The newer units operate at between 95-98% efficiency depending on the manufacturer. In some instances, it is more cost effective for companies to write off unrealized depreciation in order to receive the benefits of the newer efficiencies. Simple steps like removing abandoned cable, which creates an air dam, blanking panels, and blocking cable penetration openings are beneficial.
To meet these increasing demands for bandwidth in the data center, 10G applications are growing. Market researcher The Dell’Oro Group projects 10-Gbit Ethernet (10GbE) over copper will grow to 42% of an 8.8 million unit total 10GbE by 2010. These applications operate over copper, fiber, and twinax over varied distances. In the data center, typical distances are shorter than they are in the average horizontal application and certainly shorter than backbone or “vertical” applications.
PROTOCOL OPTIONS
There are several options available that will run at 10 Gb/s. The first is Infiniband. The single biggest advantage of Infinband is that it has far lower latency than TCP/IP based applications as there is much less overhead in this protocol. This protocol also addresses physical layer requirements and there are a variety of interfaces recognized. Infiniband is gaining popularity in clustered and grid computing environments not only for storage, but as a low latency, high performance LAN interconnect.
The next option is 10GBASE-T. The challenge of operating 10Gb/s over copper is due to the least common denominator written into the 10GBASE-T standard; which is category 6 cabling.
However, for higher performing cabling systems such as category 6A (UTP and F/UTP) and category 7/class F systems, transmissions become much easier and, in fact, support the short reach mode (under 30m) low power option is also part of the standard. Fully shielded systems such as category 7/class F and category 7A/class FA may provide even lower power options as there would be less dependence on DSP’s should those applications be developed.
The cost goal in the standards is 10x the performance at 3x the cost. This means that copper electronics, when commercially affordable, not just commercially available, will provide multiple speeds for a very attractive price point. As maintenance is based on original equipment purchase price, not only will day one costs be attractive, but day two costs will also be lower.
MATURE APPLICATIONS
Fiber based applications are the most mature as 10Gb/s was first designed for backbone and vertical applications. That is not to say that they are not used in the horizontal, quite the contrary, but they are far more common in backbone applications. The most common 10Gb/s Ethernet fiber transmission in use in the data center is 10GBASE-SR. This will support varied distances based on the type of fiber installed. For the old FDDI grade fiber, distance is limited to 28 meters. For OM3 grade fiber (500/2000 50 micron) laser optimized, the distance jumps to 300m.
In order to increase the distances on the old FDDI –OM1 grades of fiber, two other fiber standards have been published. 10GBASE-LX4 and 10GBASE-LRM increase allowable distances to 300, and 220m respectively. However it is important to note that some of the electronics are more expensive than the SR counterparts, some require the addition of expensive mode conditioning patch cords, and in most cases, it is less expensive to upgrade your fiber plant to OM3 grade fiber, which is passive and
would not carry the additional financial burden of elevated maintenance day two costs.
POWER CONSUMPTION Power is currently being touted as a problem for 10GBASE-T with current chips operating at 10-17 Watts per port. It is highly irresponsible to base arguments on current power utilization, as second generation chips will have much lower power demands. The same was seen in gigabit Ethernet which from first iteration chips to current technologies saw a decrease in power from 6 Watts per port to 0.4 Watts per port today. Solarflare recently announced a shift from 17 Watts per port to second generation 6 Watts per port and Aquantia has introduced 6.5 Watts per port chips.
The IEEE is working on Energy Efficient Ethernet (802.3az) technology that will allow links to auto-negotiate down to lower speeds or go to “sleep” during periods of inactivity which will further reduce power consumption.
Experts agree that copper will remain a viable solution for a very long time and most data centers cabling will in reality be a mixture of copper and fiber for Ethernet communications.
Carrie Higbie is Siemon’s Global Director of data center solutions and services
www.datacenterdynamics.com 67
Page 1 |
Page 2 |
Page 3 |
Page 4 |
Page 5 |
Page 6 |
Page 7 |
Page 8 |
Page 9 |
Page 10 |
Page 11 |
Page 12 |
Page 13 |
Page 14 |
Page 15 |
Page 16 |
Page 17 |
Page 18 |
Page 19 |
Page 20 |
Page 21 |
Page 22 |
Page 23 |
Page 24 |
Page 25 |
Page 26 |
Page 27 |
Page 28 |
Page 29 |
Page 30 |
Page 31 |
Page 32 |
Page 33 |
Page 34 |
Page 35 |
Page 36 |
Page 37 |
Page 38 |
Page 39 |
Page 40 |
Page 41 |
Page 42 |
Page 43 |
Page 44 |
Page 45 |
Page 46 |
Page 47 |
Page 48 |
Page 49 |
Page 50 |
Page 51 |
Page 52 |
Page 53 |
Page 54 |
Page 55 |
Page 56 |
Page 57 |
Page 58 |
Page 59 |
Page 60 |
Page 61 |
Page 62 |
Page 63 |
Page 64 |
Page 65 |
Page 66 |
Page 67 |
Page 68 |
Page 69 |
Page 70 |
Page 71 |
Page 72