What is 400G?
400G is the latest standard for high speed Ethernet client interfaces. Originally known as IEEE 802.3bs, 400G was officially approved in December of 2017 and is part of a broader family of related themes such as 200G, next generation 100G, and 50G Ethernet.
400G has driven the rapid development and adoption of new pluggable optical modules and switches. Sometimes referred to as 400GE or 400G Ethernet, the new standard includes Forward Error Correction (FEC) to improve error performance. Early 400G network elements have successfully completed trials and initial deployment is expected in 2019.
How Fast is 400G?
The term “exponential improvement” is perhaps a bit over used these days, but in the case of 400G, it is entirely. Gigabit Ethernet, meaning an Ethernet connection that could transmit traffic at a rate of 1 gigabit per second, was introduced in 1999.
Terabit Ethernet is used to describe a switch fabric capable of handling numerous ports at speeds of 100 Gbps and above. True Terabit is one trillion bits per second. 400 Gbps represents a 400-fold speed increase over 1Gig Ethernet performance at the turn of the 21st century. To put this in perspective, the relative change approximates the difference in foot speed between a Galapagos tortoise and a cheetah in full pursuit.
400G Ethernet is so fast that it has outpaced the ability of a conventional laser on/laser off binary modulation scheme to keep up. To compensate, PAM-4 modulation has been developed, which utilizes four amplitude levels rather than two in order to double the overall bit rate. Since the gap between signal levels is now much smaller, PAM-4 is more susceptible to noise.
400G means more than just new Ethernet ports and modulation advancements. The paradigm shift necessitates changes and adjustments throughout the networking ecosystem, providing flexibility and scalability of bandwidth deployment in new and unique ways.
Before 100G Ethernet, testing client optics was a much simpler task than it has become today. Bit error rates (BER) could be quantified for each channel, with “zero” errors over a pre-defined time period often used as the pass/fail criteria. With non-return to zero (NRZ) giving way to PAM-4 modulation and FEC, 400G testing and validation have become much more complex. The sheer bandwidth increase alone has raised the bar for testing substantially.
400G Testing Challenges
Higher speeds and the utilization of PAM-4 modulation bring amazing improvements in throughput, but can also lead to some of the inherent challenges of 400G testing. PAM-4 modulation introduces added complexity at the physical layer. Links now always have errors, so simply quantifying the errors or testing based on “zero” errors no longer suffices.
The increased speeds and use of FEC technology mean some modules with higher raw error rates will operate error-free post-FEC and others will not. A more sophisticated understanding of the error distribution and statistics is required to determine acceptable from unacceptable error patterns and determine true root causes. The FEC logic is complex and large. It needs to be tested for both logical validation as well as dynamic power performance.
400G also brings an increased integration of elements, such as the QSFP-DD and OSFP pluggable optics modules. The QSFP-DD, for example, is a marvel of complexity, with integrated lasers and drivers, high performance photodiodes and microcontrollers integrated into a very small form factor. At the same time, these additional elements necessitate strategies capable of 400G testing and validating of these components individually as well as within the context of the overall network structure.
With the additional Ethernet complexity inherent to 400G, it is important to maintain control of the costs associated with test equipment and test cycle times. 400G testing tools that stay ahead of the curve can mitigate this concern, providing ready-made test options that speed time to market for new products supporting the 400G transition while dampening associated development and manufacturing test costs that can hamper competitive pricing models.
400G Testing Tools
Scalability, flexibility, and upgradability are essential elements of an effective 400G test solution. The NEWTON OPTICS ONT 400G Module, based on the latest 400G/200G (IEEE 802.3 bs) standard, includes advanced error analysis features and a QSFP-DD test slot. Field programmability means ease of updates as the standards evolve. The ONT 400G Module also includes FEC and PAM-4 modulation support. This test solution provides an ideal platform to support design, development and validation for those at the cutting edge of the high-speed network ecosystem.
High density 100G (N-PORT) plays an important role in 400G testing by feeding the 400G “beast” with lower bit rates. The N-PORT module is a 4-port device for test and system verification with four native and independent QSFP28 and SFP28 ports. Advanced test applications and coverage aid in the development and testing of new components and modules while providing service providers and network equipment manufacturers with a valuable test tool for existing and emerging technologies.
Dense Wavelength Division Multiplexing (DWDM) has significantly increased fiber optic bandwidth. Using this method, a single fiber channel can transmit data at speeds of 400 Gb/second or more. With the chain only as strong (or as fast) as its weakest link, the development of 400G Ethernet now bridges the bandwidth gap between the core routers and the DWDM equipment. The 400G Ethernet interface allows the full capacity of the network elements to be met at the correct density, for seamless and unencumbered throughput. Modern switch application-specific standard parts (ASSPs) such as the Broadcom Tomahawk family can switch over 12 Terabits of traffic in one IC. 400G interfaces are a good match between this huge bandwidth capability and front panel bandwidth density.
Flex Ethernet (FlexE) is a client interface standard first published by the Optical Internetworking Forum (OIF) in 2016. As the name implies, the intent is to provide a standard flexible enough to facilitate connectivity between the Ethernet and physical interface (server) by introducing a “shim” through the MAC and PCS layers. This allows a variety of MAC rates to be supported, independent of the physical interface. FlexE provides a means of bonding multiple links. For example, 400G can be delivered as an individual pipe, two x 200G links or 4 x 100G links.
The ITU-T standards G.709 and G.709.1 for Optical Transport Networks (OTN) provides recommended interfaces and line rates for optical network elements connected through optical fiber links.
OTN B100G is an extension of this standard for data rates beyond 100 Gbps. ITU-T has utilized some of the key ideas from IEEE 802.3 and the Optical Internetworking Forum (OIF) to determine how the same pluggable modules are to be used on OTN interfaces. This is commonly referred to as “FlexO”, a mechanism that provides an OTN structure for 400GE and above, as well as FlexEthernet traffic.
Who Should Care About 400G?
The efficiencies to be gained through 400G implementation will ripple throughout the high-speed networking ecosystem. This includes chip and module manufacturers, test equipment and services industries, internet mega-corporations and telecom service providers who anxiously wait and rely upon these improvements for their life-blood.
Web 2.0 companies providing cloud services will leverage 400G to meet the density needs of their growing data centers. Likewise, telecommunications providers must keep up with an ultra-connected user base at their own enormous data centers. These large-scale players are now driving the move toward 400G at an accelerated pace, to keep up with server speed requirements. Optical module developers will benefit from the demand for more versatile and compact product offerings.
The industry changes brought on by 400G may be largely invisible to end users, but the advent of 400G will allow networks to keep pace with the expectations for high-speed, seamless performance. Streaming video, virtual gaming, and the Internet of Things (IoT), some using the upcoming 5G standards, are just a few of the applications that will benefit from the 400GE networking.
100G and Beyond
Compatibility between 100G and 400G modules can simplify testing and upgrades, and also makes good business sense. The first 100G Ethernet solutions were introduced in 2010, with a slow ramp-up through 2016.
The introduction of QSFP28, a quad small form factor hot-pluggable transceiver module capable of carrying 28G per lane, was instrumental in bringing 100G into the mainstream by 2017. Cost reduction and design enhancement for 100G modules has enabled backwards compatible technology like QSFP-DD, a new module type similar to standard QSFP, but with an additional row of contacts enabling an electrical interface with twice the number of lanes.
The advancement in pluggable optic components that has slowly taken root for 100G will make 400G Ethernet more effective as well. QSFP-DD is an optical transceiver which enables 6.4 Tb/s on a 1 RU host system card that can support 400GE while providing double the 100G port density of QSFP28. OSFP (optical small form factor pluggable) supports 400G power requirements and includes an integrated heatsink to meet the thermal demands.
The cutting-edge science developed with 400G in mind, including PAM4 modulation and KP4 FEC, can also be used to increase density and reduce cost for 100GE. As these technologies mature, 100G product offerings are expected to take full advantage of improvement opportunities engendered by 400G development.
More than Bandwidth
The monumental speed improvements inherent with 400G is a giant leap forward in Ethernet capability. However, increased speed and bandwidth is just the tip of the iceberg. 400G not only delivers more bandwidth, it delivers the right bandwidth at the right density.
Demands on cloud computing and telecom providers will continue to push data center servers to their physical limits. Eliminating the bottleneck that Ethernet once presented impacts the entire networking landscape in immeasurable ways. Innovations like PAM-4 have made this enhancement possible, while introducing a new set of obstacles for 400G testing and validation practices. Continuing to meet these challenges means leading the change towards a new era of network performance