High-port and low-latency optical switches for disaggregated data centers: the Hipoλaos switch architecture

The emergence of disaggregation in data center (DC) architectures as a way to increase resource utilization introduces significant challenges to the DC switching infrastructure, which has to ensure high bandwidth and low-latency communication along with high-radix connectivity. This paper examines the network requirements in disaggregated systems and reviews the credentials of state-of-the-art high-radix optical switch architectures. We also demonstrate a novel optical packet switch design, called Hipoλaos, that satisfies these requirements by combining N-port broadcast-and-select and N × N arrayed waveguide grating router-based forwarding schemes in N2-port connectivity configurations. The Hipoλaos switch provides sub-μs latency and high throughput performance by utilizing distributed control and optical feed-forward buffering in a modified Spanke architecture. Feasibility of a 256 port Hipoλaos layout with a four-packet buffering stage is experimentally demonstrated at 10 Gb/s, revealing error-free performance with a mean power penalty value of 2.19 dB. Simulation analysis is carried out for a 256 node system and eight traffic profiles; this analysis reveals a low latency value of only 605 ns, with throughput reaching 85% even when employing only two-packet buffers. Finally, the architecture’s scalability up to 1024 × 1024 designs is discussed, along with a power consumption analysis and a roadmap toward an integrated version of the switch.

[1]  Tencent Explores Datacenter Resource-Pooling Using Intel® Rack Scale Architecture (Intel® RSA) , 2015 .

[2]  J. Gripp,et al.  Demonstration of an integrated buffer for an all-optical packet router , 2009, 2009 Conference on Optical Fiber Communication - incudes post deadline papers.

[3]  K. Bergman,et al.  The Data Vortex Optical Packet Switched Interconnection Network , 2008, Journal of Lightwave Technology.

[4]  J. Luo,et al.  Scaling photonic packet switches to a large number of ports [invited] , 2012, IEEE/OSA Journal of Optical Communications and Networking.

[5]  V. A. Shchukin,et al.  4×40 Gb/s 2 pJ/bit Optical RX with 8ns Power-on and CDR-Lock Time in 14nm CMOS , 2018, 2018 Optical Fiber Communications Conference and Exposition (OFC).

[6]  Kostas Katrinis,et al.  dReDBox: Materializing a full-stack rack-scale system prototype of a next-generation disaggregated datacenter , 2018, 2018 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[7]  Wei Li,et al.  Low-loss and low-crosstalk 8 × 8 silicon nanowire AWG routers fabricated with CMOS technology. , 2014, Optics express.

[8]  G. Zervas,et al.  Optically Disaggregated Data Centres with Minimal Remote Memory Latency: Technologies, Architectures, and Resource Allocation , 2017 .

[9]  Lei Qiao,et al.  32 × 32 silicon electro-optic switch with built-in monitors and balanced-status units , 2017, Scientific Reports.

[10]  S. Pinna,et al.  Broadband Operation of High-Speed All-Optical Gated Wavelength Shifter , 2012, IEEE Photonics Technology Letters.

[11]  S. J. B. Yoo,et al.  Scalable and Distributed Contention Resolution in AWGR-Based Data Center Switches Using RSOA-Based Optical Mutual Exclusion , 2013, IEEE Journal of Selected Topics in Quantum Electronics.

[12]  William J. Dally,et al.  Principles and Practices of Interconnection Networks , 2004 .

[13]  N. Calabretta FPGA-Based Label Processor for Low Latency and Large Port Count Optical Packet Switches , 2012, Journal of Lightwave Technology.

[14]  H. J. S. Dorren,et al.  Scaling low-latency optical packet switches to a thousand ports , 2012, IEEE/OSA Journal of Optical Communications and Networking.

[15]  N. Pleros,et al.  On-Chip SOI Delay Line Bank for Optical Buffers and Time Slot Interchangers , 2018, IEEE Photonics Technology Letters.

[16]  Kai Wang,et al.  Demonstration of fully enabled data center subsystem with embedded optical interconnect , 2014, Photonics West - Optoelectronic Materials and Devices.

[17]  Nikos Pleros,et al.  Silicon photonic 8 × 8 cyclic Arrayed Waveguide Grating Router for O-band on-chip communication. , 2018, Optics express.

[18]  C. Dragone,et al.  Demonstration of a 15*15 arrayed waveguide multiplexer on InP , 1992, IEEE Photonics Technology Letters.

[19]  Christoph Hagleitner,et al.  Network-attached FPGAs for data center applications , 2016, 2016 International Conference on Field-Programmable Technology (FPT).

[20]  Dawei Wang,et al.  High-speed optical switch fabrics with large port count. , 2009, Optics express.

[21]  Randy H. Katz,et al.  Heterogeneity and dynamicity of clouds at scale: Google trace analysis , 2012, SoCC '12.

[22]  H. J. S. Dorren,et al.  Scaling photonic packet switches to a large number of ports , 2011, 2011 ICO International Conference on Information Photonics.

[23]  Scott Shenker,et al.  Network Requirements for Resource Disaggregation , 2016, OSDI.

[24]  Christoph Hagleitner,et al.  An FPGA Platform for Hyperscalers , 2017, 2017 IEEE 25th Annual Symposium on High-Performance Interconnects (HOTI).

[25]  Kyungsook Y. Lee,et al.  A New Benes Network Control Algorithm , 1987, IEEE Trans. Computers.

[26]  Reza Nejabati,et al.  The Benefits of a Disaggregated Data Centre: A Resource Allocation Approach , 2016, 2016 IEEE Global Communications Conference (GLOBECOM).

[27]  T. Aalto,et al.  Multicast-Enabling Optical Switch Design Employing Si Buffering and Routing Elements , 2018, IEEE Photonics Technology Letters.

[28]  Salah Ibrahim,et al.  Burst-mode optical label processor with ultralow power consumption. , 2016, Optics express.

[29]  Nikos Pleros,et al.  High-port low-latency optical switch architecture with optical feed-forward buffering for 256-node disaggregated data centers. , 2018, Optics express.

[30]  Ken-ichi Sato,et al.  Large-Scale and Simple-Configuration Optical Switch Enabled by Asymmetric-Port-Count Subswitches , 2016, IEEE Photonics Journal.

[31]  P. Maniotis,et al.  Optically-Enabled Bloom Filter Label Forwarding Using a Silicon Photonic Switching Matrix , 2017, Journal of Lightwave Technology.

[32]  T. Alexoudi,et al.  Thick-SOI Echelle grating for any-to-any wavelength routing interconnection in multi-socket computing environments , 2017, OPTO.

[33]  R. Luijten,et al.  An Optical Packet-Switched Interconnect for Supercomputer Applications ∗ , 2004 .

[34]  Timo Aalto,et al.  Low-loss spiral waveguides with ultra-small footprint on a micron scale SOI platform , 2014, Photonics West - Optoelectronic Materials and Devices.

[35]  A. Wonfor,et al.  First demonstration of automated control and assessment of a dynamically reconfigured monolithic 8 × 8 wavelength-and-space switch [invited] , 2015, IEEE/OSA Journal of Optical Communications and Networking.

[36]  Georgios Zervas,et al.  Optically disaggregated data centers with minimal remote memory latency: Technologies, architectures, and resource allocation [Invited] , 2018, IEEE/OSA Journal of Optical Communications and Networking.

[37]  A. Rohit,et al.  Monolithically Integrated 8 × 8 Space and Wavelength Selective Cross-Connect , 2014, Journal of Lightwave Technology.

[38]  David Valentine,et al.  For the machine , 2017, Interpretation of Commercial Contracts in European Private Law.

[39]  Roberto Proietti,et al.  Scalable Optical Interconnect Architecture Using AWGR-Based TONAK LION Switch With Limited Number of Wavelengths , 2013, Journal of Lightwave Technology.

[40]  N. Pleros,et al.  40 Gb/s NRZ Wavelength Conversion Using a Differentially-Biased SOA-MZI: Theory and Experiment , 2011, Journal of Lightwave Technology.

[41]  Franck Cappello,et al.  Characterizing Cloud Applications on a Google Data Center , 2013, 2013 42nd International Conference on Parallel Processing.

[42]  J E Simsarian,et al.  Photonic terabit routers: The IRIS project , 2010, 2010 Conference on Optical Fiber Communication (OFC/NFOEC), collocated National Fiber Optic Engineers Conference.

[43]  R. Pitwon,et al.  Converged photonic data storage and switch platform for exascale disaggregated data centers , 2017, OPTO.

[44]  Yusuf Leblebici,et al.  A 56Gb/s burst-mode NRZ optical receiver with 6.8ns power-on and CDR-Lock time for adaptive optical links in 14nm FinFET CMOS , 2018, 2018 IEEE International Solid - State Circuits Conference - (ISSCC).

[45]  Nicola Calabretta,et al.  Towards Petabit/s All-Optical Flat Data Center Networks Based on WDM Optical Cross-Connect Switches with Flow Control , 2016, Journal of Lightwave Technology.

[46]  Hiroyuki Tsuda,et al.  6.4-THz-spacing, 10-channel cyclic arrayed waveguide grating for T- and O-band coarse WDM , 2016, IEICE Electron. Express.

[47]  K. Pagiamtzis,et al.  Content-addressable memory (CAM) circuits and architectures: a tutorial and survey , 2006, IEEE Journal of Solid-State Circuits.

[48]  N. Pleros,et al.  Dual SOA-MZI Wavelength Converters Based on III-V Hybrid Integration on a $\mu{\rm m}$-Scale Si Platform , 2014, IEEE Photonics Technology Letters.

[49]  L. Coldren,et al.  An 8$\,\times\,$ 8 InP Monolithic Tunable Optical Router (MOTOR) Packet Forwarding Chip , 2010, Journal of Lightwave Technology.

[50]  Yojiro Mori,et al.  Large-Scale Optical Switch Utilizing Multistage Cyclic Arrayed-Waveguide Gratings for Intra-Datacenter Interconnection , 2017, IEEE Photonics Journal.

[51]  H. Jonathan Chao,et al.  A Petabit Bufferless Optical Switch for Data Center Networks , 2013 .

[52]  Rodney S. Tucker,et al.  Wavelength routing-based photonic packet buffers and their applications in photonic packet switching systems , 1998 .

[53]  George Porter,et al.  Is memory disaggregation feasible? A case study with Spark SQL , 2016, 2016 ACM/IEEE Symposium on Architectures for Networking and Communications Systems (ANCS).

[54]  K A Williams,et al.  Monolithic active-passive 16 × 16 optoelectronic switch. , 2012, Optics letters.

[55]  Qixiang Cheng,et al.  Hybrid MZI-SOA InGaAs/InP Photonic Integrated Switches , 2018, IEEE Journal of Selected Topics in Quantum Electronics.

[56]  Jaafar M. H. Elmirghani,et al.  Future Energy Efficient Data Centers With Disaggregated Servers , 2017, Journal of Lightwave Technology.

[57]  Dazeng Feng,et al.  Compact single-chip VMUX/DEMUX on the silicon-on-insulator platform. , 2011, Optics express.

[58]  Larry A. Coldren,et al.  An 8 8 InP Monolithic Tunable Optical Router ( MOTOR ) Packet Forwarding Chip , 2010 .

[59]  Kang G. Shin,et al.  Efficient Memory Disaggregation with Infiniswap , 2017, NSDI.

[60]  Kostas Katrinis,et al.  Rack-scale disaggregated cloud data centers: The dReDBox project vision , 2016, 2016 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[61]  Tzi-cker Chiueh,et al.  Marlin: A memory-based rack area network , 2014, 2014 ACM/IEEE Symposium on Architectures for Networking and Communications Systems (ANCS).

[62]  C. Vagionas,et al.  WDM-Enabled Optical RAM at 5 Gb/s Using a Monolithic InP Flip-Flop Chip , 2016, IEEE Photonics Journal.