Maximizing the impact of emerging photonic switches at the system level

Photonic switches promise to have a large impact on future HPC and datacenter networks. However, even if we assume ideal zero-energy photonic switches, we cannot achieve significant system-wide benefits if we do not change the rest of the system architecture. This motivates co-design between emerging photonic switches and the rest of the system in order to adapt the system network to best make use of unique features of photonic switches, as well as tailor photonic switches to better support system-wide trends such as resource disaggregation. In this paper, we discuss the architectural impact of several properties of photonic switches. For each, we provide an overview of what system-level capabilities they enable, how they can be adapted to support ongoing trends, and what other synergistic advancements would produce a better system-wide improvement. In this way, we illustrate the potential benefit of closer collaboration between the photonic and architecture communities.

[1]  Joel Mambretti,et al.  Optical Switching Middleware for the OptlPuter , 2003 .

[2]  L. Chen,et al.  Photonic Switching for Data Center Applications , 2011, IEEE Photonics Journal.

[3]  Nader Mokari,et al.  Optimal Qos-aware network reconfiguration in software defined cloud data centers , 2017, Comput. Networks.

[4]  Hiroaki Kobayashi,et al.  3D on-chip memory for the vector architecture , 2009, 2009 IEEE International Conference on 3D System Integration.

[5]  Reza Nejabati,et al.  The Benefits of a Disaggregated Data Centre: A Resource Allocation Approach , 2016, 2016 IEEE Global Communications Conference (GLOBECOM).

[6]  Sue Malcolm.  Lighting up time , 1982, Nature.

[7]  Ioannis Tomkos,et al.  Optical interconnection networks in data centers: recent trends and future challenges , 2013, IEEE Communications Magazine.

[8]  Ioannis Tomkos,et al.  A Survey on Optical Interconnects for Data Centers , 2012, IEEE Communications Surveys & Tutorials.

[9]  Ren-Hung Hwang,et al.  Design of SDN-Enabled Cloud Data Center , 2015, 2015 IEEE International Conference on Smart City/SocialCom/SustainCom (SmartCity).

[10]  Dustin W. Demetriou,et al.  Trends in High Performance Computing: Exascale Systems and Facilities Beyond the First Wave , 2019, 2019 18th IEEE Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronic Systems (ITherm).

[11]  Keren Bergman,et al.  Flexfly: Enabling a Reconfigurable Dragonfly through Silicon Photonics , 2016, SC16: International Conference for High Performance Computing, Networking, Storage and Analysis.

[12]  John Shalf,et al.  Trends in Data Locality Abstractions for HPC Systems , 2017, IEEE Transactions on Parallel and Distributed Systems.

[13]  Tingye Li,et al.  Optical Fiber Communication-The State of the Art , 1978, IEEE Trans. Commun..

[14]  Zhehui Wang,et al.  Systematic Exploration of High-Radix Integrated Silicon Photonic Switches for Datacenters , 2019, 2019 IEEE/ACM International Conference on Computer-Aided Design (ICCAD).

[15]  Amin Vahdat,et al.  Helios: a hybrid electrical/optical switch architecture for modular data centers , 2010, SIGCOMM '10.

[16]  John Shalf,et al.  DOE Advanced Scientific Computing Advisory Subcommittee (ASCAC) Report: Top Ten Exascale Research Challenges , 2014 .

[17]  Guido Maier,et al.  Deep Learning-Based Traffic Prediction for Network Optimization , 2018, 2018 20th International Conference on Transparent Optical Networks (ICTON).

[18]  Konstantina Papagiannaki,et al.  c-Through: part-time optics in data centers , 2010, SIGCOMM '10.

[19]  J.E. Midwinter,et al.  Optoelectronic interconnects in VLSI: the reality of digital optical computing? , 1990, IEEE LCS.

[20]  Phuoc Tran-Gia,et al.  Power Consumption Analysis of Data Center Architectures , 2011, GreeNets.

[21]  Qi Li,et al.  Silicon photonics for exascale systems [OFC 2014 Tutorial] , 2014, OFC 2014.

[22]  John Shalf,et al.  Bandwidth steering in HPC using silicon nanophotonics , 2019, SC.

[24]  Keren Bergman,et al.  Autonomous Dynamic Bandwidth Steering with Silicon Photonic-Based Wavelength and Spatial Switching for Datacom Networks , 2018, 2018 Optical Fiber Communications Conference and Exposition (OFC).

[25]  Nathan C. Abrams,et al.  Ultralow-crosstalk, strictly non-blocking microring-based optical switch , 2019, Photonics Research.

[26]  F. Mueller,et al.  Proactive process-level live migration in HPC environments , 2008, 2008 SC - International Conference for High Performance Computing, Networking, Storage and Analysis.

[27]  Takatsugu Ono,et al.  Evaluating the Impact of Energy Efficient Networks on HPC Workloads , 2019, 2019 IEEE 26th International Conference on High Performance Computing, Data, and Analytics (HiPC).

[28]  Manuel Ujaldon.  HPC Accelerators with 3D Memory , 2016, 2016 IEEE Intl Conference on Computational Science and Engineering (CSE) and IEEE Intl Conference on Embedded and Ubiquitous Computing (EUC) and 15th Intl Symposium on Distributed Computing and Applications for Business Engineering (DCABES).

[29]  Jiao Zhang,et al.  Updating Data-Center Network With Ultra-Low Latency Data Plane , 2020, IEEE Access.

[30]  Tae Hee Han,et al.  Power and Signal-to-Noise Ratio Optimization in Mesh-Based Hybrid Optical Network-on-Chip using Semiconductor Optical Amplifiers , 2019 .

[31]  Robert B. Ross,et al.  Enabling Parallel Simulation of Large-Scale HPC Network Systems , 2017, IEEE Transactions on Parallel and Distributed Systems.

[32]  Andrea Bianco,et al.  Scheduling traffic for maximum switch lifetime in optical data center fabrics , 2016, Comput. Networks.

[33]  Alex C. Snoeren,et al.  RotorNet: A Scalable, Low-complexity, Optical Datacenter Network , 2017, SIGCOMM.

[34]  Jeffrey S. Vetter,et al.  Opportunities for Nonvolatile Memory Systems in Extreme-Scale High-Performance Computing , 2015, Computing in Science & Engineering.

[35]  Qixiang Cheng,et al.  Silicon Photonic Switch Topologies and Routing Strategies for Disaggregated Data Centers , 2020, IEEE Journal of Selected Topics in Quantum Electronics.

[36]  Leonid Oliker,et al.  Analyzing Ultra-Scale Application Communication Requirements for a Reconfigurable Hybrid Interconnect , 2005, ACM/IEEE SC 2005 Conference (SC'05).

[37]  Janine Bennett,et al.  Scalable Collectives for Distributed Asynchronous Many-Task Runtimes , 2018, 2018 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW).

[38]  Juvencio Arnaldo Manjate,et al.  Energy-Efficient Data Center Networks , 2018, 2018 IEEE 17th International Symposium on Network Computing and Applications (NCA).

[39]  Kristinn B. Gylfason,et al.  MEMS for Photonic Integrated Circuits , 2020, IEEE Journal of Selected Topics in Quantum Electronics.

[40]  P. Dumon,et al.  Silicon microring resonators , 2012 .

[41]  S. Yoo,et al.  Architecture and Performance Studies of 3D-Hyper-FleX-LION for Reconfigurable All-to-All HPC Networks , 2020, SC20: International Conference for High Performance Computing, Networking, Storage and Analysis.

[42]  Sascha Hunold,et al.  Autotuning MPI Collectives using Performance Guidelines , 2018, HPC Asia.

[43]  Dzmitry Kliazovich,et al.  GreenCloud: A Packet-Level Simulator of Energy-Aware Cloud Computing Data Centers , 2010, GLOBECOM.

[44]  Jian Wu,et al.  OCBridge: An Efficient Topology Reconfiguration Strategy in Optical Data Center Network , 2018, 2018 Optical Fiber Communications Conference and Exposition (OFC).

[45]  Tara Javidi,et al.  End-to-end scheduling for all-optical data centers , 2015, 2015 IEEE Conference on Computer Communications (INFOCOM).

[46]  Benjamin G. Lee,et al.  Silicon Photonic Switch Fabrics: Technology and Architecture , 2019, Journal of Lightwave Technology.

[47]  Qixiang Cheng,et al.  Silicon Photonic 2.5D Multi-Chip Module Transceiver for High-Performance Data Centers , 2020, Journal of Lightwave Technology.

[48]  Chin-Tau Lea,et al.  A Scalable AWGR-Based Optical Switch , 2015, Journal of Lightwave Technology.

[49]  Ahmed Louri,et al.  Extending the Power-Efficiency and Performance of Photonic Interconnects for Heterogeneous Multicores with Machine Learning , 2018, 2018 IEEE International Symposium on High Performance Computer Architecture (HPCA).

[50]  Qixiang Cheng,et al.  Recent advances in optical technologies for data centers: a review , 2018, Optica.

[51]  T. S. Eugene Ng,et al.  A Tale of Two Topologies: Exploring Convertible Data Center Network Architectures with Flat-tree , 2017, SIGCOMM.

[52]  Takashi Inoue,et al.  SOA-Integrated Silicon Photonics Switch and Its Lossless Multistage Transmission of High-Capacity WDM Signals , 2019, Journal of Lightwave Technology.

[53]  Leonid Oliker,et al.  Communication Requirements and Interconnect Optimization for High-End Scientific Applications , 2007, IEEE Transactions on Parallel and Distributed Systems.

[54]  M. Ujaldón.  HPC Accelerators with 3 D Memory , 2016 .

[55]  S. Namiki,et al.  Lossless Operation of SOA-Integrated Silicon Photonics Switch for 8 × 32-Gbaud 16-QAM WDM Signals , 2018, 2018 Optical Fiber Communications Conference and Exposition (OFC).

[56]  Madeleine Glick,et al.  Your Data Center Is a Router: The Case for Reconfigurable Optical Circuit Switched Paths , 2009, HotNets.

[57]  Yang Wang,et al.  A Multi-Floor Arrayed Waveguide Grating Based Architecture With Grid Topology for Datacenter Networks , 2020, IEEE Access.

[58]  Stefan Schmid,et al.  Survey of Reconfigurable Data Center Networks: Enablers, Algorithms, Complexity , 2019, SIGA.

[59]  Yoichi Taira.  Technology for Optical Co-Packaging , 2020, 2020 Pan Pacific Microelectronics Symposium (Pan Pacific).

[60]  Ming C. Wu,et al.  Large-scale silicon photonic switches , 2016, 2016 21st OptoElectronics and Communications Conference (OECC) held jointly with 2016 International Conference on Photonics in Switching (PS).

[61]  John Shalf,et al.  PINE: Photonic Integrated Networked Energy efficient datacenters (ENLITENED Program) [Invited] , 2020, IEEE/OSA Journal of Optical Communications and Networking.

[62]  Keren Bergman.  Empowering Flexible and Scalable High Performance Architectures with Embedded Photonics , 2018, 2018 IEEE International Parallel and Distributed Processing Symposium (IPDPS).

[63]  Rami G. Melhem,et al.  On the Feasibility of Optical Circuit Switching for High Performance Computing Systems , 2005, ACM/IEEE SC 2005 Conference (SC'05).

[64]  Alex C. Snoeren,et al.  Topology Switching for Data Center Networks , 2011, Hot-ICE.

[65]  Hans Eberle,et al.  Separated High-Bandwidth and Low-Latency Communication in the Cluster Interconnect Clint , 2002, ACM/IEEE SC 2002 Conference (SC'02).

[66]  Christian Baks,et al.  An 8×8 Silicon Photonic Switch Module with Nanosecond-Scale Reconfigurability , 2020, 2020 Optical Fiber Communications Conference and Exhibition (OFC).

[67]  Xinyu Niu,et al.  EXTRA: Towards the exploitation of eXascale technology for reconfigurable architectures , 2016, 2016 11th International Symposium on Reconfigurable Communication-centric Systems-on-Chip (ReCoSoC).