Herb and James encouraged everyone to learn more and get involved. Here are links related to their discussion:
Why Must We Care About Heterogeneous Computing?
Three Key Papers that together explain quite well the shift in computing that makes heterogeneous computing the very definition of modern computing systems:
- The Free Lunch Is Over: A Fundamental Turn Toward Concurrency in Software, by Herb Sutter, 2004
- Welcome to the Jungle - or - A Heterogeneous Supercomputer in Every Pocket, by Herb Sutter, 2011
- A New Golden Age for Computer Architecture, by John L. Hennessy, David A. Patterson; Communications of the ACM, February 2019, Vol. 62 No. 2, Pages 48-60
Herb and James strongly recommend reading and studying all three!
Moore's Law and Chiplets
As we quipped, debating if and when Moore's Law is dead is a popular sport. Gordon Moore predicted Heterogeneous Computing in his seminal 1965 white paper, “Cramming More Components Onto Integrated Circuits,” writing that as chip density and complexity progressed, eventually “it may prove to be more economical to build large systems out of smaller functions, which are separately packaged and interconnected. That day is here, and UCIe and CLX standards help make it an open industry-wide adventure!
- Universal Chiplet Interconnect Express (UCIe) system
- Compute Express Link
- Original Moore's Law paper that predicted "it may prove to be more economical to build large systems out of smaller functions, which are separately packaged and interconnected."
Learn about C++ and Heterogeneous Computing
- Critical Vocabulary: The Pillars of Concurrency, by Herb Sutter, Dr. Dobbs, July 2, 2007 (note: a PDF with all pages is cached on github)
- Includes History of C++ First Usage in HPC - results were aired on CNN:
- Reflecting on the 25th Anniversary of ASCI Red and Continuing Themes for Our Heterogenous Future, by James Reinders, HPCWire April 2022
- History - Atomics once needed to be outside C++, and are now in C++:
- TBB - first book 2007 (before C++ had atomics) and then updated in 2019 (will update for heterogeneous in 2023)
- atomic Weapons: The C++ Memory Model and Modern Hardware, by Herb Sutter, 2013
- The Future - Parallelism support in C++ stepping toward heterogeneous support:
- Proposal p2300 (senders/receivers) being discussed for C++26 - viewable by all - get involved!
- SYCL standard (and extensive book) a C++ template library approach to adding C++ support - an active open community - get involved!
- Mentioned as extending C++ for concurrency or parallelism: OpenMPI, OpenMP, and oneTBB
The QR code for this page is:
James' recent book:
