[The overview of PIMFlow]

With the end of Dennard scaling, we are witnessing a major shift in the computer system and microarchitecture design towards exploiting more specialized and lightweight “accelerators” of different types instead of relying on mostly general-purpose processors. Such heterogeneous systems pose an unprecedented challenge for the entire software stack to provide programmability and portability while delivering performance. 

We work on rethinking compiler and runtime technologies for heterogeneous systems with emerging architectures such as compute-augmented memory (NDP/PIM) (joint work with SK Hynix, KAIST, SNU, and other universities and start-ups). PIMFlow (CAL ’22, CGO ’23) proposes software layers specifically designed to accelerate compute-intensive convolutional layers on PIM-DRAM. XLA-NDP (CAL '23) introduces a compiler and runtime solution for NDPX to maximize parallelism based on GPU and NDPX costs.

Related publications: 

• J. Park, H. Sung, XLA-NDP: Efficient Scheduling and Code Generation for DL Model Training on Near-Data Processing Memory, IEEE Computer Architecture Letters (CAL), 2023.

• Y. Shin, J. Park, S. Cho, and H. Sung, PIMFlow: Compiler and Runtime Support for CNN Models on Processing-in-Memory DRAM. International Symposium on Code Generation and Optimization (CGO),  2023.

• Y. Shin, J. Park, J. Hong, and H. Sung, Runtime Support for Accelerating CNN Models on Digital DRAM Processing-in-Memory Hardware, Computer Architecture Letters (CAL), 2022.

[The overview of One-shot Tuner]

Generating high-performing codes for increasingly heterogeneous hardware calls for more flexible and adaptive ways to model program behaviors with different optimization decisions than traditional heuristic-based cost models. 

We take data-driven approaches where the code-performance relationship is accurately learned from code representation and profiling results. CogR (PACT ’19) guides the OpenMP runtime scheduler by predicting whether an OpenMP target region will execute faster on CPU on GPU using a deep-learning based predictor model, while MetaTune extends an auto-tuning framework in a deep-learning compiler, TVM, to reduce autotuning overheads and generate better-optimized codes for tensor operations. Most recently, One-shot Tuner (CC ’22) showed how online auto-autotuning overheads can be practically eliminated with a NAS-inspired performance predictor model trained with a small set of samples (open-sourced). 

Related publications: 

• J. Ryu, E. Park, and H. Sung, One-shot tuner for deep learning compilers. ACM SIGPLAN International Conference on Compiler Construction (CC), 2022.

• J. Ryu and H. Sung, MetaTune: Meta-Learning Based Cost Model for Fast and Efficient Auto-tuning Frameworks, arXiv, 2021

• H. Sung, T. Chen, A. Eichenberger and K. K. O'Brien, POSTER: CogR: Exploiting Program Structures for Machine-Learning Based Runtime Solutions, International Conference on Parallel Architectures and Compilation Techniques (PACT), 2019.

[The overview of NavCim design space exploration flow]

Design space exploration is a key research topic for application-specific accelerators, but has not been extensively explored in the context of processing-in-memory hardware design.  

Our paper (ICCAD '23) introduces a heterogeneous analog computing-in-memory architecture that supports multiple tile and subarray sizes (“big-tile, little-tile”) with an end-to-end design space exploration tool to optimize latency, power, and area at the same time (open-sourced). 

Our recent paper (PACT '24) proposes NavCim, a comprehensive ACiM design space exploration mechanism that advances the prior work in terms of search efficiency, search space coverage, and optimization metrics. NavCim introduces predictive modeling of ACiM hardware performance and uses the PPA prediction models instead of running simulators, significantly reducing search overheads.

Related publications: 

• J. Park, Y. Shin, H. Sung, Multi-Objective Architecture Search and Optimization for Heterogeneous Neuromorhpic Architecture,  IEEE/ACM International Conference on Comptuer-Aided Design (ICCAD), 2023

• J. Park*, B, Kim*, H. Sung (*: co-first authors), NavCim: Comprehensive Design Space Exploration for Analog Computing-in-Memory Architectures, International Conference on Parallel Architectures and Compilation Techniques (PACT), 2024.

Modern supercomputers harness massive parallelism provided by host CPU’s and specialized computing elements such as GPU’s and NPU’s. 

In collaboration with ETRI and KISTI, we work on building Korea’s own next-generation supercomputers and providing OpenCL programming support with optimizing compilers and runtime.  

TBU