By Wayne Gillam / UW ECE News
UW ECE’s Certificate Program in GPU-Accelerated Computing & Visualization is designed for working professionals and graduate students. This three-quarter, online program takes students on a deep dive into the graphics processing unit, or GPU, a critical piece of modern computer architecture.
Starting autumn quarter 2024, UW ECE will again be providing the Certificate Program in GPU-Accelerated Computing & Visualization for working professionals and graduate students. This three-quarter, online program is affiliated with the Professional Master’s Program, and it takes students on a deep dive into the graphics processing unit, or GPU, which is a critical piece of modern computer architecture. Besides enabling the graphics seen on desktop computers, laptops, and mobile devices, GPUs also provide computing power for a wide range of other data-intensive applications, such as gaming, cloud computing, scientific computing, machine learning and artificial intelligence, robotics, and much more.
The Certificate Program is taught by UW ECE Affiliate Assistant Professor Colin Reinhardt, who is a UW alumnus (BSEE ‘05, BS Physics ‘05, Ph.D. ‘10) and an engineer in the U.S. Navy, where he leads a team of software engineers and scientists that specialize in GPU computing for physics-based modeling and simulation. Reinhardt is an expert on GPU application programming interfaces, or APIs, such as CUDA, OpenCL, Vulkan, and OpenGL. He also has expertise in a wide range of GPU applications, including scientific computing and visualization, physically based rendering, 3D-modeling and simulation, predictive rendering, machine/deep learning, and neural networks. Below are questions and answers with Reinhardt about GPUs and the Certificate Program that explain more details about this exciting course of study.
What is a graphics processing unit, or GPU?
UW ECE Affiliate Assistant Professor Colin Reinhardt
A graphics processing unit, which is commonly called “GPU” for short, is a specialized microprocessor subsystem that most computers today have, whether it’s a desktop workstation, laptop, or a mobile smartphone. All these devices have GPUs inside of them, and they work alongside the computer’s central processing unit, or CPU (which is the main “brain” of your computer). You can think of the GPU as the workhorse of your computer for many compute-intensive tasks.
Fundamentally, the GPU is about massively parallel multiprocessing capability. It allows you to do a lot of computation, faster. But there are challenges in how you write the code and how you structure your algorithms to use this power effectively. For example, there are certain conditions that must be met for an algorithm to be parallelizable. And then, if you want your algorithm to be parallelized, how do you do that efficiently for a given GPU architecture? That’s a major focus in this Program, particularly in the GPU computing course.
What sorts of things do GPUs make possible?
The original, driving purpose for GPUs was to accelerate the production of 3D graphics. They are optimized for this and are still used heavily for that purpose. Another place that GPUs are critical is in image processing. So, in scientific fields, for example, GPUs are used for doing physically based algorithms to process video. Anytime there are large numbers of pixels and large images being processed, such as for additional post-processing, special effects, or image analysis — GPUs are well suited for all those problems.
Biomedical applications are another big area. GPUs are used for biomedical image analysis, image fusion, and 3D-computed tomography. And as GPU capabilities progress, the ability to deliver solutions progresses as well. We’re now seeing real-time 3D visualization of fused biomedical images that include positron emission tomography, computed tomography, and magnetic resonance imagery. All these different modalities that are imaging into the human body and producing critical diagnostic and therapeutic imagery are fundamentally driven and accelerated by GPUs.
One of the biggest applications of GPUs today is for machine learning and artificial intelligence. Deep learning models, natural language processing models, and large language models, such as ChatGPT, DALL-E, and Microsoft CoPilot all critically depend on GPUs for training. GPUs also help engineers and scientists deal with massive amounts of data, known as “big data,” because GPUs enable us to compute faster. Furthermore, one step removed, they allow us to train these deep learning models and other machine learning tools at massive scale, which can then help to process more data even faster, and even in real time. So, overall, I think GPUs are a critical tool that are only going to continue to become more important and powerful as the technology continues to progress.
Could you tell me more about your background as it relates to GPUs and this Certificate Program?
I’m an alumnus of the Department. I received a double bachelor’s degree in physics and electrical engineering from the UW in 2005, and then I went on to earn my doctoral degree from UW ECE in 2010. As a graduate student, I did research in an area related to GPUs, free space optical communication, which is a very compute-intensive field.
After completing my doctoral degree, I went to work for the U.S. Navy, where I continued to work in free space optical communications and related areas involving LiDAR, laser systems, and a lot of computational and numerical modeling of the atmosphere. These are places where computational demands are extremely high. So, use of the GPU in my work became very important, and I started to explore using these tools for the work I did with the Navy. That led me to become familiar with GPU capabilities early on. I also had a desire to teach, give back, and support the UW. So, in 2017, I became an affiliate assistant professor in the Department. Today, in addition to my teaching duties at UW ECE, I still work as a Navy engineer, where I lead a distributed team of software engineers and scientists that do GPU computing and hardware-accelerated algorithm development.
Teaching in the Professional Master’s Program is very inspiring for me because of the quality of the students that I’ve had a chance to work with. I’ve been very impressed with the creativity, skill, and the diligence of the students that I’ve worked with over the years in this Program. I’ve seen a lot of impressive projects done by students in these classes that are applicable to many different fields ranging from mechanical engineering to biomedical and scientific applications.
There’s a huge industry demand for students who have this knowledge and can put it into practice. So, in my curriculum, I focus on the practical skills you will need to use these tools and frameworks, as opposed to just learning a bunch of theory, where there could be a gap between that theory and how you apply it to real world tasks. All the classes that I teach have a major project-based element to them, which gives students a chance to work through the whole design-development-testing life cycle in a practical way and apply the GPU theory and methods we’ve learned.
What will students learn in this Certificate Program?
The Certificate Program is all online and consists of three courses, one each quarter. Typically, we offer these in autumn, winter, and spring, consecutively. The first quarter course is titled “Applied High-Performance GPU Computing.” It is focused on the fundamentals of how GPUs work, GPU hardware architecture as well as software, algorithm analysis and requirements, and analytical methods to determine when an algorithm is parallelizable. I also teach techniques to map your algorithm to the GPU hardware. We do this using primarily CUDA, which is a Nvidia-specific GPU language.
The second quarter course is called “GPU-Accelerated Interactive Scientific Visualization Techniques.” This title emphasizes the fact that we make use of GPU capabilities to accelerate graphics visualization at interactive rates. The course starts off with the basics of 3D-computer graphics because you need to understand the structures and the methods for a traditional 3D-computer graphics pipeline. Then, we talk about how those different steps in the 3D-computer graphics pipeline map to the GPU hardware. We look at OpenGL and Vulkan, which are the two major open-source industry standard APIs for using the GPU to do 3D-computer graphics. The focus in this course is not just on game graphics, but on more general scientific visualization, in particular, biomedical image visualization. But this is also applicable to just about any kind of general engineering visualization, or even more generally, data visualization, which moves into this important new domain of visualizing and understanding deep learning and machine learning models. So, there’s some interesting methods that utilize 3D and multi-dimensional visualization techniques to look inside deep learning networks and try to understand what they’re doing. We will look at some of those methods as well as more traditional, scientific visualization with fluid dynamics and computational electromagnetics.
The third quarter is what I call “Advanced GPU Computing and Visualization.” In this course, we bring together both scientific visualization and GPU computing. This is the culmination of the curriculum, and it integrates what you’ve learned in the first two quarters. We discuss how to develop a system that might be doing both visualization and heavy computing simultaneously. Most production-scale systems need to do this kind of thing. But there are additional challenges, such as tying together your algorithms between, say, Vulkan and CUDA or Vulkan and OpenCL. Furthermore, we cover some advanced techniques leveraging new, specialized GPU hardware features. For instance, we look at programming tensor cores, which offer even more speedup for particular computation types, and we’ll also look at the programming of the new Nvidia ray-tracing cores. This is hardware-accelerated ray tracing inside the latest Nvidia RTX GPUs. This is an exciting new area that’s just starting to be leveraged.
I also teach what I call “black art techniques.” These are programming skills that are generally not taught in schools or in theoretical textbooks. They are not well documented. Manufacturers are extremely careful with their hardware architecture information. They don’t publish it. They protect it and don’t disclose it. So, it’s difficult to learn. This is something that I wrestle with as an engineer myself: How do you know what’s happening in this black box that your code is running on? And so, there are reverse engineering and micro-benchmarking techniques that can be used to tease out particulars that could be critical in optimizing your code. We get into some of those things as well as how you can infer or learn particulars that could be very useful in writing highly performant code on the device when the vendors are not going to tell you that information.
Who is a good fit for this Program?
This Program is a good fit for any student who is interested in increasing their marketability in machine learning, 3D graphics, image processing, or computer vision. All these areas are well suited to having a strong foundation in low-level GPUs and scientific visualization. This Program is also for professionals who are wrestling with compute-intensive problems, where either their current code or systems are not fast enough (maybe they’re not processing as much data as they want to or need to, maybe their accuracy is limited). All these areas can be enhanced and accelerated by using GPUs. So, anybody who is a professional wrestling with those kinds of problems and wants to learn techniques to improve their system is another good candidate.
How could skills gained in this Program benefit an individual’s career?
I think that the skills learned in this program can benefit an individual’s career by providing them with a low-level understanding of GPU architecture as well as how to write tuned code for GPUs and think in parallel software development framework, which is not something that is traditionally taught in software engineering. All these things will help an individual become a valuable contributor to software or system development teams or potentially continue their education to do further research in the field.
What sorts of tools and technologies will students have access to?
We use a lot of tools and technologies in the Program. First, we use Amazon Web Services, or AWS, as a framework for the whole curriculum development. We also use it to provide GPUs that the students will use in doing their exercises and doing their programming. So, you don’t need to have a device with an Nvidia GPU to do this course. You will get access to a real GPU on the AWS cloud, and students have dedicated use of Nvidia GPU time on their cloud instances. So, that’s nice. And because it’s through AWS, you can log in remotely. You could do this from a variety of places and access the system from many types of client devices.
Then, for the actual curriculum content, students will obviously be studying GPU architecture. In particular, we will be looking at the Nvidia GPUs, but I will also talk about the other key players in the GPU market space, which are AMD and Intel. We’ll look at their GPU architectures and the similarities and differences between what they are doing compared to Nvidia.
We will look primarily at CUDA, the latest generation of the CUDA toolbox, and we’ll be using that to program the Nvidia GPUs. We’ll also talk about OpenCL, which is an open industry Khronos standard analogous to CUDA but device agnostic. So, it’s not tied to any one vendor, although I will say Intel and AMD are strong proponents of OpenCL.
Then, when it comes to scientific visualization, we’ll be focusing on two other Khronos standards, which are OpenGL, which was the traditional, dominant API for 3D-graphics programming, but in the last few years has been replaced and superseded by Vulkan, which is the new, modern GPU-centric graphics API. So, we will be using Vulkan.
And then we’ll be using several other Nvidia tools that come along with the CUDA toolkit to do analysis and profiling. We’ll learn the CUDA compiler tool, some of the CUDA extensions, and some of the other CUDA libraries that are included that have pre-written, GPU capabilities. There are also other profilers and analysis tools that we’ll be using.
Is there anything else students might want to know?
They might be curious about prerequisites. The official set of required and recommended prerequisites for the track is provided on the website. However, I just want to comment that there is a margin of flexibility, where, if you are coming in slightly underprepared, it is possible to compensate by doing a bit of extra work alongside the regular curriculum to supplement and enrich your understanding of the background knowledge areas. But it does take a serious commitment to doing the extra work and not falling behind in your assignments!
For example, having an intermediate to advanced skill level in C and C++ programming is strongly recommended. It is highly beneficial and will make your life easier if you have that level of experience, but it is not necessarily a showstopper. I have had students who didn’t know C and C++ very well coming in. They had a little bit more of a learning curve to surmount, but they were able to do it. I certainly will do what I can to assist. So, if you’re motivated and willing to put in the extra work, you can still complete the Program.
The same thing applies for the stated recommended prerequisite about having computer architecture background experience. We do get into topics such as microprocessor architecture, memory hierarchies, and cache architectures, which are important on these devices to make them run most efficiently. But again, it’s certainly something that a student could pick up on their own while they do the coursework. So, in the spirit of parallel programming, if you’re willing to learn and work at the same time, you can do it. But it will require more biological neural flops! Sorry, I had to squeeze in a computer/AI nerd joke!
For more information, visit the UW ECE Certificate Program in GPU-Accelerated Computing & Visualization webpage and the UW ECE Professional Master’s Program webpage. Learn more about UW ECE Affiliate Assistant Professor Colin Reinhardt on his bio page.