Advanced Micro Devices, Inc. (AMD) Earnings Call Transcript & Summary

All right. Hello, everyone. Thank you for joining us today. My name is Jeffrey and I will be your host for today's webinar. Today, we're going to be talking about AI Engine test harness and Versal system integrator. So today is our final session of the Vitis Developer Series. So after today, you'll be able to go back and watch all 3 of our webinars that we've run over the past few weeks. But today's presentation is scheduled to last roughly 30 to 40 minutes with about 15 to 20 minutes of Q&A at the end. So we're going to have plenty of time for Q&A. Go ahead and drop your questions in the question section down below the video player, and we'll try to get you at the end of the webinar. Before we jump in, I'm just going to give a couple of pits about the interface. In the video player here, you can control the volume, resolution, all that jazz there. And then if you scroll down, you'll see a list of tabs that includes the overview tab, surveys and documents, as well as the questions have been -- we've probably talked about. So in the overview tab, you'll find a list of links to the Vitis GitHub, the tutorials and documentation. And then in documents, you will find a PDF copy of today's slides and then in the survey tab, you will find a survey to fill that out for us that will help us out as we try to bring in the best webinars in the most relevant content possible. And last but not least, you will be able to come back and watch this on demand any time as soon as the webinar concludes today. All right. With all that housekeeping out of the way, I would like to hand the ball over to Faisal to begin the presentation. Take it away.

Hello, everyone. Welcome to the third and final session of Vitis Developer Webinar Series 2023. My name is Faisal. I am a technical marketing manager at AMD, focusing on DSP# and AI Engine. In today's presentation, we will review the AI Engine development flow, introduce AI Engine test harness, demonstrate the AI Engine test harness using a designing example. And finally, we will demonstrate Versal system integration using Vitis tool flow. First off, we will explain the recommended AI Engine development flow. AMD recommends the following design flow. First, the user starts developing the AI Engine craft using AMD libraries and/or custom kernels. Then the user validates the functional accuracy of the AI Engine application using untimed x86 simulation. The test bench for x86 simulation can consist of basic file I/O operations or of more powerful [indiscernible] generators written in Python or C, C++. Next, user can move to AI Engine simulation to validate the performance of the graph and optimize the design. The AI Engine simulator models the timing and resources of AI Engine array while using transaction-level system [indiscernible] models for the AI Engine, network on chip or NOC and the DDR memory. This allows fast performance analysis of the AI Engine application, which cycle approximate timing information and accurate estimation of the AI Engine resource usage. The same files are generated when compiling for AI Engine simulation or for hardware runs. Thus, the kernel simulation is running on the same executable, which will run on hardware. It is important to monitor performance results and meet performance calls at this stage because the system performance will not improve as we progress through the development stages. However, meeting performance in the aiesimulator at this early stage in the design is not indicative of the final system performance because the AIE simulates I/O traffic, assuming best-case operating conditions. The same file I/O operations or Python, C, C++ traffic generators as x86 simulation can be used in the AI Engine simulation. Once the AI Engine application is validated in isolation, the user integrates the AI Engine component into the rest of the system including their programmable logic, PL and the processor subsystem, PS. The user shall then validate the functional accuracy and performance of the entire system and hardware emulation, AI Engine is simulated using system C AI Engine simulator and the PL kernals are simulated as RTL code or system CDLM based on the types of models available or selected. And the embedded software code running on the PS is emulated using [indiscernible]. The results of combining these different simulation environments together, the abstraction of Vitis hardware emulation is very close but not fully cycle accurate. The scope of the Vitis hardware emulation also defines its purpose. Hardware emulation allows you to simulate the entire design and test the interaction between the PL, PS and the AI Engine prior to running our physical hardware. Because hardware emulation provides full debug visibility into all aspects of the application, it is easier to debug complex problems in this environment than in real hardware. Finally, the user tests the full system and hardware. Some of the challenges in this flow for an AI Engine developer specifically in step #3 net include the fact that the AIE simulation can become slow for complex graphs with large data sets. There is limited verification scope and testing of the AI Engine graph on hardware requires a full device design, including programmable logic, PL, and the processing subsystem PS. For this reason, we are introducing AI Engine test harness, a new tool that allows users to deploy their AI Engine subsystem on hardware with minimal effort. This allows hardware validation earlier in the design cycle and gives users the ability to run cycle accurate characterization of their designs on longer wave forms avoiding long simulation times. More on this in the upcoming slides. The goal of this webinar is to focus on introducing the AI Engine test harness and discuss the Versal system integration. For this reason, the primary focus shall be on the last 3 steps in the diagram shown. So what is the AI Engine test harness? And what does it consist of? AI Engine test harness is a design framework that helps developers seamlessly transition their AI Engine graphs from simulation environment to testing on hardward. This allows users to focus on AI Engine development and not build throwaway test infrastructure code. Another advantage is designs will be running on hardware, which significantly increases the run time compared to simulation. Also, users will get accurate performance measurements since their designs are running on real hardware. Test harness is compromised of data mover infrastructure code to transfer data between DDR and AI Engine through a set of predefined CLIOs, set of software APIs to facilitate development of the application, use the [indiscernible] device and run tests, a script to easily package the user's AI Engine compiled design, data movers, test application and then generate the boot image for device under test. Today, AI Engine test harness supports up to 72 DMA channels used for data transfer to and from DDR, 36 in each direction. Effective throughput is 5 gigabytes per second using a 128-bit wide [indiscernible] 312.5 megahertz. Each channel has its own performance counters to count cycles, which helps profile the real throughput of the AI Engine application. Users can also validate the correctness of the output data from the design against the expected outputs. The programmable logic part of the test harness made up of data movers, it's free compiled. This results in fast build times when the AI Engine graph needs to be compiled and linked than packaged. On the requirement side of things, data size needs to be less than or equal to 128 kilobytes and must be multiple of 16 bites. Test harness is currently targeting the VCK190 board. The AIE graph must use PLIO interfaces, i.e., GMI interfaces are not supported. In this slide, we highlight the build flow to use test harness and what steps the users need to be aware of. Only minor changes are required by the designer to use the test harness. In the top-level graph, test harness header must be included before it's compiled by the AIE compiler to produce the libadf.a. Graph must also be modified to use one of the available PLIOs provided by the test harness. Test-bench program, which uses test harness APIs to drive hardware tests is compiled using GCT. [indiscernible] script is used to package these outputs along with test vectors into an SD card image, which can then be deployed on hardware to verify the design. In the following slides, we will demonstrate how to use the AI Engine test harness using an available design example. A channelizer is a commonly used digital signal processing algorithm often seen in wireless communication systems. It takes in a broadband signal as input and splits it into channels of equal bandwidth at the output. Each of the resulting channels is decimated to the new lower rate relative to the input sample rate based on the number of channels and the oversampling factor. A channelizer can be implemented efficiently utilizing an [indiscernible] structure highlighted in the figures shown. This relies on a filter bank followed by endpoint DFT, where [ end ] is a number of channels. Polyphase channelizer we are looking at today has a sample rate of 10 giga samples per second. The AI engine clock rate that we chose for this design is 1.25 gigahertz. This means that the super sample rate equals 8. Filter bank is a 128 tap polyphase filter supporting 16 channels. This is marked into 8 AI Engine tunnels. DFT block is a standard discrete 4-year transform operation, and that is not into 16 AI Engine tunnels. HLS blocks mapped into programmable logic are responsible for data permutation, buffering and performing cycle shifts as well as the transfer of data from memory to design and back. On the bottom right, we can see the resource utilization for this design split between programmable logic resources and AI Engine cores. In the following slides, we will take the top portion of the design, which are the kernels mapped to AI engine, integrate them to test harness infrastructure then run them on hardware. Afterwards, to highlight the Versal system integration flow, we will integrate the complete design, which includes the programmable logic and the processing subsystem then run it on hardware. We will begin by integrating the AI Engine sub grafts of our design, which are compromised of the filter bank and DFT into the test harness infrastructure. The polyphase channelizer contains 2 AI Engine sub grafts, each requiring 8 plus 8 PLIOs for data transfer. The top-level graph must be modified to use from the list of available PLIOs. The host application can be developed to create data from the input text file, pass it to test harness data movers, run the design, read the data at the output the design, compare output data against a golden reference and then confirmed completion or run. In order to run this design using the test harness, the user must clone the AI Engine test harness, [indiscernible] repository, ensure the latest version of Vitis tools is installed, set up the platform repo parts environment variable, source and setups [ .sh ] to set additional environment paths needed and then navigate into where the design of interest exists. In order to generate the bitstream ready to run on hardware, run the following make command using the make file flow. This shall compile the AI Engine graph using AIE compiler, lay the output with the pre-compiled data movers mapped into programmable logic, compile the test bench application using the GCC compiler, and then package the design into a bootable SD card. Next, the generated bit stream needs to be passed into an SD card provided with VCK190, [indiscernible] is an open source option that can be used for this. Communicate with the board [indiscernible] open source software can be used. Used device manager to identify available a new serial-type connection in PuTTY with 115200. Set the switch [indiscernible] and the red rectangle as shown, so the board boots up an SD card mode, turn on the board using the switch and the yellow rectangle. After the device is powered on, mode of sequence will start to be displayed on PuTTY terminal session. After that is complete, log in using petalinux as user name and password. Log into [indiscernible] change directory to path shown and then source the run_script.sh. This shall execute APIs and the host.cpp. And in this case, it will count the number of cycles to pump input data through the PLIOs, impaired expected versus actual output of the design, confirm completion of execution on hardware with "test passed" message. Finally, we will demonstrate the Versal system integration using the Vitis tool flow. Here is an overview of the Vitis tool flow for Versal devices. Each engine must be compiled through a specific set of tool chains. For programming the AI Engines, the Vitis tool come with AI Engine tools just to compile and simulate the designs. Here, x86 SIM can be used to validate the functional correctness of a design while aiesimulator has the ability to analyze performance. For programming the programmable logic PL, Vitis can be used to create custom IPs from C, C++ functions using HLS compiler. While this can package IPs created using HLS or RTL IPs as kernels, which can be added to the system during the linking stage. For creating a platform users need to create a hardware design in [indiscernible] and then generate the full platform using Vitis. If the target is running Linux, Linux tool is used to generate the kernel image rootfs and sysroot. For a program in the processing subsystem, the Vitis tool is used to compile the application using GCC tool chain. V++ later at the designer integrate their AI Engine, HLS and -- TL kernals into an existing extensible platform. In this step the platform chosen by hardware designer is provided or they can opt to use one of the many extensible based platforms provided by AMD and have V++ build the hardware design for them. V++ package is the final step, which packages all components generated in the previous step into an image, which can be booted and run on chosen targets. Available targets are hardware emulation or hardware. Hardware emulation can be performed using aiesimulator for AI Engines, [indiscernible] for the processing subsystem and provide a simulator for programmable logic. In this slide, we show the various components of the polyphase channelizer design and how they are mapped against the Versal architecture. We will use Vitis unified ITE to integrate all design components and generate a boot image to test in hardware emulation flow as well as on the physical device. To run this example, make sure Vitis 2023.1 software is installed and then clone Vitis tutorials public GitHub repository. Launch the Vitis unified IDE by using the command shown here. From 2023.2 onwards, Vitis unified IDE is the default [indiscernible]. You can specify the workspace at the time of launching the tool or open the workspace after the tool is launched. Let us begin by integrating the filter bank and DFT from our polyphase channelizer design. Once the Vitis unified IDE is launched, begin by creating a new AI Engine component. This can be done by clicking file, new components, AI Engine. In this example design, all AI Engine kernels are connected using a single top-level graph. AI Engine components are compiled into libadf.a file. Imports all AI Engine design files using this menu compromising the filter bank and DFT. Select the top level application from this [indiscernible] on menu. Also select the platform shown since we are targeting VCK190. The AI Engine component can be built targeting x86 SIM or hardware, use the bottom build option as we would like to demo the complete design and hardware emu and hardware. After compiling the design, users can inspect the report section. This contains useful information, including a summary of hardware resources used, graph and a review among other useful reports. The summary section shows the summary of the AI Engine component build status, target device, along with the clock rate is shown at the top. The AI Engine resource utilization is also shown. This includes tiles used for compute, buffers and stream interconnect. It also includes the number of interface channels used to send data to and from the AI Engine top-level graph and whether they were PLIO versus GMIO. Graph view shows a graphical view of the AI Engine kernels and their connections. And review shows a graphical view of the AI Engine kernels and they're mapping to the device [ chords ]. As previously mentioned, the filter bank and this design uses 8 AI engine tiles and is mapped to the center of the device array. The DFT uses 16 tiles and is mapped to the left side. Next, we will integrate the design components mapped into the programmable logic. In this step, we shall add HLS components of the design. HLS components are kernels mapped into the programmable logic and run on [indiscernible]. They are written in C, C++ and are synthesized into RTL. To add HLS components click file, new component, HLS. We shall show steps to import and compile a single HLS block, but these steps will need to be repeated for every HLS block in the design. Import all HLS design files for this block using this menu and point to the top level wrapper function. Use the same base platform previously selected, select Vitis kernel flow with run additional packaging step to create a top [indiscernible] output. Click the run button under the see synthesis to build this HLS component. Repeat these steps for all of the HLS plots in this design. Next, we will add the processor subsystem component of the design, implemented by the host program. The host program interacts with the AI Engine kernels and kernels in the PL region. This can be done by clicking file, new component application. We can see that the OS is limits and the processor is ARM Cortex-A72. Set the path to this sysroot used to cross-compile PS application. After adding the [indiscernible] component, right click on sources, import, files, then point to the host.cpp fund and PS apps directory. Also, import AIE control, xrt.cpp, which is a generated file after compiling AIE component. The xrt.cpp gives information about the graph to the PS application. Under our [indiscernible] component settings, edit C makelist.text to include AIE kernel task. Next, we will add a system project component. System project integrates the platform along with the 3 processing domains of an AMD Versal device, and those are the AI Engine, the programmable logic and the processing subsystem components into a single heterogeneous system. The system project refers to the components, defines and stores the linking configuration required to build a system and defines and stores the packaging configuration required to package system. Use the same base platform previously selected, users will have the option of generating custom kernel image, route FS and sysroot. For this demo, we are targeting the, VCK190 and will download and use pre-built images available on the web -- on the Xilinx website. We will link all of the HLS components of this design. There are 5 HLS components. Link the AI engine components, link the processor subsystem component. The next step is to set the connectivity between PL kernels as well as between PL and AIE graft. To do this edit binary container config under system project sources. System.config under Vitis Final should be used as a reference. For more information referred to UG1393. The package settings configuration file contains information used by Vitis compiler, V++ for packaging the system and their system project sources, edit package.config to include signal input and output files into the final build. These will be packaged and used during hardware emulation and when running the design on real hardware. The design is now ready to be built. Select build all under hardware emulation flows. Users will have the option of recompiling the individual components of the system before they are integrated. This includes the AI Engine, as well as NPS components. In order to run the hardware emulator, a launch configuration must be created. This has used to establish the specific requirements for the run or [indiscernible] session you're about to start. Click on [indiscernible] under their system project settings, then add a launch configuration as shown. Start the emulator and wait for the Linux command prompt. Once the command prompt is displayed, click run. As can be seen in the output window, the design is simulated till completion. All of the samples are matching expected outputs. After sanitizing our design on the hardware emulator, we will compile and run the design on the physical device. Click build all under hardware. Once compilation is completed plus the produced image and to provide a SD card using [indiscernible], insert card into the board and power up, connect to the board over a cereal using [indiscernible], log-in to the board using the shown user name and password, then again as [ root ], change directory into path shown and then execute the out component. Terminal window will display execution steps of the application, including a comparison between golden office files versus actual output from the hardware execution. In this webinar session, we reviewed the AI Engine development flow. We introduced the AI Engine test harness, a design methodology to rapidly take AI Engine graft from simulation to hardware, discussed Vitis tool flow and how it is used in Versal system integration, and we demoed the steps with an example design. The AI Engine test harness is available through our public GitHub repository. It contains example designs integrated to test harness and ready to be compiled and tested. There is also a documentation page containing more background information about the test harness and a step-by-step guide on how to get started. Vitis tutorials is also a great reference containing various reference designs and feature tutorials demonstrating the Vitis flow.

All right. We are ready to go for some Q&A then. Thank you very much. Before we get there, I just wanted to give a couple of reminders and things about the interface here. So -- if you have any questions, go ahead and scroll down underneath the video player, and you will be able to submit any questions you like in there. There are a few tabs for you down there. There's questions, resources and -- or documents and the overview tab and the survey tab. In the overview tab, you can find all the links that we just talked about in that summary there, so you can go to the GitHubs and get started there. You can also download a PDF probably the slides in the documents tab and you can fill out the survey for us if you be so kind in the survey tab. Now in the questions tab, let's go ahead and take a look there. Not many questions coming in so far, but -- and we have a good amount of time. So if you have any questions, please go ahead and drop those in there. Let me see here. We've got a couple of questions about the availability of on-demand. Yes, you'll be able to come back and watch any of the developer series webinars on demand, and you can find that -- be it the same link that brought you here. Let's see here, you can also find more of our webinars at [indiscernible]. And let me see, we've got question in. We actually have extra set of hands on to answer some of these Q&A. So we have Florent, who's our colleague of [indiscernible]. Let me see. So there's a question about do you recommend any books covering AI/ML and using the AMD toolchain?

So that would not be our expertise to AI Engine, but I'm not sure there are any -- but there are a lot of resources about [indiscernible] AI, which is our main tool to do machine learning [indiscernible]. So you can have a look on [indiscernible] look for the [indiscernible] AI resources.

All right. So Florent, system migration of the PS plus PL plus AIE, is the connection between those engines done in a script?

So I would not say this is a script, which is connect the engine. We have what we call a configuration file. So that's a text file in which we are documenting the connections between all the domain, not really the PS because the PS is already -- is connected through the document. Between PL and AI Engine, we have configuration file, which describes the connectivity between all the engines.

Got it. Okay. And then, Faisal, could you touch on this question here. So their understanding is that the test harness setup is to prove how time accurate execution of developed algorithms on AI Engines will be set. Is that correct assumption?

Yes. So harness would basically allow you to run the AI Engine kernels on hardware. So while the simulation would be cycle approximate, they're running on hardware be cycle accurate. So this would be an opportunity for you to validate designs and run them on physical hardware.

Awesome. Great. All right. I think we have hit -- let's see because I think there was a follow-up there. [indiscernible] the question. So back to Florent, let's see. Is there any performance comparison between the implementation of a unique algorithm on AI Engine and the PL side that makes migration to Versal from Zynq UltraScale+ any meaningful migration there?