NVIDIA Corporation (NVDA) Earnings Call Transcript & Summary

Hi, everyone. Thank you so much for joining the seminar. I am Navyaa. I'm an engineer on Developer Tools team here at NVIDIA, and I'm going to be hosting the session today. The goal of the seminar is to go over remote debugging support in Nsight Visual Studio Code Edition. In this webinar, we will go over remote debugging in 3 scenarios: first, where the target is a DRIVE OS Linux board; the second where the target is a BlackBerry QNX board; and the last where the target is a Blackberry QNX board but we set it up using containers. In all these cases, we cross-compile our program on the host, but we run and debug it on the target. I will cover all the scenarios for which our extension and Nsight VS Code Edition implement support. And other than those, I will spend some time talking about some of the scenarios covered by the support Microsoft provides. I will also briefly talk about how our extension's remote support is different from that provided by Microsoft and which might be better suited for your purpose. For regular remote debugging, where we develop, run and debug on the same machine, it's best to use Microsoft's official SSH extension. The SSH extension can be found on the marketplace. I will not go into too much detail here. Microsoft does provide awesome documentation that goes over how to set this up. The best part about this is that once everything is set up, you're able to develop, run and debug on the remote machine, almost as if you were doing it locally. For the case where you would like to develop on 1 machine and run and debug on another machine, Nsight VS Code's Remote Support is best suited. This becomes especially useful in cases where it is hard to directly develop on the machine where you want to run. For instance, BlackBerry QNX boards or Jetson devices. In this webinar, I will demonstrate debugging scenarios using devices that run DRIVE OS. My aim is to provide practical insights into debugging within the DRIVE OS environments and to augment developer workflows. Rest assured, the debugging concepts are universally applicable and the principles can be adapted to other systems. At this point, I assume everyone has Nsight VS Code Edition installed. If not, it's on the VS Code Marketplace and please install it because everything I speak about from here on out would be based on the assumption that you have the extension installed. I will be demonstrating how to do this using Nsight VS Code Edition, but for those of you who have, in the past, used CUDA-GDB remote debugging, this is equivalent to running CUDA-GDB locally and connecting to a CUDA-GDB server running on the target we want to debug on. Similar to how it would be if you were using the command line interface, there are 2 main components to be able to run a remote debug session. First is instantiating CUDA-GDB server on the target machine, and the second is connecting to it from the host. For instantiating CUDA-GDB server, you could log into the host machine and start CUDA-GDB server on the command line and that would work just fine. However, we have implemented VS Code autostart task to automate this process and make it easier for users. There are a handful of VS Code tasks that we have predefined for various scenarios. You could use them as is or you could easily customized them based on your requirements. To see the autostart tasks, you would open or create a task.json in the .vscode folder of the program or project you have opened into VS Code workspace. Then you would open the command pilot by pressing control shift P. Once the command pilot opens, you would search tasks. And in there, you would select configure default build task. Once you select that, you will have a few pre-configured tasks show up. We have 3 tasks for DRIVE OS Linux targets and 2 for QNX targets. I will go over the QNX autostart tasks in the next section. Three tasks here are Nsight autostart local host. This task will automatically start a CUDA-GDB server on local host. The second is Nsight autostart remote This task will start a CUDA-GDB server on the remote host. And the last is Nsight autostart secure copy executable binary remote. This task will secure copy the latest binary of the executable under the target and then start CUDA-GDB server on the Linux target. For the purposes of the demo I'm about to give, I will be using the third option. I'm going to demo debugging using Nsight VS Code with a CUDA sample Vector Add, which can be found on GitHub. I'm going to cross-compile this locally. One thing to keep in mind when cross-compiling for debugging is to always built with DBG1. This builds with debug symbols. Also, when remote debugging, you want to make sure that CUDA-GDB and CUDA-GDB server are from the same CUDA version. For this demo, I'm using an autostart task to copy over the executable and start CUDA-GDB server on a remote machine. I'm going to edit the command based on how I have set up SSH to the remote machine. As you can probably tell, this task will copy the binary and start CUDA-GDB server. For the variables that show up is config colon variable names. You could replace these in the task.json or fill out the values in settings.json as I have done. The merit to defining these in settings.json is that you can then also use them and launch.json. Variables like host, port executable are needed in both places so this ends up being super convenient. Before we start CUDA-GDB server, I will fill out, launch.json with all the information required to connect to the CUDA-GDB server running remotely. We need to specify the host and the port that CUDA-GDB server is running on. We also need to point to the executable binary that we want to debug, and for faster remote debugging, you also may want to set sysroot or solib-search-path. Moving on, I will actually start the task by pressing control shift B. The task copies over the executable binary and starts a CUDA-GDB server on the remote machine. You can see this as a terminal that the CUDA-GDB server is now running on the remote machine. For this, I'm going to launch the debugging session that will connect to the CUDA-GDB server running on the target and we'll hit a few breakpoints. You can see that we're connected to the remote machine. On the remote machine, it shows that we are debugging using a different system, and this further confirms that we are connected. And you see that the breakpoints work, just like they would work in a local debug session, we can continue. We can see values for locals. We can step over. We could debug just exactly like we would in a local fashion, except for the fact that here we're actually running and debugging on a remote machine. Now we move on to the case where the remote device is a QNX board, and we connect to it using a Linux client. Just like in the previous case, there are 2 main components to be able to run a remote debug session. First is instantiating CUDA-GDB server on the target machine, and second is connecting to it from the host. If you were able to directly connect to the QNX board, that is perfect. However, with the setup that I'm using, I need to connect to a Linux Host before I can connect to the QNX board. This is because the QNX device that I use can only be accessed using a specific host, so I have an extra layer here. I connected the board's host using VS Code's Remote Support. Once I'm connected to the correct host, I use Nsight Visual Studio Code's internal remote support to connect to CUDA-GDB server, which is running on the QNX device. While setting up, I had to install certain additional libraries to access debugging tools. Depending on how you set up DRIVE OS on your system, what you need to install specifically will differ. I know most people use a container to set up DRIVE OS and we will address that in the next section. For now, if you're setting up DRIVE OS without using the container, these are the additional libraries that will give you debugging tools you will need. Similar to the previous case, we have implemented VS Code autostart tasks to automate the process of remotely instantiating a CUDA-GDB server. To see the autostart tasks, you would open or create a task.json in the .vscode folder of the project or program you have opened in your workspace. Then you opened the command pilot by pressing control shift P together. Once the command pilot opens, search tasks, and in there, you will select configure default build task. You can see that we have 2 QNX tasks for QNX targets. Nsight autostart remote QNX. This task will start a CUDA-GDB server on the QNX target. The second task we have is Nsight autostart secure copy CUDA-GDB server remote QNX. This task will secure copy a CUDA-GDB server binary onto the target and then start a CUDA-GDB server on the QNX board. For the purposes of this demo, I will be using the second option. I will be copying over a CUDA-GDB server binary and then instantiating CUDA-GDB server. When starting CUDA-GDB server on a QNX target, we don't have to specify the executable binary, so we do not need that in this case, unlike when we're trying to run on a Linux board. Before we start the demo, I just wanted to share where I got my build instructions from. It's not as simple as saying make DBG equals 1, so I'm adding a link to the documentation, which you can access. It goes into more details about cross-compiling for a QNX device. Moving on to the demo. I am debugging using Nsight VS Code with a CUDA sample Matrix Mul, which can be found on GitHub. We'll start by cross-compiling this locally. The cross-compiling is using the same instructions that I just spoke about on the last slide before the demo started. Again, with remote debugging, you want to make sure that CUDA-GDB and CUDA-GDB server are from the same CUDA version. For the demo, I'm using an autostart task to copy over the CUDA-GDB server binary and start it on the remote machine. This time around, I don't need to edit the command. Also I prefilled my settings.json with all the information needed to run the task. Same as in the previous section, the variables that show up is config colon variable names. You could replace these in the task.json directly or fill these out in the settings.json as I have done. Variable values like host, port, executable are needed in both the launch.json and settings.json, so this ends up being really useful, especially if you want to edit the port value or the executable value later on. Then I will show you the launch.json, which is all the information required to connect to the CUDA-GDB server running remotely. Same as the previous case, we need to provide the host and port that we want to connect to. We also need to points to the executable binary that we want to debug. After remote debugging, we may also want to set sysroot or solib-search-path. After this, I make sure we have a CPU breakpoint and a GPU breakpoint set. And I actually start the autostart task by pressing control shift B. This task copies over the CUDA-GDB server binary and then instantiates it on the QNX device. Then I launched the debugging session locally that will connect to the CUDA-GDB server running on the target, and we hit a few breakpoints. You can see that we're connected to the remote machine. You can see that the breakpoints work like they would in a local debug session. We can continue to see values for locals, step over and debug exactly like we would in a local session, except that here, the actual running and debugging is happening on the remote board. In this case, the first breakpoint we hit was a CPU breakpoint and then a GPU breakpoint. Again, locals work, views work. They update as you go, everything, the exact same as a local debugging session, except that you're actually running on the remote QNX device. In this last part of the webinar, we will go over remote debugging on the QNX target with the difference being that this time, I set up DRIVE OS using a container. For this demo, I flashed my QNX board with DRIVE OS 6.0.8.1, and I used CUDA safe 11.4. For me, the toolkit was installed and ready to use in the tool chain that I used to set up the container. I have also linked to the documentation that I used to set up. There is similar documentation for Linux and QNX and different versions of DRIVE OS, too. We will use Microsoft's extension called Dev Containers to set up our environment in a container and then we will use Nsight Visual Studio Code's internal remote support to set up a debugging session. Just like in the previous cases, there are 2 main components to be able to run a remote debug session. First is instantiating CUDA-GDB server on the target machine and the second is connecting to it from the host. If you're able to directly connect to the QNX board, that is perfect. However, with the setup that I'm using, I need to connect to a Linux Host before I can connect to the QNX board. This is because the QNX device that I'm using can only be accessed using a specific host. So my container exists on the host that I need to connect to the board from. I connect to the board's host using VS Code remote support. Once I'm connected to the correct host, I use Nsight Visual Studio Code's internal remote support to connect to CUDA-GDB server. Similar to the previous cases, we have VS Code autostart tasks to automate the process of remotely instantiating a CUDA-GDB server to make it easier for users. You see the autostart tasks, you will open or create a task.json in the .vscode folder of the program or project that you have in your VS Code workspace. Then you open the command pilot by pressing control shift P together. Once the common pilot opens, search tasks. And in there, you can select configure default build task. You can now see that there are 2 QNX tasks for QNX targets Nsight autostart, remote QNX. This task will start a CUDA-GDB server on the QNX target. Inside your copy CUDA-GDB server remote QNX. This task will secure copy to be server binary onto the target and then start CUDA-GDB server on the QNX board. For the purposes of this demo, I will be using the second option. I will be copying over a CUDA-GDB server binary and then instantiating CUDA-GDB server. Then starting CUDA-GDB server on a QNX target, we don't have to specify the executable binary, so we don't have that extra case unlike when we're trying to do that on a Linux board. Before we start the demo, I just want to share where I got my build instructions from. Again, it's not as simple as saying make DBG equals 1. So I'm adding a link to the documentation, which you can access. Moving on to the demo, I want to start by showing my devcontainer.json. I use Microsoft's container support and have their Dev Containers extension installed. After that, I set up my container with the DRIVE OS image, the correct mound points, environment variables and the VS Code extensions that I want inside the container. Once I set it up fully, I opened the command pallet using control shift P and reopen this workspace in a container. The extension takes care of starting the container. And if we have everything correctly set up, the workspace will then open up in a running container. I am now going to debug a CUDA sample Matrix Mul using Nsight VS Code Edition. I start by cross-compiling the sample. I used the same command that I talked about just before the demo started. Note that I prefilled the launch.json, which is all the information required to connect to the CUDA-GDB server running remotely. Same as in the previous cases, we need to provide the host and the port that we want to connect to. We also need to point to the executable binary that we want to debug. And on DRIVE OS QNX systems, we want to set sysroot and solib-search-path. I also prefilled my settings.json with the information needed. Same as in the previous sections, for the variables that show up as config colon variable names, you could replace those in the task.json and launch.json or fill the values out in the settings.json as I have done. Variable values like host, port, executable are needed in both launch.json and task.json so this ends up being really useful. In my task.json you can see that I will be using an autostart task to copy over the CUDA-GDB server binary and start it on the remote machine. This time, again, I do not need to edit the command. After this, I make sure we have a GPU breakpoint and a CPU breakpoint set. Moving on, I actually start the autostart task by pressing control shift B. The task copies over CUDA-GDB server binary and starts it on the remote machine. Then I launched a debugging session that will connect to the CUDA-GDB server running on the target and hit a few breakpoints. You can see now that we are connected to the remote machine. You see that the breakpoints work like they would in a local debugging session. We can hit continue. We can see values for locals. We can step over and debug just like we would in a local session, except that here, we are actually running and debugging on the remote board. In this case, the first breakpoint that we had was a CPU breakpoint and then a GPU breakpoint. Again, locals work, views work and they update as you go. Everything is the same as a local debugging session, except the fact that you're actually running on the remote machine. To wrap up, here are some links to get the extension to go through documentation and to reach us with questions and/or feedback. For questions, I recommended that you send them via GitHub Issues or the NVIDIA Forum. That is the quickest way to reach us. And I hope the session was informative. Whether or not you're using DRIVE OS, I hoped you learn debugging tips that you could use. We are here and open to any questions that you may have now.

Hi, everyone. Thanks for joining our webinar today. During the course of the webinar, we received some fantastic questions. So I'm going to go ahead and read some of the questions that we received, and Navyaa, your presenter, will go ahead and provide some additional insight and answers. As a reminder, as we go through this Q&A, it is still open. So please feel free to submit any additional questions that you would like to have answered. So Navyaa, great job. I'll start off with this question. On autostart tasks, how flexible is the system in terms of customizing these tasks based on specific debugging needs or scenarios? Could you go into more detail on some of the options for configuring autostart tasks?

That's a great question. Autostart tasks are completely flexible. There are basically many bash scripts so you can change everything. You can change nothing. It completely depends on what you're trying to do. Currently, we have 5 different prefilled ones. So one is for starting a CUDA-GDB server on a local host if you're testing or whatever. For Linux L4T Targets, we have 1 scenario that just instantiates a CUDA-GDB server, and we have another 1 that copies over the executable binary and then instantiates the CUDA-GDB server. For QNX, we also have 2 different autostart tasks, one where we start a CUDA-GDB server and one where we copy over a CUDA-GDB server and then we started. And for the QNX scenario, we don't have a autostart task where we copy over the executable binary. And that's because on QNX, we do not need to specify the executable binary when starting a remote debugging session.

Great. Our second question is, since we discussed DRIVE OS and QNX debugging separately, I'm wondering how Nsight VSCE handles cross-platform debugging for applications built on multiple targets.

Right. So as long as you have a program that has been cross-compiled with debug symbols, you should be able to reproduce these across the board, whether it's Linux, whether it's L4T, whether it's QNX, if you cross-compile correctly, you use your debug symbols any system that you can successfully SSH to, you should be able to set up debugging in a very similar fashion.

Our next question is, on the importance of ensuring that CUDA-GDB and CUDA-GDB server are from the same CUDA version for remote debugging. Can you explain the best way for us to ensure that there isn't a mismatch? And any details on what could fail if there is one?

Okay, yes. So one of the things that I would say is the general rule of thumb is to use CUDA-GDB and CUDA-GDB server from the same toolkit. So if you navigate to wherever you have the CUDA Toolkit installed in your system, which is I think by default is user local CUDA. And you pick out the same -- if you pick up CUDA-GDB and CUDA-GDB server from the same CUDA version, you should be totally fine. If there is a version mismatch for whatever reason, if you have like multiple CUDA versions installed or you specified like an absolute path to 1 CUDA-GDB and you have the different CUDA-GDB server, it might work. It's -- There is -- we've seen weird errors come up in the past, but we will definitely not have access to 100% of the debugging features. But it might seem like it's working. So making sure that you're using CUDA-GDB and CUDA-GDB server from the same CUDA version, very important. Because those errors when they show up, they manifest in different ways. You'd think something is wrong with the stack frame or I've seen different errors saying, the breakpoint hasn't been able to set or you would think that the debug symbols haven't been loaded properly, but it has to do with a mismatched version. But those are hard problems to debug. So I definitely recommend being extra sure about that.

All right. We have another question. Could you go into more detail on what attributes are we able to specify in the launch JSON?

Right. So the launch JSON is important if you are running any debugging session on VS Code. So it starts with stuff that is essential like -- stuff like the executable binary or the debug -- like the paths to CUDA-GDB, that is always required even on a local debugging session. On a remote debugging session, stuff that's required that you would put in the launch.json is the host, the port of where the CUDA-GDB server is running and other information that you want to be able to successfully connect to the CUDA-GDB server with. However, the launch.json is supercool, it allows you to do all these extra things like you can set [indiscernible] launch, to true or false, you can hide API errors ignore them or stop them. You can specify the current working directory. You can specify args or -- and it commands, which will get past to CUDA-GDB. We have more exhaustive list with the documentation. But basically, any setting that you would be able to do on CUDA-GDB, you should be able to specify with the launch.json. But like I said, there is an exhaustive list in the documentation, which also gives examples. And then one thing that I will mention is that you can specify the environment in the launch.json. I feel like that is a very cool thing. It's something that we recently added. And I don't know if you remember but you did see it in the demo. I think that, that can be super useful as well.