NVIDIA Corporation (NVDA) Earnings Call Transcript & Summary

Hi, everyone. Thank you so much for joining seminar. I am Navyaa. I'm an engineer on developer tools team here at NVIDIA, and I'm going to be hosting the session today. The goal of the seminar is to go over Remote Debugging support in Nsight Visual Studio Code Edition. In this webinar, we will go over Remote Debugging in three scenarios: first, where the target is a DRIVE OS Linux board; the second where the target is a Blackberry QNX board; and the last where the target is a Blackberry QNX board, but we set it up using containers. In all these cases, we cross compile our program on the host, but we run and debug it on the target. I will cover all the scenarios for which our extension Nsight VS Code Edition implement support. And other than those, I will spend some time talking about some of the scenarios covered by the support Microsoft provides. I will also briefly talk about how our extension's remote support is different from that provided by Microsoft and which might be better suited for your purpose. For regular remote debugging, where we develop, run and debug on the same machine, it's best to use Microsoft's official SSH extension. The SSH extension can be found on the marketplace. I will not go into too much detail here. Microsoft does provide awesome documentation that goes over how to set this up. The best part about this is that once everything is set up, you are able to develop, run and debug on the remote machine almost as if you're doing it locally. Further case where you would like to develop on one machine and run a debug on another machine, Nsight VS Code's remote support is best suited. This becomes especially useful in cases where it is hard to directly develop on the machine where you want to run, for instance, Blackberry QNX boards or Jetson devices. In this webinar, I will demonstrate debugging scenarios using devices that run drive OS. My aim is to provide practical insights into debugging within the drive OS environments and to augment developer workflows. Rest assure, the debugging concepts are universally applicable and the principles can be adapted to other systems. At this point, I assume everyone has Nsight VS Code Edition installed. If not, it's on the VS Code marketplace and please install it because everything I speak about from here on out would be based on the assumption that you have the extension installed. I will be demonstrating how to do this using Nsight VS Code Edition, but for those of you who've in the past used CUDA-GDB remote debugging, this is equivalent to running CUDA-GDB locally and connecting to a CUDA-GDB server running on the target we want to debug on. Similar to how it would be if you were using the command line interface, there are two main components to be able to run a remote debug session. First is instantiating CUDA-GDB server on the target machine. And the second is connecting to it from the host. For instantiating CUDA-GDB server, you could log into the host machine and start CUDA-GDB server on the command line, and that would work just fine. However, we have implemented VS Code autostart task to automate this process and make it easier for users. There are a handful of VS Code tasks that we have predefined for various scenarios. You could use them as is or you could easily customize them based on your requirements. To see the autostart tasks, you would open or create a tasks.json in the .vscode folder of the program or project you have opened in the VS Code workspace. Then you would open the command pilot by pressing control-shift-P. Once the command pilot opens, you would search tasks and in there you would select to configure Default Build Task. Once you select that, you will have a few pre-configured tasks show up. We have three tasks for Drive OS Linux targets and 2 for QNX targets. I will go over the QNX autostart tasks in the next section. Three tasks that are useful here, are Nsight autostart local host. This task will automatically start a CUDA-GDB server on the local host, the second is Nsight autostart remote. This task will start a CUDA-GDB server on the remote host. And the last is Nsight autostart secure copy executable binary remote. This task will secure copy the latest binary of the executable under the target and then start CUDA-GDB server on the Linux target. For the purposes of the demo I'm about to give, I will be using the third option. I'm going to demo debugging using Nsight VS Code with a CUDA sample [ sector ad ], which can be found on GitHub. I'm going to cross-compile this locally. One thing to keep in mind when cross-compiling for debugging is to always dealt with dbg=1. This builds with debug symbols. Also, when remote debugging, you want to make sure that CUDA-GDB and CUDA-GDB server are from the same CUDA version. For this demo, I'm using an autostart task to copy over the executable and start CUDA-GDB server on a remote machine. I'm going to edit the command based on how I've set up as I switch to the remote machine. As you can probably tell, this task will copy the binary and start CUDA-GDB server. For the variables that show up, this can fake code and variable names. You could replace these in the tasks.json or fill out the values in settings.json as I have done. The merit to defining these in settings.json is that you can then also use them in launch.json. Variables like host, port, executable are needed in both places. So this ends up being super convenient. Before we start CUDA-GDB server, I will fill out launch.json with all the information required to connect to the CUDA-GDB server running remotely. We need to specify the host and the port that CUDA-GDB server is running on, we also need to point to the executable binary that we want to debug. And for faster remote debugging, you also may want to set its route or solve search path. Moving on, I will actually start the task by pressing control-shift-B, the task copy is over the executable binary and starts a CUDA-GDB server on the remote machine. You can see this is a terminal that the CUDA-GDB server is now running on the remote machine. Through this, I'm going to launch the debugging session that will connects to the CUDA-GDB server running on the target and we'll hit the few breakpoints. You can see that we're connected to the remote machine. On the remote machine, it shows that we are debugging using a different system and this further confirms that we are connected. And you see that the big points worked just like they would work in a local debug session. We can continue. We can see values for locals. We can step over. We could keep up just exactly like we would in a local session except for the fact that here we're actually running and debugging on a remote machine. Now we move on to the case where the remote device is a QNX board, and we connect to it using a Linux client. Just like in the previous case, there are two main components to be able to run a remote debug session. First is instantiating CUDA-GDB server on the target machine and second is connecting to it from the host. If you were able to directly connect to the QNX board, that is perfect. However, with the setup that I'm using, I need to connect to a Linux host before I can connect to the QNX board. This is because the QNX device that I use can only be accessed using a specific host. So I have an extra layer here. I connected the port's host using VS Code's Remote Support. Once I am connected to the correct host, I use Nsight Visual Studio Code's internal remote support to connect to CUDA-GDB server, which is running on the QNX device. While setting up, I had to install certain additional libraries to access debugging tools, depending on how you set up drive OS on your system, what you need to install specifically will differ. I know most people use a container to set up drive OS, and we will address that in the next section. For now, if you're setting up drive OS without using the container, these are the additional libraries that will give you debugging tools you will need. Similar to the previous case, we have implemented VS Code autostart tasks to automate the process of remotely instantiating a CUDA-GDB server. To see the autostart tasks, you would open or create a tasks.json in the .vscode folder of the project or program you have opened in your workspace. And you open the Command Pilot by pressing control-shift-P. Once the Command Pilot opens, search tasks. And in there, you will select configure default build task. You can see that we have two tasks for QNX targets, Nsight Autostart Remote QNX. This task will start a CUDA-GDB server on the QNX target. The second task we have is Nsight Autostart Secure Copy CUDA-GDB Server Remote QNX. This task will secure copy of CUDA-GDB server binary onto the target and then start a CUDA-GDB server on the QNX device. For the purposes of this demo, I will be using the second option. I will be copying over a CUDA-GDB server binary and then instantiating it. When starting CUDA-GDB server on a QNX target, we don't have to specify the executable binary. So we do not need that in this case unlike when we're trying to run on a Linux board. Before we start the demo, I just wanted to share where I got my build instructions from. It's not as simple as saying make dbg=1. So I'm adding a link to the documentation which you can access. It goes into more details about cross-compiling for a QNX device. Moving on to the demo. I am debugging using Nsight VS Code with a CUDA sample matrix model, which can be found on GitHub. We will start by cross-compiling this locally. The cross-compiling is using the same instructions that I just spoke about on the last slide before the demo started. Again, with remote debugging, you want to make sure that CUDA-GDB and CUDA-GDB server are from the same CUDA version. For the demo, I'm using an autostart task to copy over the CUDA-GDB server binary and start it on the remote machine. This time around, I don't need to edit the command. Also, I prefilled my settings.json with all the information needed to run the task. Same as in the previous section, the variables that show up, as config called in variable names, you could replace these in the tasks.json directly or fill these out in the settings.json, as I have done. Variable values like host, port, executable are needed in both the launch.json and settings.json, so this ends up being really useful, especially if you want to edit the port value or the executable value later on. Then, I will show you the launch.json, which is all the information required to connect to the CUDA-GDB server running remotely. Same as the previous case, we need to provide the host and port that we want to connect to. We also need to point to the executable binary that we want to debug. As to remote debugging, we may also want to set this route or sole at search path. After this, I make sure we have a CPU breakpoint and a GPU breakpoint set. And I actually start the autostart task by pressing control-shift-B. This task copies over the CUDA-GDB server binary and then instantiates it on the QNX device. And I launched the debugging session locally that will connect to the CUDA-GDB server running on the target, and we hit a few breakpoints. You can see that we're connected to the remote machine, you can see that the break points work like they would in a local debug session. We can continue to see values for locals, step over and debug exactly like we would in a local session, except that here the actual running and debugging is happening on the remote board. In this case, the first big point we had was a CPU breakpoint and then a GPU breakpoint. Again, locals work, views work, they update as you go, everything, the exact same as a local debugging session, except that you're actually running on the remote QNX device. In this last part of the webinar, we will go over remote debugging on the QNX target, with the difference being that this time I set up drive OS using a container. For this demo, I flashed my QNX board with drive OS 6.0.8.1, and I use cuda-safe-11.4. For me, the toolkit was installed and ready to use in the tool chain that I used to set up the container. I have also linked to the documentation that I used to set up. There is similar documentation for Linux QNX and different versions of Drive OS, too. We will use Microsoft's extension, called Dev Containers, to set up our environment in a container, and then we will use Nsight Visual Studio Code's Internal Remote Support to set up a debugging session. Just like in the previous cases, there are 2 main components to be able to run in remote debug session. First is instantiating CUDA-GDB server on the target machine. The second is connecting to it from the host. If you're able to directly connect to the QNX board, that is perfect. However, with the setup that I'm using, I need to connect to a Linux host before I can connect to the QNX Board. This is because the QNX device that I'm using can only be accessed using a specific host. So my container exists on the host that I need to connect to the board from. I connect to the board's host using VS Code's Remote Support. Once I'm connected to the correct host, I use Nsight Visual Studio Code's internal remote support to connect to CUDA-GDB server. Similar to the previous cases, we have VS Code Autostart tasks to automate the process of remotely instantiating CUDA-GDB server to make it easier for users. To see the Autostart tasks, you would open or create a tasks.json in the .vscode folder of the program or project that you have in your VS Code workspace. Then you open the Command Pilot by pressing control-shift-P together. Once the Command Pilot opens, search tasks. And in there, you can select, configure default build task. You can now see that there are 2 QNX tasks for QNX targets, Nsight Autostart, 2 remote QNX. This task will start a CUDA-GDB server on the QNX target. Nsight Autostart secure copy CUDA-GDB server remote QNX. This task will secure copy a CUDA-GDB server binary onto the target and then start CUDA-GDB server on the QNX Board. For the purposes of this demo, I will be using the second option. I will be copying over a CUDA-GDB server binary and then instantiating CUDA-GDB server. When starting CUDA-GDB server on the QNX target, we don't have to specify the executable binary. So we don't have that extra case, unlike when we're trying to do that on a Linux board. Before we start the demo, I just want to share where I got my builds instructions from. Again, it's not a simple as saying make dbg=1, so I'm adding a link to the documentation, which you can access. Moving on to the demo, I want to start by showing my devcontainer.json. I use Microsoft's container support and have their dev containers extension installed. After that, I set up my container with the Drive OS image, the correct [ count ] points, environment variables and the VS Code extensions that I want inside the container. Once I set it up fully, I open the Command Pilot using control-shift-P and reopen this workspace in a container. The extension takes care of starting the container. And if we have everything correctly set up, the workspace will then open up in a running container. I am now going to debug a CUDA sample matrix model using Nsight VS Code Edition. I start by cross-compiling the sample. I used the same command that I talked about just before the demo started. Note that I prefilled the launch.json, which is all the information required to connect to the CUDA-GDB server running remotely. Same as in the previous cases, we need to provide the host and the port that we want to connect to. We also need to point to the executable binary that we want to debug. And on Drive OS QNX systems, we want to set this tool and solid search path. I also prefilled my settings.json with the information needed. Same as in the previous sections for the variables that show up this can fit colon variable names, you could replace those in the tasks.json and launch.json or fill the values out in the settings.json, as I have done. Variable values like host, port, executable are needed in both launch.json and tasks.json, so this ends up being really useful. In my tasks.json, you can see that I will be using an Autostart task to copy over the CUDA-GDB server binary and started on the remote machine. This time again, I do not need to edit the command. After this, I make sure we have a GPU breakpoint and a CPU breakpoint set. Moving on, I actually start -- I want to start task by pressing control-shift-B. The task copies over the CUDA-GDB server binary and starts it on the remote machine. Then, I launched a debugging session that will connect to the CUDA-GDB server running on the target and hit a few breakpoints. You can see now that we are connected to the remote machine. You see that the breakpoints work like they would in a local debugging session. We can hit continue. We can see values for locals. We can step over and debug just like we would in a local session except that here we are actually running and debugging on the remote board. In this case, the first breakpoint that we had was a CPU breakpoint and then a GPU breakpoint. Again, locals work, views work, and the update is [ you go ]. Everything is the same as a local debugging session, except the fact that you're actually running on the remote machine. To wrap up, here are some links to get the extension to go through documentation and to reach us with questions and/or feedback. For questions, I recommend that you send them via GitHub Issues or the NVIDIA forums, that is the quickest way to reach us. And I hope this session was informative. Whether or not you're using Drive OS, I hope you learn debugging tips that you could use. We are here and open to any questions that you may have now.

Thank you, everyone, for tuning into the webinar today. At this point, we'll be moving on to the Q&A segment. We have some questions to kick things off for the presenter Navyaa to answer. While she's doing that, we encourage everyone to please continue submitting questions via the chat box on your screen. I'd also like to point you to the resources box on your screen. This will have some resources for you to learn more and download Nsight Visual Studio Code Edition, as well as Drive OS and Blackberry QNX. So without further ado, I think we'll jump into our first question here. Like I said, please continue to add more questions into the chat box, and we'll get to them. So Navyaa, the first one here, on Autostart tasks, how flexible is the system in terms of customizing these tasks based on specific debugging needs or scenarios? Could you go into more details on some of the options for configuring Autostart tasks?

Sure. So Autostart tasks are super flexible. There are basically many task scripts that we've prefilled based on existing workflows that people use for debugging CUDA-GDB. And so you can edit them entirely, and you cannot touch them at all, and they should still work. And currently, we have 5 prefilled tasks. The first one will start a CUDA-GDB server on the local host. The second one is for starting a CUDA-GDB server on Linux or L4T target. The third one copies over the latest executable binary and then starts CUDA-GDB server on a Linux or L4T target. The fourth one is for instantiating CUDA-GDB server on a QNX target. And the last one copies over a CUDA-GDB server binary and then starts CUDA-GDB server on the remote target. The last one is made that way because in QNX we don't need to specify the executable binary when starting the CUDA-GDB server. This is very different from Linux or L4T where we always have to specify the executable binary before we can connect to a running CUDA-GDB server.

Excellent. The next question here, since you discussed Drive OS and QNX debugging separately, I'm wondering how Nsight Visual Studio Code Edition handles cross-platform debugging for applications built for multiple targets.

As long as the program that you want to debug has been cross-compiled correctly for the platforms you're hoping to debug on, and as long as the platform that you're hoping to run on is accessible to you via SSH. You should be able to use the exact same steps to debug on a Linux device, L4T, QNX. The debugging principles and the steps to set it up remain unchanged. It's just that you always have to cross-compile and have CUDA-GDB and CUDA-GDB server set up currently.

That's great. That actually segues nicely into the next question, which is on the importance of ensuring that CUDA-GDB and CUDA-GDB server are from the same CUDA version for remote debugging, can you explain the best way for us to ensure that there isn't a mismatch? And any details on what could fail if there is?

Normally, I would recommend using CUDA-GDB and CUDA-GDB server from the exact same CUDA toolkit. As long as you're downloading the CUDA toolkit directly or using a tool chain to flash, our target device, you should be totally fine. If you're manually cropping them for whatever reason, you need to be extra careful because sometimes when CUDA-GDB and CUDA-GDB server are from a different version, you'll see a lot of errors like debug symbols not loading properly, or breakpoints being unable to set. And those errors can be pretty hard to debug because they give little information on the root cause. And so that is just something that I would be extra, extra careful about. You can navigate to user local bin or wherever else you have your CUDA toolkit installed and double-check in there.

Great. Our next question. Could you go into more details on what attributes you're able to specify in the launch.json?

Right. So the launch.json is for specifying attributes, some of which are necessary for being able to start a debugging session and some of which are optional but can really help debugging. So I'll start with the ones that are necessary for a local debugging session. So if you're not doing remote development, you probably want to always specify the executable binary path to CUDA-GDB, for whatever CUDA-GDB it is that you want to use. For remote debugging, you also have parameters like host, port, and you also probably want to set [indiscernible] search path just -- because it's a lot faster that way. And then moving on to the optional parameters, you can always at current working directory, break on launch, you can specify arguments to be passed to CUDA-GDB before the interior is launched, you can do that with OS, you can do that with edit commands. And then you can also set environment variables using the environment parameter -- attribute and the launch.json. And there is a more exhaustive list, this is actually -- there's more that you can specify and the exhaustive list is available in our documentation. I think that's all.

Great. Yes, links to documentation should be in the resources as well. It actually has a related question. For attributes like sysroot or rx, how do you specify multiple values in the launch.json?

So for sysroot and rx specifically, you can do that by separating the multiple values using semicolons. But as a general rule of thumb, you would want to be consistent with whatever it is that you do in CUDA-GDB. So in this case, as you can -- I don't know if you know, but even if you were doing this on CUDA-GDB, you would do it by separating the multiple values with semicolon. So we make a very conscious effort to stay as consistent as possible with CUDA-GDB and give users the same experience. So yes, as a rule of thumb, you would do that and that is how we support it.