NVIDIA Corporation (NVDA) Earnings Call Transcript & Summary

Hi, everyone, and thank you for joining the NVIDIA VCR Webinar. My name is Sheri Nimtz, and I'm the Global Alliance Manager on NVIDIA's XR team. Before we begin, I want to cover a few housekeeping items. At the bottom of your screen, you can find various widgets for use in your event. Once open on the screen, they are resizable and movable. [Operator Instructions] Attempting to precisely repeat a user's VR experience is time-consuming, if not impossible. At NVIDIA, we learned this early on from our own VR performance tests and creating marketing content. We needed a way to easily capture and play back VR sessions. So we created NVIDIA VCR to make replaying VR sessions both accurate and painless. And during our webinar session today, we'd like to review how NVIDIA VCR can help you with your VR experiences and development. I'd like to share with you how we and a few of our partners use NVIDIA VCR to improve end user productivity and how developers extend NVIDIA VCR features. We will start today's session with Ingo Esser, who created NVIDIA VCR and will share how VCR is used by the end user as well as the developers; followed by our partners, Autodesk and Agile Lens, who used NVIDIA VCR to enhance workflows in their applications in the automotive and media and entertainment spaces. And then we will close out the webinar today with a brief Q&A, where Danny Tierney from Autodesk and Eytan Manor from Agile Lens will also join us for our live chat session. So Ingo, the floor is yours.

Thank you, Sheri. Hi, everyone. My name is Ingo Esser, and I'd like to talk to you about NVIDIA VCR, our VR Capture and Replay tool, what it is, how it works and what it can do for you. Here's a preview of what you will be able to do with VCR. What you see here is a collaborative VR session in Autodesk VRED with 4 VR participants. What's special about this is that these participants are all acting through a VCR replay so that no HMD or even humans are involved in this session. Let me first start with why we created VCR. While working on VR performance data gathering, we realized there's a few issues with VR. It is quite impossible to repeat a motion perfectly multiple times. You can see me trying to recreate a triangle over and over here, and they never really match. So this makes the data gathered through manual VR perf testing hard to compare. Manual testing also doesn't scale well. Testing a configuration matrix with many dimensions takes days or weeks. And if you need multiple people, you need multiple HMDs. And HMDs are a rare and costly resource. It would be good to not need the hardware while you're testing. The first test we were doing, we did with emulated HMDs and app automation. SteamVR has a null driver that can show a desktop image, so that removed the need for an HMD. Some apps can also be scripted in Python, which is a great start, but that wasn't general enough, so we developed VCR. The capture component collects hardware information and input data and stores all of this while the users are interacting. And the replay component uses this data to emulate a virtual HMD and replay the input data, recreating the complete VR session perfectly. You can see this in this video on the site now. This is VCR replay going and overdrawing and overdrawing a triangle perfectly multiple times, so doing a lot better than me. The basic components of VCR are the capture and the replay component. The capture component is an OpenVR background app that runs while a user is interacting with an application through SteamVR. While it starts up, it stores the properties of the HMD and the controllers that are connected to a hardware properties file. It then continues by logging the motion and the buttons and any controller movement into a tracking file, so creating a pair of hardware properties and tracking data. The replay component is an OpenVR driver that emulates in HMD and controller -- and the controllers using the properties file that we collected before. In this stage, we do not need any hardware already. So we disconnect our hardware, we install the VCR replay driver, and that reads the hardware properties file to emulate the same hardware that we've used to record the whole session. Once everything has started up, it reads the tracking data and replays the motions and the buttons and everything. Maybe it's best to explain how VCR works by showing a short demo of what the capture and the replay component are doing. So here, we have the start of a capture session. So I have an HMD on my head, I have the controllers in my hand, and the capture session has already started. It has iterated through the hardware that is connected through SteamVR, collected the data of the hardware that is connected. It has written all of that into the hardware file already. And it has shown a little pop-up here in the bottom of my view saying that it's starting the VCR capture. And with that, I know I can now start interacting with the scene because anything from that point onward is going to be logged into the tracking data file. So let me start the video now. And here, I am going to start interacting with this application by choosing a tool, choosing a color and then drawing something into space here. And VCR is logging all of my head motion, controller motion and all the button inputs and all of that. Now the session is over, I get a little pop-up that VCR is stopping the recording. I'm quitting the app and SteamVR. I'm unplugging the HMD. Then I'm installing the VCR driver. And now what's popping up is VCR replay, reading the hardware file and emulating the HMD and the controllers. And then I tell VCR to start replaying what I recorded before. And it's taking the tracking data file and recreating the VR session that I recorded before the same way that it has happened before. Just to show that it's recreating the data quite perfectly, I'm replaying the same interaction again to show that it's quite nicely matching the previous replay and the -- that the image is sort of the same again. While creating VCR recordings, we noticed that it's sometimes hard to perform complex tasks perfectly so that they can be replayed later. For me, this happened very often. I push a wrong button, move somewhere wrong, just mix something up, and these errors could make the whole recording useless or at least inefficient because they would be in there, and me fixing these errors would also be in the recording. The idea of segmentation is that more complex tasks can be broken into a -- smaller, simpler tasks. This way, the user can make error and just start another segment and do the same motion again and just later skip this segment but while replaying. While capturing a VR session, there is either a keyboard key or a configurable controller button to tell VCR to create a new segment. And these segments later then can be fed to the replay driver in any order that's needed. And that's something that can be useful also for automation or testing because if you were, for example, working on a specific feature, you can create some segments that get you to the right place in your scene and then have some segments that execute or rather run through the codes of the feature that you're working on. And you can rerun these segments, running your code over and over, so you can work on that nicely. To explain this segmentation feature a little bit better, I'd like to show another demo. So here, first, we have a video of me creating segments of different teleporting actions, drawing some letters into the air, segments of choosing and then putting away tools and choosing colors and things like this and all of these different segments are collected by VCR. You can see that there's a little pop-up coming up every now and then when a new segment is generated. So that shows you that VCR is doing what you want it to do. And I've used the left controller button, which I am not using in this interaction or in this recording at all, to create the segment, so I don't have to reach for a keyboard all the time. Here's a little segment, sort of looking around in the scene, looking at the result of what I was doing. And then I also created a handful of segments of some more letters because while I was recording the first set of letters, I made a few mistakes, so I'll just have some backups so that I can be sure that everything is fine. Now we have a collection of segments, letters, teleportation, all of these different things, and we could replay those manually to create the result that we would like. But that is tedious and can actually be quite error prone. I've tried making this work for this recording here, but I've made so many mistakes that I just ended up writing a script for it. VCR helps with this automation by generating flag files to report its status. There's the ready flag that it's a file that the script can look for that shows that VCR replay is idle. So the replay driver is waiting for a tracking file and is not doing anything right now. And then there's a busy file, a busy flag file that shows that VCR has picked up a tracking file and is busy replaying it. So a script to iterate through a handful of segments now would be -- for example, that you -- for every item in your segment list, first wait for VCR to be ready with the ready flag, then feed it that item and then wait for a busy flag so that you know that you -- that VCR has picked up that item that you gave it to. And then just wait for the ready flag again to see that this item has finished playing before you continue. There are other configuration options. For example, the replay speed, which is also something that we can see in the next section or in the next video. And there's a lot of other parameters that you can configure VCR with. Please refer to the documentation for those. This would be a little bit too much to mention right now. In the video here now, you can see I've written the script to go through all of these different segments. I've also configured VCR to increase the speed of replaying to, I think, 3x and it's replaying the drawing actions for all of these different letters in a different order and it does that in quite a higher speed actually. And then it replaced some action of turning and going to a place where we can see, and then the script actually configures VCR to go slower, half the speed and then replays this looking around action, showing the results of what we sort of replayed. And we can see that we can use segments in any kind of order we want. We just need to know what they're doing to get to the right result that we need to have. The VCR package also contains a filtering sample. If you need to modify the data that you have created within a capture session, this is possible through this filtering sample. It's available in full source code to modify and build. And currently, it shows how to read, modify and write the data by just averaging the position of data. The sample lives in the sample folder. And the function that is most important for that is the filter tracking data function, which is collecting some positional data around an item and then just averages that to sort of smooth out the motion, sort of take away the high-frequency components of motion. This filter feature can be used for a lot of different use cases. For example, if you want to record a VR video, so record a video of somebody going through VR so said -- that somebody else can later watch it, you could smooth the movement of the head motion, for example, to make it nicer to look at. You could modify the input by adding or removing controller interactions at certain places in time. It's possible to combine or split tracking files. If you have a recording that is -- that contains some garbage or you forgot to segment something, you could do actually a filter that splits at certain times or things like this, or you could modify the replay speeds inside of the tracking file. So you have some action going to a certain place, then you would like to have a slow-mo of some place, you could do that within the tracking file by modifying it through a filter. To show the filter in action, I am first recording a very wiggly interaction, which has a high-frequency component, these wiggles, and a low-frequency component that's these sweeping motions back and forth. And then feeding all of that through the filter shows the following video. So you can see that the high-frequency component is taken away pretty nicely, while the low-frequency component is quite nicely preserved. So this is what you would like if you, for example, want to filter head motion to make it a little nicer for a VR video. As my last example or last video, I would like to show something that I did in VCR. Here, you see me creating segments of different actions in VCR, and I'm recording segments for hopping to a certain viewpoint. And then for that viewpoint, I am creating interactions like opening and closing doors or waving to other viewpoints and things like that, just sort of all fitting to the viewpoint that I hopped on before. I have now pairs of teleport and interact files. I've distributed these pairs to multiple systems and set up a collaboration session within Autodesk VRED. The participants or the systems that are running these participants are actually cloud systems. I've set up AWS instances for this, which don't have HMDs connected. And each of these participants now first perform their teleport and then loop their according interaction. Putting this all together, we get the following video. Each of those VCR participants are looping through their respective interaction segments. And they are opening and closing doors or waving to each other and into the camera. And because these segments are differently long, the resulting interactions are changing. If you'd like to synchronize that, if you'd sort of like to orchestrate a synchronized action on all of these different systems, you could use the VCR busy and ready flags to make sure that all of these different participants are synchronized in some way. We've discussed a lot of different VCR features. The capture and the replay components are probably the most important ones. We can segment the data. We can filter the tracking data. And VCR itself is quite configurable. Please refer to the manual for more information on that. VCR can be applied in many use cases. I've talked about VR development. If you want to code a VR app, and you need to write your code, compile it, go into the HMD, test your code by going to a certain place, then testing your feature and then maybe going in and out of VR or in and out of your HMD over and over just to check your code for correctness. This can be very tedious, and it can actually break flow pretty badly, and so VCR can help you nicely by shortening this, this [ code debox cycle ]. You write your code, you run it, VCR takes over and then gets you to the right place and then executes your code. And in that time, you can look at the data running through your developer environment and you can see what's happening. And if VCR can help you, yourself with this shortening the [ code debox cycle ], it can also serve a team of developers. So every developer could get an HMD once, record their VCR session for whatever code they are working on, pass the HMD on to their colleague and then not need an HMD for a long time while they're working on the code that they're working on. QA can use this really nicely. Basic functionality testing for automated QA can be really well done with VCR. But also if a bug appears, the QA team can create reproducer for engineering using VCR and just pass the VCR recording to the engineering team who don't need an HMD probably then to look at the code that is buggy, look at the bug and fix it and then just pass the bug back to the QA team, which also uses VCR recording to verify the bug fix. And the motivation for this whole VCR tool, the performance work, obviously, is this can be covered really nicely. So you can have big configuration matrices using VCR replay. and anything that you create this way going through different hardware configurations, going through different software settings and things like that, that all produces nicely comparable results because the recordings are always the same. In the next 2 sections, Alex and Marc are going to also show a lot of other different use cases for VCR. And a handful of them, we didn't really anticipate, and it was really nice to see how VCR is getting used in different other situations. You can get VCR by going to the NVIDIA developer site and applying for early access. Feedback is very welcome. So thank you very much, and now Alex is going to show how the Autodesk VRED team is using VCR to optimize their workflows.

Thank you, Ingo. Hi. My name is Marc Arends, and I'm working as a software engineer at Autodesk on the VRED product line. Today, I want to show you how we use NVIDIA's VCR together with VRED. For this, I will give you a brief introduction to what VRED is. And after that, I will show you how VCR changed the way we debug our software and how it can be used to create presentations with VRED. I will also give you a short outlook of how we plan to use VCR in the future. So what is VRED? Well, it's a tool for product visualization, which is mostly used in the automotive industry, but not exclusively, of course. It imports a variety of CD and other data formats and allows the user to create and edit scenes that include different variants of a product. This is helpful to review all the different configurations of a car, for example, like the different car paints or the different materials that are used in the interior. And VRED comes in 2 versions, VRED Professional and VRED Design with different feature sets. And there are also other related VRED products, but these 2 are the ones we usually work with to create and edit the scene. And VRED allows to view a scene in virtual and mixed reality also. And this is also possible with multiple users over the network, so you can collaborate in VR. And of course, we read this also a classic desktop application that can render high-fidelity images and videos. And the user can choose different renderers for that. There is an OpenGL rasterizer for real-time purposes. We have a CPU ray tracer. And in VRED Professional, there's also a GPU ray tracer. Reviewing a scene in VR requires some tools for the user, and it must be possible to navigate the scene, point at things. It must be easy to change the different variance of a reviewed object. Otherwise, an additional operator would be needed on a PC. And VRED already comes with a set of VR tools for the user, and this include a teleport tool, a pointer and the menu for even more tools and all the options you can change in the scene. And in this video, you can see our teleport tool, and it is possible to turn your armrest to choose the orientation after the teleport and see it here with the turning green arrow. And you also have a pointer that I mentioned before, that can be seen by other users. And it can also be used to interact with the scene. In this case, you can open the cockpit of this car. And here, you see the VR menu that has some other tools available and also has the options for switching the variants of the car or jumping to a different viewpoint. So this is the out-of-the-box experience we offer to the VR user. Developing these tools included a lot of testing and debugging. Here, you can see our old debugging loop. It consists of putting the headset on, running the software, hitting a breakpoint, putting the headset off, finding the error, changing some code and repeating that. In fact, our scenario was even worse. Most of our VR equipment is located in a special VR lab, and we have our development machines in our office. And every time you change something or recompile a code, you need to move between office and VR lab and this takes a lot of time. And as I mentioned, multiple users can collaborate in VR and VRED. And debugging sessions with multiple users requires hardware for every user, and you need help from other developers and of course, their time and which makes case even more complex to test and to debug. And when we developed these tools, we didn't have VCR yet. But with VCR, we have a new debugging loop, and it will look like this. In testing software, hit a breakpoint, look for the error and change the code and repeat that. No more headsets putting on and putting the headset off again, or walking between offices. This, of course, requires that you record one of your VR sessions in advance to but being able to just play back a session without actual VR hardware saves a lot of time. And actually, we have already made use of this advantage. We came to a point where we needed to revisit our VR tools to prepare the code for future features. This required some refactoring, which means, in our case, splitting up the code, reorganizing and simplifying it. but it should still work the same way as before. So the user should not see a difference at all. So I recorded how I use these tools in VR and could compare the playback in the new and the old code. The case with multiple users is also much easier to debug now. You can just run a recording on different machines that are connected. And for every user, you just need to run the playback. There is no need for VR hardware anymore and no need for the time of your colleagues. So you can -- one person can test this. So that's really time and money saving. And in this video, you can see how the same recording is replayed in 2 different versions of VRED that have different implementations of the teleport tool. On the left is the original code. And on your right, the new one but with a bug. When refactoring our code, we could see that the function, it allows us to adjust the view direction after the teleport was not called correctly. You can directly see the difference. You can see that on your left side, the arrow that indicates the orientation is not -- is moving. And on the right side, it is not. And debugging this was easy now. We could just set some breakpoints and step through the function while the playback was running, and all of this without the headset. And we fixed the issue in no time. We compiled the code, started software and the replay again, and then we could see that the issue was gone. And VCR did not only prove to be helpful for development tasks, it can also be used for presentations in VRED. You can, for example, record a VR session to get the natural camera movement around an object, which can then be replayed by desktop users that might not want to wear a VR headset. This way, you have an immersive camera without the need to animate it by hand. This kind of presentation is also much more dynamic as you can still change details in your scene like the car paint or the environment, and a simple view recording would be much more static in this case. And the replay can also be used on a much more capable machine with much more capital hardware. For example, hardware that allows real-time ray tracing to improve the visual quality of your presentation. And in this video, you'll see a recording that I did at home, which I sent to my colleague in Munich. Starts -- this is basically starting with my VR recording. But my colleague replayed the recording and used ray tracing to improve the visual quality. He figured out the right balance between quality and performance and could then review different variants and Full GI to get an impression how shadows and light work together with this car model. And you can see the clay model here and how the shadows are casted on a clay model. You see the current -- a nice environment and some air streams to see the aerodynamics. Here, the rims are reviewed, and it's -- you can see how the shadows are cast by the rims. Here's some ranges of the day time. You can see the car in different lighting conditions. Here are the animated airstreams, and you can see how they reflect on the car paints, thanks to ray tracing. And a small look at the cockpit. And when I recorded this, I didn't change anything on the car or in the scene. I just moved around it. And my colleague with all the switches of the environment, the car paint, he activated this air flow, et cetera. And he did all of this during playback. So he was able to decide for himself which aspect of the car he wanted to see. And as I said, this is a nice benefit you do not have with a regular video capture because you can alternate what you want to see a bit here with this kind of recording. Yes, I hope you now have an idea how we use VCR at Autodesk to develop and use VRED. I showed you how we develop this usage for improving our workflow, and I also showed you an idea how users can make use of VCR and VRED. But we also have other use cases for the future in our minds, so we think that VCR can be very useful for our QA department because they have a lot of testing to do. For every release, they need to check if everything still works, and this includes all of our VR tools and it also includes different VR headsets and controllers. And if we can now record ground truth data for them for every VR tool that we implemented and for different VR headsets and controllers they can use it to run their tests without using the actual VR hardware, and they can also automate their testing to some degree, which again saves a lot of time and money for them. Yes, that's it from me. Thank you for listening. If you have any questions left, I'm happy to answer them in the Q&A. And I will now hand over to Alex Coulombe from Agile Lens. Thank you.

Thank you, Marc. And hello, folks. My name is Alex Coulombe. And I run Agile Lens, an XR creative studio here in New York City. I'm excited to speak to you today about a few of the use cases we have of virtual reality capture and replay from NVIDIA. And yes, there's going to be a smattering of other NVIDIA shout-outs, so keep your ears pealed for those. Let's dive right in. So who am I? What's the story of Agile Lens? We don't have time for that. So I'm just going to throw a bunch of words and companies and bullet points and phrases that might mean something to you, but here's the important thing. I've been working in virtual reality for 10 years now, behind me is the Oculus Rift DK1. And by now, I've spent thousands of hours inside of virtual reality. And while I'm usually pretty thrilled to be doing that, sometimes it's a bit of a pain. Sometimes, you're working on solving one particular problem, and you change a bit of code. Put the headset on, move around, click, click, click, doesn't work. Change another bit of code. Put the headset on, move around, click, click, click, on and on and on. Sometimes I'm a little tired, and I lose track of whether the particular bug that I'm seeing came from pressing the joystick forward then entering the lobby. Or wait, was it entering the lobby then pressing the joystick sideways? I can't remember. And so with this in mind, my initial use case for NVIDIA VCR was pretty straightforward and a huge relief. I would record myself doing a thing in VR, moving here, clicking there, et cetera, stop the recording. And then the next time I change the code, instead of putting the headset back on for the thousandth time, I'm just playing back that same file that has all my input over and over again, exactly the same, perfectly replicated, entering the lobby, then pressing the joystick forward or whatever it was. I don't need to remember anymore, VCR takes care of it. This was instrumental for me recently as I prepared my talk for Unreal Fest last October. I needed to give a 2-hour presentation about the latest VR features in Unreal Engine 5.1, which wasn't even properly out. And while at the conference, I found myself rapidly trying to upgrade a bunch of my Blueprints to work with these new systems. And for those who don't know, Blueprints is just Unreal Engine's visual scripting language. This would have been a nightmare under normal circumstances. But using NVIDIA VCR, I was able to quickly and easily play back all of the recorded actions I had in virtual reality, not sure how many quotes I need here, without a headset basically to confirm that when it was time to do my presentation, things were going to work correctly. And to be clear, many of the clips in the presentation where you've seen me putting on a VR headset, that was my first time seeing that in VR because I'd always done the rest in VCR. So thanks, VCR. To demonstrate this a little further, allow me to show some iteration on a recent problem I was trying to solve involving snap turning in Unreal Engine with a character class instead of a pawn. And without going into too much detail of VR design here, this was a perfect use case for VCR because in order to properly test this, it requires moving around within the relative area of a VR play space. And there's no real way to replicate that input without Unreal Engine reading a virtual reality headset and controllers moving around a designated play space, which, of course, VCR can do. The issue I was having was that the default snap turn that comes with the VR template works great in a pawn setup. But as soon as you want to make that pawn into a character, which opens up a lot of easy new locomotion options and allows you to have things like a skeletal mesh and a character that animates, at that point, snap turn breaks. So I needed to figure out some vector math to query where exactly a user was in world space before they turned and then have the player controller move and then snap the head back to the location where they were, kind of complicated. But essentially, while that was all happening, the play space needed to adjust in the process, which could be extremely finicky. And I would say VCR have the amount of time that this iteration has taken so far to work out. I'm not totally done. But my last stumbling block here was that I realized that this couldn't all happen on the same frame and there's no way I would have figured that out if I had a headset on the whole time. Much easier to examine this while looking at my computer the whole time it's playing back in the spectator view mode that you're seeing here. So here is 9 progressive replays of the same VCR footage getting progressively better as I worked through this problem. Now for a more complex use case of NVIDIA VCR, and thanks to some language from my colleague, Eytan Manor, Software Engineer at Agile Lens, we'll be getting a little technical. So to set this up, we have a cloud computing platform that hosts live virtual shows, which combines mocap, audio, video, OSC and a variety of other data together for a synchronized experience, which is interactive to users in a browser or in VR if they use our customized NVIDIA CloudXR app. Our platform breaks with the traditional methods of putting it on a live show by making the entire production virtual. And what this means is the performance can be distributed all over the world instead of gathering under a single roof as they would traditionally. For example, with our recent production of Christmas Carol, we had one actor, Ari Tarr , playing Scrooge in Charles Dickens out of his home in Oregon and our other actor, Debbie Deer, who played the ghost and was traveling for the holidays, so found herself in Jamaica just a few days before the show started. This distribution goes the same for our attendees who can be anywhere in the world, enjoying the show from the comfort of their home, connecting to various NVIDIA GPUs in the cloud, starting with RTX A6000s and falling back to less costly cards, all the way to the RTX 4000 depending on the requirements of the experience. NVIDIA CloudXR, which we don't have time to go into much today, basically enables this connection for anyone who wants to join in consumer-level headsets like Meta Quest instead of a browser. And we've actually had over 100 simultaneous CloudXR connections on our platform, which has been very exciting to see. In many ways, this virtualized setup makes our job much easier because it's cheaper, quicker and much more scalable than dealing with a physical venue, especially since we're based in New York City. But there are challenges, for example, keeping everything connected. Audience members randomly disappearing from a physical theater would be horrifying in the real world. In a virtual theater show, it's honestly just another day of bug fixes. The solution that we built, among other things, is able to connect performers to attendees using some pretty clever load-balancing techniques. A performer would constantly be emitting these arbitrary streams of data and our platform is responsible for broadcasting to all the attendees who are watching the show. Let's zoom into this connection between performers and audience members to better understand how things technically work here. Our most important objective is to make sure that everything is done in real time. And so we've decided to utilize the UDP protocol as opposed to TCP, for example. UDP for those who don't know is a little tricky because it can't emit data chunks that are too big. It doesn't guarantee the chunks will arrive in order. And it may drop some chunks. However, it's very quick and has minimal latency. So by engineering a few adjustments, we've been able to bring this protocol to a sufficient level of reliability in our use case. Using UDP, a performer can emit any type of data stream that underlying data type, audio, video, MIDI, DMX, OSC doesn't really matter. In Christmas Carol, a performer would typically emit 3 types of data streams. We'd have motion capture using Rokoko, facial expressions using Live Link Face switches, an iPhone app, or audio input using a custom microphone solution that we built. In our platform, we take all 3 of these streams, and we combined them into a single super stream. Each chunk gets partitioned, indexed and timestamped before it exists to the network. And the super stream then goes through a data service that was deployed specifically for a particular show, like Christmas Carol, and thus, it's aware of who exactly is watching the show and can broadcast those chunks accordingly. Once the super stream makes it to the other side, the attendee side, it will be out of order. But since we took care of partitioning and indexing everything, we're then able to reconstruct the original streams. Just imagine for a moment here that I paid for some really exciting stock footage of broken pieces swirling and coming back together or maybe that scene from Jurassic Park with Mr. DNA. From there, every stream can make it back to their client application. We have the Rokoko stream, making it into the Rokoko plug-in for Unreal Engine. We have Live Link Face getting into Unreal through that Live Link setup. We have the audio coming into either the virtual speaker on the computer, where it's actually picked up by NVIDIA CloudXR quite easily, or a separate WebRTC client, which is for pixel streaming. More on this in a moment. Given that all of our data streams go through a single point, i.e., the data service, we realized that we can record our live performances and then eventually play them back. Maybe you see where we're going here. Now you've heard about how NVIDIA VCR works, but our system follows a slightly different process. First, we mount a persistent volume claim. If you aren't familiar with this term, it basically means we provision a hard drive using our cloud providers' API. We then attach an additional global timestamp to each data job, and then we pipe all the traffic into a file that's located in the hard drive we've just provisioned. As a file, that stream can later be accessed for retransmission of the data. And up until now, only recording the performance side was possible, but thanks to NVIDIA VCR, we can now record all the activity of the attendees and capture all the components of what was happening during a live show. The process is quite similar. We use the same persistent volume claim we created for the performance to store the VR run time input. And remember that each attendee has their own VR headset and virtual machine so we can have a dedicated take directory for each one. We also store a time stamp for each recording, so then when we play them all back, they stay perfectly in sync. The replaying process is actually very similar to how we would do a live stream, which is exactly what we want to keep parity there. But instead of using the analog input, like motion capture suites and everyone having VR headsets, we can now use that digital input, the streams stored and files, which allows for a one-stop shop and no need for additional software. And so on the performance side, the data service reconstructs that super stream from the recording and emits the data chunks as accurately as possible based on the attached timestamps. And on the attendee side, the replay from the NVIDIA VCR driver gets installed on each attendee virtual machine and reconstructs the stream from that correlated take directory. The replay session then starts based on the timestamp value that we stored while recording the actual attendee, just using a simple script that creates the necessary hooks. Making sure all the streams don't fall out of sync is key when replaying the performance, and this is why we rely on timestamp so much. If someone, for example, picks up an object and gives it to another person but the streams are out of sync, then because this is all playing back as though we're a live stream, the other person would not actually receive the object. So timing is critical here. Now we talked a little about our audio input and how we have a custom solution for this, let's briefly go through that setup as we find it to be a key feature when having a live performance. Now most media toolkits rely on codecs when processing data for encoding and decoding, and one of the most popular ones is called FFmpeg, which is kind of like a Swiss Army knife. Many media players use FFmpeg. Even Blender, for example, uses FFmpeg, to process and display various media. And while this is great for nonreal-time use cases because you don't have to deal with the raw stream format, which is very large and uncompressed, this is not something that suffices for real-time performance. Accordingly, we don't use any media tool kits or codecs. First, we use Chrome's implementation to receive raw microphone input directly from the web browser. And that raw audio format is called PCM, pulse-code modulation. Then we proxy the stream using UDP to a target destination. And then finally, we either pipe the stream to native audio libraries that run on the operating system or to a WebRTC pure connection. It all depends on the launch parameters of the receiver. But typically for a virtual reality CloudXR session, we'll rely on the native libraries. And for pixel stream sessions, we rely on WebRTC. And finally, NVIDIA's CloudXR driver is able to play the system sounds directly on the VR headset, which makes things very easy for us, it's just picking up whatever audio the virtual machine is hearing. And for reference, here's a table of those different audio back ends that each OS uses. On Windows, for example, there's a DLL file for playing audio directly to the speakers, and that's located in the Win32 directory. Okay. So where are we going with all this? Well, as of now, when we create a new build in Unreal Engine for one of our live shows in the cloud, we often rely on our staff and beta testers to let us know if anything is broken or misbehaving. Playing back all the audience sessions with NVIDIA VCR is great but mostly as an ambient experience. So if someone is live watching that replay, they get to feel like there's other people there, and you can see where the audience members were most excited and where they were gathered and what they were looking at. But if we actually want to figure out any bugs that happened on an audience member, we actually need to ask them to report it, which isn't their job, or manually watch each recorded stream with VCR or from video. And that's tedious and quite prone to human error. So something we've begun to work on is an automated CI/CD pipeline, and that stands for continuous integration/continuous deployment, which would automate this process and be far less prone to errors by using a clever algorithm that would go through all of our sessions and run a pixel-to-pixel comparison between the current and previous time line to see if anything had changed. And if it did, it could send us notifications, say through a Slack integration, that would include snapshots of those breaking points. At which point, it will be up to us to either approve the build deployment or decline it. And if we decline it, it could simply fall back to the last approved build and keep everything stable. So to wrap up, I just want to emphasize how useful NVIDIA VCR has been for us at saving time, headaches and neck aches from putting our headsets on and off. And I hope we've been able to demonstrate through our case studies here how it can be both an incredibly useful tool out of the box for, say, solo development, but it can also scale to much more complex utilities like capturing everything a live audience is doing during a VR show or deploying hundreds of virtual audience members to test out a new build and let us know if things are working okay. And with that, we're out of time. So I would like to thank you for viewing our presentation. A special shout-out to NVIDIA Broadcast for making it look like I've been looking at you this whole time and NVIDIA ShadowPlay for capturing a lot of this footage. And please, if you have any questions, especially technical ones, my colleague, Eytan Manor, is here during the Q&A to help you out. Thanks again, and back to you, Sheri.

Thank you, everyone, for attending this webinar. We really appreciate the time that you spent with us today. Before we transition to Q&A, we would like to remind all of you to please download the VCR SDK as well as give us any feedback. You can download the SDK at the URL on the screen as well as contact our product management team directly at the e-mail alias VCR-Outreach@nvidia.com with any questions. Thank you, and we'll now open it up for Q&A.

Hi, everyone. Thank you again for joining us today for the VCR webinar. We hope it was useful. We have about 10 minutes now to answer some questions. [Operator Instructions] Okay. So let's start. Let's see. Okay. Does VCR work with OpenXR?

Yes. So I think I will take that question. Hi, I'm Ingo Esser from NVIDIA. So VCR and OpenXR, SteamVR hasn't opened XR front end, so OpenXR applications can work through SteamVR. And so VCR is only working on the back end of SteamVR. So yes, that works. SteamVR will sort of translate the OpenXR calls to the driver -- or rather, to the capture and the driver component.

Great. Next question. Can you make the VR game with Unity Engine and test the game with this?

If I understand correctly -- or rather, Unity Engine should support SteamVR, if I remember correctly. So yes, any application that supports SteamVR will work with VCR. So you have SteamVR supporting your application, VCR sort of does not care what it's recording or replaying. It's pretty agnostic to the application that you are interacting with.

Great. Do you need to be in the app at approximately the same scene location you were in when you did the recording in order to replay?

Not only approximately, to be honest. So as VCR doesn't know what application you are interacting with or whatever it is recording and replaying, it's important to have some ways of recreating a position quite exactly. So for example, there are applications where you can jump to viewpoints or where you can sort of reset your view in the scene or things like this, and that is something you should maybe record in VCR and then sort of use that as a basic place to start your interaction with. If you are -- if your app decides to move you somewhere because of -- there's many, many different reasons that the app could do that, VCR will -- replay component will not know that, and so it will happily replay all of your interactions like one foot to the right of all the buttons you want to work with. And that can be an issue, obviously. And so be sure to make -- yes, so make sure that you are in the same location and in the same scene that you are recording. One thing that I need to say here also is that some applications actually do a calibration step for something like this at the beginning when they start up. And if you move between starting your application and starting your, recording, that calibration step may be different when you are replaying afterwards. And that may mean you record something and then the recording is -- or rather, the replay is completely off. That can be because the application you're interacting with sort of tries to understand what height your head is at or which direction you're looking at to render, let's say, UI element in that direction, so it's always in front of you. And if you turn, that may be off, and so that may become a problem.

Ingo, one point to add, if I understand the question correctly, that you suggested in the past is when we've created recordings, many VR applications will let you save a particular location. So when the recording starts, jump to that saved location. And then at the end of the recording, you could jump back to that same location. So if it loops again, you're always starting within the same place. Hopefully, that's helpful to the question as well.

Yes, absolutely. Exactly.