Fantastic Filming in VR - Behind the Scenes

Hello there! Over the years I’ve been using VR to make my videos about art. At first it was just a necessary medium for getting my points across about learning to draw. But as I got into it, I realized that there’s a whole world to the art form of VR cinematography, and slowly started to make a practice of trying to make my videos just a tiny bit better with each one. There’s a lot of parallels with traditional film-making, but some aspects unique to VR, and some even unique to how I do things. My name’s Kanjon and you’re watching the first behind-the-scenes video of Kanjon’s Clips.

If you’re worried about me ruining the magic, don’t panic: there’s still a lot of creativity that goes into each video I make, and showing you the process won’t remove that fun. I think that goes for any video producer: the creativity is unique to each video, and even if you know how it’s made, there’s still loads to discover in the writing, filming, acting, and editing of each video. A well-produced video makes that all shine, and the production itself hides in the background. I’m definitely not perfect at it, but it does make me smile each time I see someone comment about how they have no idea how my videos are animated, or how they can be so lively. That just means I’ve done it right. People will often tell you that you should just focus on the message you want to get across and the rest doesn’t matter so much… which is true to an extent, but if you add that extra layer of shine and polish, it just works so much more.

Anyway! All of the stuff I’m about to show you is how I currently do things. You’re welcome to adapt this and make your own videos, but don’t feel like this is the correct way to do VR video. It’s what works for me and my narrative-driven format. This workflow is also some of the output of my own skills in video production, 3D asset creation, and even software development. It won’t make sense for you to try this the exact same way without the exact same background I’ve got, but I do hope you’ll find little snippets to try on your own. This is also a behind-the-scenes video, not a tutorial. Most of this I’m going to gloss over, because I feel like I could teach a semester-long course on all of it and still now be done. But by all means, feel free to ask questions and I’ll get to them when I can!

Here’s my process: Writing, recording, filming, editing, publishing. To anyone who’s done a solo-project video, this might sound mostly standard. On the writing stage, yes: pretty much all of my videos are scripted. Unless I’m really “in-character”, I’m actually quite bad at improvising videos, at least in a traditional informational style. I’m ADHD as heck and jump around between thoughts. And yes, recording is separate from filming: for my workflow, recording means the part where I record my voice, and filming is the visual side. They’re done at different times, sometimes different days! If that surprises you, well, keep watching, it gets crazier. And yes, this is all actually done inside VRChat. There’s so much creativity in avatars, assets, worlds, even filming methods here, and I want to celebrate that just as much as making a video about what I want to talk about.

Writing

Probably the hardest part of any video for me is figuring out what I’m going to talk about and how to talk about it! I keep a digital pile of sticky notes with video ideas in a few places and peek at them frequently to see what feels right to write. Then it’s a plain matter of writing. The tool here really doesn’t matter. Notepad, Notes app, Obsidian, anything that gets out of your way fast to get to typing. I’ve literally written over a hundred videos, this is one thing I feel very confident in saying: don’t spend time researching the best tool or place or editor for your writing. Just start writing in the first place that pops into your head, you can organize things when you’re done. I literally do not organize my video notes until after I’ve published a video and it’s time to clean up and file things away for archival.

I start with a basic bulleted outline. Even this video right now has one, I’ll show you that on screen. For people like me, this is how I get started and know what I’m doing. Other folks may have a better time going directly to prose. Write in the way you know how, just get writing. Once I’ve got my outline done, I start typing out the video narration top to bottom. Sometimes I’ll skip the intro until I know what I want, but usually it’s just finding a quiet place free of distractions and starting to type. Even if I am feeling distracted, once I get typing, I can keep it up for quite a while. (So far on this video script, I’m doing this on my first sitting! Let’s see if I can get it all done here.)

If I have ideas for visuals or camera set-up while I’m writing, I might leave myself little scene notes. I don’t use a typical screenplay format because, honestly, I didn’t learn that up front, and I don’t need all of the overhead when my videos are primarily narration and a one-person project. For collaborations, yes, standardizing on screenplay makes more sense. Don’t make everyone else learn what your own annotations mean.

Recording

Apart from music and the occasional sound effect, everything you’re hearing in my videos comes out of this recording stage. I do my voice-over first, before I even decide where I’m going to film. This allows me to completely get into character and really focus on the sound, and not have to deal with VR shenanigans or other “on-set” filming issues.

Nowadays I hop into my home-made PVC voice booth and go. I use a microphone, audio interface, and iPad. Made a short about that, but this setup is not at all necessary to get started. Originally I did record and film at the same time, using my VR headset mic, and you can see that in my first few videos. But let’s be real the Quest Pro microphone is amazingly shitty and it didn’t take long to want something better. Those of you that have a Quest 2 or 3 and are confused, because your mic probably sounds fine, yeah me too. No idea why it’s so bad on a Pro. Anyway: I switched to my desktop gaming microphone and used that for a couple years, and that took me a long way. Use the microphone you have and work on your voice and delivery, it’s way more important than the microphone early on.

Speaking of voice, this whole character voice thing I do is kinda accidental. When I started out, I just tried speaking “clearly” and it felt okay enough. But I distinctly remember recording one night and listening to the result, and thinking to myself that it sounded like the audio demo for a Bose sound system. Professional, clear, but lacking charm and character. So I added the slightest little change to my voice and re-recorded. Published a video. Did it again, pushed my voice a little more, published a video. Each time I did this I felt a little silly when recording, but I was liking the sound coming out when I listened to the final result. So I kept doing that over many videos, until I found the sound that you’re hearing today. A tone that is energetic, whimsical, but with capacity to be serious or comforting as needed. I’ve since gone on to research more about voice acting to push myself farther, but for these videos? Just a product of practice.

The recording part might be my favorite part of the process today, because I get to really feel out the tone and energy of my video script. Even a topic that can feel boring on paper can come to life with the right kind of delivery. I do a bunch of warm-ups, then I hit record and start speaking. I might do a few takes of some sentences to correct for mistakes, nail a tone better, or just experiment, but at the end of it all I have one big long audio recording. That recording I then bring over to my desktop and open in Reaper. I used to record in Reaper too, but my voice booth isn’t near my PC and there’s no iPad version of that. I’ll cut up the recording into bits, stitching together the good takes, and removing weird vocal clicks as needed. Creatures of misaphonia, unite in removing gross mouth sounds from videos. (Oh my, sidenote, something deeply ironic and frustrating is that you’ll find a ton of video “tutorials” online about recording audio, and these videos will often have the worst audio you’ve ever heard, or so many gross mouth noises. Ruins it. Um. Anyway.)

I then cut the audio up into sections using regions in Reaper, and render those out to individual audio files. Those get used in the part I think you’re all waiting for…

Filming

Filming time! Yes, we’re here, though you’ve been watching the output of that this whole video already. With my audio clips ready, I hop into VRChat, quite often forget to set my status to do-not-disturb, and then start looking for a world matching the vibe of what I think fits the video. Cozy, adventurous, whimsical, sometimes it can take a while, sometimes I’ll find something right away. Plop down a camera, start recording, go, right? Whew.

I apologize in advance, because this is where things get very involved, very quickly. I use custom filming software that I wrote from scratch to fit my workflow. This software launches OBS with a specific profile, sets VRChat’s desktop window size correctly, routes audio using Voicemeeter, and automatically starts and stops OBS and audio playback. It is overkill for everyone but myself, because it was made to fit my own workflow that I’ve come to like over the years. There’s no secret sauce in this software: you can do this all at home with Voicemeeter and Audacity, which is how I first started, though other choices like Soundpad have also emerged.

When I select a voice clip and push “go”, I get a few seconds to get into position, OBS starts recording, then there’s a tick tick tick, and I go. The whole time filming, I’m watching a copy of the script I placed on top of my camera, and I am miming. You’re seeing it now: I have my headset, controllers, and just three full-body trackers on. When the audio clip is done, OBS stops, and I have a chance to review what I’ve done before continuing.

Similar to the recording stage, miming what I’ve already recoded lets me focus exclusively on my motions and expressions. I don’t also have to get the audio right. This is the key secret for narrative VR video and film: make the work less complex by breaking it down and it’ll all look better as a result. Tricks like this, and more I’m about to show you, have turned video production into in art form of its own for me, because making high quality videos is appealing. I like to push myself.

At its core, that’s what it is. But there’s so much I’ve done before the camera turns on that makes it look great, including… the choice of camera. Unfortunately VRChat’s camera, while it has gotten better in recent years, still lacks some of the control and accuracy of traditional photography. It’s gained exposure control but misses the mark quite a bit with things like depth of field. Luckily, there’s a super cool avatar addon called VRCLens that can go that extra mile and get you a fantastic camera for cinematography, even down to emulating certain sensors and lens shapes. It’s details like this that make VR video look that much nicer in that final output.

Outside of the camera, I bring lights with me. Many VRChat worlds don’t have great lighting but are otherwise fun places to be, or might have good lighting for VR but not for filming, which is honestly fine, it’s a VR game. That’s why you bring your own lights on set. On top of that, using an avatar with adjustable shading settings, like I’ve done with mine, lets me compensate for overly-dark worlds or directional light that may be too harsh. (For the shader nerds out there, I’m using Poiyomi, with Minimum Light and Wrap in Wrapped lighting mode. Animate and slap those on sliders in your menu. Works wonders.)

Camera and lighting set up takes a lot of practice, like, this is a whole-ass film school kinda topic and then some. I’m certainly not perfect at it but know enough to film a video in pretty much any VR world situation and still look decent. Try it out a little bit if you haven’t, but don’t obsess over getting it perfect. Even a little light to defeat a super dark world is loads better than nothing.

Some other thoughts on filming: I like to lock my position using Gogo Loco, another avatar add-on, so that I don’t hit weird polygon edges in some worlds that make my avatar snap up and down. And you’ll sometimes see me using finger tracking when I think it adds to what I’m saying, though lately I’ve been using controllers more because that tracking tends to be fickle. And finally, I often film the same voice line twice using two different camera angles, because I can sync those up later during the editing phase to switch between views. Let’s get into some of that!

Editing & Publishing

Once filming’s wrapped, it’s time for some post-production. I dump all of my video clips into DaVinci Resolve, delete the clips that I know are bad takes, and start editing things onto my timeline. I start with a template project that I made that has my outro and such built-in, but otherwise, not a lot of magic here. For clips where I have multiple camera angles, I select them, smack sync, and sync to audio. I can then use the cut page’s sync bin or multi-cam on the edit page to swap camera angles, but I’m not here to teach video editing.

One quick note, I actually film in 30 FPS and let Resolve do Optical Flow interpolation up to 60fps for my videos. It actually works quite well and I don’t think most folks have noticed! The reason I don’t film in 60 FPS is that any frame drop will be noticeable when your PC can’t keep up, which can happen a ton in many VRChat worlds. But most PCs can do at least 30 FPS just fine, guaranteeing you a “good” frame every frame. Resolve then does a pretty great job of blending between the frames in a way that you don’t notice at regular speed. Don’t believe me? Pause the video on a PC and use the period key on your keyboard to move one frame at a time. Every other frame will have some motion blur kinda look to it, and that’s the interpolation happening. At full speed, it just looks normal. This isn’t some gen-AI bullshit, I hope it’s obvious that I don’t use that anywhere, this is decades-old motion estimation techniques at work.

Anyway, I get shots into place, add some on-screen graphics where it’s relevant, drop some music in from a licensing service, and I’m good to go. I’ll render that video out, drop it onto Youtube Studio, and… theres’ still more work to do. Video cards, chapter markers, setting a description, subtitles, setting up a companion website, there’s a lot of activities to complete once a video is done that still takes quite a bit of time. Publishing a video means I want to shout it out on socials, too, so I draft up posts for that. I also spend some time drawing out a thumbnail for my videos, since this is an art channel after all. But all that stuff is a part of finding your flow with making videos. If you wanna make VR videos like I do, I hope this was helpful! I’d love to see more high-quality VR films, documentaries, and shows. The way you get there is just like drawing: by trying it out, by doing it badly and figuring out what works and what doesn’t. Also, from earlier, I did finish writing this in one sitting, neat. I didn’t finish recording this in one go though, it was like five. And it’s hot in here. Right, we’ll get back to some art stuff soon, but I hope you enjoyed this peek behind the curtain. Chase that passion cool creature, I’ll see ya.