Recording Unity video on an EC2 server
Why Record In The Cloud?
We were looking for ways to create quality content on our YouTube channel that wouldn’t cause us to take a run on the content treadmill. The “Daily Challenge” mode in King Rabbit — Race was a perfect candidate for content, as a new race is generated every day and it would show off the skills of our best players. If we could record and upload these videos in an automated fashion, we would have a trickle of juicy content every day.
Unity Recorder
The research into how to accomplish this task started with the built in Unity Recorder, a tool for capturing video straight from Unity. The unfortunate issue I saw right off the bat was the tool could only be used directly from the Editor. If we had extra licenses to run an instance of Unity in the cloud, that could work, but that wasn’t an option for us. We would need a way to record video from a running app instead.
FFmpegOut
Through some Googling I was able to find an awesome open source project called FFmpegOut by keijiro that enables video capture from a live app. The plugin is basically a wrapper for ffmpeg to feed camera render data into a video output file. I was able to add the CameraCapture component from the project directly on my existing Camera GameObject to get the capture working on my dev machine. It was a breeze to implement and very performant.
Xorg
All was well and good testing locally so I moved forward to running my app in the cloud using an AWS EC2 instance. Since a GPU is needed for the recording to work, I chose to go with a g4dn.xlarge instance (the “g” instances have a GPU attached) using Ubuntu Server as the OS. After getting my app (built using the Linux64 build platform) uploaded to the server, I ran the app using the same command I did locally. Fail. Looking at the logs, Unity was printing out a 0 x 0 resolution for the screen. This made sense as EC2 instances don’t have physical monitors attached to them.
After some more reading, I found out I would need to have a graphics card driver and Xorg installed. To install the graphics card driver, I followed the AWS-written guide here (Option 4: Nvidia gaming drivers). To install Xorg, it’s as simple as sudo apt-get install xorg.
Once those dependencies were installed and configured, the app was logging the correct resolution I was expecting. However, I was now seeing the error: Failed to initialize an FFmpeg session due to lack of async GPU readback support. Back to the drawing board.
Async Readbacks
A great feature of Unity that’s supported by FFmpegOut is the use of async readbacks when reading GPU data for use in the recording. This improves performance by preventing the GPU from blocking the main thread when reading data from it, which results in the app running smoothly while recording. Unfortunately, the graphics card attached to the EC2 instance (and/or xorg) does not support this feature and therefore will fail to render video with the FFmpegOut library.
For our use case, we would be running an asynchronous task in the cloud with no one watching, so having a slower than normal recording time would be ok. To fix this issue, I was able to patch in some code to FFmpegOut to do synchronous readbacks instead. The main piece of code I injected into the FFmpegSession.QueueFrame method looks something like this:
if (_useSyncReadback)
{
var readbackTexture = new Texture2D(_width, _height);
RenderTexture.active = rt;
readbackTexture.ReadPixels(new Rect(0, 0, _width, _height), 0, 0);
RenderTexture.active = null;
_pipe.PushFrameData(readbackTexture.GetPixelData<byte>(0));
Object.Destroy(readbackTexture);
}
else
{
_readbackQueue.Add(AsyncGPUReadback.Request(rt));
}With this patch I was now able to run the app and have it successfully record video on EC2! But that was just the first step.
Uploading to YouTube
The second step of the process would be to upload the recorded video to YouTube on a daily basis. I was able to write a node script that would start up the render server, run the render script, download the resulting video, upload to YouTube, then shutdown the server (no need to keep it running).
The node script made use of the
aws-sdklibrary to manage starting and stopping the server, and thegoogleapislibrary to upload to YouTube.
The whole process takes about 15 minutes for 3 minutes of video output on a g4dn.xlarge instance. Now whenever a Daily Challenge gets finalized, on our app server, the script is automatically run to render the final results each day.
Note: When running the server script I had to start up Xorg with the ‘&’ command so the rest of the script could continue to run (otherwise Xorg will block until it’s shutdown). e.g.
Xorg :0 &
No Audio
A side effect of rendering on an EC2 instance is that they don’t have sound cards attached, which means no audio will be recorded for the video. There may be ways around this using something like Jack, but I haven’t gone down that road yet. If you have a solution for this, please let me know! In the meantime I’m using ffmpeg to loop a song over the video.