Capturing Guidelines

Creating 3D radiance fields is done by shooting regular 2D video or a set of 2D photos. Postshot then trains a 3D radiance field model that reproduces these input images as accurately as possible. This means that the model can at most be as good as the input images. It is therefore important to capture the best possible images of the scene.

The goal of this document is to give you practical guidelines for shooting images that work well for radiance field training.

Below we will use the term capture shot to describe the process of shooting images for radiance field training.

Beware that this document is a work in progress. We will improve it as we explore this new technology and the workflows around it. As with any other part of Postshot, your feedback is much appreciated.

The Scene

Avoid moving objects, lights and shadows

Postshot's current radiance field profiles assume strictly static scenes. Regarding motion, capture shots are somewhat similar to long exposure photography. Anything that moves on the images will create blur or ghosting artefacts in the radiance field. Of course you could also use motion to create these effects on purpose.

Lens flairs are also a variant of this issue. While they are often static, lens flairs are am extreme form case of view-dependent lighting that is likely to cause problems for the radiance field. If there are angles in your capture shot in which flairs arise, you want to get rid of them quickly through appropriate movement.

Camera Settings

Video length and framerate

Aim for about 1-2 min of video or burst mode photos. In burst mode aim for about 2-3 FPS. A typical training run uses about 400 images. 100 images can be considered a small image set, 1000 a large one. In order for the tracking and training to work well, images should have at least about 30% of overlap. On the other hand, many images with more than 90% overlap are unlikely to improve the result but will increase the processing time.

Focal length

The focal length must remain constant throughout the imported image set. Unlike the other guidelines, this rule is actually mandatory. In order to get closer to the object, the camera always has to be moved.

You tend to want small focal lengths that give a lot of context in the image. For hand-held recordings, 24mm full frame is a good value. For outdoor drone shots you can use longer lenses if the distance to the object is already very large.

Keep exposure and white balance constant

The less auto-exposure and auto-white balance is used, the better. Therefore, it is worth shooting HDR if possible.

Camera path

Shoot from many angles

The goal of a capture shot is to record as many points of the scene as possible, each from as many different angles as possible. At the same time, we want to keep the number of images low.

To achieve this, a good scheme for the camera path are a set of 2-5 rings around the scene, each with a different tilt (vertical viewing angle) and pedestal (height above ground). Shoot upward when the camera is low, and downward when it's high.

When capturing rooms, move along the walls and shoot the longest view into the room.

When capturing objects, it's often helpful to include one or more large rings for context.

Always move the camera

Don't pan, tilt or roll without moving the camera at the same time.

The reason for this is that camera tracking is based on the triangulation of points. In order to form a triangle for any point in the scene, it takes two camera positions that see the point. Therefore, the extreme failure case is a camera fixed at one position while rotating (pan, tilt or roll).

Avoid blur

Prefer short exposure times and small apertures to avoid both motion blur and defocus blur. If the lighting conditions don't allow you to turn both down, prefer using a small aperture. You can compensate for a longer exposure time by moving the camera more slowly.

Using higher ISO values is also an option, as radiance fields tend to tolerate noise better than blur.

Example

Here is an example video that you can use to get an idea of the capturing process and to test training a radiance field in Postshot.

As you may see, all of the above is more what you'd call 'guidelines' than actual rules. There are several things about this shot that are not ideal.

There are lens flairs at one point and moving shadows at another. The moving shadows create holes in the ground of the radiance field, while the flairs are tolerated better. The still wash out the radiance field around their viewing direction.

The cars in the background and the leaves on the trees are also moving in the scene. But this has comparatively little influence.

And lastly, the shot could be sharper. It's a smartphone video with H265 compression. So there is both some motion blur and loss of details from the compression.