When Alignment Fails

In this blog we are going to take a look at how unwelcome variability can trigger differences in the number of tie points detected, and how duff GPS values can really cause more issues than null values.

Both were causing unpredictable results during alignment…it piqued our interest so we took a deeper look. The causes were not initially obvious but the answers are worth sharing. The cameras in question are the DJI Osmo but the findings and conclusions apply equally to any camera.

Taking each issue in turn, let’s review the impact of image stabilisation features on photogrammetry first.

Image Stabilisation

Image stabilisation exists to remove the effects of camera shake and blurry photos. Broadly, there are two types:

• Hardware based
• Electronic or digital based

These features are common in many cameras and lenses. Vibration or movement will trigger these features in an attempt to preserve image sharpness and works well for handheld photography.

And image sharpness is highly valued in photogrammetry, right?

Hardware image stabilisation can be achieved by either moving lenses, or moving the sensor or both combined and are called, OIS (optical), IBIS (in body IS) or simply IS. The physical path of light taken through the lens and onto the sensor is tweaked to remove motion and the characteristics of the lens become variable, with each variance requiring its own calibration during alignment.

The only way to make IS sourced images work for photogrammetry is to put each image in a separate camera group and calibrate individually. This really is a “can I?” and “should I? choice when considering hardware stabilised images…and the “should I?” ends in a firm “no” every time.

Digital stabilisation uses digital tricks to make video look stable. Everything is valid; tracking the horizon, rotating digitally each frame, using projective transforms and local morphing to make thing look steady. Feature points may be tracked and forced to follow linear or smooth paths… so every trick is acceptable.

Digital stabilisation is found in lower end imaging hardware. Action cameras can take stills but they have no mechanical shutter so while some have a stills mode, others take stills while they are recording video. Without a deep analysis we are not 100% sure what is going on inside the Osmo but we do know digital IS normally relies on image motion tracking… so it needs a stream of frames to work.

By engaging digital stabilisation and we disengage Brown’s lens model – the fundamental lens equations used by everyone in photogrammetry – and no optical rules can be trusted. Local distortion is applied to correct shakes so it’s pointless trying to model distortions that are not caused by the fixed optics but by trade secret algorithms. This is before we have started to consider the rolling shutter effects and distortion…which adds another level of complication we shall discuss in future blog.

Ultimately both stabilisation methods – hardware or electronic – introduce variability into the mix and the path of light isn’t constant and fixed.

Why does this matter?

Metashape will calibrate the camera on-the-fly but if we are handing over a mixed set of images, whereby the lens or sensor is subtly changing, then calibration that works for a single frame might be totally unsuitable for the rest.

So its a combination of sharpness and stability that are required.

In all instances the use of image stabilisation is not recommended for photogrammetry and we believe this explains why the number of tie points found in differing alignments was variable.

But that was not the end of the issues…GPS values were playing their part too.

Invalid GPS Values

Unless the project is blessed with hardware such as UWIS access to the GPS radio waves is impossible when underwater. With no signal it’s better to have nothing recorded in the image EXIF.

Although the GPS feature had been turned off in camera the dataset was refusing to align correctly. There were two factors at work here:

• The option Reference preselection was set to Source
• The actual GPS EXIF data recorded in every image was 0.000/0.000/0.000

The recorded latitude, longitude and altitude placed every single image on the equator in the Atlantic ocean, due south of Ghana. We don’t need Global Mapper to tell us this is nowhere near the Great Barrier Reef in Australia and Reference preselection was telling Metashape to take these values into account during alignment.

In this instance, unchecking Reference preselection resolved the issue and image alignment went from a paltry 32 to a complete set of 942 aligned. A better approach would be clearing the values from the images – no data is better than duff data any time.

If anyone from DJI is reading this…please leave GPS EXIF as null values if recording is turned off…

Data In – Data Out

With the correct settings and values the images were aligned to generate a transect of the reef:

The resulting ortho mosaic – hosted as ever by our friends at Dronelab.io – was a fine example of how this can be used to create a highly detailed map of the reef, in this case 1mm per pixel downgraded from 0.7mm scaling:

Be sure to zoom in and check out the detail.

The data isn’t geo referenced (and nowhere near Ghana!) but if you pan left you will see the coral give way to rubble and alga. The reasons for this damage are currently unknown but can be studied at distance to a fine level of detail.

Solving Problems

Both Jose and Simon relish in solving problems. Consultancy work can involve everything from refining or validating a workflow, improving accuracy and repeatability or creating custom python scripts to automate or extract more value from existing data.

From massive civil engineering projects to small scale ROV operators our clients always receive a bespoke service geared around their equipment and resulting data. The variability of the challenges keeps us mentally engaged and interested in photogrammetry. We can quickly advise if photogrammetry is not the correct method and recommend other solutions, and have saved clients capital spend on hardware that was not suitable or required. Please get in touch if you think we may be able to assist your project.

Finally, we would like to extend our thanks to Karen Joyce of James Cook University for posing the question, sharing the data and allowing us to use this consultancy example here on our blog.