In this blog we are going to take a look at how unwelcome variability can trigger differences in the number of tie points detected, and how duff GPS values can really cause more issues than null values. How do we handle failure of photogrammetry alignment?
Both were causing unpredictable results during alignment…it piqued our interest so we took a deeper look. The causes were not initially obvious but the answers are worth sharing. The cameras in question are the DJI Osmo but the findings and conclusions apply equally to any camera.
Taking each issue in turn, let’s review the impact of image stabilisation features on photogrammetry first.
Image Stabilisation
Image stabilisation exists to remove the effects of camera shake and blurry photos. Broadly, there are two types:
- Hardware based
- Electronic or digital based
These features are common in many cameras and lenses. Vibration or movement will trigger these features in an attempt to preserve image sharpness and works well for handheld photography.
And image sharpness is highly valued in photogrammetry, right?
Hardware image stabilisation can be achieved by either moving lenses, or moving the sensor or both combined and are called, OIS (optical), IBIS (in body IS) or simply IS. We tweak the physical path of light through the lens and onto the sensor to remove motion. The lens characteristics become variable, with each variance requiring its own calibration during alignment.
To ensure IS-sourced images work for photogrammetry, place each image in its own camera group and calibrate it individually. This really is a “can I?” and “should I? choice when considering hardware stabilised images…and the “should I?” ends in a firm “no” every time.
Moving Ahead
Digital stabilisation uses digital tricks to make video look stable. Everything is valid; tracking the horizon, rotating digitally each frame, using projective transforms and local morphing to make thing look steady. You can track feature points and force them to follow linear or smooth paths—every trick is acceptable when fixing photogrammetry alignment failure.
Digital stabilisation is found in lower end imaging hardware. Action cameras can take stills but they have no mechanical shutter so while some have a stills mode, others take stills while they are recording video. Without a deep analysis we are not 100% sure what is going on inside the Osmo but we do know digital IS normally relies on image motion tracking… so it needs a stream of frames to work.
What’s Next in fixing Alignment Failure?
By engaging digital stabilisation and we disengage Brown’s lens model – the fundamental lens equations used by everyone in photogrammetry – and no optical rules can be trusted. We apply local distortion to correct shakes, so it’s pointless to model distortions caused by trade secret algorithms rather than the fixed optics. This is before we have started to consider the rolling shutter effects and distortion…which adds another level of complication we shall discuss in future blog.
Ultimately both stabilisation methods – hardware or electronic – introduce variability into the mix and the path of light isn’t constant and fixed.
Why does this matter?
Metashape will calibrate the camera on-the-fly but if we are handing over a mixed set of images, whereby the lens or sensor is subtly changing, then calibration that works for a single frame might be totally unsuitable for the rest.
So, a combination of sharpness and stability is necessary.
In all instances, we do not recommend using image stabilization for photogrammetry. We believe this explains the variable number of tie points found in different alignments.
But that was not the end of the issues…GPS values were playing their part too.
Invalid GPS Values
Is the GPS Values the cause of photogrammetry alignment failure? Unless the project is blessed with hardware such as UWIS access to the GPS radio waves is impossible when underwater. With no signal it’s better to have nothing recorded in the image EXIF.
Although we turned off the GPS feature in the camera, the dataset refused to align correctly. There were two factors at work here:
- We set the option “Reference preselection” to “Source.”
- The actual GPS EXIF data recorded in every image was 0.000/0.000/0.000
The recorded latitude, longitude and altitude placed every single image on the equator in the Atlantic ocean. In fact, due south of Ghana. We don’t need Global Mapper to tell us this is nowhere near the Great Barrier Reef in Australia. furhtermore, Reference preselection was telling Metashape to take these values into account during alignment.
In this instance, unchecking Reference preselection resolved the issue and image alignment went from a paltry 32 to a complete set of 942 aligned. A better approach would be clearing the values from the images – no data is better than duff data any time.
If anyone from DJI is reading this. Please leave GPS EXIF values as null if you turn off recording.
Data In – Data Out: Photogrammetry
Diving further into the topic of photogrammetry alignment failure. With the correct settings and values, we aligned the images to generate a transect of the reef. With the correct settings and values, the images produced a detailed transect of the reef.
The resulting ortho mosaic is hosted as ever by our friends at Dronelab.io. It serves as a fine example of how to create a highly detailed map of the reef, with a resolution of 1mm per pixel, downgraded from 0.7mm scaling.
Be sure to zoom in and check out the detail.
The data lacks geo-referencing and is not near Ghana. However, if you pan left, you’ll see the coral giving way to rubble and algae, illustrating a photogrammetry alignment failure. The reasons for this damage are currently unknown but can be studied at distance to a fine level of detail.
Solving Problems in Failure of Photogrammetry Alignment
Both Jose and Simon relish in solving problems. Consultancy work can involve everything from refining or validating a workflow, improving accuracy and repeatability or creating custom python scripts to automate or extract more value from existing data.
From massive civil engineering projects to small scale ROV operators our clients always receive a bespoke service geared around their equipment and resulting data. The variability of the challenges keeps us mentally engaged and interested in photogrammetry. We can quickly advise if photogrammetry is not the correct method and recommend other solutions. Also, we have saved clients capital spend on hardware that was not suitable or required. Please get in touch if you think we may be able to assist your project.
Finally, we would like to extend our thanks to Karen Joyce of James Cook University for posing the question, sharing the data and allowing us to use this consultancy example here on our blog.
1 thought on “Photogrammetry Alignment Failure: When Alignment Fails”