Artifacts of image stabilization and becoming part of new “masters”

Regardless of the responsible service providers or assigning rightholders, as well as the archives responsible for the film heritage, this article deals with the potential challenges of automated image stabilization, which is standard in digital film restoration.

The experience gained in digital film restoration projects shows again and again that the elimination of analogous errors and damages brings along a risk of new and thus digital errors (artifacts). It is important to consider whether the risks of error, which are potentially caused by digital restoration, and the fact, that these methods are also subject to constant evolution, should not lead to an adaptation of the current depository practice, which refers exclusively and is limited to the so-called DCDM and its processing level.

Reading time ca. 10 min. Published on 9th. April 2019

Structure

• What is “foreground dominance?
• Examples from “Sissi – The Young Empress, 1956”
• Conclusion

Foreground dominance

The problem of “foreground dominance” can arise in the reduction of high-frequency image shifts, which are also perceived as “shaking” (stabilization error). For the process of stabilization, image features that are of high-contrast are analyzed as they shift from image to image. In scenes with a blurred or largely hidden background, there are ineligible image features whose movement in the image can be attributed to the global “camera movement”.
In contrast, actors define regions of local movement as part of the images. Ideally, this movement component should be distinguished from the global movement in the image (see Camera on the environment). However, most of the algorithms only distinguish statistically between these sets, thereby incorporating both components of motion at different levels into a resulting stabilization.

“Foreground dominance” occurs when local movements have a higher weighting than the global movement and thus also these local movements in the foreground – often those of actors – are proportionately stabilized. These artifacts become noticeable, for example, by contrasting movements of the background that are slightly delayed to the foreground movement.

Examples

The following examples are all from “Sissi” part 2. They were chosen because the malus is particularly clear here. However, this does not mean that all remaining scenes in the same movie are without this error.
More examples of other films can be found here ..

• Pater Brown – Der Fehler der Maschine, 1966 (strong visible defects)
• Das Doppelte Lottchen, 1950
• Quax der Bruchpilot, 1941

Shot 356

The movement of the actor (Gustav Knuth) is compensated into the background, whereby this movement contains some former foreground motion softened and inverted. For comparison, below are the camera/global motion paths of both versions.

Shot 358

The head movement was partially stabilized. Again, the background moves artificially, as the corrected version (right) shows.

Shot 495

Here, the length of the shot clearly shows that clearly visible artifacts only appear as soon as the actress (Magda Schneider) raises her shoulders. She does this twice in varying magnitudes. Followed by this there are compensating and wrong correction shifts in the background.

Shot 502

The jerky movements of the actress (Vilma Degischer) were compensated by “foreground dominance” patially into the background.

Shot 564

In this shot, the background resonates against the initiated motion of the actress (Romy Schneider).

Conclusion

Digital film restoration is not a trivial task despite highly specialized software. The industry likes to use highly automated systems that do not require any technical understanding of the algorithms applied by operators. Thus, these and other artifacts are difficult to identify and become – as shown in the examples – henceforth part of new “masters”.

Adequate procedures are applied by depositing the original digitized material (RAW-Scan). This should be applied for all funded restoration projects (see “Förderprogramm Filmerbe”), but also for others that are to be deposited digitally in the Filmarchiv in the Bundesarchiv. Only given this procedure, comprehensive quality assessments can be made for the processed data.

There are rights holders and curatorial institutions that have been following this procedure internally for many years and are archiving their digitized originals. After inquiries with the Filmförderanstalt (FFA), however, there are no further measures for quality assurance and also the deposit practice refers exclusively to a DCDM (precursor to the DCP for digital cinema), in which all processing steps are already incorporated.

The key position of a digitization project is the RAW scan at the end of the actual digitization process, thus providing the basis for a completely non-destructive processing of digital restoration in the ideal case. It attains an increased importance, since only this state, when referring to the DIN Spec 15587, can be considered as a digital representation of the former analog film. Here all characteristics of the analogous origin are best documented.
In addition, the preparatory steps before a scan can be very extensive or the material itself is no longer available for a new digitization process at a later date. Also, future improvements in the digitization technique and qualitative differences can only be proven in comparison to existing RAW scans.

The RAW-Scan as BagIt-Container

The proposed amendment does not require a new format or new procedures to be developed. The DIN-Spec, published in 2019, sets clear quality requirements and at the same time represents common practice. Like the DCDM, the RAW scan would be deposited using BagIt Containers .
It would be desirable to establish a cooperation of the Filmarchiv im Bundesarchiv and other institutions to create a documentation of already existing RAW scans of projects of various complexity. In this way clear conventions for standardization could be set up, which further clarify the existing recommendations. This step is also essential for software manufacturers to develop systems for automated quality control (Scan vs. DCDM) and related documentation.

Still thinking further, a pool of good digital RAW scans in the Filmarchiv im Bundesarchiv appears to be a valuable trove for the upcoming KI-based film restoration projects involving German research institutions. Here, a great opportunity would be missed in two ways; data for AI research on image enhancement applications and preserving cultural heritage without digitally added errors and damages.

About the author

Rainer M. Engel studied at the GERMAN FILM SCHOOL and since 2006 has been working independently in the field of technical image processing as well as teaching at various colleges and technical colleges in Germany in the area of media production. He is co-founder and honorary managing director of the SCHRIFT-BILDER gGmbH.

Developed digital film restoration automations in an open workflow to circumvent the limitations and problems of existing commercial solutions are part of his passion. Only by this approach even unsubsidised projects could be made possible; technically as well as economically. The continuous view on quality and its verifiability are essential.

References (realization / participation):
Metropolis, Winnetou I-III, Der blaue Strohhut, Mein Name ist Nobody, Edgar Wallace aso.

contact: engel@schrift-bilder.org