Sustainability of Digital Formats: Planning for Library of Congress Collections

RDD 48 is the abbreviated name for the SMPTE RDD 48:2018 MXF Archive and Preservation Format Registered Disclosure Document. (For more about the definition of a SMPTE (Society of Motion Picture and Television Engineers) RDD, see Registered Disclosure Documents.) As described in the specification published by SMPTE in 2018, "RDD 48 specifies a vendor-neutral subset of the MXF file format for the long-term archiving and preservation of moving image and other audiovisual content, including all forms of Ancillary Data, together with Associated Materials. Among other features, RDD 48 defines a means for the carriage and labeling of multiple timecodes and audio tracks; the handling of captions, subtitles, and Timed Text; a minimal core metadata set; program segmentation metadata; and embedded content integrity data." RDD 48 was sponsored and authored by the Federal Agencies Digital Guidelines Initiative (FADGI)'s AudioVisual Working Group. See History for more details about the evolution of the specification over time.

Some of the key highlights of RDD 48 include:

• Support for multiple timecodes, including legacy timecode. As noted in the specification (p. 40), "RDD 48 files can contain many timecodes (one Master Timecode, many Historical Source Timecodes). Each timecode is represented by a Timecode Track, and the timecode data can occur simultaneously in several places. The Master Timecode can be stored in the Material Package, in the Top Level Source Package and in the System Item of an Essence Container. A Historical Source Timecode can be stored in the Top Level Source Package, in the System Item of an Essence Container, in the essence on a picture, sound or data Essence Container essence, or in a Lower Level Source Package." In support of this, one of the outcomes of the RDD 48 authoring process was the registration of new timecode subdescriptors in the SMPTE Metadata Register through the SMPTE TC-31FS (SG Timecode in MXF) working group.
• The carriage of extra information, beyond the basic picture and sound. MXF files are made up of partitions – one is called the Generic Stream Partition. A GSP can carry text-based (e.g., XML) or binary (e.g., still image) “streams.” The additional data can include EBU STL, Timed Text, other associated data like images of the packaging as well as supplementary metadata.
• Metadata. RDD 48 defines four Descriptive Metadata Schemes (DMS) that can be included in an RDD 48 MXF file: RDD 48 Core Descriptive Metadata Scheme (AS_07_Core_DMS) pertains to the whole file and includes one or more identifiers for the file and its content, a high level description of the file's content (e.g., title or working title), identifies who is responsible for the file (the "keeper" in terms of long-term management), provides basic video characteristic information, identifies if captions are present, defines audio track allocations, and offers high level information about how the file was made; RDD 48 Generic Stream Partition Descriptive Metadata Framework (AS_07_GSP_BD_DMS) defines additional metadata elements for non-essence binary data stream when present; RDD 48 Generic Stream Partition Text-based Data Descriptive Metadata Framework (AS_07_GSP_TD_DMS) defines additional metadata elements for non-essence text-based data streams; and RDD 48 Segmentation Descriptive Metadata Scheme (AS_07_Segmentation_DMS) provides a description of both the individual segmented parts as well as the aggregate group of parts. Each of these is described in Annexes in the specification.
• Content integrity data. RDD 48 calls for the embedding of fixity data on the V or value data in the KLV triplets that represent frame-wrapped essences, specifically a Castagnoli CRC-32C hash value. In addition, RDD 48 defines a Cryptographic Context Set metadata scheme. See section 6.7.2. in the specification for details.
• Manifest. RDD 48 defines an XML-based Manifest to support preservation and good housekeeping by offering an inventory of the RDD 48 file's parts and expresses the relationships between them. Overall, it provides a high level inventory of the parts including their identifiers, data description, MIME type, size and location. This information can help the user to better understand the composition of the file and it will also provide machine-interpretable information for content processing if, for example, an RDD 48-aware application used values in the DataDescription element to quickly locate the correct QC file in a workflow or to delete embedded graphics (binary data) prior to distribution. See section 6.7.1. in the specification for details.

RDD 48 includes a subset of constraints called a "shim" in convention with AMWA convention called the "Baseband Shim: Single Items from Baseband Video", and it is intended to support the reformatting of older analog and digital videotapes and, for some organizations, the encoding and packaging of "live" video streams sent to an archive via a serial interface. As described in section 6.2.10.1.1, "RDD 48 Baseband Shim files are for items derived from baseband video, understood to encompass both analog baseband and uncompressed digital video, and encoders will typically process a baseband (uncompressed) signal. For high picture quality the preferred picture encodings for the baseband shim are JPEG 2000 picture encoding and uncompressed picture." The Baseband Shim "is intended to serve the most critical current use case for memory institutions: the reformatting of existing and obsolescent videotapes in their collections. The Baseband Shim is also intended to serve memory institutions (and others) who may be acquiring digital video ingested via serial interfaces, e.g., congressional high definition video transferred to the Library of Congress via HD-SDI or its equivalent. In both of these use cases, memory institutions wish to archive the highest possible quality of image and sound (uncompressed or losslessly compressed), as well as retaining source data such as multiple timecodes, captions and subtitles, and also embed metadata that will support authentication and management of the content for the long term."

RDD 48 files employ three standardized MXF Operational Patterns: OP1a, OP1b, and OP3c. OP1a is the most common implementation for RDD 48 and is used for both single item and segmented item files, OP1b [standardized through ST 391, MXF - Operational Pattern 1b (Single Item, Ganged Packages)] can also be used for both single item and segmented item files while OP3c [standardized through ST 408, Material Exchange Format (MXF) - Operational Patterns 1c, 2c and 3c] is used only for bundled collections of files. Links to purchase both standards are available from the SMPTE Store.

SMPTE RDD 48:2018 MXF Archive and Preservation Format Registered Disclosure Document.

The specification is licensed under a Creative Commons Attribution-Share Alike 4.0 International License. (CC BY-SA 4.0). The specification notes (p. 2): This document builds upon the AS-07: MXF Archive & Preservation Proposed Specification published by the Advanced Media Workflow Association (AMWA) in 2016. Following the rules of the CC BY-SA 4.0 license for adapted material, this updated document [RDD 48] continues with the same CC BY-SA 4.0 license as the 2016 AMWA Proposed Specification document.

Adoption has been slow mostly because of a scarcity of tools and applications. FADGI's embARC (metadata embedded for archival content) "enables users to audit and correct embedded metadata to comply with FADGI guidelines. V.1.1.0 added support for FFV1 in MXF to existing functionality for DPX (Guidelines for Embedded Metadata within DPX File Headers for Digitized Motion Picture Film) and MXF (SMPTE RDD 48: MXF Archive and Preservation Format) files."

RDD 48 is included in the Target Format Comparison Tables for IASA-TC 06 Guidelines for the Preservation of Video Recordings

The specification notes "The user’s attention is called to the possibility that implementation and compliance with this specification may require use of subject matter covered by patent rights. By publication of this specification, no position is taken with respect to the existence or validity of any claim or of any patent rights in connection therewith. All responsibilities for identifying patents for which a license may be required or for conducting inquiries into the legal validity or scope of those patents are assumed by the user."

Accessibility Features

RDD 48 has enhanced features that strongly support digital accessibility with its capacity for multiple timecode, caption and audio language tracks and identifiers. In addition, RDD 48 requires the translation of ANSI/CTA-608-E (CEA-608) and ANSI/CTA-708-E (CEA-708) data to SMPTE ST 2052-1 Timed Text to be carried in a Generic Stream Partition. Teletext, the text-only closed captioning system for European television, is carried in the ANC (Ancillary Data) packet and "the Top Level Source Package shall include a strong reference to the VBI Data Descriptor as detailed in SMPTE ST 436-1:2013. The descriptor shall be associated with a Data Track via the MXF Generic Descriptor (as defined in SMPTE 377-1:2011)." In RDD 48, teletext is considered a form of ANC.RDD 48 files need not include EBU-STL unless it is present in the source file. If EBU-STL is present in the source file, it is to be maintained in the RDD 48 file. See section 6.2.12 Captions, Subtitles, and Timed Text of the specification for specific details.

A robust set of sample files, some of which are graded as gold, silver, copper and lead depending on their error tolerance, is available for PAL and NTSC, including for FFV1 in MXF Generic Container mapping for Amd 1, 2022. These files were created as part of a contract with AVP and their subcontractor EVS, first Product Development Manager Valérie Popie and later Engineering Manager Nicolas Bernard, to create a set of graded sample files in 2016 and 2017 based on RDD 48. These sample files were reviewed by Oliver Morgan of Metaglue, Inc. and Cube-Tec. Sample files starting in 2018 were created by Oliver Morgan of Metaglue. All sample files are in the public domain as the product of US federal government research. If a license is required, an appropriate license is CC0 1.0 Universal license for worldwide use and reuse as this parallels the license used for FADGI guidelines and documents.