Sustainability of Digital Formats: Planning for Library of Congress Collections

The High-Throughput JPEG 2000 (HTJ2K) encoding provides a modification to the JPEG 2000 Part 1 encoding by introducing a new HT block coder, which serves as a drop-in replacement for the original JPEG 2000 block coder, known as J2K_C and is standardized in ISO/IEC 15444-15:2019.

Per HTJ2K's "High Throughput JPEG 2000 (HTJ2K): Algorithm, Performance and Potential" white paper, this new encoding provides an order of magnitude increase in throughput to JPEG 2000, "approximately 10x for moderate to high compressed bit rates and more than 30x for lossless coding." Because HTJ2K serves as a drop in replacement for the J2K-1 algorithm, the HT block coder can be used with all technologies described in the JPEG 2000 family of standards. In that case, HTJ2K preserves all features of JPEG 2000 Part 1 encoding besides quality scalability but includes support for resolution scalability, multi-spectral and hyper-spectral content, and non-iterative rate control among others. See J2K_C and JP2_FF for a review of the features associated with JPEG 2000 Part 1. HTJ2K also provides for reversible transcoding to and from J2K-1.

Because of HTJ2K’s efficiency and high throughput, it supports needs of professional video capture, mobile and satellite imaging applications, efficiency image capture, preview, and browsing. See JPH_FF for more details on the usage scenarios for HTJ2K and the JPH file format.

See Notes for more information about the encoding and testing.

Comments welcome.

Fully disclosed. This is an ISO published standard, specified in ISO/IEC 15444-15: Information technology — JPEG 2000 image coding system — Part 15: High-Throughput JPEG 2000 and co-published as ITU-T T.814.

As of June 2019, the International Telecommunications Union’s (ITU) prepared the above-mentioned ISO standard as the ITU-T T.814 standard. This standard is specified in the ITU-T T. 814 Information technology – JPEG 2000 image coding system: High-throughput JPEG 2000.

As stated in the ISO/IEC 15444-15 standard, "Recommendation ITU-T T.814 (2019) is a common text with ISO/IEC 15444-15:2019, both in their first edition."

As of October 10, 2022 this ISO standard is in published status. Per the defined scope language in ISO/IE 15444:15, "specifies an alternate block-coding algorithm that can be used in place of the block-coding algorithm specified in ISO/IEC 15444-1. This alternate block-coding algorithm offers a significant increase in throughput at the expense of slightly reduced coding efficiency, while a) allowing mathematically lossless transcoding to and from codestreams that use the block-coding algorithm in ISO/IEC 15444-1, and b) preserving codestream syntax and features specified in the JPEG 2000 Part 1 Codec."

Per the JPEG HTJ2K primer white paper, SMPTE has added support for the new High Throughput JPEG 2000 encoding in MXF files in the SMPTE 422 standard.

The Kakadu SDK supports HTJ2K and the JPH_FF file format in conjunction with supported features from the following JPEG 2000 standards; Part 1, Part 2, Part 3, and Part 9. See Notes for links to related ISO specifications.

There are also a variety of open source implementations of HTJ2K including but limited to:

• OPENJPH
• OPENHTJS
• OpenHTJ2K
• MatHTJ2K
• Grok
• OPENJPEG
• HTJ2K Reference Software

Comments welcome.

Per HTJ2K's "High Throughput JPEG 2000 (HTJ2K): Algorithm, Performance and Potential" white paper, the HTJ2K encoding on the JPEG 2000 is a royalty free standard. Kakadu LTD, as owner of the intellectual property behind the technologies of the HT block coding algorithm and the FBCOT framework, has made royalty free declarations to the ITU and ISO. Kakadu’s objective is to maintain the royalty free status for the entirety of JPEG 2000 standards.

Depends upon algorithms and tools to read; will require sophistication to build tools.

See JPH_FF.

See JPH_FF.

See JPH_FF.

See JPH_FF.

Joint publication of HTJ2K specifications by ITU and ISO/IEC. The JPEG High-Throughput encoding is published as ISO/IEC 15444 Part 15 and as ITU T.814. Per the defined scope language in ISO/IE 15444:15, "this alternate block-coding algorithm offers a significant increase in throughput at the expense of slightly reduced coding efficiency, while a) allowing mathematically lossless transcoding to and from codestreams that use the block-coding algorithm specified in Rec. ITU-T T.800 | ISO/IEC 15444-1, and b) preserving codestream syntax and features specified in Rec. ITU-T T.800 | ISO/IEC 15444-1."

ITU published an equivalent technical report as T.814 Information technology – JPEG 2000 image coding system: High-throughput JPEG 2000. The summary of T.814 includes, "Rec. ITU-T T.814 | ISO/IEC 15444-15 specifies a high-throughput (HT) block-coding algorithm that can be used in place of the block-coding algorithm specified in Rec. ITU-T T.800 | ISO/IEC 15444-1. The HT block-coding algorithm increases decoding and encoding throughput and allows mathematically lossless transcoding to and from the block-coding algorithm specified in Rec. ITU-T T.800 | ISO/IEC 15444-1. This is achieved at the expense of some loss in coding efficiency and substantial elimination of quality scalability."

Additional related specifications:

• ISO/IEC 15444-1:2019 Information technology — JPEG 2000 image coding system — Part 1: Core coding system.
• ISO/IEC 15444-2:2021 Information technology — JPEG 2000 image coding system — Part 2: Extensions.
• ISO/IEC 15444-3:2007 Information technology — JPEG 2000 image coding system: Motion JPEG 2000 — Part 3.
• ISO/IEC 15444-4:2021 Information technology — JPEG 2000 image coding system — Part 4: Conformance Testing. Per the HTJ2K white paper page 5, this standard is the subject of an active amendment to augment Part 1 codestreams and conformance testing with Part-15.
• ISO/IEC 15444-9:2005 Information technology -- JPEG 2000 image coding system: Interactivity tools, APIs and protocols - Part 9.A newer version of this specification is under development. See next specification.
• ISO/IEC 15444-16:2021 Information technology — JPEG 2000 image coding system — Part 16: Encapsulation of JPEG 2000 images into ISO/IEC 23008-12. A revision is currently underway to support more flexible warping of all JPEG 2000 code-streams including HTJ2K.
• o T.800 : Information technology - JPEG 2000 image coding system: Core coding system.

Regarding HTJ2K.com. The HTJ2K site provides a high level overview of the encoding and related file format while also consolidating resources for interested researchers and users. The site is owned and maintained by Kakadu Software Pty, who own the property rights over the HT block coding algorithm that is the key component of the new encoding.

Regarding encoding. As stated in the "High Throughput JPEG 2000 (HTJ2K): Algorithm, Performance and Potential" white paper, the HT block coding algorithm is faster than traditional J2K-1 implementations. The J2K-1 algorithm is based upon the Embedded Block Coding with Optimized Truncation (EBCOT) algorithm used in conjunction with the Discrete Wavelength Transform (DWT). The HTJ2K encoder uses the Fast Block Coding with Optimized Truncation (FBCOT) algorithm to achieve targeted compression size but also works in conjunction with DWT to produce sub-bands and LL sub-band. The HT algorithm, like J2K-1, adopts a coding pass structure (see page 12 and 13 of the white paper), with Cleanup, SigProp, and MagRef coding passes. Based on the end to end encoding and decoding system, the new HTJ2K code-stream can be almost 10 times faster than the J2K-1 algorithm with much larger speedups, "on the order of 30x for lossless compression and decompression." (see page 4 of the white paper). The new HT block coding algorithm can be used with many of technologies described in the JPEG 2000 standards family while retaining most existing features, while also supporting completely lossless transcoding to and from code-streams that have been generated using the original J2K-1 algorithm.

For a diagram of the overall architecture, see Figure 2 on page 4 of the JPEG primer white paper or Figure 2 on page 12 of the May 2020 High Throughput JPEG 2000 white paper. The HTJ2K compression and decompression system preserves the existing architecture and syntax of JPEG 2000. The white papers describe in more detail the components of the HTJ2K architecture and the process of input images to the decompressed images.

Regarding testing. The following summarizes JPEG’s 2022 published case study on High-Throughput JPEG 2000’s performance on laptops and desktops. A 2018 15-inch MacBook Pro with 16GB RAM and Intel i9 processor was used for speed testing with four primary software applications.

• BlackMagic DaVinci Resolve Studio 17.4.4, a video editing application for macOS, Windows, and Linux.
• Apple Compressor, Apple’s proprietary software application that integrates with Final Cut Pro
• Kakadu v8.2.1 applications, the case study used kdu_vcom_fast and kdu_vex_fast applications that are included in the Kakadu software developer toolkit for JPEG 2000 encoding and decoding.

As indicated in Figure 1 and Figure 2 from the JPEG 2022 case study, HTJ2K performed comparably to Apple ProRes, which surpassing ProRes in distortion efficiency. This summary won’t dive into the mathematical specifics of the rate-distortion performance, but further details can be found in the case study on page 4.

The following list summarizes the development work of the ISO/IEC 15444-15 standard from the initial Call for Proposal (CfP) in 2017 to its submission to ISO in early 2019. This list summarizes the ISO/IEC JPEG meetings that occur four times a year to discuss and develop related standards.

• 75th ISO/IEC JTC1/SC29/WG1 (JPEG) Meeting – March 2017. JPEG committee begins work on Part 15 of JPEG 2000 standard, with the stated of project to, "identify and standardize an alternate block coding algorithm that can be used as a drop-in replacement for algorithm defined in JPEG 2000-Part 1," per the Association of Computing Machinery’s (ACM) JPEG meeting overview. A draft Call for Proposals was issued for public comment and made available on the JPEG website.
• 78th ISO/IEC JTC1/SC29/WG1 (JPEG) Meeting – February 2018. The JPEG’s publicly distributed list of Ad hoc groups present for the 78th ISO/IEC/WG1 meeting includes the Ad Hoc Group on High Throughput JPEG 2000 (HTJ2K). The stated mandates and objectives from that meeting were identified as follows: Process submissions to the CfP, organize independent performance of the evaluation procedure on submitted Block Coder Libraries, and process comments to ISO/IEC 15444-1:2016 PDAM ballot.
• 80th ISO/IEC JTC1/SC29/WG1 (JPEG) Meeting – July 2018. The JPEG committee announces that ISO/IEC 15444-15 High-Throughput JPEG 2000 (HTJ2K) is in "Draft" status.
• 81st ISO/IEC JTC1/SC29/WG1 (JPEG) Meeting – October 2018. The JPEG committee announces that ISO/IEC 15444-15 High-Throughput JPEG 2000 (HTJ2K) has reached Draft International Status. The JPEG committee also announces that they have begun development on conformance codestreams and reference software.
• 83rd ISO/IEC JTC1/SC29/WG1 (JPEG) Meeting – March 2019. The JPEG committee announces that ISO/IEC 15444-15 High-Throughput JPEG 2000 (HTJ2K) has been submitted to ISO for immediate publication as an International Standard.