Sustainability of Digital Formats: Planning for Library of Congress Collections

Introduction | Sustainability Factors | Content Categories | Format Descriptions | Contact
Format Description Categories >> Browse Alphabetical List

High Efficiency Video Coding (HEVC) Family, H.265, MPEG-H Part 2

>> Back
Table of Contents
Format Description Properties Explanation of format description terms

Identification and description Explanation of format description terms

Full name High Efficiency Video Coding (HEVC) Family, H.265
Description

High Efficiency Video Coding (HEVC) is an international standard defined jointly by ISO/IEC (as ISO/IEC 23008-2) and ITU-T (as H.265) and also referred to as MPEG-H Part 2. It was first published in 2013. HEVC was seen as an evolution of earlier video coding specifications, namely ITU-T H.261 (MPEG-1), ITU-T H.262 (MPEG-2), ITU-T H.263 (MPEG-4, Visual Coding) and ITU-T H.264 (MPEG-4, Advanced Video Coding). HEVC was developed in response to the growing need for higher compression of moving pictures for various applications such as streaming, videoconferencing, digital storage media and television broadcasting. The HEVC syntax was designed to enable the use of the coded video representation in a flexible manner for a wide variety of network environments and to enable a high compression capability for a desired image or video quality.

The family of HEVC encodings comprises a large number of profiles (see below) aimed, according to the specification, at different application areas, including, but not limited to: broadcast (cable TV on optical networks / copper, satellite, terrestrial, etc.); camcorders; content production and distribution; digital cinema; home cinema; internet streaming, download and play; medical imaging; mobile streaming, broadcast and communications; real-time conversational services (videoconferencing, videophone, telepresence, etc.); remote video surveillance; storage media (optical disks, digital video tape recorder, etc.); and wireless display. Each profile is defined by constraints on the syntax as defined in the full HEVC (H.265) specification.

A concise summary of the structures available in the HEVC syntax is provided in High Efficiency Video Coding (HEVC) Test Model 16 (HM 16) Encoder Description Update 13 (January 2020): "HEVC has a block-based hybrid coding architecture, combining inter [temporal, see Notes below] and intra [spatial, see Notes below] prediction and transform coding with high efficiency entropy coding. However, in contrast to previous video coding standards, HEVC employs a quadtree coding block partitioning structure that enables a flexible use of large and small coding, prediction, and transform blocks. HEVC also allows for improved intra prediction and coding, adaptive motion parameter prediction and coding, a new loop filter and an enhanced version of context-adaptive binary arithmetic coding (CABAC) entropy coding over that defined by previous standards. New high-level structures have also been designed to aid parallel processing." The introduction to HEVC in a 2017 video titled Introducing HEIF and HEVC, presented at the Apple WWDC 2017 conference for developers at which Apple announced the new default formats being introduced for image and video capture on Apple mobile devices, discusses some key features that make HEVC different from the AVC (H.264) codec that preceded it. Evaluation of High Efficiency Video Coding (HEVC) for 3GPP services, a technical report from the 3rd Generation Partnership Project, also provides a useful description of HEVC, focusing on comparison to AVC.

HEVC is a important advance on AVC (H.264), in that it supports higher bit depths and enhanced chroma formats, including the use of full-resolution chroma (4:4:4). It might be assumed that a greater number of pixels in the source data signal leads to larger files and greater demand for transmission bandwidth. However, key innovations designed to support efficient compression mean that this is not necessarily the case. Chapter 9 of High Efficiency Video Coding (HEVC): Algorithms and Architectures describes in detail comparisons using objective and subjective measures in compression performance of the reference software for HEVC and AVC for a varied test set of video sequences. The objective evaluation used average differences of R-D (rate-distortion) curves based on PSNR (peak signal to noise ratio) measured against bit rate. Based on this objective measure, an overall performance improvement of of HEVC over AVC was between 22% and 43% for four different choices of coding parameters aimed at different use cases. Results of subjective evaluation tests indicated that an even higher bit rate saving in the ranges of 55–87% could be achieved. A separate objective evaluation of video sequences in the larger 4k resolution (3,840 x 2,160 pixels) showed up to 76% improvement.

Innovations highlighted in descriptions of HEVC include the following:

  • One of the major features that contribute significantly to the coding efficiency of H.265 (HEVC), particularly for high resolution formats, is the usage of flexible block sizes for coding and transforms. Unlike H.264 (AVC), where the basic coding block is a macroblock of fixed size 16x16, H.265 (HEVC) defines a Coding Tree Unit (CTU) of a maximum size of 64x64. Each CTU can be divided into smaller units in a hierarchical manner (known as a quad-tree) and can represent smaller blocks of size 4x4. Similarly, the transforms used in H.265 (HEVC) can have different sizes, starting from 4x4 and going up to 32x32.
  • Intra coding in HEVC allows 35 prediction modes – the DC [discrete cosine] mode, the planar mode and 33 directional (or "angular") modes. See subclause 8.4.2 of version 7 of the HEVC specification. AVC included 9 modes - the DC mode and 8 directional modes. See subclause 8.3.1.1 of version 13 (6/2019) of the AVC specification. The added angular modes can model different directional lines and structures in an image more accurately and the planar mode can smooth regions with gradually changing colors.  The increased flexibility improves both objective coding efficiency and visual quality as edges can be better predicted and ringing artifacts around the edges are reduced. To avoid contouring artifacts, a new interpolative prediction option is included to improve the visual quality.  Discrete Sine Transform (DST) is utilized instead of traditional Discrete Cosine Transform (DCT) for 4x4 intra transform blocks.  For full details, see Intra Coding of the HEVC Standard (2012).
  • Prediction residuals after inter coding within individual blocks may be further compressed to remove spatial correlation inside the block before it is quantized, typically discarding less important visual information while forming a close approximation to the source data.
  • HEVC has greater flexibility and adaptivity in its transform and quantization design and includes some additional coding modes in which the transform stage and sometimes also the quantization stage are skipped altogether. HEVC allows certain part of the coded picture to be coded in a lossless manner.
  • In common with its AVC predecessor, HEVC contains an in-loop deblocking filter, where the blocking artifacts around the transform edges in the reconstructed picture are smoothed to improve the picture quality and compression efficiency. This filter has been simplified and made more parallel-friendly for HEVC.  Moreover, HEVC introduces a new filtering stage called the sample-adaptive offset (SAO) filter, which can provide both an objective and subjective improvement in video quality.
  • HEVC uses a single entropy coding engine, based on Context Adaptive Binary Arithmetic Coding (CABAC). CABAC in HEVC shares many similarities with CABAC as specified for AVC, but contains several modifications that improved coding efficiency and lowered implementation complexity, especially for parallel architectures.
  • Any bitstream compliant with HEVC is organized into network abstraction layer (NAL) units, which are self-contained packets that allow the video layer to be identical for different transmission environments. Each NAL unit is associated with a particular temporal layer. The syntax is constrained in such a way that a decoder capable of decoding at a given frame rate is expected to discard NAL units associated with intermediate pictures used for higher frame rate decoding. This feature is known as "temporal scalability." See section on hierarchical encoding into temporal layers from an Apple presentation for developers at WWDC 2017 or the diagram Two HEVC encoder methods for block artifact reduction: Fig. 4.

Versions and profiles: As of November 2020, there are seven versions of the HEVC specification. Most of the updates have introduced new profiles, as listed below. In a profile name that ends in a number, that number indicates the maximum bit depth per channel for color representation. If a profile does not include a chroma indication, and is not for a monochrome picture, the profile is limited to 4:2:0 chroma support.

  • ITU-T H.265 (V1) (04/2013). Version 1 of the HEVC specification introduced Main, Main Still Picture, and Main 10 profiles. These are all limited to the 4:2:0 chroma format. The Main and Main Still Picture profiles are limited to 8 bits per color channel.
  • ITU-T H.265 (V2) (10/2014). Version 2 of the HEVC specification introduced a large number of profiles, of which most extended bit-depth and/or chroma support and are often referred to as a group, as Range Extension (RExt) profiles. 4:2:2 profiles addressed needs for camera-captured video for broadcast. 4:2:0 and 4:2:2 with bit depth 12 addressed expectations for ultra-high definition (UHD) equipment and services, for which support for high dynamic range (HDR) video would be needed. See Overview of the Range Extensions for the HEVC Standard: Tools, Profiles, and Performance (2016). The new profiles were:
    • Monochrome, Monochrome 12, Monochrome 16
      The Monochrome profile, which allows 8 or 10 bits per pixel, is suitable for alpha (transparency) channels and depth maps. Monochrome images with higher bit depth are used in medical imaging applications.
    • Main 12
    • Main 4:2:2 10, Main 4:2:2 12
    • Main 4:4:4, Main 4:4:4 10, Main 4:4:4 12
      Supporting both inter-picture and intra-picture coding. 4:4:4 chroma variants are important not only for for studio and professional content production, but also for applications involving remote screen-sharing.
    • Main Intra, Main 10 Intra,  Main 12 Intra,  Main  4:2:2 10 Intra, Main 4:2:2 12 Intra, Main 4:4:4 Intra, Main 4:4:4 10 Intra, Main 4:4:4 12 Intra, Main 4:4:4 16 Intra
      The Intra profiles are only permitted to use intra-picture prediction, resulting in coding that only includes I-frames.
    • Main 4:4:4 Still Picture, Main 4:4:4 16 Still Picture
    • High Throughput 4:4:4 16 Intra
      Intended for codecs embedded in professional cameras.
    Additional profiles added in version 2 of HEVC include:
  • ITU-T H.265 (V3) (04/2015). Version 3 of the HEVC specification introduced one profile.
  • ITU-T H.265 (V4) (12/2016). Version 4 of the HEVC specification introduced several new profiles and defined additional supplemental enhancement information (SEI) payloads and additional color representation identifiers. This revision also corrected various minor defects in the prior content of the specification. The following profiles were added to Annex A.
    • High Throughput 4:4:4, High Throughput 4:4:4 10, High Throughput 4:4:4 14
    • Screen-Extended Main, Screen-Extended Main 10. The screen-extended profiles, referred to collectively as HEVC-SCC, take advantage of the fact that screen content frequently contains no sensor noise, and such content may have large uniformly flat areas, repeated patterns, highly saturated or a limited number of different colors, and numerically identical blocks or regions among a sequence of pictures. These characteristics offer opportunities for significant improvements in compression efficiency over a coding system primarily designed for camera-captured content. See, for example, Overview of Screen Content Video Coding: Technologies, Standards, and Beyond (2016) or Overview of the Emerging HEVC Screen Content Coding Extension (2015).
    • Screen-Extended Main 4:4:4, Screen-Extended Main 4:4:4 10
    • Screen-Extended High Throughput 4:4:4, Screen-Extended High Throughput 4:4:4 10, Screen-Extended High Throughput 4:4:4 14
    The following scalable profiles were added to Annex H:
    • Scalable Monochrome, Scalable Monochrome 12, Scalable Monochrome 16
    • Scalable Main 4:4:4
  • ITU-T H.265 (V5) (02/2018). Version 5 of the HEVC specification added SEI message payloads, including messages to support omnidirectional 360° video as used in augmented reality (AR) and virtual reality (VR) applications. See , for example, Towards Bandwidth Efficient Adaptive Streaming of Omnidirectional Video over HTTP: Design, Implementation, and Evaluation (2017). Minor defects in the prior content of the specification were corrected. Profiles added were:
    • Monochrome 10
    • Main 10 Still Picture
  • ITU-T H.265 (V6) (06/2019). Version 6 of the HEVC specification adds additional SEI messages that include SEI manifest and SEI prefix messages, and corrections to various minor defects in prior content. No profiles were added.
  • ITU-T H.265 (V7) (11/2019). Version 7 of the HEVC specification adds additional SEI messages for fisheye video information and annotated regions, and also includes corrections to various minor defects in prior content. No profiles were added.

To support "islands of interoperability," HEVC defines not only profiles, but also levels and tiers. The discussion here is based not only on the HEVC specification but also on content found in High Efficiency Video Coding (HEVC): Algorithms and Architectures. While profiles define the syntax and coding features that can be used for the video content, a significant consideration is the degree of capability within a given feature set. Levels are levels of capability defined to establish the picture resolution, frame rate, bit rate, buffering capacity, and other aspects that are matters of degree. The lowest levels have only low resolution and low frame rate capability—e.g., a typical video format for level 1 may be only 176x144 resolution at 15 frames per second, whereas level 4.1 would be capable of 1920x1080 HDTV at 60 frames per second and level 6.1 would be capable of 8192x4320 video resolution at up to 120 frames per second. As of Version 7, Table A.8 of the HEVC specification lists the following general levels: 1, 2, 2.1, 3, 3.1, 4, 4.1, 5, 5.1, 5.2, 6, 6.1, and 6.2. Note that when a bitstream is indicated to conform to a specific level it is also considered as conforming to higher levels. Levels higher than 4 have variants for Main and High tiers, with a key difference being that the High tier defines a much higher maximum size for coded picture buffers (CPB), which increases the bit rates decoders must be capable of handling. The Still Picture profiles may use a special level (8.5) that indicates no limit on picture size or complexity.

One use of levels is in specifications based on HEVC encoding. For example, DICOM specifies the use of Main or Main 10 profiles at level 5.1 and ATSC 3.0 - A/341 refers to levels 3.1, 4.1, 5, 5.1, and 5.2 for Main 10 and Scalable Main 10 profiles. The compilers of this resource have not made an attempt to explore the relative adoption of particular profiles, tiers, and levels. Comments welcome.

Production phase Can be used at any phase of video production. Incorporated into some cameras and mobile devices for capture (initial state). Employed in formats used for end-user delivery (final-state), particularly for streaming. Can also serve middle-state postproduction and archiving uses. Has profiles to support different use cases.
Relationship to other formats
    Used by HEIF_HEIC, High Efficiency Image File Format, HEIC/HEIX brands
    Used by Other formats not described on this website that can include HEVC bitstreams, such as OMAF (Omnidirectional MediA Format, ISO/IEC 23090-2).
    Has subtype Domain-specific standards that are constrained subsets of HEVC functionality not described separately on this website. HEVC is incorporated into ATSC 3.0 from the Advanced Television Systems Committee, Inc. and listed among Data Structures and Encoding permitted in the DICOM (Digital Imaging and Communications in Medicine) standard.

Local use Explanation of format description terms

LC experience or existing holdings The Library of Congress has a handful of images in the HEIC format (a subtype of the HEIF format that uses HEVC encoding) in its digital storage system. These are likely to have been acquired in the course of archiving websites.
LC preference No format incorporating video or still image content in the HEVC (H.265) encoding is listed in the Library of Congress Recommended Formats Statement (RFS) for Video - File-Based and Physical Media, Photographs - Digital, or Other Graphic Images - Digital.

Sustainability factors Explanation of format description terms

Disclosure International standard, fully disclosed. Developed and maintained under the auspices of the Joint Collaborative Team on Video Coding (JCT-VC), a collaboration of the Motion Picture Experts Group (MPEG) and the ITU Telecommunication Standardization Sector (ITU-T). MPEG functioned as ISO/IEC JTC1/SC 29/WG11 through June 2020. After re-organization of SC29 in 2020, responsibility for maintenance of the HEVC standard is under working group WG5 (MPEG Joint Video Coding Team(s) with ITU-T SG 16). The scope of ITU-T Study Group 16 is multimedia coding, systems and applications.
    Documentation

The HEVC specification is published as ISO/IEC 23008-2: Information technology — High efficiency coding and media delivery in heterogeneous environments — Part 2: High efficiency video coding (aka MPEG-H Part 2) and as ITU-T H.265: High efficiency video coding. See ITU Recommendation H.265 (approved 11/2019) for full text of Version 7 of ITU-T H.265. See ISO/IEC 23008-2:2020 catalog record and preview for the equivalent ISO/IEC specification published in August 2020.

The HEVC section of the MPEG website has been a rich source of drafts, white papers, and intermediate documentation associated with the initial development of HEVC and its steady extension to cover additional use cases by adding profiles. The website has been largely frozen as of June 2020 as a result of the re-organization of ISO/IEC JTC1/SC 29.

While the specification describes the syntax and a suggested decoding process for coded video compliant with the standard, a possible encoding process is described in documentation for reference software, often referred to as HM (for HEVC Test Model). See, for example, High Efficiency Video Coding (HEVC) Test Model 16 (HM 16) Encoder Description Update 13 (January 2020), which serves as an overview of HEVC Version 1 and the Range Extensions of HEVC Version 2, and also provides an encoder-side description of the HM-16.21 software version. The HM software is open source, maintained at https://vcgit.hhi.fraunhofer.de/jct-vc/HM.

Two books by members of the JCT-VC development group were published in 2014 and 2105 as informative companions to the formal HEVC specification.

  • High Efficiency Video Coding (HEVC): Algorithms and Architectures, Editors: Vivienne Sze, Madhukar Budagavi, Gary J. Sullivan. Has separately authored chapters on different aspects of HEVC, including: block structures and parallelism; intra-picture prediction; inter-picture prediction; transform and quantization; in-loop filters; and entropy coding. Also includes compression performance analysis in comparison to AVC (H.264) using subjective and objective methods.
  • High Efficiency Video Coding: Coding Tools and Specification, by Mathias Wien. Includes history and technical background on video coding and comparisons with AVC in all the chapters that relate to aspects of video coding (such as temporal and spatial predictive coding, color representation, etc.). Introduces the range extension (RExt) profiles included in version 2 of the HEVC specification.
Adoption

Some standards organizations other than ISO/IEC and ITU have incorporated support for particular HEVC profiles into their own standards:

HEVC has been adopted in many parts of the video creation and distribution business. The Wikipedia entry for High Efficiency Video Coding lists many of the early adopters. A selection of other examples follows:

MainConcept provides a plugin HEVC encoder for the widely used FFmpeg video toolkit. An encoding library for HEVC is available from x265.com. x265 source code is available for developers who would like to contribute to the project, or build open-source applications under the terms of the GNU GPL v2 license. Commercial licenses are available for companies who want to use x265 in video hardware and software products. An open source encoder for HEVC has also been made available by the VideoLAN project. See Videolan -- H.265. The compilers of this resources have not determined whether these projects are connected. Comments welcome.

As is the case for many moving image formats, the intra-picture features of HEVC can be used to compress still images. One such use of the HEVC is in the default format with a .heic extension used by Apple in its iPhones and iPads since 2017. This format employs the High Efficiency Image File Format (HEIF) as a container for a single image or collection of images, where the individual image items are compressed using certain HEVC profiles (Main, Main Still Image, and Main 10). See HEIF_HEIC for information about use of HEVC for still images in cameras from Sony and Canon and cellphones from Samsung.

    Licensing and patents

The underlying HEVC encoding is heavily encumbered with patents. According to HEVC Advance Passes 10,000 Patent Milestone - announces Toshiba Corp. Joins as a Licensor, a March 2020 press release, there are around 17,000. Patents have proved a general problem for video coding schemes. See, for example, Patents, Standards Brew Licensing Woes (June 2019) from EE Times. Rather than there being a single patent pool, there are three, as shown in a blog post from January 2018 by Leonardo Chiariglione.

In January 2019, a study of HEVC / H.265 standard-essential patent royalty rates was released by Unified, an international membership organization that seeks to improve patent quality and deter unsubstantiated or invalid patent assertions in defined technology sectors (Zones). One zone for Unified's activities is video codecs. The report, based on a methodology Unified describes as objective economic evaluation (OVAL), claims that it "demonstrates that HEVC rates should be comparable to or less than the cost of licensing the main patent pool for AVC (MPEGLA), HEVC’s predecessor codec." The report found that "each of the three major licensors are asking for licensing rates that likely exceed what the total HEVC rate should be, both individually and when taken together."

It is clear to the compilers of this resource that the patent situation has been an impediment to adoption. In particular, the lack of built-in support for HEIF formats in browsers and in operating systems other than Apple's iOS and Mac OS seems likely to be related to the challenges and risks associated with the patent landscape. Comments welcome. Although related to the earlier AVC (H.264) coding scheme, the patent-related battle between Nokia and Lenovo in Germany in 2020 illustrates the risks. See, for example, Setback for Nokia in German patent battle with Lenovo, a November 2, 2020 post by Reuters. Nokia has not joined any of the HEVC patent pools.

Transparency Depends upon complex algorithms and tools to read; will require sophistication to build tools.
Self-documentation Depends on capabilities of container or wrapper in which coded video or image data is stored.
External dependencies HEVC defines a self-contained compression encoding for a video track or still picture.
Technical protection considerations Depends on capabilities of container or wrapper in which coded video or image data is stored or the protocol used for streaming HEVC content.

Quality and functionality factors Explanation of format description terms

Still Image
Normal rendering See HEIF_HEIC for still image application of the HEVC encoding.
Moving Image
Normal rendering Designed to support choice of balance between picture quality and file size or required bandwidth for transmission. A wide variety of profiles and levels address different use cases.
Clarity (high image resolution) HEVC can accommodate very large images and ultra-high definition (UHD) video. Picture quality including color representation and required transmission rate (bandwidth) depend on profile and level used for encoding.
Functionality beyond normal rendering Many of the features identified above in Description are beyond normal video functionality, including support for 3D functionality, stereoscopic (multiview) functionality, scalability to adapt to varying network conditions, and omnidirectional functionality for application of virtual or augmented reality.

File type signifiers and format identifiers Explanation of format description terms

Tag Value Note
Filename extension See note.  Depends on format for container file.
Indicator for profile, level, version, etc. See note.  See subclause 7.3.3 of the HEVC specification for the syntax for indicators of profile, tier, and level in a coded video sequence in a terse profile-tier-level structure consisting of a number of elements, mostly 1-bit flags. Examples of the flag elements are general_max_422chroma_constraint_flag and general_max_420chroma_constraint_flag. If both are recorded as a single bit with value 0, the picture may use the chroma 4.4.4 format. These indicator structures will be included in the Video Parameter Set (VPS) and the Sequence Parameter Set (SPS), with the possibility of occurring for each layer of a multi-layer image. The rules for determining the scope of a particular profile-tier-level structure and how to interpret the individual elements if the values for elements in the different instances of the structure differ or are not present are extremely intricate. See Notes below for more information about these parameter sets. A published specification for conformance testing is at H.265.1 : Conformance specification for ITU-T H.265 high efficiency video coding. Commercial software for checking conformance to profiles and levels, etc. is available at Allegro DVT | H.265/HEVC Elementary Streams.
Other See note.  Characteristics of an individual coded video sequence and its encoding are stored very tersely in two parameter sets within the coded video, namely a Video Parameter Set (VPS) and a Sequence Parameter Set (SPS). These require HEVC-aware tools to identify and extract. See Notes below for more information about these parameter sets. Many workflows involving HEVC encoded content will generate metadata in a more usable format to be stored with the coded video in a particular container format.

Notes Explanation of format description terms

General

Terminology The terminology used in the HEVC specification and descriptions of the encoding is complex and sometimes extends or modifies terms found in glossaries such as the Glossary of Video Terms and Acronyms from Tektronix, the FADGI Glossary, or the Wikipedia entry for Video compression picture types. The notes below relate to terms for which the compilers of this resource did not find definitions in these sources that covered HEVC usage.

  • Quadtree (often hyphenated). A mathematical term for a tree in which a parent node can be split into four child nodes, each of which may become parent node for another split into four child nodes. For an example of a quadtree as used in HEVC, see Block Structures and Parallelism Features in HEVC: Figure 3.
  • CTU (Coding Tree Unit). In HEVC a picture is divided into CTUs, which are square blocks except possibly for the bottom row and right-hand column. A CTU may be further divided into 4 child CTUs.
  • Tiles and Slices. HEVC has two additional types of partition that may be used for a picture. They consist of sequences of CTUs. A tile is a rectangular region of CTUs within a rectangular grid of tiles that form a picture; tiles are treated like independent pictures. A slice is a sequence of CTUs that can be decoded independently from other slices of the same picture. A slice can either be an entire picture or a region of a picture. One of the main purposes of slices is resynchronization in the event of data losses. In contrast to common usage, a slice in HEVC is not limited to a single row of blocks in a rectangular grid. See subclause 6.3 in the HEVC specification or Block Structures and Parallelism Features in HEVC: Figure 6.
  • I-slices (intra slices) are independently compressed HEVC slices. An I-slice is decoded using intra prediction only. An extension of the more common term, I-frame.
  • P-slices (predictive slices) employ inter-picture coding, based on differences from previous pictures in the video source. An extension of the more common term, P-frame.
  • B-slices (bi-predictive slices) are bidirectionally predicted slices using differences between the current picture and both preceding and following pictures to derive coded content for the current slice. An extension of the more common term, B-frame.
  • Intra coding uses various spatial prediction modes to exploit spatial statistical dependencies in the source signal for a single picture or tile.
  • Inter coding uses motion vectors for block-based prediction to exploit temporal statistical dependencies between different pictures. In HEVC, inter prediction can apply to block and slice substructures, not only to entire pictures or tiles.
  • SEI: supplemental enhancement information. An SEI message contains information (often referred to as the message payload) that is not necessary simply to decode the samples of coded pictures, but is a standardized, structured message to a decoder that can assist in processes related to decoding, display, or other purposes. Some advanced features, such as 3-D, will not function as intended in an application that does not recognize the corresponding SEI messages.
  • Levels consist of a defined set of constraints on the values that may be taken by HEVC syntax elements and variables or the value of a transform coefficient prior to scaling. A common "general" set of levels is defined for all HEVC profiles. Hence most aspects of the definition of each level are common across different profiles. Within a coded video sequence, profiles, levels, and tiers are identified in very terse fashion in parameter sets and not accessible to simple tools. See File type signifiers above.
  • Tiers are specified categories of level constraints imposed on values of the syntax elements in the bitstream, where the level constraints are nested within a tier and a decoder conforming to a certain tier and level would be capable of decoding all bitstreams that conform to the same tier or the lower tier of that level or any level below it.
  • CPB (coded picture buffer) is a buffer containing decoding units in decoding order specified in the hypothetical reference decoder in Annex C of the HEVC spec. The size of CPB in content coding generated by an encoder is an important factor for decoders. Levels specify a maximum CPB size in bits.
  • Parameter sets in HEVC are similar to the parameter sets in H.264/AVC, and share the same basic design goals—namely bit rate efficiency, error resiliency, and supporting systems layer interfaces. There is a hierarchy of parameter sets in HEVC, including the Sequence Parameter Set (SPS) and Picture Parameter Set (PPS) which are similar to their counterparts in AVC. HEVC introduced a new type of parameter set called the Video Parameter Set (VPS). The parameter sets are stored in special network abstraction layer (NAL) units, with specific NAL unit types encoded as binary integers at the beginning of the unit. See Table 7-1 in the HEVC specification or Table I in Overview of the High Efficiency Video Coding (HEVC) Standard for a mapping of the integer codes to human-readable names. For compression efficiency, these parameter sets are very tersely coded, primarily using 1-bit flags and binary integers. Note that if an HEVC coded video sequence or still image is embedded in an ISO_BMFF container as specified in ISO/IEC 14496-15, the VPS and SPS parameter sets will be found in an HEVC configuration item property in a box of type hvcC.
  • SPS (Sequence Parameter Set). Contains parameters that apply to an entire coded video sequence, and do not change from picture to picture within a coded video sequence. The SPS has codes for characteristics such as bit-depth, chroma format, picture width and picture height. Has NAL unit type of 33.
  • VPS (Video Parameter Set). A new parameter set defined in HEVC, which applies to all of the layers of a bitstream. A layer may contain multiple temporal sub-layers. Has NAL unit type of 32 and is stored before the SPS.
  • PPS (Picture Parameter Set). There is one PPS per picture. Among the details stored in this parameter set are: number of tile rows; number of tile columns; flags related to whether particular extensions, e.g., for 3D or screen content coding(SCC) are used; and other parameters needed for decoding the picture. Has NAL unit type of 34.
History

The article titled Overview of the High Efficiency Video Coding (HEVC) Standard, published in 2012 before the HEVC specification had been approved as a standard, discussed the antecedent standards. The summary that follows is adapted from that discussion.

Some important video coding standards have evolved in a series of well-known ITU-T and ISO/IEC standards. The ITU-T produced H.261 and H.263; ISO/IEC produced MPEG-1 and MPEG-4 Visual; and the two organizations jointly produced the H.262/MPEG-2 Video and H.264/MPEG-4 Advanced Video Coding (AVC) standards. Throughout this evolution, efforts have been made to maximize compression capability, while considering the computational resources that were practical for use in products at the time of anticipated deployment of each standard.

The two standards that were jointly produced had been widely deployed. The standard preceding HEVC, H.264/MPEG-4 AVC, was developed in the period between 1999 and 2003, and extended in several important ways from 2003–2009. By 2012, AVC had been widely used for many applications, including broadcast of high definition (HD) TV signals over satellite, cable, and terrestrial transmission systems, video content acquisition and editing systems, camcorders, security applications, Internet and mobile network video, Blu-ray Discs, and real-time conversational applications such as video chat, video conferencing, and telepresence systems. By 2012, AVC had largely displaced the older H.262/MPEG-2 Video standard within its more limited application domains.

Key drivers behind the development of HEVC were: the growing popularity of HD video, and the emergence of yet larger formats (e.g., 4k×2k or 8k×4k resolution); and the networking issues caused by the growing volume of video applications targeting mobile devices and the transmission needs for video-on-demand services. HEVC was designed with a focus on two key issues: increased video resolution and increased use of parallel processing architectures, while still addressing applications that had been supported by AVC.

The first version of the HEVC (H.265) specification was published in April 2013. Editions of the standard are listed below:

Two more recent video encodings have been developed, namely Versatile Video Coding (VVC) and Essential Video Coding (EVC). VVC, also known as ITU-T H.266, aims at improved compression over HEVC, yet higher spatial resolution, and full support for immersive applications. The first edition was approved in 2020 as ITU-T H.266 (08/2020). See also VVC Overview from Fraunhofer HHI. The objective of EVC is to be a royalty-free standard.


Format specifications Explanation of format description terms


Useful references

URLs


Last Updated: 08/12/2022