Sustainability of Digital Formats: Planning for Library of Congress Collections

A Scene Layer Package (SLPK) file is an openly published file format for 3D geospatial data, designed as a package for storage or exchange that captures all resources for a "scene layer" structured according to the Indexed 3D Scene Layer (I3S) specification. The I3S/SLPK specification originates from Esri ( formerly known as Environmental Systems Research Institute, Inc.). It has been in use in products in Esri's ArcGIS range since 2016. In September 2017, it was approved by the Open Geospatial Consortium (OGC) as a community standard: OGC Indexed 3d Scene Layer (I3S) and Scene Layer Package Format Specification.

An I3S Scene Layer, sometimes also referred to as an I3S dataset, can contain arbitrarily large amounts of 3D geospatial data of a specific type. I3S is designed specifically to support rapid streaming and access to large caches of 3D content over the web; the specification includes an API to support streaming. A common application is interactive visualization, allowing a user to move through a 3D scene. See the example of buildings in Montreal. SLPK is the corresponding file format, using the extension .slpk. I3S and SLPK are defined in a single specification document. Because the Sustainability of Digital Formats Web site focuses on the long-term preservation of digital content, this format description is for SLPK, sometimes termed the "persistence format" for an I3S dataset. The content of an SLPK file for a Scene Layer corresponds exactly to the resources available for the I3S scene layer through a streaming "scene service" or other application based on the I3S API. I3S is designed to facilitate delivering only the data needed to present the view and level of detail called for by user interactions and the SLPK format supports the same objective. A hierarchical arrangement of nodes is implemented via a hierarchy of index files (3dNodeIndexDocuments), one for each node. This structure facilitates efficient location and extraction of nodes and associated resources individually, by traversing the tree from the root node.

An I3S Scene Layer may be one of several layers in a "web scene" presented through a screen service. An SLPK file is designed to correspond to a single I3S Scene Layer. Each I3S Scene Layer corresponds to a single layer type; each layer type corresponds to a profile [see table in 6.2 I3S – Overview]. The combination of layer type and profile describe the behavior of the layer and how it is represented in the SLPK file. As of late 2018, the following layer types are supported in ArcGIS:

• 3D Object Scene Layers represent large collections of geo-referenced 3D models representing features such as building exteriors. Associated profile is Mesh-pyramids. Used for city models. An example is of buildings in San Francisco.
• Point Scene Layers represent vector GIS points with either two dimensional or three dimensional coordinates. Point scene layers can mark points using either 2D symbols or 3D symbols in which each symbol is a 3D model. Often used in city models to mark hospitals, schools, etc. An example is for trees in San Diego Parks. Associated profile is Points.
• Integrated Mesh Scene Layers can be used to represent textured triangular mesh data and are typically used to describe surfaces, such as the "skin" of the earth. Such layers are often derived from drone, aerial, or satellite imagery. Integrated meshes may be as small as a single building or as large as entire countries or states. Associated profile is Mesh-pyramids. See Yosemite area integrated mesh data extracted from satellite imagery by VRICON. See also example of animation of fly-through generated from mesh data from VRICON.
• Point Cloud Scene Layers provide fast display of large volumes of symbolized and filtered point cloud data. They are optimized for the display and sharing of many kinds of sensor data, including lidar. Associated profile is Pointclouds (not documented in the OGC standard).

The first three layer types above are included in the OGC standard. Point Cloud Layers have been supported in ArcGIS Pro since version 1.4 (released in January 2017). Such a new layer type can be introduced as an extension to I3S without requiring a change to the SLPK package specification. For examples of all four types, see OGC Indexed 3D Scene Layer Examples. See more technical detail about the layer types as implemented by Esri at Indexed 3D Scene Layers.

The structure of an IS3 layer is a hierarchical index structure of nodes, in which each node’s "payload" may contain features with associated geometry, textures and attributes. The layer type and profile define which particular geometries and attributes are required or allowed and whether textures are applicable. For details of the general node-based structure, see 7.3 Indexed Scene Layers - Organization and Structure or the Note on I3S Layer Organization below.

The specification for a Scene Layer Package (SLPK), together with a helpful diagram, is in 10.1 Scene Layer Packages within the I3S/SLPK specification. The package is based on a constrained form of the ZIP-PK format and represents the hierarchical structure in a hierarchy of folders. SLPK requires the use of the ZIP64 variant of the ZIP format, to accommodate large files. The ZIP compression method must be either no compression (aka "store") or DEFLATE64. Standard DEFLATE is acceptable as a fallback if DEFLATE64 is not available, but will only work with smaller SLPK files.

At the top level, the ZIP-based SLPK format holds:

• A metadata.json file that holds essential technical metadata for the SLPK package. This is the only JSON file in the package that may not be compressed with GZIP.
• A 3dSceneLayer.json.gz file that provides descriptive and administrative metadata for the Scene Layer as a whole. The file is typically compressed with GZIP.
• A nodes folder containing all node resources in a tree structure. All Node Index resources are stored as JSON, with each JSON file typically compressed individually with GZIP. Value schemas are used for JSON files to ensure that the content of a JSON property follows a fixed pattern. As described below in the Note on I3S Layer Organization, the sub-resources associated with each node are encoded either as JSON or in a binary form defined in the I3S/SLPK standard. Except for Texture Data, these sub-resources may all be compressed individually with GZIP.

Esri supports generation of IS3 layers and SLPK files in many of its ArcGIS products. The document Esri provided to OGC to justify adoption of I3S/SLPK as a community standard (Community Standard Justification: 16-133r1) lists a number of Esri products with support as of September 2016.

As of November 2018, https://github.com/Esri/i3s-spec indicates that four vendors can generate I3S and SLPK for "integrated mesh" layers: Bentley (ContextCapture); Vricon; Pix4D; Melown Photogrammetry. For more examples of support for particular layer types, see Useful References below.

An evaluation matrix for support in several GIS "solutions" for vector tiling in OGC TestBed 13: Vector Tiles Engineering Report (February 2018) suggested that the only support for SLPK as input or output was by Esri. In addition to Esri I3S, solutions investigated were Mapbox Vector Tiles, Cesium 3D Tiles, Ecere Gnosis, and Geoserver.

However, tools for manipulating and transforming SLPK files exist. In FME 2018.1, released in late 2017, FME from Safe Software introduced an IS3/SLPK writer. In November 2018, the open-source Point Data Abstraction Library (PDAL) introduced a reader for SLPK files for point clouds.

According to a presentation by Esri staff at the 2017 Esri User Conference, I3S and SLPK have no commercial dependencies. Anyone can: inspect I3S layers with common tools such as JSON browsers, zip file readers, and text editors; create an I3S layer according to the specification; or read an I3S layer created by themselves or another application.

Esri (Environmental Systems Research Institute, Inc.) granted the Open Geospatial Consortium (OGC) a nonexclusive, royalty-free, paid up, worldwide license to copy and distribute the I3S/SLPK specification under a Creative Commons ShareAlike (CC BY-SA) license.

None beyond software that can import data in this format. However, in early 2019, it seems unlikely that any software except Esri products can fully support reading, rendering visualizations of, and writing for all layer types.

In practice, the large size of SLPK files may cause problems for generic tools.

The SLPK file format is not designed to hold a general dataset, but can incorporate data associated with its geo-referenced nodes as attributes.

Basic data types supported for attributes are: integers (Int16, Int32, Int64), unsigned integers (UInt8, UInt16, UInt32, UInt64), floating point numbers (Float32, Float64), and String (UTF-8). Attribute values can also be arrays of the basic data types. Dates (as strings) and various data types used as identifiers and pointers are also supported.

Data types supported in JSON-based resources include: Integer (Int32), Float (Float64), String (UTF-8), Date (ISO 8601 timestamp YYYY-MM-DDThh:mm:ss.sTZD with fixed timezone), and various data types used to identify sub-resources and support links among them.

The SLPK format supports 3D geometry definition using several approaches, including: triangular meshes and 3D objects defined by faces and vertices. Point clouds are supported in Esri applications and have been proposed as an extension to the OGC specification for I3S/SLPK.

The concept of Level of Detail (LoD) is intrinsic to the I3S standard, to support efficient visualization at different levels of zoom. Scene Layers may include levels of detail that apply to the layer as a whole and serve to generalize or summarize information for the layer, similar to image pyramids and raster tiling schemes. See 7.4 Level of Detail Concept in the I3S/SLPK specification for details.

The binary representation of geometry used in SLPK employs the TypedArray construct defined in the 6th edition (2015) of ECMAscript (aka Javascript).

I3S Layer Organization: I3S organizes information using a tree structure in which each node’s payload may contain features with associated geometry, textures and attributes. The information for a node is stored in multiple individually accessible resources. The Node Index Document is a lightweight resource representing a node, its topology within the tree, and references to other sub-resources. These index documents facilitate efficient navigation of the tree from a node to its parent, children, and siblings to locate the nodes required to generate the view called for by the rendering application and/or its user by panning or zooming. The root node represents the layer as a whole, an interior node has a parent and at least one child. Each interior node logically contains or covers the set of information covered by the nodes below it and participates in a path to the leaf nodes below it. Interior nodes may contain generalized or reduced representation of the information contained in descendant nodes. The Feature Data sub-resource for a node is a text resource that contains the identifiers for the set of features within a node. It can store the geometry and attributes for all of the features in the node either by value or as references into geometry and attribute sub-resources for the node. Nodes may also have Geometry, Attribute and Texture sub-resources appropriate to the layer type or profile. Brief descriptions of these sub-resources follow:

• GeometryData: Employs basic geometry types: points, lines, triangles. Can define faces and vertices. Represented in Geometry.bin, a binary format defined in the I3S/SLPK standard.
• FeatureData: Describes geospatially positioned features with references to geometry attributes and to other attributes identifying symbols, styling, or materials that could guide the rendering of appearance. A feature is usually representative of a feature present in the real, geographic world. Represented in JSON.
• TextureData: Images used as texture maps. I3S supports multiple texture formats, including JPEG, PNG, and optimized compressed texture formats such as S3TC. Represented in Textures.bin, a binary format defined in the I3S/SLPK standard, capable of holding several texture images.
• AttributeData: Tabular data associated with features used for attribute-based rendering or representation by symbols. Represented in a structured binary form defined in the I3S/SLPK standard. Example: Trees in San Diego Parks. Each tree positioned on the map has values for attributes such as: Tree genus; Crown height, Crown diameter. The attributes are used both to display this data and to scale the tree model displayed.
• SharedData: Shared resources identify models or textures that can be shared among features within the same scene layer. Represented in JSON.

The I3S structure also includes information about Levels of Detail, information that can guide an application as to which nodes and resources to use when a user zooms in or out within the display of a scene. A node in the I3S scene layer tree is somewhat analogous to a tile in a raster tiling scheme.

3D Model Characteristics: The quality and functionality factors used above in relation to support for 3D model support in SLPK are based on an analysis from 2008 in An Overview of 3D Data Content, File Formats and Viewers, a technical report (ISDA08-002) from the National Center for Supercomputing Applications (NCSA). According to this analysis, the key aspects of a 3D model are: geometry, appearance, scene, and animation. For brief descriptions of these factors, see the table for 3D model formats under Quality and functionality factors. The descriptions there are drawn from the 2008 report and from 8 Most Common 3D File Formats of 2018 (an article from C|net), which uses the same factors to distinguish formats.

Web Scenes and scene layers were introduced by Esri in concept in late 2014 with the pre-release of ArcGIS Pro 1.0, offically released in January 2015. The specification was first mounted by Esri as an open source project on github in April 2015. ArcGIS Pro 1.x upgrades followed in: June 2015 (1.1); March 2016 (1.2); July 2016 (1.3); January 2017 (1.4). Support was steadily added for scene layers for different layer types. In version 1.4, the file extension .slpk replaced .spk for SLPK files. A major release, to ArcGIS Pro 2.0 came in June 2017; see What's new in ArcGIS Pro 2.0 for information about incompatibilities. Presentations introducing the I3S specification were made at the Esri Developers Summit in March 2017, and the Esri User Conference in July 2017. See Useful References below.. In September 2017, OGC published OGC Indexed 3D Scene Layer (I3S) and Scene Layer Package Format Specification as a community standard.

A draft of an I3S extension to handle point clouds is available. The following layer types are also planned for future inclusion: Line Features; Polygon Features.