Sustainability of Digital Formats: Planning for Library of Congress Collections

The KryoFlux Stream (Stream) file format is a proprietary, publicly but incompletely documented format for each of the sectors found on a floppy disk. Stream files were not conceived of as a standalone file format per se, as they are an exact copy of the byte Stream Protocol between the KryoFlux device and the host system when communicating over a USB link which is then saved by the host system when the disk is imaged. They are hardware-specific to KryoFlux devices.

The KryoFlux manual defines Stream files, on page 28, as:

"pure, unanalyzed raw data. Be warned that writing of raw data can be neither verified, nor can [Disk Took Console] give any feedback if the data read was good in the first place. In other words: garbage in, garbage out. Since data on floppy disks is digital data stored on a basically analogue carrier, creating 1:1 perfect clones for preservation and archival is not a trivial task. Unlike controlled clones (using a preservation container that has been verified for authenticity and integrity as the result of an analysis based on a well-known format – like IPF), KryoFlux will create a clone based on entropy analysis. A major challenge in the replication process is that the data can be read from the disk differently than it was written."

Stream files are not meant for long-term preservation due to their hardware specificity. The combination of their proprietary format and large file size poses challenges. However, an exception exists for copy-protected disks. Additionally, the page 39 of the Archivist’s Guide to KryoFlux states that decisions around the use of Stream files for long-term preservation should be made institutionally. They share example situations when storing Stream files for preservation were made by different institutions. Preserving these accurately requires the use of Stream files due to their non-standard coding and formatting. Unlike traditional file systems, Stream files cannot be mounted or viewed directly; interpretation requires the use of KryoFlux software.

In comparison to a disk image file, a Stream file created from a whole disk will create a file of significantly larger size. From the Archivist's Guide to KryoFlux:

"a set of stream files for a 1.44 megabyte (MB) floppy disk may be around 20-25 MB. While such a footprint is much smaller many other file types created in contemporary digital environments, it is considerably larger than the maximum capacity of a floppy disk."

Stream files are many .raw files that represent segments of data. Meaningful information in these files begins with an Out Of Band (OOB) header, followed by the type and the data itself. Anything before the initial header can be disregarded. The encoding details of Stream files can be found on Software Preservation Society Stream Protocol.

In a Stream file, data is organized into variable-length blocks, each starting with a Block Header specifying interpretation. Two block types exist:

• In Stream Buffer (ISB) blocks convey timing values of sampled flux reversals.
• Out Of Stream Buffer (OOB) blocks aid in interpretation and verification, transmitting critical information like Index Signals timing or KryoFlux hardware information.

Further information is available in the Device Behavior section of the specification on page 11.

“ Stream Protocol”, published by the Software Preservation Society. This webpage:

"describes and documents the Stream files produced by the KryoFlux device. It is a technical document, describing a binary Stream, but some effort has been made to make it more accessible to a wider audience, including references to further information."

“ KryoFlux Stream File Documentation”, by Jean Louis-Guerin. Revision 1.1. 01/12/2013. This document is listed on the official KryoFlux webpage as “Stream Protocol Documentation,” however, the document itself states that the information is the author’s opinion based on references. Comments welcome.

Adopted by many companies and institutions including BitCurator, Duke University, The University of Texas, Yale, the New York Public Library, Princeton, and the University of California, Los Angeles (UCLA). Use of Stream files is limited by the institutions who also utilize KryoFlux software.

KryoFlux software is licensed with strict restrictions. However, the Stream file format is not licensed. The license agreement specifies that: "The software product and accompanying documentation (the program’s object code, source code and documentation, explicitly excluding the file format definitions (e.g. but not limited to IPF, STREAM, DRAFT) included within, are collectively referred to as the “Technology”) is a technology and does not imply any restrictions, warranty, [sic], obligation or any other link or association with what it may contain (the data ingested or encapsulated by the Technology is referred to as the “Content”)."

See the official license for more information.

Disc mounting software is required to read this format. KryoFlux hardware and software with their rights and licenses are required to write this format.

KryoFlux, and other emulator tools, can convert Stream files into other formats.

Source: Wikidata entries:

• KryoFlux: https://www.wikidata.org/wiki/Q59608340
• KryoFlux Stream: https://www.wikidata.org/wiki/Q29000561

The Software Preservation Society’s KyroFlux page states that the software:

“allows you to check data against integrity information implied by the sector formats. It allows imaging of disks to stream [sic] files and to a more emulator-friendly image formats at the same time. So for a dual format Amiga/Atari ST disk, KryoFlux can produce stream files, a .ADF and a .ST all on-the-fly while dumping the disk. It also allows use of stream files, dumped previously, as a ‘virtual floppy’ in order to produce sector dumps at a later time.”

The specificationstates that:

“The most important information to retrieve from a Stream File is:

The provided Index information allows to compute the precise Index Time (time between to index signals) as well as to find the Index Position in reference to the current data flux reversals."

This format was created within the KryoFlux forensic floppy disk controller software. KryoFlux was developed by The Software Preservation Society and was released in 2009.