For: Digital talking book distribution analysis. Task 2 - System design (Final report : June 29, 2006)
Section 4 - Hybrid Book Distribution System
This section of the report presents the envisioned design for the Hybrid DTB distribution system. Because this approach uses circulation of DTBs via duplication-on-demand delivery from NLS contractor-operated Centers as well as circulation of mass-duplicated DTBs from network libraries, the changes required to library and NLS operations are more extensive than they would be for the All Mass-Duplication system. However, almost all aspects of the mass-duplication portion of the Hybrid system are identical to the corresponding operations under the All Mass-Duplication system, the primary difference being the number of DTB titles and copies that would be managed by the libraries. The major components of operations for the Hybrid system are first presented, followed by the estimated major costs for audio book duplication and distribution.
In most respects, library operations for the DTB mass-duplication portion of the Hybrid system will be identical to those for operations under the All Mass-Duplication system, which are described in Section 3.1 of this report. Those areas which would be different are described below, as are the requirements for interaction with DOD Centers.
4.1.1 Book Production and Circulation: Although total annual production of new titles would be the same as under the All Mass-Duplication system, i.e., 2,000 titles per year, nominally 800 titles would be mass-duplicated in average quantities of 1,100 copies per title rather than 2,000 titles at an average quantity of 911. Thus libraries would receive approximately 880,000 copies per year rather than slightly over 1.8 million copies per year. Further analyses have not changed the conclusion from Phase 1 that a Title-Based Hybrid distribution model in which approximately 40% of new titles would be mass-duplicated is the best system for implementation, in which libraries would handle approximately 80% of total DTB circulation and DOD Centers would handle 20%.
Appendix 14 presents the results from a special analysis of title-specific circulation data (also used in Phase 1 of the study), that shows the expected circulation, collection size, inventory turnover, cumulative circulation, cumulative collection size, and cumulative inventory turnover for all ranges of the Hybrid model. For the 40% mass-duplicated titles Hybrid option, the expected annual inventory turnover will be approximately 1.6 times, as compared to about 1.0 for the All Mass-Duplication system. Such a result is expected, since libraries will be storing and circulating the most popular titles. If a 30% title mass-production Hybrid model were used, annual turnover would be about 1.9 times per year; if 50% (likely the maximum that would ever be employed in a Hybrid system), then a turnover of about 1.5 would be realized.
Appendix 15 presents variations on the Hybrid model. The columns in the table represent the proportion of new titles that would be mass-duplicated and distributed from libraries, and the rows each represent titles in 1-year age classes. These data allow the evaluation of the expected effects of a combined Title-Based and/or Time-Based Hybrid distribution model.
For the baseline plan, which is implementation of the Title-Based Hybrid with 40% of titles being mass-duplicated, the table shows that the total circulation that could expected to be handled by the libraries would be 81.5%. This is shown in the "FY 1988" row and the "40%" column. The statistics shown for FY 1988 (for each columns in the table) include activity for books shipped during FY 1988 and during all years prior to FY 1988. Since the cumulative proportion of total circulation is shown in the table, the FY 1988 row represents the expected circulation for titles of all ages in the collection.
However, if "migration" were to occur after, e.g., 8 years, libraries would remove copies of titles older than 8 years from their collections and forward them to the NLS cartridge and container reconditioning contractor(s). About 67.2% of total circulation could be expected to come from network libraries and about 33% from DOD Centers in such a combined Title-Based and Time-Based Hybrid model.
Alternatively, in a pure Time-Based Hybrid system, all 2,000 new titles would be mass-duplicated and distributed to libraries. After a given time threshold all titles (or all but a very few perennially popular titles) would migrate to DOD Center delivery from library delivery. The rightmost column in the table, i.e., for 100% of titles being mass-duplicated, shows the results for such a system. In this system, if titles remained in library circulation for 7 years and then migrated to DOD Center delivery, it would be expected that 81.4% of circulation would come from libraries and the remainder (about 19%) from DOD Centers, effectively achieving the same results, in terms of the allocation of circulation workload between libraries and DOD Centers, as the 40% Title-Based Hybrid system.
Based upon input from the DLTPG, some type of Time-Based migration of titles from libraries to DOD Centers may be feasible. Because there are so many possible combinations to consider, and because there is nevertheless genuine uncertainty about the feasibility of such migration (i.e., network libraries and NLS agreeing on (1) which titles would not migrate, and (2) when the migration would occur), we continue to recommend implementation of a Title-Based Hybrid system, but this does not preclude later implementation of a Time-Based migration component if later deemed feasible.
4.1.2 Changes to Library Information Systems. ManTech has discussed the required changes to library information systems necessary for operation under the Hybrid distribution model, specifically the changes that would be necessary to facilitate information exchange between the libraries and DOD Centers. The overall conclusion from system vendors and representatives is that these changes will be of minor-to-moderate difficulty. Ranked on a Likert Scale of 1-10 (10 being the most difficult), the responses ranged from 2-to-5 (only one 5), with most being 3’s and 4’s. Because implementation of DOD will probably not occur until FY 2012 or later, or perhaps never (FY 2010 was assumed in the Phase 1 final report), system enhancements necessary to support DOD operations can be made in a later, second enhancement after the necessary changes are made to support the All Mass-Duplication system described in Section 3.
Information exchange with the DOD Centers must be completely automated. A DTB title record in the library’s system must indicate if the "Library" or "Center" is responsible for circulation of that title. If the library, then circulation will be handled as described in Section 3. If the Center, then the following types of transactions and information exchange will occur between the libraries and DOD Centers.
- "Order" (library to Center), i.e., a library will place an order with the Center for a DOD book; the DOD Center will then perform checks for validity and completeness, and assign its own transaction number to the order.
- "Status" (Center to library), i.e., the library order will be classified as "accepted" or "rejected" by the Center; if rejected, the reason for rejection will be cited, probably using codes, e.g. "Address Incomplete". The status will be reported back to the libraries for all Orders received.
- "Shipped" (Center to library), i.e., the order has been shipped-out to a patron from the Center via USPS.
- "Undeliverables" (Center to library), i.e, undeliverable returned to the Centers by the USPS.
- "Returns" (Center to library), i.e., a book has been read by a patron and returned to a Center.
- "Inquiry" (Library to Center), i.e., the library inquiring about the status of an order.
- "Lost" (Library to Center), i.e., the library will inform the Center that a book/copy is lost.
Communications between the libraries and DOD Centers will be web-based (i.e., via the Internet), using records in XML format. A "Periodic Polling" approach will be employed, whereby library systems will send transaction records to a server(s) at the DOD Centers and will "pick-up" transaction records written to the server(s) by the DOD Centers. In this manner, some of the more fundamental security and system access issues, specifically difficulty associated with the DOD Centers having security access to 132 library information systems, will be minimized (it is probable that not all 132 libraries will communicate directly with the DOD Centers). Further considerations of information systems and the information exchange between libraries and DOD Centers are discussed in Section 4.3.
A nominal 20% of DTB circulation will ultimately be provided by the DOD
Centers under the Hybrid distribution option, and two geographically separated
production facilities will be required (primarily because of risk diversification
concerns). The mission of these facilities will be to fill demand for books
that are not distributed by libraries. These are the slowest-moving titles
that will account for about 20% of reader demand. This total includes some
of the library RC orders that are now sent to the MSCs for order filling.
The design throughput capacity of a DOD Center is as follows:
|Number of Copies|
|Total annual throughput - 20% of 20,000,000||4,000,000|
|Copies per day @ 250 working days per year||16,000|
|Copies per day for each of two facilities||8,000|
|Copies per minute for a net 400 minute workday||20|
The achievable throughput of a DOD production line will therefore be between 20 and 30 copies per minute, depending solely upon the speed of duplication. The design specification for the duplicator line will therefore be at least 20 copies per minute. All upstream and downstream production tasks will also be accomplished in this time frame to prevent bottlenecks, and duplicate workstations and equipment will be provided as needed and to the maximum extent possible.
The handling units in this instance are the DTB cartridge and the container, and there will never be more than one cartridge in a shipment. The container will be made of a rigid durable plastic, and will have a lid fastener that is secure to prevent opening in transit, but can be readily opened and closed by the patron. It will have the same fastener design that is used on the container for RC books. There will be no container labeling other than the address card and an adhesive print label that uniquely identifies the DOD Center so that containers with lost address cards can still be returned.
The cartridge labeling will include all of the print information that is now on a cassette label, plus a bar-coded unique transaction number which is assigned by the computer system at the time of order entry. Braille labeling will be provided for Braille readers, who constitute 5% of the readership, and for other readers who have indicated a desire to have Braille labeling on their DOD books, estimated as another 5% of the readership. Thus a total of about 10% of the orders filled will have Braille labels.
The cartridge is designed to be readily conveyable through the automated receiving reconditioning, labeling, duplication, and packing operations. It will have an accommodation that will make the labels mechanically and automatically removable when the cartridges are being prepared for recirculation.
A DOD Center will have two distinct production areas, which are operationally independent, as follows:
- Input operations, including receiving, check-in of patron returns, and reconditioning of cartridges;
- Output operations, including label printing, cartridge labeling, duplication, check-out of patron shipments, packing and shipping. The two operations will have the same basic operating schedules, and each will have a variable workload. However, all output operations will always end on or before the 5:00 PM deadline for same-day shipment of patron orders. DOD service response times will therefore compare favorably with present response times for library shipments.
A sizeable storage area for reconditioned cartridges is located between the two operating areas. This buffer is needed to counter the variations in cartridge input and output. When workflow is in balance, the area will be half-full. An 8-hour supply of cartridges will be adequate to provide this buffering capability. Two-hundred-forty new cartridges will be added to the buffer supply on an average day, to provide for an expected attrition rate of 3% of circulation.
All DOD Center DTB input will be received from a local BMC, and all DTB output will be sent to the BMC. The handling unit for this workflow will be a large metal open-top tow-line container called a BMC (Bulk Mail Container, also referred to as an "OTR"). A BMC measures 42" wide x 60" long x 69" high, and will hold up to 2,200 DTB containers. About 20 BMCs will be received and processed each week. The incoming BMC’s will also contain patron non-deliveries, and some other packages, which must first be sorted out.
The BMC’s will be unloaded using a specially designed dumping machine that does not require a high lift. The machine will lift the BMC some 3" off the floor and then rotate it slowly to dump the contents onto a 36" wide heavy-duty pan-type belt conveyor that rises to table height to push the books onto a slick-steel sort table. The table will be some 10 feet long and 36" wide, and will have 3" high side rails on 3 sides. The conveyor belt and tilt mechanism on the dumper will be manually controlled to keep the work table full. Packages other than returned DTBs will then be removed, and the patron non-deliveries will be taken to the DOD office for special handling.
There will be two fully-mechanized cartridge reconditioning lines in a DOD Center, each of which will have a rated capacity of 12 copies per minute. Two receivers will be required to feed these lines, which will be monitored by an attendant. This production schema will also provide the ability for one line to continue operating if the other goes down.
A receiver will first obtain a container from the receiving sort table and remove the address card from the container. This can be done either manually or with vacuum assist. The receiver will then place the container in a magazine that is an integral part of the conveyor that feeds the workstations. The container will be oriented in the magazine to conform with the needs of downstream operations. On signal, conveyor controls will then singulate a container from the magazine onto the conveyor. The conveyor will be designed to hold the container in place during transit, but later operations may require that the container be clamped in place at the workstations.
The first station on the line is container opening, at which time the container straps are unlatched. This will be a fully automated operation. The opened container will then proceed on the conveyor to an inspection workstation, where it will be removed from the conveyor by an inspector. The inspector will remove the cartridge from the container and stack the empty container on a cart. The attendant will later take a full cart to the packing area and the containers will be reused for shipping the same day.
The inspector will make a visual inspection of the cartridge to insure that there is no damage (primarily to the Universal Serial Bus (USB) connector), and to insure that there is a barcode on the cartridge (to verify that it is not a mass-duplication cartridge that belongs to a library). The cartridge will then be placed in a magazine that is an integral part of the conveyor that feeds cartridges to the label removal workstation. At this point, the cartridge will be oriented label-up and with the USB connector trailing.
Conveyor controls will cycle a cartridge onto the conveyor and the barcode on the label will be read while the cartridge is in motion, so as to record return of the copy, and the library will be so advised via a "Return" transaction. The cartridge will be held stable on the conveyor, but will have to be clamped in place at the label removal station due to the force required to remove the label. The cartridge and the label will be designed to facilitate this removal, which will be fully automated, and will take less than 5 seconds.
The final station on the conveyor line will automatically place the reconditioned cartridges in work-in-process storage tubes that will contain 80 cartridges (based upon an assumed thickness of 3/8"). When a storage tube becomes full, it will be automatically cycled out of the station and an empty tube will be cycled into place. The full tubes of reconditioned cartridges will be stored in a staging area near the cartridge labeling infeed conveyors.
Cartridge Label Printing
Orders will be telecommunicated to a DOD Center by the libraries, and after checks for validity and completeness, the Center will assign a unique transaction number to each Order. The transaction number will link the title ordered to the patron order in the DOD Center database, and will appear on a cartridge label in barcode format (and, if space on the cartridge label permits, also in human-readable format). The label data for the approximately 40,000 titles in the DOD Center database will be stored on label servers (assumed to be dedicated devices, but they could possibly also be stored on other system servers), and the production control system will be programmed to direct the automated printing of the labels.
The cartridge labels will be prepared in pressure-sensitive strip format and in duplication sequence (which will be either FIFO, or sorted by title duration i.e., write time), depending upon the sequencing scheme used for the actual duplication process. Patron orders coded Rush will have top priority, and a cartridge label will be promptly printed as soon as a rush order is received. The up-front labeling operation will not run full-time, and two label print lines will be needed to provide the required printing capacity. Each line will have a Braille embosser, and the control system will activate the embosser only when a Braille label is required.
The cartridge labeling will be conveyorized and automated, and there will be two identical labeling lines to provide the required labeling capacity, and also to provide the ability for one line to continue to operate if the other line goes down. The blank cartridges that are to be labeled will be in the work-in-process storage tubes previously cited, and an induction station at the head of the line will meter the cartridges onto the conveyor. The cartridges could either be held in place while the label is being applied, or could be labeled while in motion.
At the final workstation on the cartridge labeling conveyor, the labeled cartridges will be automatically placed in a work-in-process storage tube. When a storage tube becomes full, it will be automatically cycled out of the station and an empty tube will be cycled into place. The full tubes of labeled cartridges will be stored in a staging area near the duplicator infeed conveyors.
Cartridge duplication is expected to be the bottleneck operation in DOD production. To ease this bottleneck, the duplication and packing operations will therefore be scheduled to operate a full 480 minutes a day, as needed. The extra staffing required for this schedule will come from the labor pool of the contractor, and four part-time people will be needed.
The write speed of flash memory is assumed to be at least 4 MB/s by the time steady-state operations are achieved; this value may indeed turn out to be higher, since tests run by the NLS Engineering Department in January, 2006 indicated average write speeds of about or slightly greater than 4 MB/s. For the erase speed of flash memory, it is assumed for planning purposes that only several seconds (1-5) will be required to erase the average size DTB. Given the average size of a NLS DTB, i.e., about 120 MB in compressed format, it will require approximately 30 seconds to first erase and then write an average size DTB title in duplication operations. Additional time will be needed to cycle a cartridge into and out of the duplicator, and the total cycle time required to duplicate a cartridge is yet to be determined.
Book duplication times are a key input to systems design, and based upon a required cycle time of 30 seconds per copy, there will be 10 duplicators. Labeled cartridges will either be fed to the duplicators from a common infeed conveyor (ref. Battelle report), or possibly fed to each of the duplicators individually using a multiple-queue, multiple-server approach. This later approach: (1) has the advantage of not having to sort orders by duration time, but rather by FIFO sequence; (2) would make the workflow smoother if one or several duplicators go down; and (3) may facilitate smoother production for the 10% of cartridges that will require Braille labels if modifications to duplicators are necessary RE handling cartridges labeled with Braille labels. There will be five duplicators on each of two duplication lines.
As previously noted, cartridges will be labeled prior to duplication, and the labeled cartridges will be stored in tubes. The cartridges will be automatically fed into the duplicators, where they will first be erased and then duplicated. The loaded cartridges will be automatically exited from the duplicators onto two output conveyors that lead to the packing/shipping workstations. The stations will be located along the conveyors, and cartridges will accumulate on the conveyors ahead of the stations.
Cartridge Packing and Shipping
There will be two packing stations on each of the two duplicator lines, and each station will have a barcode scanner, a high-speed address card printer, and a supply of empty DTB containers. After removing a cartridge from the conveyor, the packer will pass the cartridge through a barcode scanner to signal that the order has been shipped, and the library will be so advised. The scanning will also notify the printer at the station that an address card is needed, and the cartridge will be packed while the address card is being printed. Packed and labeled containers will be placed on a takaway conveyor that leads to the shipping area, where the containers will be loaded directly into an outgoing BMC.
It may be possible to read the barcode automatically and place the cartridge into a container dependent upon the final cartridge and container design (TBD) and the mailing card could be automatically inserted into the holding slot on the container. Another variation on this function is to first insert a blank card into a container and then print the address information (with the container either stationary or preferably in motion).
Production Line Configuration
The production line for a DOD Center with a throughput capacity of 8,000 DTB copies per day, as just described, is in essence two production lines of 4,000 DTB copies per day running in parallel. This is the configuration that was first contemplated for a DOD Center in the Phase 1 planning study.
The estimated staffing for a Hybrid distribution DOD Center in full operation will be 14 people, consisting of 10 production personnel, 3 office personnel, and one supervisor, as shown in Appendix 16. This would be a daytime 1-shift operation, thereby providing patrons with the same or better shipping response times as now, and there would be no carryover at the end of the week.
The estimated size of each of the two production facilities required for the Hybrid DOD distribution system is 4,400 square feet, and details of the functional space allocations are provided in Appendix 17. In making these projections, we have assumed that a DOD Center will be a compatible part of the operations of a larger contractor. The 4,400 square feet is therefore only the net production and office area required, and does not include common areas, such as receiving and shipping docks and employee facilities.
The estimated capital equipment costs for a DOD Center are shown in Appendix 18. With the single exception of "Computer System – Software," no costs would be shared by another DOD Center. But with two distribution centers, the costs for this system, which will require a custom design by a Systems Developer/Integrator, will be shared between the two operations.
The estimated total costs to duplicate a DTB on demand are shown in Appendix 19. The major components of cost, which are for labor, facilities, equipment, materials and profit, are all shown. Labor costs are the dominant variable, and the average hourly rates used in the cost model are derived from comparable hourly rates as recently listed by the Bureau of Labor Statistics (BLS).
Based upon these components and an annual volume of 2,000,000 copies per Center, a value of $0.65 per DTB copy (which is net of the cost of the flash memory cartridge and the container, which will be provided by NLS as GFE) is derived and is used as an estimate for planning purposes. Because in any year after steady-state operating levels are reached only 3% of circulation from DOD Centers will be books using new cartridges (to replace those lost) versus 97% using reconditioned cartridges, the assumption has been made in the cost estimates that all cartridges used by the DOD Centers will be reconditioned.
Shipping demand and patron returns in a DOD Center are expected to vary by day of week. But just how much they will vary is yet to be determined. For scheduling purposes, it is best to have a stable workload, and to do this, there would have to be a carryover order backlog from day-to-day, but not necessarily from week-to-week. This backlog of unfilled orders should be labeled cartridges in storage tubes that are staged in front of the duplicators, where they will be available for immediate duplication.
We have also considered the impending problem that a reader would have in distinguishing the approximate one cartridge in five that should be returned to a DOD Center rather than to a network library. Since there are no liberties with cartridge design, this differentiation must come from the label. While there is some merit in using a different color, the most defining solution would be to provide a tactile border on the Braille portion of the cartridge label for cartridges produced for Braille readers (estimated at about 10% of demand). The lack of a Braille label on the other cartridges from the DOD Center would be the differentiating feature for the other copies circulated, i.e., mass-duplicated DTBs circulating from libraries would all have Braille labels.
Containers used by the DOD Centers would not have external labeling of any type (only the mailing card). The absence of both the print and Braille labels from the containers will distinguish the DOD Center containers from the library containers.
In this section, the functional requirements of the information system necessary to facilitate the exchange of information between network libraries and DOD Centers, and the production of DTBs on a DOD-basis within the Centers, are presented. An emphasis is placed upon the functional requirements for the system rather than the specification of particular hardware and software components. Because it will be several years before set-up of the DOD Centers will begin and, given the rapid rate of evolution of information system technology, hardware and software components that are the most appropriate today may and probably will be replaced by other more efficient components in the future.
A high-level technical design of data communications between LASs and DOD Center information systems is therefore presented, which does not address design specifics such as the most appropriate source code to be used, or specific programming functions required, to develop an information system that provides the capabilities described. It is rather intended to provide NLS management and technical staff, and vendors and representatives of the LASs, a description of what information system functionality will be required to facilitate DTB distribution on a duplication-on-demand basis in the future Hybrid system. It is also intended to provide the framework for specifications for the information system, which are being developed under Task 3 of this project, and which will form the basis for a Statement-of-Work (SOW) for development and implementation of the system by a Systems Developer/Integrator.
More emphasis is placed upon the requirements for network library-DOD Center information exchange than upon those necessary for production support within the Centers. This approach was taken because Flash Drive cartridge duplication technology is now in its infancy and will evolve considerably between the present and when the DOD Centers will actually be set-up and go online, especially with regard to automated labeling and loading of Flash Memory cartridges. The Systems Developer/Integrator who will develop and implement the custom-designed production control information system at the DOD Centers will incorporate state-of-the-art Flash Drive cartridge duplication technology that will be available at that time.
In the future Hybrid DTB distribution system, the information systems used by network libraries and DOD Centers must be able to exchange information via data telecommunications in order to facilitate the distribution of DTBs on an on-demand basis. This information exchange between the LASs and DOD Center information systems must be completely automated. This is not to say that human oversight and monitoring of such exchanges will not be required, but the performance of the data exchange tasks themselves must be fully automated and not require manual intervention.
At a meeting held with library information system vendors and representatives on April 24, 2006, the information that needs to be exchanged between network libraries and DOD Centers was defined, various possibilities for how that information could be exchanged were discussed, and several fundamental solutions were decided upon regarding the same. These fundamental solutions were as follows:
- Information exchange between network libraries and DOD Centers should be by web-based (i.e., via the Internet) data telecommunications. While other possibilities were discussed, because almost all network libraries have some type of Internet access, the volume of data to be exchanged will be relatively minor (even by today’s standards), and other methods would likely impose incremental costs upon network libraries, web-based data communications was selected as the means for DOD Center-network library information exchange.
- Data communications between network libraries and DOD Centers should use transaction records in XML format. This approach was considered to be effective because data communications using XML records is common and has a proven track-record in many applications. It would also be efficient because all library information system vendors and representatives have experience using XML records, and could therefore "hit the ground running" when such enhancements to library information systems become necessary.
- A "Periodic Polling" approach will likely be necessary to facilitate data communications between network libraries and DOD Centers, whereby network library information systems will send transaction records to servers in the DOD Centers’ information systems, and will "pick-up" transaction records written to the servers by the DOD Centers’ information systems. These tasks would be scheduled and fully automated. In this manner, most fundamental information system security and access problems will be minimized, specifically the difficulty associated with information systems in the DOD Centers having security access to information systems that support 132 network libraries.
At present, while not all network libraries possess the communications infrastructure necessary to facilitate the data exchange as described, the vast majority do possess these capabilities. The vendors and representatives of the five library information systems expected to be in use in the future system indicated that the system enhancements necessary would be of minor-to-moderate difficulty to implement. These vendors and representatives are currently in the process of collecting information on the communications infrastructures of the information systems of their client libraries.
The following terms are used in the discussions below and are defined for the context in which they are used.
- SOAP: Simple Object Access Protocol (SOAP), is a lightweight protocol intended for exchanging structured (XML) information in a decentralized, distributed environment.
- Load Balancing: A technique (usually performed by load balancers) which allocates work between multiple computers, processes, disks, or other resources in order to achieve optimal resource utilization and decrease computing time.
- Replication: Refers to the provision of redundant resources (software and/or hardware components) in order to improve reliability and provide fault-tolerance.
- WSDL: The Web Services Description Language (WSDL) is an XML format published for describing Web services. Version 1.1 has not been endorsed by the World Wide Web Consortium (W3C), however, it has released several drafts for Version 2.0 that will be a W3C recommendation and thus will be endorsed by the W3C.
- LAS: Library Automation System. These are the clients that access the DOD Center information systems.
- Service Provider (DOD Center): From a business perspective, this is the owner of the service. From an architectural perspective, this is the platform that hosts access to the service. It is also referred to as a service execution environment or service container. Its role in the client-server message exchange patterns is that of a server.
- Service Requestor (LAS): From a business perspective, this is the business that requires certain functions to be satisfied. From an architectural perspective, this is the application that is looking for and invoking or initiating an interaction with a service. The requestor role can be played by a browser driven by a person, or a program without a user interface, e.g. another web service. Its role in the client-server message exchange patterns is that of a client.
- Discovery Agency: This is a searchable set of service descriptions where service providers publish their service descriptions. The service discovery agency can be centralized or distributed. A discovery agency can support both the pattern where it has descriptions sent to it and where the agency actively inspects public providers for descriptions. Service requestors may find services and obtain binding information (in the service descriptions) during development for static binding, or during execution for dynamic binding. For statically bound service requestors, the service discovery agent is in fact an optional role in the architecture, as a service provider can send the description directly to service requestors. Likewise, service requestors can obtain a service description from other sources besides a service registry, such as a local file system, FTP site, URL, or WSIL document.
- Message Exchange Pattern (MEP): An MEP is a specialized form of Feature that describes a generalized pattern of message exchange between two services.
- Web Service: A Web Service is a computational service, accessible via messages of definite, programming-language-neutral and platform-neutral format, which has no special presumption that the results of the computation are used primarily for display by a user-agent.
In the future Hybrid DTB distribution system, a DTB’s title record in a library information system will indicate whether the library or a DOD Center is responsible for circulation of that title. If a Center is responsible, then the library and DOD Center must exchange certain fundamental information via data telecommunication transactions on a routine, on-going basis.
A LAS will send correctly formatted requests to a DOD Center when ordering a DTB, and receive information from a DOD Center in a similar manner. The LAS provider will be presented with a well-defined set of requirements that will need to be integrated into their applications. The introduction of a new communication protocol is the primary foundation for effective information exchange with the DOD Centers, and this foundation will be based upon Web services within a Service Orientated Architecture (SOA).
Web services specifications define the details needed to implement services and interact with them. However, SOA is an approach to building distributed systems that deliver application functionality as services to end-user applications or to build other services. A service in the DOD Center SOA will be a LAS function packaged as a reusable component for use in the overall DOD Center business process. It will either provide information or facilitate a change to business data from one valid and consistent state to another.
Using SOAP, services can be invoked that stress interoperability and location transparency. A service has the appearance of a software component, in that it looks like a self-contained function from the LAS perspective. However, the service implementation may actually involve many steps executed on different computers within one enterprise, or on computers owned by a number of business partners. A service might or might not be a component in the sense of encapsulated software. Like a class object, the requester application is capable of treating the service as one.
DOD Center Web services will be based on invocation using SOAP messages which are described using WSDL over a standard protocol such as HTTP. Use of Web services will be the best approach for communications between the external business partners and the DOD Centers.
The binding from the LAS to the DOD Center information systems will loosely couple the service. This means that the LAS will have no knowledge of the technical details of the DOD Center system, such as the programming language and deployment platform used. The LAS will invoke operations by way of messages—a request message and the response—rather than through the use of APIs or file formats. Such a loose coupling allows software on each side of the conversation to change without impacting the other side, provided that the message schema stays the same.
The service interaction must also be well-defined. WSDL is a widely-supported way of describing the details required by a service requester for binding to a service provider. The service descriptions focus on operations used to interact with the following:
- A service,
- Messages to invoke operations,
- Details of constructing such messages, and
- Information on where to send messages for processing details of constructing such messages.
WSDL does not include any technology details of the implementation of a service. A LAS will neither know nor care whether the service will be written in Java code, C#, or some other programming language. It can describe a SOAP invocation using HTTP. The common definition for WSDL allows development tools to create common interfaces for various styles of interaction, while hiding the details of how it invokes the service from the application code.
Services will be independent, self-contained requests, which do not require information or state from one request to another when implemented (STATELESS). Services will not be dependent on the context or state of other services. When dependencies are required, they are best defined in terms of common business processes, functions, and data models, not implementation artifacts (like a session key). A LAS will require persistent state between service invocations, but this should be separate from the DOD Center.
The granularity of operations is an important design criterion. The use of coarse-grained interfaces for external consumption is recommended, whereas fine-grained interfaces might be used inside the enterprise. A coarse-grained interface might be the complete processing for a given service, such as SubmitDTBOrder, where the message contains all of the business information needed to define a DTB order. A fine-grained interface might have separate operations for: CreateNewDTBOrder, SetShippingAddress, AddItem, and so forth.
While the fine-grained interface offers more flexibility to a LAS, it also means that patterns of interaction may vary among different LASs. This could make support relatively more difficult for the DOD Center. A coarse-grained interface guarantees that the service requesters will use the service in a consistent manner. SOA does not require the use of coarse-grained interfaces, but recommends their use as a best practice for external integration. Service choreography can be used to create a coarse-grained interface that runs a business process consisting of fine-grained operations.
DOD Center SOA designs will span the various LASs and might cross enterprise lines. The security capabilities and requirements when using the Internet and how to link across partners’ security domains need to be considered. Internet protocols such as HTTP are not designed for reliability (guaranteed delivery and order of delivery), the LAS--> DOD Center -->LAS communication must ensure that data are delivered and processed on time. When this is not possible, the DOD Center must know that a request has not been processed.
The recommended approach will implement SOAP, which is a protocol for exchanging XML-based messages over a computer network, normally using HTTP. SOAP forms the foundation layer of the Web Services stack, and provides a basic messaging framework upon which more abstract layers can be built. The messaging framework will also be well defined, and will describe all of the possible states of a transaction as LAS to DOD events, and vice versa, occur. When this framework is established, information exchange between the LASs and DOD Centers via data telecommunications will be established.
To provide the functionality required for the future system, the NLS DOD Center information system (and LASs) will require:
- Web Services and SOAP for the transport and delivery of XML Messages, and
- A connection to the Internet in order to both send and receive data.
The basic DOD Center system architecture will model the interactions between three roles:
- Service provider
- Service requestor
- Discovery Agent
Data communications between a LAS and a DOD Center will be modeled after a basic Web Services architecture that defines an interaction between software agents as an exchange of messages between service requesters (LASs) and service providers (DOD Centers). Requesters are software agents that request the execution of a service. Providers are software agents that provide a service. Agents can be both service requesters and providers. Providers are responsible for publishing a description of the service(s) they provide. Requesters must be able to find the description(s) of the services.
Interactions between the LASs and the DOD Center systems will involve the following operations:
In this scenario, the DOD Center will host a network-accessible software module (an implementation of a Web Service). The DOD Center will define a service description for the Web Service and Publish it to a LAS. A LAS will use a Find operation to retrieve the service description locally or from the discovery agency (i.e. a registry or repository) and use the service description to Bind with the DOD Center system and invoke or interact with the Web Service implementation. The DOD and LAS roles are logical constructs, and a service may exhibit characteristics of both.
A LAS and DOD Center information system will interact using one or more MEPs that define the sequence of one or more messages exchanged between them. A service description will be hosted by a discovery service, to which a DOD Center system will Publish the description, and from which a LAS will Find the description. The description will include data type and structure information, identify the MEP, and contain the address of the service provider.
The DOD Center system will encapsulate the request logic for the service in a method (or function) then setup a process that listens for requests to the service; such requests will be in SOAP format and contain the service name and any required parameters. The Transport Layer will be HTTP. The listener process, which for simplicity will be written in the same language as the service method, will decode an incoming SOAP request and transform it into an invocation of the method. It will then take the result of the method call, encode it into a SOAP message (response) and send it back to a LAS. Conceptually, this arrangement looks like the following:
The incentive for managing state is because various problems or complexities, which are listed below, may result if it is not properly managed:
- LAS --> DOD Center-->LAS communication delays
- DOD Center server may be loading (and hence response time) is unpredictable
- Locks might not be retained for duration of activity
- Servers will be controlled by different organizations
- Modules might not trust each other
- Failures will be more likely
Using explicit state identifiers will be the most straightforward and effective technique for managing the conversation state in DOD Center information system Web Service interactions. Although the name suggests a single ID, fully-managed conversations require more information, especially when associated with service requests. As a result, a state identifier is really a control class that maintains the following set of state-related information with respect to each request-response interaction:
- Unique identifier of the LAS,
- Identifier of the Web Service conversation originated for that LAS
- Identifier of the service request within that conversation
- Indicator of the conversation phase between a LAS and DOD Center system (Starting, Continuing, and Ending), which will be used by the service for proper state maintenance.
The recommended method for correlating requests is to have the LASs add the state identifier information — a unique identifier of the service request (Request ID), identifier of the conversation (Conversation ID), and conversation phase indicator (Phase) — to the request message, and have the DOD Center system copy the identifier information to the response message passed via callback, so that the requester can correlate the reply message to the request message. This will be done by passing the identifier information in the SOAP message header.
The identifier information will be added as a new group of sub-elements into a message's SOAP header by a LAS. The DOD Center system will intercept the SOAP message, and then extract the header with appropriate state identifier information, leaving the message body untouched and unaffected.
Adding the identifier information to a SOAP header will keep related code separate from the document processing and associated business logic, as well as the concrete implementation of service interfaces.
Simple State Diagram with DOD not Available
State Diagram with Incorrect State
1 - "Order" (sent from a library to a Center). A library will place an order with a Center for a DOD DTB. After full implementation of the DOD Centers is achieved, each of two Centers will receive from network libraries about 8,000 Order transactions per working day (assuming 250 working days per year and an equal allocation of workload between the two centers). Orders will be received at a Center from the approximate 66 libraries that it serves (i.e., assuming that each Center handles 50% of the 132 network libraries in the system). The data elements required for the Order transaction record are shown below.
|Library_Code||Alphanumeric: 4 Characters||NLS Library Code|
|Order_Date||Date-Time format||YYYY-MM-DD HH:MM:SS|
|Library Transaction Number||VarChar : 0-25 Characters (TBD)||Unique for the library|
|Title_ID||VarChar : 8 Characters||NLS Title Number, e.g. "DB070000"|
|Patron_Name||VarChar : 0-32 Characters||All in one field|
|Patron_Address||VarChar : 0-32 Characters|
|Patron_Address2||VarChar : 0-32 Characters|
|Patron_City||VarChar : 0-18 Characters|
|Patron_Zip + 4||VarChar : 9 Characters||USPS Zip+4 code|
|Patron_State||VarChar : 2 Characters||USPS Code|
|Patron_Country||VarChar : 3 Characters|
|Patron_Braille_Reader||Integer: 1 Character||0 = No , 1= Yes|
|Patron_Priority||Integer: 1 Character||e.g. 1-3, where 1 is "normal," 2 is Rush for non-Veterans, and 3 is Veterans|
|Order_Status||VarChar : 0-255 Characters, TBD||Updated by the DoD System (TBD)|
Most of the field sizes cited above must either conform to a size necessary to contain a standard code (e.g., the 2-position alphanumeric USPS state abbreviation) or to other constraints (e.g., the 32-character maximum size for Patron_Address, which is required in order to print the field, on a single line, in sufficiently-sized font for USPS use, on a 3" x 5" cardstock mailing card that will be used in DOD Center DTB mailing containers, i.e., the same as that which will be used in network libraries). The Library Transaction Number, however, could be of arbitrary size (within the limits of manageability) and could, and probably will, vary among libraries.
2 - "Status" (sent from a Center to a library). After receipt of an Order transaction record from a library, the DOD Center information system must perform checks for validity and completeness of the record, and assign its own unique Transaction Number to the order. An Order transaction will be classified either as "accepted" or "rejected" by a Center. If rejected, the reason for rejection must be cited, probably by using standard codes or descriptions, e.g. "Address Incomplete." The Status will be reported back to the libraries by a Center for all Orders received. It is likely that the only data elements necessary in a Status transaction record will be: (1) Library Transaction Number, (2) Center Transaction Number, (3) the Accept/Reject status, and (4) the Reject Reason/Code (if applicable). The volume of Status transactions will be exactly the same as the number of Order transactions, i.e., about 8,000 per day per Center, because Status is reported for every Order received. It is likely that the number of rejected Orders will be at most 1% of the number of Orders, or about 80 transactions per working day per Center, and could be as low as 0.1% of Orders or about 8 transactions per day per Center.
3 - "Shipped" (sent from a Center to a library). After a DOD order has been shipped to a patron from a Center via USPS, the Center must notify the library of the same along with the date shipped. It is likely that the only data elements necessary in a Shipped transaction record will be: (1) Library Transaction Number, (2) Center Transaction Number, and (3) the Date Shipped. The volume of Shipped transactions will be virtually the same as the number of Order transactions, on average over time; it will actually be the number of Order transactions less the number of rejected orders. Since the number of rejects may be as low as 0.1% of Orders, it is conservatively estimated that there will be 8,000 Shipped transactions per working day per Center.
4 - "Undeliverable" (sent from a Center to a library). A DOD order may be returned to a Center by the USPS as "undeliverable." If so, then the Center must notify the library of the same along with the date that the copy was returned undeliverable. It is likely that the only data elements necessary in an Undeliverable transaction record will be: (1) Library Transaction Number, (2) Center Transaction Number, and (3) the Date Returned Undeliverable. The volume of Undeliverable transactions is unknown, but for planning purposes should be assumed to be about 1% of orders shipped, or about 80 transactions per working day per Center; it may turn out to be less than this estimate (libraries will nominally be circulating four times the number of copies as the Centers, and will likely receive undeliverable copies and correct the patron addresses in their databases before a Center receives an undeliverable for the same patron).
5 - "Return" (sent from a Center to a library). After a DOD book has been returned by a patron to a Center via the USPS, the Center must notify the library of the same along with the date of the return. It is likely that the only data elements necessary in a Return transaction record will be: (1) Library Transaction Number, (2) Center Transaction Number, and (3) the Date Returned. The volume of Return transactions will be approximately 96% of the number of orders shipped (the latter of which, as noted, is conservatively assumed to be the same as the number of orders received), or about 7,680 books per working day per Center. It is assumed that 3% of orders shipped will be lost and never returned, and that 1% will be undeliverable, hence the estimate of 96%.
6 - "Inquiry" (sent by a library to a Center). It is possible that a library may wish to inquire about the status of an order being processed by a Center (i.e., an Accepted order), although this should occur relatively infrequently. It is likely that the only data elements necessary in such an Inquiry transaction record will be: (1) Library Transaction Number, (2) Center Transaction Number, (3) Date of Inquiry, and possibly (4) Reason for Inquiry (using some type of standardized codes or descriptions). The volume of Inquiry transactions will probably be very low, possibly of the same magnitude as that for Rejected Order transactions, i.e., 0.1% to 1% of orders placed, or 8 to 80 such transactions per working day per Center.
7 - "Lost" (sent by a library to a Center). A DOD Center information system will not decide when a DTB copy is lost (e.g., after some specific time on-loan). Rather, a library will inform a Center that a copy (loaned to their patron) is lost, using the same criteria applied to determine that a copy is lost as those applied to mass-duplicated DTBs distributed by libraries. It is likely that the only data elements necessary in a Lost transaction record will be: (1) Library Transaction Number, (2) Center Transaction Number, and (3) Date Declared Lost. The volume of Lost transactions is assumed to be 3% of Shipped transactions (the latter of which is conservatively assumed to be the same as the number of Order transactions), or about 240 Lost transactions per working day per Center.
The above types of transactions will not constitute the entirety of communications between libraries and Centers, e.g., as cited above, another type of communication may be "DOD server not ready/accessible." Furthermore, whether such classes of transaction records are sent to (or "picked-up" by) libraries, or whether such transaction information is written/appended to “master” order records sent to (or picked-up by) libraries, must be determined by the Systems Developer/Integrator. However, the cited transactions do constitute the entirety of the types of information that must be exchanged, and the volume of those exchanges, between libraries and Centers in order for DOD operations to function properly in the Hybrid DTB distribution system.
SOAP messages should be less than 10 kb in size, and this metric should be used to determine required hard drive capacity. This amounts to approximately 1,048,576 messages per 10 GB of hard drive space. When planning for transaction space capacity, it is recommended that 20% be added as a buffer to the 10 kb per transaction size. This buffer will serve as reserve capacity.
Statistics will be derived from a transaction log that may average 1 kb per line. At this time, it is somewhat unclear how many workdays of logs should be retained, but we anticipate 1 year. An order will remain "open" on the DOD Center system until a copy is either returned undeliverable, returned by a reader, or declared lost by a library.
The method used for data exchange over the Internet will be based upon sending encrypted data in order to prevent intrusion by using SSL. SSL runs on layers beneath application protocols such as HTTP, SMTP and NNTP, and above the TCP or UDP transport protocol, which form part of the TCP/IP protocol suite. While it can add security to any protocol that uses reliable connections (such as TCP), it is most commonly used with HTTP to form HTTPS. HTTPS is used to secure World Wide Web pages for applications such as electronic commerce. It uses public key certificates to verify the identity of endpoints.
Every SOAP message has a SOAP envelope and SOAP encoding. The SOAP envelope is a data structure that can be used to carry any XML document. SOAP encoding is used to encode non-XML data into an XML document, so that it can be transmitted within a SOAP envelope. Typically, this encoding is intended to be used in an application like a Remote Procedure Call (RPC).
This design should consider:
- From the sender's perspective: When sending a message, how is the identity of the intended recipient authenticated?
- From the recipient's perspective: When receiving a message, how is the identity of the sender and the content of the message authenticated?
Message authentication guarantees that a transmitted message was not modified en route and the identity of the creator is not misrepresented. Typically, message authentication can be accomplished by appending a digital signature or Message Authentication Code (MAC) to the transmitted message. Message authentication guarantees nothing about who sent the message.
The information system design will handle errors using the SOAP Fault message. A SOAP Fault message is a normal SOAP message with a single, well-known element inside the body: soapenv:Fault. The presence of that element acts as a signal to processors to indicate that something has gone wrong.
The Fault code is the first thing to be examined since it conveys, in a general sense, the nature of the problem. Fault codes are QNames, and SOAP defines the set of legal codes as follows (each item is the local part of the QName—the namespace is always the SOAP envelope namespace):
Sender (LAS) — The problem was caused by incorrect or missing data from the LAS. For instance, if a service required a security header in order to do its work and it was called without one, it would generate a Sender fault. In this instance, the LAS will then have to make a change to the message before resending.
Receiver (DOD Center) — Something went wrong on the DOD Center server while processing the message, but it wasn't directly attributable to the message contents. For example, a necessary resource like a database was down, or a thread wasn't available, etc. A message causing the DOD Center system fault might succeed if resent at a later time.
MustUnderstand — This fault code indicates that a header was received that was targeted at the receiving node, marked MustUnderstand = "true", and not understood.
VersionMismatch — The VersionMismatch code is generated when the namespace on the SOAP envelope that was received isn't compatible with the SOAP version on the receiver. This is the manner in which SOAP handles protocol versioning, the latter of which is discussed further below.
The Fault code resides inside the Code element in the fault, in a sub-element called Value. The SOAP 1.2 specification contains further information on implementing the Soap Fault Message.
It is recommended that a SOAP debugger be used to examine data traffic between the libraries and DOD Centers. A debugger will act as a Web Services proxy between the LAS and the DOD Center system, allowing inspection of WSDL files and SOAP messages.
The Debugger will also be useful for tracking live issues. It can be setup in both the LASs and the DOD Center systems. An example is shown below.
The protocol designs for network library - DOD Center data exchange will probably evolve over time. It is therefore necessary that some type of "versioning" be incorporated into the protocol design so that network library information systems implementing the protocols can smoothly adjust to version upgrades.
All protocols will be defined on the day of design implementation. Some protocols like SOAP will handle version management. For example, a SOAP/1.1 node receiving a SOAP Version 1.2 message will, according to SOAP/1.1, generate a version mismatch SOAP fault based upon a SOAP/1.1 message construct. This should not occur since there will be special notices if protocol versions change. For this design, protocol version changes will impact production and will adhere to a full (TBD) Configuration Management cycle.
Protocol Throttling may occur if a system becomes unstable. In the event of an overloaded system, certain mechanisms will be put into place in order to accommodate potential throttling. The DOD Center server will reject a request if the server load is over 50% Central Processing Unit (CPU) utilization. A simple CPU check command before processing will suffice. When the load is determined to be too high, the DOD Center system will pool other servers to fulfill the request. In the event that a server is not found, a LAS will be sent a Server Down message.
If a process "hangs" during function invocation, the primary DOD Center servers should recognize the delay and kill the process and retry. If there is still no response in the allotted time, then the server will be considered unstable and the message will be routed to an available server.
Some website links regarding SOAP performance are the following:
Design and implementation decisions made by SOAP infrastructure vendors can have a considerable impact upon system performance.
- Design Web Service interface to minimize network traffic. A 'coarse-grained' API is better, as it minimizes the number of requests that a client has to make to get information.
- Large SOAP messages are a performance bottleneck due to time spent parsing them. Payload size should be kept as small as possible.
- Complex SOAP messages are a performance bottleneck due to time spent serializing/de-serializing messages. Payload complexity should be kept low. However, payload complexity and payload size are often design tradeoffs.
- SOAP intermediaries (gateways, proxies) should minimize the parsing of messages.
- Better XML parsing techniques.
- Document/Literal style SOAP messages are smaller and less complex than RPC/SOAP messages.
- Security has performance costs. Not all SOAP traffic needs to be secure. The performance costs of an end-to-end security (i.e. WS-Security) are, in most cases, higher than those for a transport level security mechanism like SSL.
- Caching is a way to improve performance for processor-intensive services, although this is applicable only for read-only types of services.
- Persistent connections are good for performance in situations where a large number of messages of small payload size are handled. For larger messages, this has less of an effect. HTTP keep-alive is a way to request that a HTTP connection persists, though this is a default in HTTP/1.1.
- Streaming connections are good for performance in situations where a large payload size must be handled. HTTP "chunked encoding" is a kind of streaming, and is supported by HTTP/1.1.
- Binary encoding of some payload elements should be considered.
Information systems should keep data for 1-year cycles, and then the data should be archived remotely. As previously stated, all orders will remain open in the DOD Center system until a copy has either been returned undeliverable, returned by a reader, or designated lost by a network library. CD and DVD copies of DTB title masters will also be archived, probably both on and off site.
If a client or server is first powered on it should refer to a message queue. In the restart process, the server or client will establish an Internet connection and process the messages in the queue. This will be the same scenario if the client or server would be down for several hours. Note that there will be redundancy in place on the server side and some messages will be sent to available servers. In the event of a fully down client or server, a restoration plan will need to put in place. It is suggested to use a mirror drive on the both the Clients and network backup systems. A client restore should take less than a full business day.
It is assumed that the LASs and the DOD Center information systems will operate as companion applications. It is also assumed that communications between the LASs and DOD Centers will occur over the Internet or VPN, and that the LASs and DOD Center systems will be configured to communicate using SOAP.
The following technologies establish a framework within which the DOD Center information systems and LASs must operate in the future DTB distribution system:
- SOAP 1.2 (Character Set not addressed in this document)
- HTTP 1.1
The following are the major points concerning information system availability:
- 6:00 A.M. to 9:00 P.M. (local time), Monday through Friday.
- Some users may want to access the system for overtime work. Depending upon the number of users who access the system during off hours, a procedure for users to request off-hours system availability at least three days in advance could be put in place.
- In general, DOD Center systems will be online 24 hours a day, 7 days a week. They will nevertheless require a scheduled downtime to perform full backups, upgrades, or other housekeeping functions.
- An outage will last for up to a maximum of three hours.
The following are the major points regarding information system reliability:
- The system should support the carrying of message traffic reliably for business processes whose lifetimes commonly exceed the up-times of the components on which these processes are realized.
- Support quality-of-service assertions such as:
- That each message sent be received exactly once (i.e., once and only once), at most once, at least once, etc.
- That messages be received in the same order in which they were sent.
- That failure to deliver a message be made known to both the sender and receiver.
- Accommodate mobility of a reliable business process to different channels or physical machines.
- Support message transfer via intermediaries.
- Leverage the SOAP extensibility mechanism to achieve reliable messaging.
- Enable reliable messaging bindings to a variety of underlying reliable and unreliable transport protocols together with the Message Routing Protocol.
- Compose with other protocols in order to support security and other message delivery services.
The average throughput requirements, in terms of the average number of data communication transactions to be processes per day by the DOD Centers, by type of transaction, were shown in Section 4.3.5. The DOD Center information system that interfaces with the LASs must be capable of handling this volume of traffic, plus a differential above the average sufficient for handling "peak" loads (within a range of "reasonableness"). Exactly how this peak load should be defined is somewhat ambiguous, but the DOD information system design should plan for a capacity that can handle traffic at least 50% higher than the average workload, but no more than 100% higher.
The Web Services utilized by LASs and DOD Center information systems will need to adhere to the following security requirements:
- The systems will be required to follow LOC security directives.
Authentication: Authentication will ensure that both LAS and DOD Center systems are what they actually claim to be. Authentication will involve accepting credentials from the entity and validating them against an authority.
- Authorization: Authorization will determine whether the DOD Center system has granted access to the Web Service to a LAS.
- Data Protection: Data protection will ensure that the Web Service request and response have not been tampered with en route. It will require securing both data integrity and privacy.
- Nonrepudiation: Nonrepudiation guarantees that the message sender is the same as the creator of the message.
Client / Server Side Data Storage Requirements
Please reference Section 4.3.6.
There would be an integrated information system in a DOD Center in the envisioned future Hybrid book distribution operations. The system would exchange information with network libraries and manage and process orders for DTBs duplicated-on-demand in a semi or fully-automated production environment capable of an average daily throughput of 8,000 DTB copies.
The information system would be custom-built, and the required functionality would have to be developed by a Systems Developer/Integrator. Regardless of how the overall information system is configured vis-à-vis formal subsystems (e.g., one that interacts with LASs, and another that controls DTB production within the Center, each interacting with one another in a fully-integrated system), the core functionality requirements of the system are as follows:
- Information Exchange - Exchange information with network libraries to facilitate DOD distribution of DTBs, the envisioned processes for which have been discussed extensively above.
- Production Control - Manage all production within the DOD Centers including production planning.
- Check-In - Interface with receiving workstations for patron returns, undeliverables, and possibly for new title masters (new title master processing may occur at a workstation other than the receiving workstation).
- Print Labels - Produce print title labels for the labeling of all copies produced.
- Braille Labels - Produce Braille title labels for the labeling of about 10% of the copies produced.
- Duplication - Manage the Flash Drive cartridge dispenser/duplicator/file server/output conveyor interface.
- Shipping Labels/Checkout – Print shipping labels at the packing stations and checkout outgoing copies (which could be performed in one or two steps).
Mass-Storage Device(s)/DTB File Servers
The inventory of all DTB titles that would be used by a C center for duplication-on-demand operations will reside in large file, or "image," servers utilizing hard drive-based data storage and having very fast READ-speeds, rather than servers using optical jukebox-based storage (which have much slower READ speeds). If nominally 40,000 DTB titles are to be stored eventually on the files servers in compressed format (either AMRWB+ or older MP3 formatted titles), then the required capacity of the file servers would be a minimum of 5 TB per Center (which assumes that the average NLS DTB is about 120 MB in size in compressed format).
These file servers would be loaded with new DTB titles in compressed and "image" (which expedites the copying of files) format as received from NLS narration contractors. Eight new DTB titles would be received per average workday (assuming 250 working days per year), assuming that all new audio book titles, not just titles to be provided via DOD, would be stored on the Centers’ servers (even if not provided via DOD for several years, in the case of mass-duplicated titles that later migrate from library to DOD Center circulation).
New DTB titles would be received at the DOD Centers from NLS narration contractors via either data telecommunicated over high-speed channels and/or courier delivery on CDs, DVDs, or other portable data storage medium. Even if the new title masters are acquired via cvia data telecommunications, CD/DVD copies of the title masters would nevertheless also be shipped to the DOD c Centers and stored there on-site and/or in close proximity offsite in a backup collection for restore and recovery operations if necessary. Files for new DTB titles received by the DOD Centers would be written to the file servers after duplication operations have ceased for the workday, not while the duplicators are operating. These WRITE operations would constitute a minor daily task.
The primary production function of the file servers will be to "feed" the flash memory cartridge duplicators with DTB files as directed by the production control system. As previously stated, the DTB files will be in image format to expedite both reading from the file servers and writing to the flash cartridges being duplicated. High-speed optical cables will enable the rapid transfer of DTB files from the file servers to the duplicators. During the hours that duplicators are operating, the file servers will be exclusively performing READ operations and feeding the duplicators with DTB files. WRITE operations, i.e., the loading of new DTB files onto the file servers, will occur when the duplicators are not operating. The ability of the file servers to perform continuous, rapid READ operations will be critical to the DOD production operation, and they collectively must have the ability to READ 8,000 DTB files per working day during a nominal 8-hour shift.
These mass-storage file servers will not be used for any other purposes in the DOD Center operations. That is, neither applications software for enabling DOD Center-library information exchange, managing order processing, nor controlling actual DTB DOD production within the Centers will be housed on these file servers. Neither will network libraries nor external entities have access to the file servers in the DOD Centers. As noted in Section 3, network libraries and patrons will have access to the NLS DAMS for downloading DTBs.
It is recommended that each DOD Center have at least two completely independent file server devices, each capable of storing all the DTB title files managed by the Centers. This will be necessary for risk-reduction reasons, i.e., so that one can continue to operate if the other is down regardless of whether, from a logistics standpoint, one or two file servers will be necessary to feed the flash cartridge duplicators with DTB files when the duplicators are operating. Therefore, even if a single mass-storage device were to have multiple, independent hard drives which could function as file servers (in terms of both required capacity and READ speed), if such a device were to have a single power supply, then it would not provide the type of independent support that will be required for the recommended "duplex" DOD DTB production lines in a Center.
It is assumed that each mass-storage file server will cost approximately $75,000, and that two servers will be required for each DOD Center for the reasons cited at a total cost of about $150,000. Appendix 21 contains basic information on one such system.
The estimated costs for the DOD Centers’ information systems are included among all estimated capital equipment costs shown in Appendix 18 of the report. It is assumed that the costs for software development for all information system functionality, net of the cartridge dispenser/duplicator/mass-storage device/takeaway conveyor systems integration needs, will cost approximately $1 million (Item 26), and is amortized between two DOD Centers. Computer system hardware other than the customized computers controlling the cartridge duplicator subsystem, the mass-storage devices and the label servers, is estimated to cost $60,000 per Center (Item 25). As noted above, each Center will require two mass-storage devices, each of which is estimated to cost $75,000, for a total cost per Center of $150,000 (Item 22). Label servers (dedicated to storing and providing data on DTB titles handled by the Centers) will cost about $25,000 per Center (Item 23). Finally, the estimated costs for both computer hardware and customized software development for the cartridge dispenser/duplicator/mass-storage device/takeaway conveyor subsystem are subsumed within the total estimated costs for the duplicators themselves (Item 19); these capabilities and costs are the most uncertain aspect of the Centers’ operations because the technology for Flash Drive automated duplication is now in its infancy.
The estimated major NLS-incurred costs for audio book duplication and distribution under this system are shown in Appendix 20. Many of the estimates and assumptions used are unchanged from those used for the final Phase 1 report. Those that changed for the Mass-Duplication portion of the Hybrid system are described in Section 3 for the All Mass-Duplication system.
Those that changed for the DOD portion of the Hybrid are as follows: systemwide circulation is assumed to be 19,000,000 rather than 20,000,000 book copies per year; the 18.5% of total circulation handled by DOD Centers in a 40% Mass-Duplicated Title-Based Hybrid is simply rounded-up to 20% to be slightly more conservative; the average price for a DOD copy (net of cartridge and container) has been changed from $0.66 to $0.65 based upon updated cost estimates for the DOD Center operations described in this section; and the new container unit price has been changed from $0.50 to $0.60 based upon current and near-future expected prices (as it has been changed for the Mass Duplication portion of the Hybrid, and the All Mass-Duplication system). As was the case for the analysis shown in Section 3 for the All Mass-Duplication system and in the Phase 1 analysis for both the Hybrid and All Mass-Duplication systems, narration costs (currently between $5 and $5.5 million per year) are not included, and will not vary with the DTB distribution system implemented.
With regard to network library costs under the Hybrid system, the analysis shown in Appendix 60 of the final Phase 1 report is still essentially valid. This analysis used FY 2004 values from the NLS LCC model, which are still the most current source data available as of this writing.