SNIA has a new specification in town – focused on key value storage. SNIA on Storage sat down with Bill Martin, Co-Chair of the SNIA Technical Council and Co-Chair of the SNIA Object Drive Technical Work Group, to understand why SNIA took on this project and what are the results.
SNIA On Storage (SOS): Bill, thanks for taking the time to chat with us. To get started, can you tell me what key value storage is and how it relates to the Technical Work charter that SNIA undertakes?
Bill Martin (BM): Key value storage is a new method of storing data when compared to the traditional block storage method. You store a “Value” related to a “key (address)”, with the ability to then look up the value in the future using the “key” of the associated object.
Within SNIA, the Object Drive Technical Work Group (TWG) has the charter to establish architectures and standards for disk-, solid state-, and tape drive- based functionalities that allow them to be higher level storage nodes in emerging scale-out solutions. The TWG develops specifications that are vendor-agnostic and support existing and future functionality in drive form factors.
Under the charter, the Object Drive TWG has developed and published the Key Value Storage API Specification, a SNIA Technical Position that defines an application programming interface (API) for Key Value storage devices.
With a storage device that enables key value storage, the API allows upper level applications to take advantage of the key value store method, independent of what infrastructure is being used. The API outlines a set of functions that access key value storage the same way regardless of lower level protocol (NVMe, PCIe, or SCSI). The provider of the lower level protocol provides a library interface that translates the key value API calls into whatever the lower level function calls need. The API does the heavy lifting- freeing the application to not worry about what bus the storage is on.
SOS: Why did SNIA undertake this work?
BM: As storage is evolving from disk-based to non-volatile memory (NVM), there is a different paradigm for how data is addressed on media. Disk storage has logical and physical block address mapping. With NVM storage, the media is addressed differently; so for a key to be stored using the traditional mechanism, the host translates the key address to a logical block address, and then the storage device translates to the physical location on the media. Therefore, if we can take a key as an address and hand it to the NVM storage device, we only need to do a single translation to store or retrieve the value on the media. Eliminating the second translation reduces the time needed for mapping and storing the data on the physical device and a mapping table in the host – the “location of the data” now resides on the key value storage device, eliminating both physical storage and latency.
SOS: What work is being done to make Key Value Storage available for the operating systems?
BM: The first place the API will be used is in the Linux stack. Samsung, a SNIA and Object Drive TWG member, has developed open source software for use in the Linux stack. The drivers will be based on lower level protocol – a NVMe key value storage device.
While Linux is the first effort by reason of the customer base focus on Linux-based operating systems, subsequently there will be drivers for Microsoft.
SOS: Is SNIA doing the lower level functional standards for key value storage as well?
BM: The SNIA API programming interface is independent of the lower level commands. One lower level protocol interface is being developed by a technical work group within the NVM Express organization, and will be announced by them at a later date. Many of the same individuals working on the SNIA specification are also working on the NVM Express specification, taking advantage of a strategic alliance announced between the two organizations and DMTF in 2018 (Strategic Alliance Formed with SNIA, DMTF, and NVM Express)
SOS: Is there more work to do on key value storage?
BM: Version 1.0 of the Key Value Storage Specification is now out as a published SNIA Technical Position and available for anyone to use. The Object Drive TWG has developed another published Technical Position on IP- based drive management. Work now underway in the TWG is on a management interface for key value storage devices. The TWG is looking at the need to manage key value storage devices to find out their capabilities and select the capabilities they are interested in to effectively utilize the API to move data. Currently, the KV API Specification is a set of functions that allow storage and retrieval of data. The management work will look at how to configure storage, incorporating work done by the SNIA Scalable Storage Management (SSM) Technical Work Group on the SNIA SwordfishTM open storage management model as an extension of the DMTF Redfish system management model.
SOS: What do you see impacting storage and management in the future?
BM: I am very enthusiastic about the relationship between key value storage and computational storage. Today what we can do is address storage based on a set of keys. Because key value storage eliminates the extra key to block transaction, it will facilitate computation. In the future, we will be able to request computation be done on the values associated with the keys and return only the meaningful data.
SOS: Where can blog readers learn more about key value storage?
BM: SNIA has an exciting new Educational Library, which cross references over 700 pieces of SNIA content assets, including presentations, white papers, tutorials, webcasts, and technical specifications. I’d first recommend reading the Key Value Storage Specification, and watching this video on Key Value Storage Standardization Progress. You can then search the library for other key value storage items, and related topics such as NVMe, object drives, and Linux.