Category: Storage Management

SNIA Swordfish™ – Your Questions Answered

Diane Marsili Leave a comment

The Storage Networking Industry Association’s (SNIA’s) Storage Management Initiative (SMI) took on the topic of SNIA Swordfish™ in a live webcast titled “Introduction to SNIA Swordfish™ – Scalable Storage Management.” The replay is available here. SNIA experts Richelle Ahlvers and Don Deel, responded to questions during the webcast. Here are those questions and responses:

Q. You talked about two different ways to add storage to Redfish – hosted service configuration and integrated service configuration. When would you use one configuration instead of the other?

A. The integrated services configuration was added to clarify support with direct attach configurations using Swordfish constructs. If you have a server that has a RAID card in it, and you want to have it use a more complex storage configuration – storage pools and some notion of class of service, you would use the integrated service configuration. The hosted service configuration is used to model non-direct attach configurations, such as external storage arrays, or file services. Read More

A Q&A on Storage Management – These Folks Weren’t Too Proud to Ask!

J Metz

The most recent installment of our SNIA ESF webcast series “Everything You Wanted To Know About Storage But Were Too Proud To Ask” took on a broad topic – storage management. Our experts, Richelle Ahlvers, Mark Rogov and Alex McDonald did a great job explaining the basics and have now answered the questions that attendees asked here in this blog. If you missed the live webcast, check it out on-demand and download a copy of the slides if you’d like to take notes.

Q: What is the difference between storage and a database? Could a database be considered storage?

A: The short answer is no. The long answer relies on the fact that a database doesn’t just store data: it modifies the data to fit into its schema (table, index, etc.) A storage solution doesn’t mutate the data in any shape—the data is always preserved as is.

Q: Doesn’t provisioning a storage array mean setting it up?

A: Within the storage community, provisioning is akin to serving a cake at a party. To provision storage to a server means cutting a slice of usable capacity and allocating it to a very specific server. The record of the particular pairing is carefully recorded.

Q: Does deduplication fall into Configuration? Even when it is done only on cold data?

A: Great question! Deduplication is one of the services that a storage array may offer, therefore enabling it is configuring such service. To further clarify your question, the point of deduplication is irrelevant: it may happen with cold data (the data that is stored on the array but applications haven’t accessed it in a long time); it may happen to hot or in-flight data (frequently accessed data or data inside cache).

Q. Do Hyperscale vendors (like AWS) use any of the storage management?

A. Hyperscale vendors, like all consumers of storage, use storage management to configure their storage. They use a combination of vendor device tools and custom development scripts/tools, but are not heavy consumers of industry standard storage interfaces today. Swordfish’s RESTful interface will provide an easy-to-consume API for hyperscale vendors to integrate into their management ecosystem as vendors start delivering Swordfish-based solutions.

Q. It was mentioned that there was a ‘steep learning curve’ for previous SNIA storage management model. Any idea how much easier this is to learn?

A. One of the major advantages for Swordfish is that the RESTful API’s are standardized and can take advantage of readily available tools and infrastructure. With the JSON-based payload, you can use standard plug-ins for browsers, as well as Python scripting languages to immediately interact with the Swordfish API’s. This is a distinct difference from the SMI-S API’s, which although they are also XML-based APIs, required custom or third-party tools to interact with the SMI-S providers.

Q. You talked about how Swordfish is being designed as more user and client centric.  How are you doing this?   

A. We are starting with very specific use cases and scenarios  (rather than looking at “what is all the functionality we could expose”) as we build both the structure of the API and the amount of information returned.   We’ve also documented a lot of the basic use cases, and who might like to do them, in a user’s guide, and published that alongside the Swordfish specification.  You can find links to this at the SNIA Swordfish page: snia.org/swordfish

Q. You weren’t specific on storage management tools, and I was expecting more detail. I’m wondering why you did this at such a high level, as this really hasn’t helped me.

A. We were primarily referring to ITIL –(The Information Technology Infrastructure Library). It’s a framework designed to standardize the selection, planning, delivery and support of IT services to a business. Learn more here.

Q. While most of the products today support SMI-S, it’s not something that DevOps or Storage Admins use directly.  How, or is, Swordfish going to be different?

A. There are two primary ways we see the Swordfish API being much more accessible directly to the individual admins.  First, as a RESTful interface, it is very easy to access and traverse with the tools that they use daily – from web browsers directly, to REST plugins, to simple (or complex) python scripts.  The learning curve to start interacting with Swordfish is extremely small.  You can get a sense by going to an online “mockup” site here:  http://swordfishmockups.com – there are some simple instructions to either browse the system directly or some standard clients to make it easier.  That will give you an idea of how easy it will be to start interacting with Swordfish (plus security for a real system, of course).

Remember the “Everything You Wanted To Know About Storage But Were Too Proud To Ask” is a series. We’ve covered 8 storage topics to date and have a library of on-demand webcasts you can watch at your convenience. Happy viewing!

A Q&A on Storage Management – These Folks Weren’t Too Proud to Ask!

J Metz

The most recent installment of our SNIA ESF webcast series “Everything You Wanted To Know About Storage But Were Too Proud To Ask” took on a broad topic – storage management. Our experts, Richelle Ahlvers, Mark Rogov and Alex McDonald did a great job explaining the basics and have now answered the questions that attendees asked here in this blog. If you missed the live webcast, check it out on-demand and download a copy of the slides if you’d like to take notes.

Q: What is the difference between storage and a database? Could a database be considered storage?

A: The short answer is no. The long answer relies on the fact that a database doesn’t just store data: it modifies the data to fit into its schema (table, index, etc.) A storage solution doesn’t mutate the data in any shape—the data is always preserved as is.

Q: Doesn’t provisioning a storage array mean setting it up?

A: Within the storage community, provisioning is akin to serving a cake at a party. To provision storage to a server means cutting a slice of usable capacity and allocating it to a very specific server. The record of the particular pairing is carefully recorded.

Q: Does deduplication fall into Configuration? Even when it is done only on cold data?

A: Great question! Deduplication is one of the services that a storage array may offer, therefore enabling it is configuring such service. To further clarify your question, the point of deduplication is irrelevant: it may happen with cold data (the data that is stored on the array but applications haven’t accessed it in a long time); it may happen to hot or in-flight data (frequently accessed data or data inside cache).

Q. Do Hyperscale vendors (like AWS) use any of the storage management?

A. Hyperscale vendors, like all consumers of storage, use storage management to configure their storage. They use a combination of vendor device tools and custom development scripts/tools, but are not heavy consumers of industry standard storage interfaces today. Swordfish’s RESTful interface will provide an easy-to-consume API for hyperscale vendors to integrate into their management ecosystem as vendors start delivering Swordfish-based solutions.

Q. It was mentioned that there was a ‘steep learning curve’ for previous SNIA storage management model. Any idea how much easier this is to learn?

A. One of the major advantages for Swordfish is that the RESTful API’s are standardized and can take advantage of readily available tools and infrastructure. With the JSON-based payload, you can use standard plug-ins for browsers, as well as Python scripting languages to immediately interact with the Swordfish API’s. This is a distinct difference from the SMI-S API’s, which although they are also XML-based APIs, required custom or third-party tools to interact with the SMI-S providers.

Q. You talked about how Swordfish is being designed as more user and client centric.  How are you doing this?   

A. We are starting with very specific use cases and scenarios  (rather than looking at “what is all the functionality we could expose”) as we build both the structure of the API and the amount of information returned.   We’ve also documented a lot of the basic use cases, and who might like to do them, in a user’s guide, and published that alongside the Swordfish specification.  You can find links to this at the SNIA Swordfish page: snia.org/swordfish

Q. You weren’t specific on storage management tools, and I was expecting more detail. I’m wondering why you did this at such a high level, as this really hasn’t helped me.

A. We were primarily referring to ITIL –(The Information Technology Infrastructure Library). It’s a framework designed to standardize the selection, planning, delivery and support of IT services to a business. Learn more here.

Q. While most of the products today support SMI-S, it’s not something that DevOps or Storage Admins use directly.  How, or is, Swordfish going to be different?

A. There are two primary ways we see the Swordfish API being much more accessible directly to the individual admins.  First, as a RESTful interface, it is very easy to access and traverse with the tools that they use daily – from web browsers directly, to REST plugins, to simple (or complex) python scripts.  The learning curve to start interacting with Swordfish is extremely small.  You can get a sense by going to an online “mockup” site here:  http://swordfishmockups.com – there are some simple instructions to either browse the system directly or some standard clients to make it easier.  That will give you an idea of how easy it will be to start interacting with Swordfish (plus security for a real system, of course).

Remember the “Everything You Wanted To Know About Storage But Were Too Proud To Ask” is a series. We’ve covered 8 storage topics to date and have a library of on-demand webcasts you can watch at your convenience. Happy viewing!

A Q&A on Storage Management – These Folks Weren’t Too Proud to Ask!

J Metz
The most recent installment of our SNIA ESF webcast series “Everything You Wanted To Know About Storage But Were Too Proud To Ask” took on a broad topic – storage management. Our experts, Richelle Ahlvers, Mark Rogov and Alex McDonald did a great job explaining the basics and have now answered the questions that attendees asked here in this blog. If you missed the live webcast, check it out on-demand and download a copy of the slides if you’d like to take notes. Read More

Too Proud to Ask Webcast Series Opens Pandora’s Box – Storage Management

J Metz
Storage can be something of a “black box,” a monolithic entity that is at once mysterious and scary. That’s why we created “The Everything You Wanted To Know About Storage But Were Too Proud to Ask” webcast series. So far, we’ve explored various and sundry aspects of storage, focusing on “the naming of the parts.” Our goal has been to break down some of the components of storage and explain how they fit into the greater whole. On September 28th, we’ll be hosting “Everything You Wanted To Know About Storage But Were Too Proud To Ask – Part Cyan – Storage Management.” This time, we’re going to open up Pandora’s Box and peer inside the world of storage management, uncovering some of the key technologies that are used to manage devices, storage traffic, and storage architectures. In particular, we’ll be discussing: Read More

SNIA Highlights Persistent Memory and Scalable Storage Management at Storage Field Day 13

khauser 2 Comments

Following enthusiastic response to their first Storage Field Day in March, SNIA is returning to the lineup on June 15, 2017. Storage Field Day events bring together innovative IT organizations and independent thought leaders to share information and opinions in a presentation format that is lively – and live streamed.

SNIA will present Storage Field Day #13 at their Technology Center in Colorado Springs, CO where organizer Stephen Foskett and a dozen delegates will tour the facility and interact with SNIA members on persistent memory and scalable storage management – two hot storage topics that consumers and the industry want to learn more about.

Read More

Managing Your Computing Ecosystem Part Two

khauser Leave a comment

by George Ericson, Distinguished Engineer, Dell EMC; Member,
SNIA Scalable Storage Management Technical Working Group,
@GEricson

 

Introduction

This blog is part two of a three-part series by George Ericson, a distinguished engineer at Dell EMC. If you missed part one, you can read it here. George is an active participant on the SNIA Scalable Storage Management Technical Working Group which has been developing the SNIA Swordfish™ storage management specification. Read More

Managing Your Computing Ecosystem Part Two

Kristen Hauser

by George Ericson, Distinguished Engineer, Dell EMC; Member,
SNIA Scalable Storage Management Technical Working Group,
@GEricson

 

Introduction

This blog is part two of a three-part series by George Ericson, a distinguished engineer at Dell EMC. If you missed part one, you can read it here. George is an active participant on the SNIA Scalable Storage Management Technical Working Group which has been developing the SNIA Swordfish™ storage management specification.

SNIA Swordfish is designed to integrate with the technologies used in cloud data center environments and can be used to accomplish a broad range of storage management tasks from the simple to the advanced. SNIA is holding the very first Swordfish plugfest June 13-15 in the SNIA Technology Center in Colorado Springs.

Overview

We are making strides toward universal and interoperable management interfaces. These are not only interfaces that will interoperate across one vendor or one part of the stack, but management interfaces that can truly integrate your infrastructure management. Last time, we discussed OData, the Rest standardization. This time we will talk about Redfish for managing hardware platforms.

DMTF’s Redfish®

Redfish defines a simple and secure, OData conformant data service for managing scalable hardware platforms. Redfish is defined by a set of open industry standard specifications that are developed by the Distributed Management Task Force, Inc. (DMTF).

The initial development was from the point of view of a Baseboard Management Controller (BMC) or equivalent. Redfish management currently covers bare-metal discovery, configuration, monitoring, and management of all common hardware components. It is capable of managing and updating installed software, including for the operating system and for device drivers.

Redfish is not limited to low-level hardware/firmware management. It is also expected to be deployed to manage higher level functionality, including configuration and management of containers and virtual systems.   In collaboration with the IETF, Redfish is also being extended to include management of networks.

The Redfish Scalable Platforms Management API Specification specifies functionality that can be divided into three areas: OData extensions, utility interfaces, and platform management interfaces. These are described briefly in the following sections.

Redfish OData extensions

Redfish requires at least OData v4 and specifies some additional constraints:

  • Use of HTTP v1.1 is required, with support for POST, GET, PATCH, and DELETE operations, including requirements on many HTTP headers
  • JSON representations are required within payloads
  • Several well-known URIs are specified
    • /redfish/v1/ returns the ServiceRoot resource for locating resources
    • /redfish/v1/OData/ returns the OData service document for locating resources
    • /redfish/v1/$metadata returns the OData metadata document for locating the entity data model declarations.

Redfish also extends the OData metamodel with an additional vocabulary for annotating model declarations. The annotations specify information about, or behaviors of the modeled resources.

Redfish utility interfaces

The utility interfaces provide functionality that is useful for any management domain (for example, these interfaces are used by Swordfish for storage management). These interfaces include account, event, log, session, and task management.

The account service manages access to a Redfish service via a manager accounts and roles.

The event service provides the means to specify events and to subscribe to indications when a defined event occurs on a specified set of resources. Each subscription specifies where indications are sent, this can be to a listening service or to an internal resource, (e.g. a log service).

Each log service manages a collection of event records, including size and replacement policies. Resources may have multiple log services for different purposes.

The session service manages sessions and enables creation of an X-Auth-Token representing a session used to access the Redfish service.

The task service manages tasks that represent independent threads of execution known to the redfish service. Typically tasks are spawned as a result of a long running operation.

The update service provides management of firmware and software resources, including the ability to update those resources.

Redfish platform management interfaces

The principal resources managed by a Redfish service are chassis, computer systems and fabrics. Each resource has its current status. Additionally, each type of resource may have references to other resources, properties defining the current state of the resource, and additional actions as necessary.

Each chassis represents a physical or logical container. It may represent a sheet-metal confined space like a rack, sled, shelf, or module. Or, it may represent a logical space like a row, pod, or computer room zone.

Each computer system represents a computing system and its software-visible resources such as memory, processors and other devices that can be accessed from that system. The computer system can be general purpose system or can be a specialized system like a storage server or a switch.

Each fabric represents a collection of zones, switches and related endpoints. A zone is a collection of involved switches and contained endpoints. A switch provides connectivity between a set of endpoints.

All other subsystems are represented as resources that are linked via one or more of these principal resources. These subsystems include: bios, drives, endpoints, fans, memories, PCIe devices, ports, power, sensors, processors and various types of networking interfaces.

Conclusion

Redfish delivers a standardized management interface for hardware resources. While it is beginning with basic functionality like discovery, configuration and monitoring, it will deliver much more. It will extend into both richer services and cover more than physical resources – e.g. virtual systems, containers, and networks. Redfish is built as an OData conformant service, which makes it the second connected part of an integrated management API stack. Next up – Swordfish.

Managing Your Computing Ecosystem Part Two

Kristen Hauser

by George Ericson, Distinguished Engineer, Dell EMC; Member,
SNIA Scalable Storage Management Technical Working Group,
@GEricson

 

Introduction

This blog is part two of a three-part series by George Ericson, a distinguished engineer at Dell EMC. If you missed part one, you can read it here. George is an active participant on the SNIA Scalable Storage Management Technical Working Group which has been developing the SNIA Swordfish™ storage management specification.

SNIA Swordfish is designed to integrate with the technologies used in cloud data center environments and can be used to accomplish a broad range of storage management tasks from the simple to the advanced. SNIA is holding the very first Swordfish plugfest June 13-15 in the SNIA Technology Center in Colorado Springs.

Overview

We are making strides toward universal and interoperable management interfaces. These are not only interfaces that will interoperate across one vendor or one part of the stack, but management interfaces that can truly integrate your infrastructure management. Last time, we discussed OData, the Rest standardization. This time we will talk about Redfish for managing hardware platforms.

DMTF’s Redfish®

Redfish defines a simple and secure, OData conformant data service for managing scalable hardware platforms. Redfish is defined by a set of open industry standard specifications that are developed by the Distributed Management Task Force, Inc. (DMTF).

The initial development was from the point of view of a Baseboard Management Controller (BMC) or equivalent. Redfish management currently covers bare-metal discovery, configuration, monitoring, and management of all common hardware components. It is capable of managing and updating installed software, including for the operating system and for device drivers.

Redfish is not limited to low-level hardware/firmware management. It is also expected to be deployed to manage higher level functionality, including configuration and management of containers and virtual systems.   In collaboration with the IETF, Redfish is also being extended to include management of networks.

The Redfish Scalable Platforms Management API Specification specifies functionality that can be divided into three areas: OData extensions, utility interfaces, and platform management interfaces. These are described briefly in the following sections.

Redfish OData extensions

Redfish requires at least OData v4 and specifies some additional constraints:

  • Use of HTTP v1.1 is required, with support for POST, GET, PATCH, and DELETE operations, including requirements on many HTTP headers
  • JSON representations are required within payloads
  • Several well-known URIs are specified
    • /redfish/v1/ returns the ServiceRoot resource for locating resources
    • /redfish/v1/OData/ returns the OData service document for locating resources
    • /redfish/v1/$metadata returns the OData metadata document for locating the entity data model declarations.

Redfish also extends the OData metamodel with an additional vocabulary for annotating model declarations. The annotations specify information about, or behaviors of the modeled resources.

Redfish utility interfaces

The utility interfaces provide functionality that is useful for any management domain (for example, these interfaces are used by Swordfish for storage management). These interfaces include account, event, log, session, and task management.

The account service manages access to a Redfish service via a manager accounts and roles.

The event service provides the means to specify events and to subscribe to indications when a defined event occurs on a specified set of resources. Each subscription specifies where indications are sent, this can be to a listening service or to an internal resource, (e.g. a log service).

Each log service manages a collection of event records, including size and replacement policies. Resources may have multiple log services for different purposes.

The session service manages sessions and enables creation of an X-Auth-Token representing a session used to access the Redfish service.

The task service manages tasks that represent independent threads of execution known to the redfish service. Typically tasks are spawned as a result of a long running operation.

The update service provides management of firmware and software resources, including the ability to update those resources.

Redfish platform management interfaces

The principal resources managed by a Redfish service are chassis, computer systems and fabrics. Each resource has its current status. Additionally, each type of resource may have references to other resources, properties defining the current state of the resource, and additional actions as necessary.

Each chassis represents a physical or logical container. It may represent a sheet-metal confined space like a rack, sled, shelf, or module. Or, it may represent a logical space like a row, pod, or computer room zone.

Each computer system represents a computing system and its software-visible resources such as memory, processors and other devices that can be accessed from that system. The computer system can be general purpose system or can be a specialized system like a storage server or a switch.

Each fabric represents a collection of zones, switches and related endpoints. A zone is a collection of involved switches and contained endpoints. A switch provides connectivity between a set of endpoints.

All other subsystems are represented as resources that are linked via one or more of these principal resources. These subsystems include: bios, drives, endpoints, fans, memories, PCIe devices, ports, power, sensors, processors and various types of networking interfaces.

Conclusion

Redfish delivers a standardized management interface for hardware resources. While it is beginning with basic functionality like discovery, configuration and monitoring, it will deliver much more. It will extend into both richer services and cover more than physical resources – e.g. virtual systems, containers, and networks. Redfish is built as an OData conformant service, which makes it the second connected part of an integrated management API stack. Next up – Swordfish.