Category: Open Source

Ceph Q&A

Erin Farr
In a little over a month, more than 1,500 people have viewed the SNIA Cloud Storage Technologies Initiative (CSTI) live webinar, “Ceph: The Linux of Storage Today,” with SNIA experts Vincent Hsu and Tushar Gohad. If you missed it, you can watch it on-demand at the SNIA Educational Library. The live audience was extremely engaged with our presenters, asking several interesting questions. As promised, Vincent and Tushar have answered them here. Given the high level of this interest in this topic, the CSTI is planning additional sessions on Ceph. Please follow us @SNIACloud or at SNIA LinkedIn for dates. Q: How many snapshots can Ceph support per cluster? Q: Does Ceph provide Deduplication? If so, is it across objects, file and block storage?  A: There is no per-cluster limit. In the Ceph filesystem (cephfs) it is possible to create snapshots on a per-path basis, and currently the configurable default limit is 100 snapshots per path. The Ceph block storage (rbd) does not impose limits on the number of snapshots.  However, when using the native Linux kernel rbd client there is a limit of 510 snapshots per image. Read More

Here’s Why Ceph is the Linux of Storage Today

Erin Farr
Data is one of the most critical resources of our time. Storage for data has always been a critical architectural element for every data center, requiring careful considerations for storage performance, scalability, reliability, data protection, durability and resilience. A decade ago, the market was aggressively embracing public storage because of its agility and scalability. In the last few years, people have been rethinking that approach, moving toward on-premises storage with cloud consumption models. The new cloud native architecture on-premises has the promise of the traditional data center’s security and reliability with cloud agility and scalability. Ceph, an Open Source project for enterprise unified software-defined storage, represents a compelling solution for this cloud native on-premises architecture and will be the topic of our next SNIA Cloud Storage Technologies Initiative webinar, “Ceph: The Linux of Storage Today.” This webinar will discuss: Read More

A Q&A on the Open Programmable Infrastructure (OPI) Project

Joseph White
Last month, the SNIA Networking Storage Forum hosted several experts leading the Open Programmable Infrastructure (OPI) project with a live webcast, “An Introduction to the OPI (Open Programmable Infrastructure) Project.” The project has been created to address a new class of cloud and datacenter infrastructure component. This new infrastructure element, often referred to as Data Processing Unit (DPU), Infrastructure Processing Unit (IPU) or xPU as a general term, takes the form of a server hosted PCIe add-in card or on-board chip(s), containing one or more ASIC’s or FPGA’s, usually anchored around a single powerful SoC device. Our OPI experts provided an introduction to the OPI Project and then explained lifecycle provisioning, API, use cases, proof of concept and developer platform. If you missed the live presentation, you can watch it on demand and download a PDF of the slides at the SNIA Educational Library. The attendees at the live session asked several interesting questions. Here are answers to them from our presenters. Q. Are there any plans for OPI to use GraphQL for API definitions since GraphQL has a good development environment, better security, and a well-defined, typed, schema approach? Read More

Understanding CDMI and S3 Together

Alex McDonald
How does the Cloud Data Management Interface (CDMI™) International Standard work? Is it possible be to both S3 and CMDI compliant? What security measures are in place with CDMI? How, and where, is CDMI being deployed? These are just some of the topics we covered at our recent SNIA Cloud Storage Technologies (CSTI) webcast, “Cloud Data Management & Interoperability: Why A CDMI Standard Matters.” CDMI is intended for application developers who are implementing cloud storage systems, and who are developing applications to manage and consume cloud storage. Q. Can you compare CDMI to S3? Is it possible to be both CDMI and S3 compliant? Is it too complicated? Read More

A New Wave of Video Analytics

Jim Fister

Adoption of cognitive services based on video and image analytics is on the rise. It’s an intriguing topic that the SNIA Cloud Storage Technologies Initiative will dive into on December 2, 2020 at our live webcast, “How Video Analytics is Changing the Way We Store Video.” In this webcast, we will look at some of the benefits and factors driving this adoption, as well as explore compelling projects and required components for a successful video-based cognitive service. This includes some great work in the open source community to provide methods and frameworks, some standards that are being worked on to unify the ecosystem and allow interoperability with models and architectures. Finally, we’ll cover the data required to train such models, the data source and how it needs to be treated.

 

As you might guess, there are challenges in how we do all of this. Read More

Key Value Storage – A Talk with Bill Martin of the SNIA Technical Council

khauser Leave a comment

SNIA has a new specification in town – focused on key value storage.  SNIA on Storage sat down with Bill Martin, Co-Chair of the SNIA Technical Council and Co-Chair of the SNIA Object Drive Technical Work Group, to understand why SNIA took on this project and what are the results.

SNIA On Storage (SOS):  Bill, thanks for taking the time to chat with us.   To get started, can you tell me what key value storage is and how it relates to the Technical Work charter that SNIA undertakes?

Bill Martin (BM):  Key value storage is a new method of storing data when compared to the traditional block storage method.  You store a “Value” related to a “key (address)”, with the ability to then look up the value in the future using the “key” of the associated object. Read More

Open Source Software-Only Storage – Really.

Mario Blandini

Virtually any storage solution is more parts software than hardware. Having said this, users don’t care as much about the percentage of hardware vs. software. They want their consumption experience to be easy and fast to start up, with a pay-as-you-grow model and with the ability to scale without limits. So, it should not be a shock that real IT organizations are using software-only on standard servers to deliver storage to their customers. What’s more, this type of storage can be powered by open source.

At the upcoming SNIA Data Storage Innovation Conference, we are looking forward to discussing software-defined storage (SDS) from a user experience perspective with examples of OpenStack Swift providing an engine for building SDS clusters with any mixed combination of standard server and HDD hardware in a way that is simple enough for any enterprise to dynamically scale.

Swift is a highly available, distributed, scalable object store available as open source.  It is designed to handle non-relational (that is, not just simple row-column data) or unstructured data at large scale with high availability and durability.  For example, it can be used to store files, videos, documents, analytics results, Web content, drawings, voice recordings, images, maps, musical scores, pictures, or multimedia. Organizations can use Swift to store large amounts of data efficiently, safely, and cheaply. It scales horizontally without any single point of failure.  It offers a single multi-tenant storage system for all applications, the ability to use low-cost industry-standard servers and drives, and a rich ecosystem of tools and libraries.  It can serve the needs of any service provider or enterprise working in a cloud environment, regardless of whether the installation is using other OpenStack components.

I know what you are thinking, storage is too critical, so it will never work this way. But the same was said >25 years go when using RAID was seen as too risky given solutions would acknowledge writes while the data was in cache prior to being written to disk. The same was also said >15 years ago when VMware was seen as not robust enough to run any manner of demanding or critical application. Replicas and Erasure Codes are analogous to RAID 1 and RAID 5 respectively, and the uniquely as possible distribution of data behind a single namespace abstracts standard hardware like server virtualization.

Interested in hearing more? Come check out my DSI session, “Swift Use Cases with SwiftStack,” where we look forward to sharing how this new type of storage can work, and to suspend your disbelief that this storage can be enterprise-grade.