What the CSI is Up to at SDC

Alex McDonald

What the Cloud Storage Initiative Is Doing At SDC

The SNIA Storage Developer Conference (SDC) is less than a week away. We’re looking forward to the conference and in particular want to make note of some exciting news and events that pertain to work the CSI is doing to promote standards that will increase the adoption, interoperability and portability of data stored in the cloud.

SDC Conference session: Introducing CDMI v1.1 – Tuesday, September 16th, 1:00 p.m. by David Silk. This session introduces the new CDMI 1.1 and provides an overview of capabilities the Technical Work Group have added to the standard, and what CDMI implementers need to know when moving from CDMI 1.0.2 to CDMI 1.1.

Cloud Interoperability Plugfest – Participants at the 12th Cloud Interoperability Plugfest will be testing the interoperability of their cloud storage interfaces based on CDMI. We always have a large showing of CDMI implementations at this event, but are also looking for implementations of Amazon S3, and OpenStack Swift, Cinder and Manila interfaces.

It’s not too late to register for this Plugfest. Find out how here.

SDC 2014 is going to be exciting and educational. It’s “one stop shopping” for IT professionals who focus on the tools, technologies and developments needed for understanding and implementing efficient data storage, management and security. The CSI hopes to see you there.

 

Upcoming Plugfests at SDC

Jason Blosil

This year’s SNIA Storage Developer Conference (SDC) will take place in Santa Clara, CA Sept. 15-18.  In addition to an exciting agenda with great speakers, there is an opportunity for vendors to participate in SNIA Plugfests. Two Plugfests that I think are worth noting are: SMB2/SMB3 and iSCSI.

These Plugfests enable a vendor to bring their implementations of SMB2/SMB3 and/or iSCSI to test, identify, and fix bugs in a collaborative setting with the goal of providing a forum in which companies can develop interoperable products. SNIA provides and supports networks and infrastructure for the Plugfest, creating a collaborative framework for testing. Plugfest participants work together to define the testing process, assuring that objectives are accomplished.

Still Time to Register

Great news! There is still time to register. Setup for the Plugfest begins on September 13, 2014 and testing begins on the September 14th.

Register here for the SMB2/SMB3 Plugfest

Register here for the iSCSI Plugfest

What to Expect at a Plugfest

Learn more about what takes place at the Plugfests by watching the video interview of Jeremy Allison, Co-Creator of Samba, as he candidly talks about what to expect at an SDC Plugfest.

Learn more about the Plugfest registration process. If you have additional questions, please contact Arnold Jones (arnold@snia.org).

 

Expanding Your Data Center with FCoE – Q&A

Jason Blosil

At our recent live ESF Webcast, “Expert Insights: Expanding the Data Center with FCoE,” we examined the current state of FCoE and looked at how this protocol can expand the agility of the data center if you missed it, it’s now available on-demand. We did not have time to address all the questions, so here are answers to them all. If you think of additional questions, please feel free to comment on this blog.

Q. You mentioned using 40 and 100G for inter-switch links.  Are there use cases for end point (FCoE target and initiator) 40 and 100G connectivity?

A. Today most end points are only supporting 10G, but we are starting to see 40G server offerings enter the market, and activity among the storage vendors designing these 40G products into their arrays.

Q. What about interoperability between FCoE switch vendors?

A. Each switch vendor has his own support matrix, and would need to be examined independently.

Q. Is FCoE supported on copper cable?

A. Yes, FCoE supports “Twin Ax” copper and is widely used for server to top of rack switch connections to seven meters.  In fact, Converged Network Adapters are now available that support 10GBASE-T copper cables with the familiar RJ-45 jack.  At least one major switch vendor has qualified FCoE running over 10GBASE-T to 30 meters.

Q. What distance does FCoE support?

A. Distance limits are dependent on the hardware in use and the buffering available for Priority Flow Control. The lengths can vary from 3m up to over 80km. Top of rack switches would fall into the 3m range while larger class switch/directors would support longer lengths.

Q. Can FCoE take part in management/orchestration by OpenStack Neutron?

A. As of this writing there are no OpenStack extensions in Neutron for FCoE-specific plugins.

Q. So how is this FC-BB-6 different than FIP snooping?

A. FIP Snooping is a part of FC-BB-5 (Appendix D), which allows switch devices to identify an FCoE Frame format and create a forwarding ACL to a known FCF. FC-BB-6 creates additional architectural elements for deployments, including a “switch-less” environment (VN2VN), and a distributed switch architecture with a controlling FCF. Each of these cases is independent from the other, and you would choose one instead of the others. You can learn more about VN2VN from our SNIA-ESF Webcast, “How VN2VN Will Help Accelerate Adoption of FCoE.”

Q. You mentioned DCB at the beginning of the presentation. Are there other purposes for DCB? Seems like a lot of change in the network to create a DCB environment for just FCoE. What are some of the other technologies that can take advantage of DCB?

A. First, DCB is becoming very ubiquitous. Unlike the early days of the standard, where only a few switches supported it, today most enterprise switches support DCB protocols. As far as other use cases for DCB, iSCSI benefits from DCB, since it eliminates dropped packets and the TCP/IP protocol’s backoff algorithm when packets are dropped, smoothing out response time for iSCSI traffic. There is a protocol known as RoCE or RDMA over Converged Ethernet. RoCE requires the lossless fabric DCB creates to achieve consistent low latency and high bandwidth.  This is basically the InfiniBand API running over Ethernet. Microsoft’s latest version of file serving protocol, SMB Direct, and the Hyper-V Live Migration can utilize RoCE, and there is an extension to iSCSI known as iSER, which replaces TCP/IP with RDMA for the iSCSI datamover; enabling all iSCSI reads and writes to be done as RDMA operations using RoCE.

Q. Great point about RoCE.  iSCSI RDMA (iSER) is required from DCB if the adapters support RoCE, right?

A. Agreed. Please see the answer above to the DCB question.

Q. Did that Boeing Aerospace diagram still have traditional FC links, and if yes, where?

A. There was no Fibre Channel storage attached in that environment. Having the green line in the ledger was simply to show that Fibre Channel would have it’s own color should there be any links.

Q. What is the price of a 10 Gbp CNA compare to a 10Gbps NIC ?

A. Price is dependent on vendor and economics. But, there are several approaches to delivering the value of FCoE which can influence pricing:

  • Purpose built silicon that offloads the FC and Ethernet protocol functions offer a number of advantages including high performance, low CPU overhead, advanced features, etc., though even this depends on the vendor’s implementation.   But, these added features come with the expectation of additional cost. But, the processing of the protocols has to be done somewhere, and if you need your server CPUs to process applications instead of network protocols, then the value is justified.
  • With the introduction of Open FCoE drivers with DCB supported NICs, new options are available for customers to deploy the value of FCoE at the host. Open FCoE offloads the FC processing onto the host CPU and standard 10GbE NICs with DCB support can be used to manage the Ethernet transport functions. Where you have excess CPU capacity on your server, you might be in a position to reduce costs and deploy a software driver with  a 10GbE or faster NIC enhanced with the limited set of hardware offloads necessary to achieve full performance with Open FCoE. However, Open FCoE isn’t available with every OS or every NIC, so you need to consider OS support and availability.
  • A third consideration is that most enterprise servers include some form of advanced 10GbE networking on the motherboard that either supports purpose built silicon or DCB enabled silicon. So, depending upon which server and OS you deploy, you may have several options via embedded silicon.

 

What’s Happening with 25GbE

David Chalupsky

In July 2014, IEEE 802.3 voted to form a Study Group for 25Gb/s Ethernet.  There has been a lot attention in the networking press lately about 25Gb/s Ethernet, but many people are asking what is it and how did we get here.  After all, 802.3 already has completed standards for 40Gb/s and 100Gb/s and is currently working on 400Gb/s, so from a pure speed perspective, starting a 25Gb/s project now does look like a step backwards.

(Warning: the following discussion contains excessive physical layer jargon.)

The Sweet Spot

25GbE as a port speed is attractive because it makes use of 25Gb/s per lane signaling technology that has been in development for years in the industry, culminating in the recent completion of 802.3bj, the standard for 100GbE over backplane or TwinAx copper that utilizes four parallel lanes of 25Gb/s signaling to achieve the 100Gb/s port speed. Products implementing 25Gb/s signaling in CMOS technology are just starting to come to market, and the rate will likely be a sweet spot for many years, as higher rate signaling of 40Gb/s or 50Gb/s is still in early technology development phases. The ability to implement this high speed I/O in CMOS is important because it allows combining high-speed I/O with many millions of logic gates needed to implement Ethernet switches, controllers, FPGAs, and microprocessors. Thus specifying a MAC rate of 25Gb/s to utilize 25Gb/s serdes technology can enable product developers to optimize for both the lowest cost/bit and the highest overall bandwidth utilization of the switching fabric.

4-Lane to 1-Lane Evolution

To see how we got here and why 25Gb/s is interesting, it is useful to back up a couple of generations and look at 10Gb/s and 40Gb/s Ethernet.  Earliest implementations of 10GbE relied on rather wide parallel electrical interfaces: XGMII and the 16-Bit interface.  Very soon after, however, 4-lane serdes-based interfaces became the norm starting with XAUI (for chip-to-chip and chip-to-optical module use) which was then adapted to longer reaches on TwinAx and backplane (10GBASE-CX4 and 10GBASE-KX4).  Preceding 10GbE achieving higher volumes (~2009) was the specification and technical feasibility of 10Gb/s on a single electrical serial lane. XFI was the first followed by 10GBASE-KR (backplane) and SFI (as an optical module interface and for direct attach TwinAx cable using the SFP+ pluggable form factor).  KR and SFI started to ramp around 2009 and are still the highest volume share of 10GbE ports in datacenter applications. The takeaway, in my opinion, is that single-lane interfaces helped the 10GbE volume ramp by reducing interconnect cost. Now look forward to 40GbE and 10GbE. The initial standard, 802.3ba, was completed in 2010.  So during the time that this specification was being developed, 10Gb/s serial interfaces were gaining traction, and consensus formed around the use of multiple 10Gb/s lanes in parallel to make the 40GbE and 100GbE electrical interfaces. For example, there is a great similarity between 10GBASE-KR, and one lane of the 40GBASE-KR4 four-lane interface. In a similar fashion 10Gb/s SFI for TwinAx  & optics in the SFP+ form factor is similar to a lane of the 40GbE equivalent interfaces for TwinAx and optics in the QSFP+ form factor.

But how does this get to 25Gb/s?

Due to the similarity in technology needed to make 10GbE and 40GbE, it has because a common feature in Ethernet switch and NIC chips to implement a four-lane port for 40GbE that can be configured to use each lane separately yielding four 10GbE ports.

From there it is a natural extension that 100GbE ports being implemented using 802.3bj technology (4x25Gb/s) also can be configured to support four independent ports operating at 25Gb/s.  This is such a natural conclusion that multiple companies are implementing 25GbE even though it is not a standard.

In some environments, the existence of a standard is not a priority.  For example, when a large-scale datacenter of compute, storage and networking is architected, owned and operated by one entity, that entity validates the necessary configuration to meet its requirements. For the broader market, however, there is typically a requirement for multi-vendor interoperability across a diverse set of configurations and uses. This is where Ethernet and IEEE 802.3 has provided value to the industry for over 30 years.

Where’s the Application?

Given the nature of their environment, it is the Cloud datacenter operators that are poised to be the early adopters of 25GbE. Will it also find a home in more traditional enterprise and storage markets? Time will tell, but in many environments ease of use, long shelf life, and multi-vendor interoperability are the priorities. For any environment, having the 25GbE specification maintained IEEE 802.3 will facilitate those needs.

NVM Programming Model Recognized for Enterprise Business Innovation

Marty Foltyn

At the most recent Flash Memory Summit in August 2014, the SNIA NVM Programming Model was selected for a singular honor - industry recognition as a Best of Show for Enterprise Business Applications.  Being recognized was quite unique, as the Model is not a product or a solution as were other winners in the various categories; but rather a body of work that defines new software programming models for non-volatile memory, also known as NVM.

NVM technologies are currently advancing in such a way as to blur the line between storage and memory – which will radically transform the way software is written. The NVM Programming Model embraces this transformation, describing behavior provided by operating systems that enables applications, file systems, and other software to take advantage of new NVM capabilities.

The Model addresses NVM extensions for existing devices such as SSDs and persistent memory, describing the differences between software written for block storage (SSDs and disks) and persistent memory, and outlining the potential benefits for adapting software for persistent memory.

In presenting the Award to Doug Voigt, co-chair of the SNIA NVM Programming Technical Work Group, Jay Kramer, Chairman of the Flash Memory Summit Awards Program and President of Network Storage Advisors Inc., stated,  “Flash memory technology is currently experiencing a progression of innovations that can make a real difference in solving storage solution challenges. The industry is seeing a proliferation of new Non–Volatile Memory (NVM) functionality and new NVM technologies.  We are proud to select the SNIA NVM Programming Model for the Best of Show Award as it brings to the marketplace a new standard with a model that defines recommended behavior between various user spaces and operating system (OS) kernel components supporting NVM.”

Congratulations to the many SNIA member company contributors to the Programming Model for this honor!  For information and to download the specification, visit http://www.snia.org/forums/sssi/nvmp.

flash_mem_summ2014_award

 

Upcoming Webcast: Is FCoE the Answer to Data Center Agility?

Joe Kastura

Fibre Channel over Ethernet (FCoE) has been growing in popularity year after year. From access layer, to multi-hop and beyond, FCoE has established itself as a true solution in the data center.

Interested in learning how the Data Center is expanding with FCoE? Join us on August 20th, at 4:00 pm ET, 1:00 pm PT for our live Webcast, “Expanding the Data Center with FCoE.”  Continuing our conversation from our February Webcast, “Use Cases for iSCSI and FCoE,” which is now available on demand. This live SNIA Webcast examines the current state of FCoE and looks at how this protocol can expand the agility of the data center.

  • We’ll take an unbiased look at the data center using FCoE, covering:
  • The history and evolution of convergence
  • Using FCoE as a storage overlay
  • Single-hop, multi-hop and beyond
  • 40G/100G  – Where does it fit
  • Futures:
    • OpenStack
    • Defining Network Functions Virtualization (NFV)
    • Mapping NFV to FCoE
  • Real-world Use Cases

This will be a vendor-neutral live presentation. Please join us on August 20th and bring your questions for our expert panel. Register now.

 

 

 

Join the Solid State Storage Initiative August 4-7 at Flash Memory Summit 2014

Marty Foltyn

The SNIA and the Solid State Storage Initiative (SSSI) invite SNIA members and non-members alike to attend Flash Memory Summit 2014, August 4-7, 2014 at the Santa Clara Convention Center.

SNIA at Flash Memory Summit offers an all star keynote lineup, including SNIA Member companies Dell, Diablo, Fusion-io, IBM, Intel, Marvell, Micron, NetApp, PMC-Sierra, Samsung, and SanDisk.  SSSI members will lead panels and sessions on SSD, NVDIMM, and NVM Programming.

A SNIA Education Day on Monday, August 4 in Room 203/204 of the Santa Clara Convention Center will feature award-winning SNIA Tutorials on Flash and Storage where attendees can learn about secure storage, SSD workload testing, benefits of Flash storage to the enterprise, PCI Express, and Flash storage architectures from SNIA member experts.  This Education Day is complimentary to all FMS attendees.

Following the Education Day, all are welcome to attend a Solid State Storage Reception Monday evening from 5:30 pm – 7:00 pm in Room 203/204 featuring updates on the solid state disk market, an NVDIMM presentation, and an NVM Programming Model overview.  Visit displays that highlight SNIA Solid State Initiative programs, including Non Volatile Memory Programming, Performance Testing,  and Workload I/O Capture.  Learn how you can participate in the exciting 2014 programs of the SSSI.

A Non-Volatile DIMMs:  When Flash Isn’t Fast Enough Hands-On Lab presented by the NVDIMM SIG and SIG member companies AgigaTech, Netlist, and SMART Modular will illustrate how a category of NVDIMMs function in server and storage systems and how they can be integrated into a standard server platform.

And don’t forget to stop by SNIA SSSI Booth 808 in the Exhibit Hall to check out  five static and two live NVDIMM displays and new whitepapers, brochures, and news on SSDs.

Register now at www.flashmemorysummit.com

Use the code “SNIA” to sign up today and receive $100 off Full Conference, 3-Day Conference, and One-Day Technical Program registration. 

What a Solid State We Store In

Marty Foltyn

Note:  This blog entry is authored by SNIA Solid State Storage Initiative Governing Board member Gilda Foss, who serves on the SNIA Solid State Storage Initiative [SSSI] Governing Board as well as her role as Industry Evangelist in the CTO Office at NetApp, Inc. 

Solid state drives use semiconductor chips, as opposed to magnetic media, to store data.  The chips that solid state drives use are non-volatile memory meaning that the data remains even when the system has no power.  I’ve written about solid state drive technology in the past and I will continue to, for it represents the first major advancement in primary storage in a very long time.  Serving on the Governing Board of the SNIA Solid State Storage Initiative, it allows me to help foster the growth and success of the market for solid state storage in both enterprise and client environments. Our goals are to be the recognized authority for storage made from solid state devices, to determine and document the characteristics of storage made from solid state devices, and to determine and document the impact of storage made from solid state devices on system architectures.

So what can you expect if you were to ever upgrade to an SSD?  Well, for starters your computing experience will be transformed with screaming fast random access speeds, multi-tasking proficiency, as well as fantastic reliability and durability… and you can choose between an external SSD or even a hybrid drive so you’ve got some options.  A new SSD will make your system faster because the boot times will decrease, launching apps will be lightening fast, opening and saving docs will no longer drag, copying and duplicating file speeds will improve, and overall your system will have a new ‘pep in its step’.  Furthermore, to promote being green, SSDs consume far less power than traditional hard drives, which means they also preserve battery life and stay cooler.  Who doesn’t want and need that? They’re also very quiet, with none of the spinning and clanking you get with HDDs – for obvious reasons. SSDs are cooler and quieter, all the while being faster.

Since modern SSDs are Flash-based, there is no real hard-defined difference between Flash and SSD.  Rather, as mentioned previously, Solid State Disk is essentially storage that doesn’t require moving parts and Flash is what allows that to exist.  SSDs use Flash instead of RAM these days, since it’s a type of memory that’s super fast and doesn’t require continuous power, making it non-volatile.  A match made in solid-state heaven.

There are some fundamental aspects that folks expect from a robust flash-based storage solution.  First off, I/O performance and efficiency for many applications, including database acceleration, server and desktop virtualization, and cloud infrastructure.  You should also expect to speed up overall IT performance, boost responsiveness of performance-critical applications, and reduce power costs and over-provisioning.  Furthermore, you will obviously use more high-capacity, low-cost SATA drives while improving utilization of your data center space.  If you can achieve all your flash-based goals without changing your IT infrastructure management processes, then you’ve really got it good.

Flash storage has customarily had substantial aging issues. In a nutshell, a user could only write to the memory a certain number of times before they would just lose that section of the drive coupled with the fact that performance would degrade over time, too.  However, a lot of these issues were resolved and companies started manufacturing SSDs out of Flash memory instead of out of RAM.

I’ve stated in the past that many people in the industry believe that flash SSDs will eventually replace traditional hard drives.  By the time this happens other characteristics, such as slower write time and added cost, will likely have been eradicated or significantly diminished. Even today, an SSD can extend the life of a laptop battery, reduce the weight of the system, make it quieter, and increase read performance.  When properly and optimally engineered, SSDs are now at least as reliable as traditional spinning hard drives.  Relating to the faster speed, think of one starting up in seconds versus minutes. Even the slowest current SSD gives you much improved real-world performance than does the fastest conventional hard drive, perhaps even 100x as fast.  This allows for better user productivity, allowing for more work to get done in a fraction of the time.  Furthermore, using flash in enterprise storage servers means you can support more users, do more work, and use less power so it’s no wonder that it’s become an important technology for business transactions.   It’s a solid win-win-win.

SSSI’s 2014 Mission

This SNIA initiative was formed in September 2008 and its mission is to foster the growth and success of the market for solid state storage in both enterprise and client environments. Our goals are to be the recognized authority for storage made from solid state devices, to determine and document the characteristics of storage made from solid state devices, and to determine and document the impact of storage made from solid state devices on system architectures.  Additionally, the SSSI collects solid state technical requirements of storage system vendors and communicate to SSD manufacturers for common features, behavior, and robustness.  The initiative collaborates with academia and the research labs of member companies to understand how advances in solid state memory will impact storage made from solid state memory as well as to educate the vendor and user communities about storage made from solid state devices.

The SNIA SSSI also coordinates education activities with the Education Committee, performs benchmark testing to highlight the performance advantages of solid state storage, create peer reviewed vendor neutral SNIA Tutorials, and create vendor-neutral demonstrations.  The SSI also leverages SNIA and partner conferences, collaborate with industry analysts, perform market outreach that highlights the virtues of storage made from solid state devices.  The initiative determines what technical work should be performed within SNIA technical working groups to further the acceptance of storage made from solid state devices.  Furthermore and very importantly, the SSSI determines the standards that will be necessary to support the industry usage of SSDs by performing interoperability plug-fests as necessary in support of standards development.

Collaboration between other SNIA organizations is also key.  The SSSI works with the Storage Management Initiative (SMI) to understand how SMI-S can be used to manage storage made from solid state devices.  We also work with the Green Storage Initiative (GSI) to understand how storage made from solid state devices will impact energy use in computer systems.  The work that the SSI does with the Technical Council helps create the desired technical working groups and provides external advocacy and support of these technical working groups.

Finally, the SSSI collaborates with other industry associations via SNIA’s Strategic Alliances Committee (SAC) on SSD-related technical work in which they are involved as well as coordinates with SNIA Regional Affiliates to ensure that the impact of the SSS Initiative is felt worldwide.  For more information, please visit http://www.snia.org/forums/sssi

Object Storage 101 – Questions and Answers

AlexMcDonald

At our recent live ESF Webcast, “Object Storage 101,” we talked about the what, how, and why behind storage technologies. Over 200 people attended the event. If you missed it, it’s now available on-demand. It was an interactive session and we did not have time to address all the questions, so here are answers to them all. If you think of additional questions, please feel free to comment on this blog.

Q. Would Object Storage be a feasible solution for only the nearline storage tier?

Typically Yes. If we think about the latency needed for real-time transactions, these are best served using a cache storage tier such as NAND or large arrays of RAM. Object stores are excellent methods to store and retrieve large data sets within single/multiple containers. Note: most systems support offset reads so you don’t need to access an entire object to get to the section of interest.

Q. Where is the index to find the location of an object that is stored? Is it stored locally or stored distributedly or replicated among each clusters?

Storage of the Index or Metadata of objects that are stored, if used, typically is replicated throughout the system. Also, if the Metadata is lost, typically, these can be re-built as a maintenance function.

Q. How is the object stored/broken up? Aside from being stored by metadata (like name, size, etc) … what is the process of the fragmentation…breaking it up …as described during this erasure coding segment?  Once it’s assigned some unique identifier … ie. an x-ray picture…. how is it addressed? (if not by block/bit/byte/level)?

Currently, Objects are stored using one of two methods of data protection either Replication or Erasure Coding. Some systems use both. That said, there are several algorithms used today to Erasure Code protect Objects. When using Reed-Solomon methods, you need to specify the number of “Data” Fragments and the number of the “Parity” fragments that will be created. The Size of each “Data” fragment is closely related to the Object size divided by the number of “Data” fragments requested. Each “Parity” fragment will be same size of each of the “Data” fragments created. The protected Object size is the sum of the “Data” fragments plus the “Parity” fragments created. Each of these fragments (Data and Parity) is stored on a different server for the purpose of avoiding a single point failure. The application that created the Object that will be accessing the Object store is responsible for keeping track of the ID of the Object and the Namespace the ID was stored in. Typically the Application will create an ID however, when an Application “Puts” an Object using an existing ID, the older stored Object using that same ID is overwritten. Typically, access into an Object Store using a RESTful Interface using commands like “Put, Get, Delete, List” over HTTP.

Q. Will Object storage drive network scale—further adoption of 10GE and 40GE or is 1GE enough?

Yes. If we think about the interconnection between the Control Plane and Data Plane of these systems (Orchestration and Object Storage Devices), better the connectivity the higher the performance.

Q. Is the number of fragments set or configurable?  What are the trade-offs of requiring fewer fragments for recovery besides perhaps processing overhead?  Are there any gotchas to watch out for/consider?

Yes. Storage policies are configurable. The number of “Parity” fragments defines the data loss risk. The more “Parity” fragments requested the lower this risk but this increases the storage resource needed for the Object. Eliminating single point failures is a key consideration. For example, if your Object Storage system has 10 servers, a storage policy using 9 of 12 will have 2 fragments of this Object located on 2 servers. In this case any single server failure would not cause data loss but may cause higher latency. However, if 3 servers would fail, you would lose access to your data until the servers were recovered. If the drives of the failed servers were not recovered then data loss would occur.

Q. Is erasure encoding used instead of Hash tagging?

No. Hash Tagging is a method of generating a unique number given a specific input of data, this number is used to find the location of the Object to be stored. Erasure Coding is the method used to create the fragments. So think of Hash tag as the seed to the address needed to find the fragments.

Q. How large are the fragments?

A rough estimate is the Object size divided by the number of fragments to re-hydrate the object. (e.g. 1GByte Object stored using a 8 of 12 policy would have a fragment size of 1GByte/8 =~ 125MByte

Q. What do you see as the requirement for the interconnect between the Object storage arrays/boxes to be? Very large pipes as in multiple 40G links or something lower?

It depends on the use case or Service Level Objective for the system. If your system design uses a Proxy service and Erasure Coding, then your back end network throughput (the network connecting the Proxy and Object Storage Devices – Storage Servers) will aggregated (Multiply). In this case the network throughput is based on the number of “Data” fragments being used. If you use Replication, then the back end network throughput will not aggregate. This multiplication factor, if present, is key to an efficient network strategy. In Non-Proxy based Object Storage designs or replication based Object Storage systems the network strategy will scale with network bandwidth to the limitation of the HDDs ability to server data.

Q. What about access control and security at the object level?  Is that typically part of the model?

Typically, access control methods are at the gateway or entry point of a Namespace. The access method used is up to the vendor of the Object Store.

Q. What is the presentation mode at the host level? i.e. a drive mapping or similar

Typically presentation methods are a RESTful API via HTTP. This used “PUT, GET, DELETE, LIST” semantics.

Q. Can you explain the differences/similarities between object storage, CDMI and software defined storage?

Object Stooge defined a system (Software + Hardware) to storage Objects. CDMI defends a method used to access/connect your application to an Object Storage system. Software Defined Storage describes using standard high volume servers with software for the purpose of storing data.

Q. Why can’t a traditional approach be used to Object Storage for its durability?

Traditional storage approaches such as direct attached storage (RAID Sets) do not scale. Once you run out of space, managing additional storage on separate systems becomes the issue.

Q. Aren’t all types of data going to need the accessibility required by users? For example, isn’t everything going to need to placed in an object store?

There is a lot of debate on this issue. The goal of an Object Store is two fold. 1) Drive down the cost/Byte and 2) keep content readily accessible.

Q. How to we avoid losing the Metadata from the data? Also, is there something like sub-meta data, where a small amount of Metadata is contained within the data and the larger Metadata is stored somewhere else?

Some Object storage systems support Extended File Attributes, which is a file system feature that allows the Applications to store “Metadata” about an Object which is then bound to the Object within the storage environment. These Extended File Attributes (XATTR’s) can be queried separately and can be used by your application as you see fit. The management of the XATTR’s is handled by the local file system and accessed by the Object Storage software via the RESTful API using HTTP.

Q. Is maintaining multiple copies mainly for durability or can it be used for performance enhancement (parallel access), or is that irrelevant?

Absolutely!  Management of copies/replicas can serve multiple purposes.  Replication across racks, datacenters, geographies, etc. can provide resiliency against failures at those levels.  Replication can also be used to provide object access in close proximity to the requester.  In the X-ray example discussed in the Webcast, we might set up a replica local to the medical practice for the first 90 days, in order to provide a low latency (time to first byte) copy during the initial treatment.  Additional copies can be kept at remote sites in order to provide fault tolerance.

Q. Is there a standard methodology for migrating from a file-system based methodology to an object store?

The short answer is no.  In general an application that is currently developed to use file or block based storage will need to be re-architected in order to take advantage of an object storage system/service.  There is, however, a growing category of products referred to as “cloud gateways” that can provide a bridge to object storage by presenting a filesystem to the existing application, while writing and reading via a RESTful API to a backend object storage system/service.

Q. Is it safe to say that in order to use object storage the application needs to be “object storage aware”? Unlike a traditional storage where the application doesn’t necessarily need to be familiar with the storage or file system since that is handled at a lower layer.

Yes, however as indicated in the question regarding migration of applications above, it is possible to implement a “cloud gateway” solution that will provide the translation from RESTful API to a CIFS/NFS fileshare, thus not requiring any application changes.  I would disagree with the premise that traditional applications don’t need to be familiar with the underlying storage.  Traditional file-based applications must understand the location (fileserver, folder, filename, etc.) in order to gain access to the appropriate data.

Q. I’m hearing a lot of ‘what’ and ‘how’ but not so much ‘why’ about object storage. Can we hear some real-world examples of applications in industry today that are running better because of object storage?

An example of an application running today with object storage behind it, and why:  Web Based Media Asset Management/Distribution.  This particular use case tends to deal with billions of files/objects that can vary in size from very small thumbnail images to massive 4k HD movie files.  The ability to deliver these to multiple platforms (phone, laptop, set top box, etc.) across multiple geographies is something that is well suited for object storage.  Traditional file and/or block based storage environments may hit scale limitations in dealing with the number of files/objects, in addition the ability to have a single namespace maintained across multiple locations/datacenters is something that is exceedingly complex for storage environments other than object stores.

Q. Replicating an object two or three times would exponentially increase storage costs, wouldn’t it?  The more copies the higher the costs?

Certainly more copies would use more storage, and as a result most object stores provide different durability schemes based upon the performance/availability tradeoffs the data owner is willing to make.  Recovering a single object from a replica is significantly faster than rebuilding an object from geo-distributed EC fragments. Also, as discussed in the question above related to replicas to drive performance, replication can serve the purpose of placing objects as close to the consumer as possible, minimizing time to first bye and increasing the overall throughput of an application.

Q. If I have an app that access a CIFS share, is there a way to translate it into object store?

Please see answer to question: “Is there a standard methodology for migrating from a file-system based methodology to an object store?” Short answer: Yes, via a “cloud gateway” product.

Q. Is there a confluence point of Object and File based storage – specifically in NAS where object storage can be multi-protocol (NFS, and REST)?

While there are some object storage solutions that provide their own native cloud-gateway capability (NAS protocol to the application, RESTful API to the object store).  There are very few that provide a “file/object duality” capability allowing applications to manipulate an object as both an object and a file.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

M.2 Webcast – Get the Latest Info on the New SSD Card Form Factor!

Marty Foltyn

The SNIA Solid State Storage Initiative is partnering with SATA-IO and NVM Express to present a panel of experts from Objective Analysis, Micron, TE Connectivity, Intel, Calypso, and Coughlin Associates to give you the latest information on M.2, the new SSD card form factor. You will leave this webinar with an understanding of the M.2 market, M.2 cards and connection schemes, NVM Express, and M.2 performance; you’ll also be able to ask questions of the experts.

Join us on June 10, 2014 at 10:00 am.  Register at http://snia.org/news_events/multimedia#webcasts!