Category: Storage

The Changing World of SNIA Technical Work – A Conversation with Technical Council Chair Mark Carlson

khauser Leave a comment

carlson_mark_resizeMark Carlson is the current Chair of the SNIA Technical Council (TC). Mark has been a SNIA member and volunteer for over 18 years, and also wears many other SNIA hats.   Recently, SNIA on Storage sat down with Mark to discuss his first nine months as the TC Chair and his views on the industry.

SNIA on Storage (SoS):  Within SNIA, what is the most important activity of the SNIA Technical Council?

Mark Carlson (MC): The SNIA Technical Council works to coordinate and approve the technical work going on within SNIA. This includes both SNIA Architecture (standards) and SNIA Software. The  work is conducted within 13 SNIA Technical Work Groups (TWGs).  The members of the TC are elected from the voting companies of SNIA, and the Council also includes appointed members and advisors as well as SNIA regional affiliate advisors. SNIA_Technology_Infographic_4

SoS:  What has been your focus this first nine months of 2016?   

MC: The SNIA Technical Council has overseen a major effort to integrate a new standard organization into SNIA.  The creation of the new SNIA SFF Technology Affiliate (TA) Technical Work Group has brought in a very successful group of folks and standards related to storage connectors and transceivers. This work group, formed in June 2016, carries forth the longstanding SFF Committee work efforts that has operated since 1990 until mid-2016.  In 2016, SFF Committee leaders transitioned the organizational stewardship to SNIA, to operate under a special membership class named Technology Affiliate, while retaining the long standing technical focus on specifications in a similar fashion as all SNIA TWGs do.

SoS:  What changes did SNIA implement to form the new Technology Affiliate membership class and why?

MC: The SNIA Policy and Procedures were changed to account for this new type of membership.  Companies can now join an Affiliate TWG without having to join SNIA as a US member.  Current SNIA members who want to participate in a Technology Affiliate like SFF can join a Technology Affiliate and pay the separate dues.  The SFF was a catalyst – we saw an organization looking for a new home as its membership evolved and its leadership transitioned.  They felt SNIA could be this home but we needed to complete some activities to make it easier for them to seamlessly continue their work.   The SFF is now fully active within SNIA and also working closely with T10 and T11, groups that SNIA members have long participated in.

SoS:  Is forming this Technology Affiliate a one-time activity?

MC: Definitely not.  The SNIA is actively seeking organizations who are looking for a structure that SNIA provides with IP policies, established infrastructure to conduct their work, and 160+ leading companies with volunteers who know storage and networking technology.

SoC:  What are some of the customer pain points you see in the industry?

MC: Critical pain points the TC has started to address with new TWGs over the last 24 months include: performance of solid state storage arrays, where the SNIA Solid State Storage Systems (S4) TWG is working to identify, develop, and coordinate system performance standards for solid state storage systems; and object drives, where work is being done by the Object Drive TWG to identify, develop, and coordinate standards for object drives operating as storage nodes in scale out storage solutions.  With the number of different future disk drive interfaces emerging that add value from external storage to in-storage compute, we want to make sure they can be managed at scale and are interoperable.TC org chart 2016

SoS:  What’s upcoming for the next six months?

MC: The TC is currently working on a white paper to address data center drive requirements and the features and existing interface standards that satisfy some of those requirements.  Of course, not all the solutions to these requirements will come from SNIA, but we think SNIA is in a unique position to bring in the data center customers that need these new features and work with the drive vendors to prototype solutions that then make their way into other standards efforts.  Features that are targeted at the NVM Express, T10, and T13 committees would be coordinated with these customers.

SoS:  Can non-members get involved with SNIA?

MC:   Until very recently, if a company wanted to contribute to a software project within SNIA, they had to become a member. This was limiting to the community, and cut off contributions from those who were using the code, so SNIA has developed a convenient Contributor License Agreement (CLA) for contributions to individual projects.  This allows external contributions but does not change the software licensing. The CLA is compatible with the IP protections that the SNIA IP Policy provides to our members.  Our hope is that this will create a broader community of contributors to a more open SNIA, and facilitate open source project development even more.

SoS:  Will you be onsite for the upcoming SNIA Storage Developer Conference (SDC)?

MC: Absolutely!  I look forward to meeting SNIA members and colleagues September 19-22 at the Hyatt Regency Santa Clara.  We have a great agenda, now online, that the TC has developed for this, our 18th conference, and registration is now open.  SDC brings in more than 400 of the leading storage software and hardware developers, storage product and solution architects, product managers, storage product quality assurance engineers, product line CTOs, storage product customer support engineers, and in–house IT development staff from around the world.  If technical professionals are not familiar with the education and knowledge that SDC can provide, a great way to get a taste is to check out the SDC Podcasts now posted, and the new ones that will appear leading up to SDC 2016.

Linear Tape File System Now an International Standard

khauser Leave a comment

By David Pease, Co-Chair SNIA Linear Tape File System Technical Working Group

In 2011 the Linear Tape File System (LTFS) earned IBM an Engineering Emmy Award after being recognized by FOX Networks for “improving the ability of media companies to capture, manage and exploit content in digital form, fundamentally changing the way that audio and video content is managed and stored.”  Now, the International Standardization Organization (ISO) has named LTFS an International Standard (ISO/IEC 20919:2016).

LTFS’s road to standardization was a long one.  It started with IBM and the LTO (Linear Tape Open) Consortium jointly publishing the LTFS Format Specification as an open format in April, 2010, the day that LTFS was announced at the NAB (National Association of Broadcasters) show in Las Vegas.  In 2012, at the invitation of the Storage Networking Industry Association (SNIA), we formed the SNIA LTFS Technical Work Group, with a specific goal of moving towards international standardization.  The LTFS TWG and SNIA proceeded to publish several revisions of the LTFS Format Specification, inviting all interested parties to join the work group and contribute, or to comment on the specification before formal publication.  In 2014 SNIA helped the LTFS TWG format the then-current version of the specification (V2.2) to ISO standards and worked with the ISO organization to publish the specification as a draft standard and solicit comments.  After review and comments, the LTFS Format Specification was approved by ISO as an international standard in April of 2016 (just 6 years after it was first announced).

We are thrilled by the recognition of LTFS as an ISO standard; it is one more step towards guaranteeing that the LTFS format is a truly open standard that will continue to be available and usable for the foreseeable future.  In my opinion, two of the major inhibitors to the widespread use of tape technology for data storage have been the lack of a standard format for data storage and interchange on tape, and its perceived difficulty of use. LTFS addresses both of these problems by providing a general-purpose, open format that can easily be used like any other storage medium.

As the world’s data continues to grow at an increasing pace, and the need for affordable, large-scale storage becomes more important, the standardization of LTFS will make the use of tape for long-term, affordable storage easier and more attractive.

Use Case: Making Digital Media Storage Open and Future-Proof

Just as in personal photography, the last couple of decades have seen a major shift from analog and film technologies to digital ones in the Media and Entertainment industry, where modern cameras record directly to digital media. This has led to the need for new technologies to replace traditional film as a long-term storage medium for television and movies.

Film has some specific advantages for the Media & Entertainment industry that a new technology needs to replicate, including long shelf life, inexpensive, and zero-power storage, and a format that is “future-proof.” Tape storage is a perfect match to several of these criteria, including long (30+ years) shelf life, and zero-power, inexpensive storage. However, a stumbling block for the wide-spread acceptance of tape for digital storage in the media and entertainment business had been the lack of an open, easy-to-use, future-proof standard for the format of the data on tape. You can imagine an entertainment company using proprietary storage software, for example, only to run into problems like the provider going out of business or increasing its software costs to an unacceptable level.

We created LTFS to be an open and future-proof format from the beginning: open, because when we published the format, we made it publicly available at no charge, and future-proof because the format is self-documenting and can be easily accessed without the need for proprietary software.

Being an international standard should make anyone who is considering the use of LTFS even more comfortable with the fact that it is an open standard that is not owned or controlled by any single company, and is a format that will continue to be supported in the future.  As such, becoming an international standard has the potential to increase the use, and therefore the value, of LTFS across industries.

For more information about the work of the SNIA LTFS TWG, please visit www.snia.org/ltfs.

Podcasts Bring the Sounds of SNIA’s Storage Developer Conference to Your Car, Boat, Train, or Plane!

khauser Leave a comment

SNIA’s Storage Developer Conference (SDC) offers exactly what a developer of cloud, solid state, security, analytics, or big data applications is looking  for – rich technical content delivered in a no-vendor bias manner by today’s leading technologists.  The 2016 SDC agenda is being compiled, but now yousdc podcast pic can get a “sound bite” of what to expect by downloading  SDC podcasts via iTunes, or visiting the SDC Podcast site at http://www.snia.org/podcasts to download the accompanying slides and/or listen to the MP3 version.

Each podcast has been selected by the SNIA Technical Council from the 2015 SDC event, and include topics like:

  • Preparing Applications for Persistent Memory from Hewlett Packard Enterprise
  • Managing the Next Generation Memory Subsystem from Intel Corporation
  • NVDIMM Cookbook – a Soup to Nuts Primer on Using NVDIMMs to Improve Your Storage Performance from AgigA Tech and Smart Modular Systems
  • Standardizing Storage Intelligence and the Performance and Endurance Enhancements It Provides from Samsung Corporation
  • Object Drives, a New Architectural Partitioning from Toshiba Corporation
  • Shingled Magnetic Recording- the Next Generation of Storage Technology from HGST, a Western Digital Company
  • SMB 3.1.1 Update from Microsoft

Eight podcasts are now available, with new ones added each week all the way up to SDC 2016 which begins September 19 at the Hyatt Regency Santa Clara.  Keep checking the SDC Podcast website, and remember that registration is now open for the 2016 event at http://www.snia.org/events/storage-developer/registration.  The SDC conference agenda will be up soon at the home page of http://www.storagedeveloper.org.

Enjoy these great technical sessions, no matter where you may be!

Open Source Software-Only Storage – Really.

Mario Blandini

Virtually any storage solution is more parts software than hardware. Having said this, users don’t care as much about the percentage of hardware vs. software. They want their consumption experience to be easy and fast to start up, with a pay-as-you-grow model and with the ability to scale without limits. So, it should not be a shock that real IT organizations are using software-only on standard servers to deliver storage to their customers. What’s more, this type of storage can be powered by open source.

At the upcoming SNIA Data Storage Innovation Conference, we are looking forward to discussing software-defined storage (SDS) from a user experience perspective with examples of OpenStack Swift providing an engine for building SDS clusters with any mixed combination of standard server and HDD hardware in a way that is simple enough for any enterprise to dynamically scale.

Swift is a highly available, distributed, scalable object store available as open source.  It is designed to handle non-relational (that is, not just simple row-column data) or unstructured data at large scale with high availability and durability.  For example, it can be used to store files, videos, documents, analytics results, Web content, drawings, voice recordings, images, maps, musical scores, pictures, or multimedia. Organizations can use Swift to store large amounts of data efficiently, safely, and cheaply. It scales horizontally without any single point of failure.  It offers a single multi-tenant storage system for all applications, the ability to use low-cost industry-standard servers and drives, and a rich ecosystem of tools and libraries.  It can serve the needs of any service provider or enterprise working in a cloud environment, regardless of whether the installation is using other OpenStack components.

I know what you are thinking, storage is too critical, so it will never work this way. But the same was said >25 years go when using RAID was seen as too risky given solutions would acknowledge writes while the data was in cache prior to being written to disk. The same was also said >15 years ago when VMware was seen as not robust enough to run any manner of demanding or critical application. Replicas and Erasure Codes are analogous to RAID 1 and RAID 5 respectively, and the uniquely as possible distribution of data behind a single namespace abstracts standard hardware like server virtualization.

Interested in hearing more? Come check out my DSI session, “Swift Use Cases with SwiftStack,” where we look forward to sharing how this new type of storage can work, and to suspend your disbelief that this storage can be enterprise-grade.

 

Hitachi Data Systems’ Hu Yoshida Featured in Keynote at SNIA Annual Members Symposium

khauser Leave a comment

The Storage Networking Industry Association (SNIA) Annual Members Symposium is a must attend event for both SNIA members and interested colleagues. Held at the Westin San Jose in San Jose California from January 19-22, 2016, the Symposium offers four full days of collaboration, networking, and knowledge about the latest advances in cloud storage, Ethernet storage, solid state storage, storage management, green computing, and more. Register and learn more at http://www.snia.org/events/symp.

A highlight of every SNIA Symposium are keynotes from leaders in the industry, and 2016 is no exception. Whether you are a SNIA member, or thinking about hubert-yoshidaSNIA membership, you are invited to attend a very special keynote presented on January 19 at 8:45 am on IT Trends that Matter in 2016, presented by Mr. Hubert Yoshida, CTO of Hitachi Data Systems. Mr. Yoshida will comment on how innovative technology companies that quickly capitalize on business opportunities and satisfy the demands of today’s empowered consumer have caused a wave of disruption. In 2016, businesses will turn to IT for solutions that will keep them competitive. Chief Information Officers will invest in faster delivery of applications and analytics, and transform IT by leveraging the third platform, social, mobile, analytics, and cloud. Mr. Yoshida will advise on how to avoid distractions and remain focused on the IT trends that matter in 2016.

Hubert Yoshida is responsible for defining the technical direction of Hitachi Data Systems. Currently, he leads the company’s effort to help customers address data life cycle requirements and resolve compliance, governance and operational risk issues. He was instrumental in evangelizing the unique Hitachi approach to storage virtualization, which leveraged existing storage services within Hitachi Universal Storage Platform® and extended it to externally-attached, heterogeneous storage systems.

Mr. Yoshida is well-known within the storage industry, and his blog has ranked among the “top 10 most influential” within the storage industry as evaluated by Network World. In October of 2006, Byte and Switch named him one of Storage Networking’s Heaviest Hitters and in 2013 he was named one of the “Ten Most Impactful Tech Leaders” by Information Week.

Hitachi Data Systems is a valued member of SNIA.  Currently,  Thomas Rivera, Technical Associate – File, Content & Cloud Solutions at Hitachi Data Systems, serves as the Secretary of the SNIA Board of Directors.

For the full agenda of activities and events at the SNIA Symposium visit http://www.snia.org/events/symp.

New Webcast: The Life of a Storage Packet (Walk)

J Metz

Wonder how storage really works? When we talk about “Storage” in the context of data centers, it can mean different things to different people. Someone who is developing applications will have a very different perspective than someone who is responsible for managing that data on some form of media. Moreover, someone who is responsible for transporting data from one place to another has their own view that is related to, and yet different from, the previous two.

Add in virtualization and layers of abstraction, from file systems to storage protocols, and things can get very confusing very quickly. Pretty soon people don’t even know the right questions to ask! That’s why we’re hosting our next SNIA Ethernet Storage Webcast, “Life of a Storage Packet (Walk).”

Join us on November 19th to learn how applications and workloads get information. Find out what happens when you need more of it, or faster access to it, or move it far away. This Webcast will take a step back and look at “storage” with a “big picture” perspective, looking at the whole piece and attempting to fill in some of the blanks for you. We’ll be talking about:

  • Applications and RAM
  • Servers and Disks
  • Networks and Storage Types
  • Storage and Distances
  • Tools of the Trade/Offs

The goal of the Webcast is not to make specific recommendations, but equip you with information that will help you ask the relevant questions, as well as get a keener insight to the consequences of storage choices. As always, this event is live, so please bring your questions, we’ll answer as many as we can on the spot. I encourage you to register today. Hope to see you on November 19th!

A Deep Dive into the SNIA Storage Developer Conference – The File Systems Track

khauser Leave a comment

SNIA Storage Developer Conference (SDC) 2015 is two weeks away, and the SNIA Technical Council is finalizing a strong, comprehensive agenda of speakers and sessions. Wherever your interests lie, you’re sure to find experts and topics that will expand your knowledge and fuel your professional development!SDC15_WebHeader3_999x188

For the next two weeks, SNIA on Storage will highlight exciting interest areas in the 2015 agenda. If you have not registered, you need to! Visit www.storagedeveloper.org to see the four day overview and sign up.

Year after year, the File Systems Track at SDC provides in depth information on the latest technologies, and 2015 is no exception. The track kicks off with Vinod Eswaraprasad, Software Architect, Wipro, on Creating Plugin Modules for OpenStack Manila Services. He’ll discuss his work on integrating a multi-protocol NAS storage device to the OpenStack Manila service, looking at the architecture principle behind the scalability and modularity of Manila services, and the analysis of interface extensions required to integrate a typical NAS head.

Ankit Agrawal and Sachin Goswami of TCS will discuss How to Enable a Reliable and Economic Cloud Storage Solution by Integrating SSD with LTFS. They will share views on how to integrate SSD as a cache with a LTFS tape system to transparently deliver the best benefits for Object Base storage, and talk about the potential challenges in their approach and best practices that can be adopted to overcome these challenges.

Jakob Homan, Distributed Systems Engineer, Microsoft, will present Apache HDFS: Latest Developments and Trends. He’ll discuss the new features of HDFS, which has rapidly developed to meet the needs of enterprise and cloud customers, and take a look at HDFS at implementations and how they address previous shortcomings of HDFS.

James Cain, Principal Software Architect, Quantel Limited, will discuss a Pausable File System, using his own implementation of an SMB3 server (running in user mode on Windows) to demonstrate the effects of marking messages as asynchronously handled and then delaying responses in order to build up a complete understanding of the semantics offered by a pausable file system.

Ulrich Fuchs, Service Manager, CERN, will talk about Storage Solutions for Tomorrow’s Physics Projects, suggesting possible architectures for tomorrow’s storage implementations in this field, and showing results of first performance tests done on various solutions (Lustre, NFS, Block Object storage, GPFS ..) for typical application access patterns.

Neal Christiansen, Principal Development Lead, Microsoft, will present Support for New Storage Technologies by the Windows Operating System, describing the changes being made to the Windows OS, its file systems, and storage stack in response to new evolving storage technologies.

Richard Morris and Peter Cudhea of Oracle will discuss ZFS Async Replication Enhancements, exploring design decisions around enhancing the ZFS send and ZFS receive commands to transfer already compressed data more efficiently and to recover from failures without re-sending data that has already been received.

J.R. Tipton, Development Lead, Microsoft, will discuss ReFS v2: Cloning, Projecting, and Moving Data File Systems, presenting new abstractions that open up greater control for applications and virtualization, covering block projection and cloning as well as in-line data tiering.

Poornima Gurusiddaiah and Soumya Koduri of Red Hat will present Achieving Coherent and Aggressive Client Caching in Gluster, a Distributed System, discussing how to implement file system notifications and leases in a distributed system and how these can be leveraged to implement a client side coherent and aggressive caching.

Sriram Rao, Partner Scientist Manager, Microsoft, will present Petabyte-scale Distributed File Systems in Open Source Land: KFS Evolution, providing an overview of OSS systems (such as HDFS and KFS) in this space, and describing how these systems have evolved to take advantage of increasing network bandwidth in data center settings to improve application performance as well as storage efficiency.

Richard Levy, CEO and President, Peer Fusion, will discuss a High Resiliency Parallel NAS Cluster, including resiliency design considerations for large clusters, the efficient use of multicast for scalability, why large clusters must administer themselves, fault injection when failures are the nominal conditions, and the next step of 64K peers.

Join your peers as well – register now at www.storagedeveloper.org. And stay tuned for tomorrow’s blog on Cloud topics at SDC!

SNIA’s Events Strategy Today and Tomorrow

khauser Leave a comment

David Dale, SNIA Chairman

Last month Computerworld/IDG and the SNIA posted a notice to the SNW website stating that they have decided to conclude the production of SNW.  The contract was expiring and both parties declined to renew.  The IT industry has changed significantly in the 15 years since SNW was first launched, and both parties felt that their individual interests would be best served by pursuing separate events strategies.

For the SNIA, events are a strategically important vehicle for fulfilling its mission of developing standards, maintaining an active ecosystem of storage industry experts, and providing vendor-neutral educational materials to enable IT professions to deal with and derive value from constant technology change.  To address the first two categories, SNIA has a strong track record of producing Technical Symposia throughout the year, and the successful Storage Developer Conference in September.

To address the third category, IT professionals, SNIA has announced a new event, to be held in Santa Clara, CA, from April 22-24 – the Data Storage Innovation Conference. This event is targeted at IT decision-makers, technology implementers, and those expected to influence, implement and support data storage innovation as actual production solutions.  See the press release and call for presentations for more information.  We are excited to embark on developing this contemporary new event into an industry mainstay in the coming years.

Outside of the USA, events are also critically important vehicles for the autonomous SNIA Regional Affiliates to fulfill their mission.  The audience there is typically more biased towards business executives and IT managers, and over the years their events have evolved to incorporate adjacent technology areas, new developments and regional requirements.

As an example of this evolution, SNIA Europe’s events partner, Angel Business Communications, recently announced that its very successful October event, SNW Europe/Datacenter Technologies/Virtualization World, will be simply known as Powering the Cloud starting in 2014, in order to unite the conference program and to be more clearly relevant to today’s IT industry. See the press release for more details.

Other Regional Affiliates have followed a similar path with events such as Implementing Information Infrastructure Summit and Information Infrastructure Conference – both tailored to meet regional needs.

The bottom line on this is that the SNIA is absolutely committed to a global events strategy to enable it to carry out its mission.  We are excited about the evolution of our various events to meet the changing needs of the market and continue to deliver unique vendor-neutral content. IT professionals, partners, vendors and their customers around the globe can continue to rely on SNIA events to inform them about new technologies and developments and help them navigate the rapidly changing world of IT.

Flash Webcast Q&A

David Fair

Our recent Webcast: Flash – Plan for the Disruption was very well received and well attended. We thank everyone who was able to make the live event. For those of you who couldn’t make it, it’s now available on demand. Check it out here.

There wasn’t enough time to respond to all of the questions during the Webcast, so we have consolidated answers to all of them in this blog post from the presentation team. Feel free to comment and provide your input.

Q. Are you going to cache both read and writes in NetApp FlashCache?
A. Flash Cache is a level 2 Read cache and it is used to accelerate random read operations. NetApp offers an additional capability called Flash Pool which caches both random reads and random overwrites. Both technologies are part of the NetApp Virtual Storage Tier family within the Data ONTAP operating environment.

Q. Is eMLC flash available today?
A. Yes, a number of Flash vendors are shipping eMLC today.

Q. Also can you review the write cycle performance of SLC vs. MLC?
A. Write cycles for SLC are typically around 100,000. With eMLC, write cycles of 30,000 per bit can be achieved.

Q. Has specific analysis been conducted on what applications and relative data can be cached at the server versus at the storage controller (tolerance for latency, user patience, etc.)?
A. This varies but server caching will typically be used for applications with the most hot spots such as databases. If there is a particular requirement for ultra low latency such as in OLTP environments, server caching may be appropriate. Server caching can also yield significant benefit to increase VM density. Generally, server caching will be deployed to accelerate a specific application while storage controller caching will be used to accelerate storage which is shared across multiple applications.
Q. Does the data running over the network storage PDUs or Ethernet Layer2/IP traffic?
A. Ethernet Layer 2 in this demo, thought it could have been scaled to for L3 IP routed traffic.
Q. What is the difference between flash tier and flash cache?
A. A flash tier is persistent storage whereby datasets are pinned to flash technology for some period of time (or permanently). In Automated Storage Tiering, data may be migrated to and from the flash tier based on the temperature of the data. A flash cache, on the other hand is a caching technology in which the most frequently accessed data is copied to flash for data access but then evicted as the data cools down. Data is copied to the flash cache either on the basis of calculated data temperature or on a first-in first-out basis.
Q. Given the large advantages of flash on power (direct), cooling, and DC footprint, why do enterprise data centers not just completely switch out their HDDs? It seems like there is a good ROI even without considering performance. Is it the operational complexities that make this challenging?
A. For many applications, this is not cost justified given the significant price difference of the SSD and HDD devices. Since hot data typically amounts to less than 20% of total data, a small amount of flash can be deployed successfully. In the caching case, this can be around 1%.

Flash Webcast – Are You Ready for the Disruption?

David Fair

There’s no doubt that flash is a game changer. Even a relatively small percentage of flash can drive a significant improvement in peak storage performance. How are you planning for the disruption? Join me and my SNIA colleague, Paul Feresten, for a live Webcast next week, Thursday, September 20th (11:00 a.m. ET, 8:00 am. PT) as we discuss the impact of flash. We’ll take a look at how flash is being deployed in storage systems, key considerations and tradeoffs, performance benefits, trends in non-volatile memory and more. And because it’s live we’ll take your questions on the spot. We hope to see you there. Register now.