Why the FCoE – iSCSI Debate Continues

Jeff Asher

Why the FCoE – iSCSI Debate Continues

This is my first blog post for SNIA-ESF.  As a Principal Storage Architect, I have been doing extensive research on the factors that are driving FCoE vs. iSCSI choices over the last several years. The more I dive into the topic, the more intriguing the debate becomes. In fact, this blog is a preview of an upcoming white paper I’m writing and a Webcast SNIA is hosting on February 18th. If you agree this debate is interesting, I encourage you to attend. Details on the Webcast are at the end of this post.

A Look Back at FCoE and iSCSI History

There are two entrenched standards for block storage protocols over Ethernet networks.  FCoE was ratified in 2009, while iSCSI was ratified in 2004.  Of course, various vendors and early adopters supported these protocols before ratification, so the history of these protocols is a couple of years longer than it looks, respectively.  While iSCSI simply encapsulates the SCSI protocol in IP, FCoE operates lower in the network stack and to do so required many enhancements to Ethernet.  While iSCSI runs on any IP network (mostly Ethernet these days), FCoE requires Data Center Bridging and Converged Network Adapters all running at 10 Gbps or faster.

All of the Data Center Bridging enhancements that make FCoE possible, like lossless Ethernet, benefit all of the protocols using Ethernet as the transport protocol.  DCB doesn’t just make FCoE possible, but it improves iSCSI at the same time  (see the SNIA-ESF blog, How DCB Makes iSCSI Better). So given that modern servers, networks, and storage may all be connected by hardware capable of running FCoE, that same network is also able to run iSCSI, as well as other network traffic.  Nothing precludes them from running simultaneously on the same network either.  The leading storage vendors that offer both FCoE and iSCSI target systems allow administrators to present the same LUN over either protocol with little effort, so a transition from one protocol to the other is not difficult.

Strengths and Weaknesses

So which network protocol is the right choice?

Each protocol has strengths and weaknesses when judged relative to each other.  FCoE has higher throughput at lower host CPU utilization than iSCSI and FCoE doesn’t have to process the TCP/IP stack as iSCSI does. iSCSI is relatively simple to setup and troubleshoot when compared to FCoE because zoning is not a factor and IP connectivity (although not optimized for storage traffic) is likely in place already.  Also, while FCoE has a comprehensive set of existing tools available to ease troubleshooting, there aren’t as many qualified people to use them in most enterprises.  Ease of use, plus the ability to use low cost NICs and switches, gives iSCSI a cost advantage.  (However, if you check out our SNIA-ESF webcast, “How VN2VN Will Help Accelerate Adoption of FCoE,” you’ll hear about new technologies that reduce the costs of deploying FCoE.) FC, and by extension FCoE, are perceived to be enterprise-grade, suitable for all workloads; and while iSCSI is being widely adopted at the enterprise level, it is still perceived by some not to be ready for Tier-1 applications.  The graph below is excerpted from the report “Intel 10GbE Adapter Performance Evaluation” prepared by Demartek for Intel in September 2010.  This data is consistent with the rest of the report findings and is only intended to be representative of the results from comparative iSCSI and FCoE testing.  The report is interesting reading and I recommend looking at it for more information. This graph shows IOPS and CPU utilization for JetStress tests running against NetApp storage over multi-path iSCSI and FCoE.  Note that latencies were all similar and running the tests against EMC storage showed similar results.

FCoE-iSCSI_Data

Many other factors must be considered, but according to industry pundits- as well as my own personal experience – in the majority of cases either protocol is adequate for the task at hand, and that is to effectively transfer block data across an Ethernet network.

Maximizing Throughput

The reality is, most servers, applications, and storage arrays simply won’t take advantage of FCoE’s superior performance or any storage protocol running over 10GbE.  iSCSI and NAS protocols are very fast and are typically sufficient to meet most application requirements.  But this is not meant to be a SAN vs NAS post – besides years of history, thousands of happy end users, and billions of continued investment show that both work well enough to meet most business needs.  The commonly deployed storage systems and hosts are simply not configured with enough hardware to saturate multiple 10 gigabit network links.  While this is rare today, it is going to become more common to see systems capable of saturating 10GbE pipes in the near future, especially as flash memory, either in all-flash arrays or tiered storage systems, find more application.  (Hear more on the impact of flash in our SNIA-ESF webcast, “Flash – Plan for the Disruption”). At least as it relates to spinning media disk systems – network bandwidth increases faster than storage system throughput can keep up.  So consider the storage system to be the bottleneck or limiting factor when evaluating storage network performance.  After all, in most data center environments, the ratio of servers and applications to storage systems is high. So, it’s reasonable to expect the storage system to be the bottleneck.  The absolute throughput of FCoE and iSCSI, when pushing a storage system to its limits, is not sufficient alone to be used as the sole basis for the decision between the two protocols except, for a few edge cases.  Bottom line: Whether the storage system is the bottleneck or the network is the bottleneck the performance relationship between FCoE and iSCSI does not change.

These edge cases tend to be extremely IO intensive database workloads and big data applications, such as Hadoop.  Citing the graph above, FCoE is about 15-20% faster on identical hardware than iSCSI.  Granted this is a single graph of a single test, but the data is consistent across tests performed by IBM using Emulex network interfaces.  If absolute throughput and efficiency (both network and CPU) are the only criteria when deciding between block protocols, FCoE looks like the choice.  Since these cases are rare – because complexity, supportability, and even politics are almost always considered – the decision is not so obvious.  Again, beyond the scope of this article, NAS protocols should be considered when determining the proper protocol for an application also.

Is There a Clear Winner?

While FCoE can claim technical superiority, iSCSI has the edge in cost and supportability.  The number and range of systems supporting iSCSI connectivity is greater, particularly at the entry level.  What’s more, the availability of people that can troubleshoot end-to-end connectivity for iSCSI is also much greater.  (The “ping” command diagnoses most iSCSI connectivity problems.)  Also, do a resume search on Monster or LinkedIn and the number of people that can configure VLANs dwarfs the number that can properly zone a Fibre Channel network.  Greater familiarity reduces the support and operating cost of iSCSI.

IDC predicts that FCoE revenue will ramp very quickly through 2016. (If available to you, see the IDC Worldwide Enterprise Storage Systems 2012-2016 Forecast Update.)  As customers decide to transition existing Fibre Channel networks to an Ethernet infrastructure, deploying FCoE would be a comfortable choice due to existing IT expertise and functional expectations of the Fibre Channel protocol.

Both iSCSI and FCoE are capable storage protocols and choosing one over the other will likely be dependent upon budget, IT skill set, and application requirements

Don’t forget to join us on Feb. 18th

Again, I encourage you to attend our February 18th Webcast, “Use Cases for iSCSI and FCoE –Where Each Makes Sense.”  Analysts from Dell’Oro Group will share their latest market research on this topic and I’ll dive into use cases for both iSCSI and FCoE. It’s a live event, so please come with your toughest questions. I hope you’ll join us!

 

 

2013 in Review and the Outlook for 2014 – A SNIA ESF Perspective

Jason Blosil

Technology continues to advance rapidly. Making sense of it all can be a challenge. At the SNIA Ethernet Storage Forum, we focus on storage technologies and solutions enabled by and associated with Ethernet Networks. Last year, we modified the charters of our two Special Interest Groups (SIG) to address topics about file protocols and storage over Ethernet. The File Protocols SIG includes the prior focus on Network File System (NFS) related topics and adds discussions around Server Message Block (SMB / CIFS). We had our first webcast last November on the topic of SMB 3.0 and it was our best attended webcast ever. The Storage over Ethernet SIG focuses on general Ethernet storage topics as well as more information about technologies like FCoE, iSCSI, Data Center Bridging, and virtual networking for storage. I encourage you to check out other articles on these hot topics in this SNIAESF blog to hear from our member experts as well as guest posts from leading analysts.

2013 was a busy year and we are already kickin’ it in 2014. This should be an exciting year in IT. Data storage continues to be a hot sector especially in the areas of All-Flash and Hybrid arrays. This year, we will expect to see new standards coming out of the T11 committee for Fibre Channel and possibly FCoE as well as progress in high speed Ethernet networks. Lower cost network interconnects will facilitate adoption of high speed networks in the small to midsize business segment. And a new conversation around “Software Defined…” should push a lot of ink in trade rags and other news sources. Oh, and don’t forget about the “Internet of Things”, mobile solutions, and all things Cloud.

The ESF will be addressing the impact on Ethernet storage solutions from these hot technologies. Next month, on February 18th, experts from the ESF, along with industry analysts from Dell’Oro Group will speak to the benefits and best practices of deploying FCoE and iSCSI storage protocols. This presentation “Use Cases for iSCSI and Fibre Channel: Where Each Makes Sense” will be part of an upcoming BrightTalk Summit on Storage Networking. I encourage you to register for this session. Additionally, we will be publishing a couple of white papers on file-based storage and a review of FCoE and iSCSI in storage applications.

Finally, SNIA will be kicking off its first year of the new user conference, Data Storage Innovation Conference. This will be one of the few storage focused user conferences in the market and should be quite interesting.

We’re excited about our growing membership and our plans for 2014. Our goal is to advance application of innovative technologies and we encourage you to send us mail or comment below with topics that are of interest to you.

Here’s to an exciting 2014!

New SIG Formed to Accelerate Adoption of NVDIMM

Marty Foltyn

SNIA has just announced a new special interest group around NVDIMM to:

  • Provide  education on how system vendors can “design-in” NVDIMMs.
  • Communicate existing industry standards, and areas for vendor differentiation.
  • Help technology and solution vendors whose products integrate NVDIMMs to communicate their benefits and value to the greater market.
  • Develop vendor-agnostic user perspective case studies, best practices, and vertical industry requirements to help end user customers understand how products and solutions can meet performance, cost, and efficiency goals.

Initial members of the NVDIMM SIG include  vendors AgigA Tech, IDT,  Inphi,  Intel, Micron, Microsoft Corporation,  Netlist,  Pericom, Samsung, SK Hynix,  SMART Modular Technologies, and Viking Technology.

A new webpage under the Solid State Storage Technology Community on the SNIA website at www.snia.org/nvdimm provides a knowledge resource for presentations, white papers, FAQs, and  webcasts on NVDIMM contributed by SIG companies. Those interested in joining the NVDIMM SIG should contact nvdimmsigchair@snia.org.

SSD Education Afternoon Monday January 27 at SNIA Symposium in San Jose

Marty Foltyn

Interested in the latest information on SSD technology?  Join the SNIA Solid State Storage Initiative Monday January 27 for lunch and an afternoon of the latest on:

  • Flash/SSD technology
  • SCSI Express
  • SAS
  • NVM Express
  • SATA Express
  • SSD performance
  • SSD Markets

Lunch begins at noon, with presentations from 1:00 pm – 4:00 pm.  There is no charge to attend this session at the Sainte Claire Hotel in downtown San Jose CA. You can attend in person – register at www.snia.org/events/symp2014 or by WebEx (click here for details and the agenda).

Add to your NVDIMM Knowledge – Attend the January 28 Summit

Marty Foltyn

Over 150 individuals participated in the BrightTALK Enterprise Storage Summit NVDIMM webcast.  If you are eager for more information on NVDIMM, you will want to attend an upcoming SNIA Event – the Storage Industry Summit on Non–Volatile Memory.

This Summit will take place at the Sainte Claire Hotel in San Jose, CA on January 28th as part of the SNIA Annual Members’ Symposium, and will offer critical insights into NVM, including NVDIMMs, and the future of computing. This event is complimentary to attend and you can register here.

The Summit will take place from 8:15 AM to 5:30 PM and speakers currently include:

  • Nigel Alvares, Senior Director of Marketing, Inphi
  • Bob Beauchamp, Distinguished Engineer and Director Hardware Technology and Architecture, EMC
  • Matt Bryson, ABR Investment Strategy, LLC, SVP-Research
  • Jeff Chang, Vice President, Marketing & Business Development, AgigA Tech
  • Tom Coughlin, Founder, Coughlin Associates
  • Mark Geenen, President, TrendFocus
  • Jim Handy, Analyst, Objective Analysis
  • Jay Kidd, CTO, NetApp
  • Eden Kim, CEO, Calypso
  • Tau Leng, VP/GM, Supermicro
  • Jeff Moyer, Principal Software Engineer, Red Hat
  • Wes Mukai, VP of Cloud Computing, System Engineering, SAP
  • Jim Pinkerton, Lead Partner Architect, Microsoft
  • Adrian Proctor, VP Marketing, Viking Technology
  • Andy Rudoff, Senior Software Engineer, Intel
  • Esther Spanjer, Director, Marketing Management, SanDisk
  • Garret Swart, Database Architect, Oracle
  • Nisha Talagala, Lead Architect, Fusion-IO
  • Doug Voigt, Distinguished Technologist in Storage, HP

Visit http://www.snia.org/nvmsummit for more information and we hope you will join us in San Jose!

Solid State Storage Highlighted at Storage Visions 2014

Marty Foltyn

The SSSI began 2014 at the Storage Visions Conference. With a theme of Fast is Beautiful, the 2014 conference explored the latest storage technologies and methods to improve performance and efficiency, lower costs, and improve reliability.

Solid state storage was a key component of exhibits and technology sessions.IMG_6719

SSSI showcased our latest white papers on solid state storage. See them here.  SSSI members Fastor Systems and Greenliant Systems demonstrated innovative SSD solutions.

At Storage Visions,  Paul Wassenberg, SNIA SSSI Chair, discussed findings of a new SSSI survey rating SSD features. An unexpected discovery was the need for education on issues such as data encryption, power management, and others. SSSI will work to provide more information on these issues to help users to make informed choices about SSDs.  Join our dialog on related topics via a dedicated LinkedIn group .

SNIA’s continuing work in NVM programming and PCIe SSD created a buzz.  Jim Pappas of Intel moderated a panel on Bringing Non-Volatile Memory To Tomorrow’s Storage Architectures; Adrian Proctor of Viking Technology gave an NVDIMM overview on a Storage Developments Drive New Storage Options panel; and Eden Kim of Calypso Systems spoke on PCIe SSD activities on a Future Content – What’s Ahead for Content Storage panel.  Explore these areas further at our upcoming SNIA Annual Members Meeting, January 27-30, 2014 at the Sainte Claire Hotel in San Jose CA where a Storage Industry Summit-Focus on NVM will take place Tuesday January 28. Registration is complimentary.

Fibre Channel over Ethernet (FCoE): Hype vs. Reality

Casey Quillin Sameh Boujelbene

It’s been a bit of a bumpy ride for FCoE, which started out with more promise than it was able to deliver. In theory, the benefits of a single converged LAN/SAN network are fairly easy to see. The problem was, as is often the case with new technology, that most of the theoretical benefit was not available on the initial product release. The idea that storage traffic was no longer confined to expensive SANs, but instead could run on the more commoditized and easier-to-administer IP equipment was intriguing, however, new 10 Gbps Enhanced Ethernet switches were not exactly inexpensive with few products supporting FCoE initially, and those that did, did not play nicely with products from other vendors.

Keeping FCoE “On the Single-Hop”?

The adoption of FCoE to date has been almost exclusively “single-hop”, meaning that FCoE is being deployed to provide connectivity between the server and the Top of Rack switch. Consequently, traffic continues to be broken out one way for IP, and another way for FC. Breaking out the traffic makes sense—by consolidating network adapters and cables, it adds value on the server access side.

A significant portion of FCoE switch ports come from Cisco’s UCS platform, which runs FCoE inside the chassis. In terms of a complete end-to-end FCoE solution, there continues to be very little multi-hop FCoE happening, or ports shipping on storage arrays.

In addition, FCoE connections are more prevalent on blade servers than on stand-alone servers for various reasons.

  • First, blades are used more in a virtualized environment where different types of traffic can travel on the same link.
  • Second, the migration to 10 Gbps has been very slow so far on stand-alone servers; about 80% of these servers are actually still connected with 1 Gbps, which cannot support FCoE.

What portion of FCoE-enabled server ports are actually running storage traffic?

FCoE-enabled ports comprise about a third of total 10 Gbps controller and adapter ports shipped on servers. However, we would like to bring to readers’ attention the wide difference between the portion of 10 Gbps ports that is FCoE-enabled and the portion that is actually running storage traffic. We currently believe less than a third of the FCoE-enabled ports are being used to carry storage traffic. That’s because the FCoE port, in many cases, is provided by default with the server. That’s the case with HP blade servers as well as Cisco’s UCS servers, which together are responsible for around 80% of the FCoE-enabled ports. We believe, however, that in the event that users buy separate adapters they will most likely use that adapter to run storage traffic—but they will need to pay an additional premium for this – about 50% to 100% – for the FCoE license.

The Outlook

That said, whether FCoE-enabled ports are used to carry storage traffic or not, we believe they are being introduced at the expense of some FC adapters. If users deploy a server with an FCoE-enabled port, they most likely will not buy a FC adapter to carry storage traffic. Additionally, as Ethernet speeds reach 40 Gbps, the differential over FC will be too great and FC will be less likely to keep pace.

About the Authors

Casey Quillin is a Senior Analyst, Storage Area Network & Data Center Appliance Market Research with the Dell’Oro Group

Sameh Boujelbene is a Senior Analyst, Server and Controller & Adapter Market Research with the Dell’Oro Group

Your Chance to Learn about PCIe Storage Protocol Analysis Test and Connectivity Tools and How to Hear On-demand The Latest on NVDIMM

Marty Foltyn

Join the SNIA Solid State Storage Initiative for an Open SSSI PCIe SSD Committee call

Who:  John Weidemier, Teledyne LeCroy

What:  PCIe storage protocol analysis test and connectivity tools

When:  Monday December 9, 2013 at  4:00 PM PST

Where:   via WebEx. at http://snia.webex.com Meeting Number: 792 152 928 password: sssipcie  and by dialing in to teleconference: 1-866-439-4480 Passcode: 57236696#

Why:  This OPEN call is an invitation to non SSSI Members to learn more about the exciting work of the PCIe Solid State Drive Committee and 2014 activities of the SSSI.

And, the great NVDIMM talk given on 12/5/13 as part of the BrightTALK Enterprise Storage Summit is now available for on-demand viewing! The session ranked 4.8 out of a 5.0 for quality technical content featuring information on a wide number of vendors who are creating NVDIMM products and solutions.

Click on this link  https://www.brighttalk.com/webcast/663/95329.

SNIA’s Events Strategy Today and Tomorrow

khauser Leave a comment

David Dale, SNIA Chairman

Last month Computerworld/IDG and the SNIA posted a notice to the SNW website stating that they have decided to conclude the production of SNW.  The contract was expiring and both parties declined to renew.  The IT industry has changed significantly in the 15 years since SNW was first launched, and both parties felt that their individual interests would be best served by pursuing separate events strategies.

For the SNIA, events are a strategically important vehicle for fulfilling its mission of developing standards, maintaining an active ecosystem of storage industry experts, and providing vendor-neutral educational materials to enable IT professions to deal with and derive value from constant technology change.  To address the first two categories, SNIA has a strong track record of producing Technical Symposia throughout the year, and the successful Storage Developer Conference in September.

To address the third category, IT professionals, SNIA has announced a new event, to be held in Santa Clara, CA, from April 22-24 – the Data Storage Innovation Conference. This event is targeted at IT decision-makers, technology implementers, and those expected to influence, implement and support data storage innovation as actual production solutions.  See the press release and call for presentations for more information.  We are excited to embark on developing this contemporary new event into an industry mainstay in the coming years.

Outside of the USA, events are also critically important vehicles for the autonomous SNIA Regional Affiliates to fulfill their mission.  The audience there is typically more biased towards business executives and IT managers, and over the years their events have evolved to incorporate adjacent technology areas, new developments and regional requirements.

As an example of this evolution, SNIA Europe’s events partner, Angel Business Communications, recently announced that its very successful October event, SNW Europe/Datacenter Technologies/Virtualization World, will be simply known as Powering the Cloud starting in 2014, in order to unite the conference program and to be more clearly relevant to today’s IT industry. See the press release for more details.

Other Regional Affiliates have followed a similar path with events such as Implementing Information Infrastructure Summit and Information Infrastructure Conference – both tailored to meet regional needs.

The bottom line on this is that the SNIA is absolutely committed to a global events strategy to enable it to carry out its mission.  We are excited about the evolution of our various events to meet the changing needs of the market and continue to deliver unique vendor-neutral content. IT professionals, partners, vendors and their customers around the globe can continue to rely on SNIA events to inform them about new technologies and developments and help them navigate the rapidly changing world of IT.

Learn about NVDIMM in BrightTalk’s Enterprise Storage Summit

Marty Foltyn

If 2013 was the year of software-defined everything, of everything-as-a-service, and big data – what’s on the horizon for storage in 2014? What do we still need to know about the innovations of this year? What hard realities have been missing from the hype? In the  Enterprise Storage Summit, https://www.brighttalk.com/summit/2271, a bi-annual event from BrightTALK, top thought leaders from all over the globe gather to share their insights into one of the most difficult areas of IT infrastructure and shed light on some of the most crucial topics facing enterprises today, including backup/recovery, hardware innovations, continued transition to cloud storage and the implications of big data.

On December 5 at 9:00 am Pacific/12:00 noon Eastern, learn how Non-Volatile DIMMs, or NVDIMMs, provide a persistent memory solution with the endurance and performance of DRAM coupled with the non-volatility of Flash. This webinar, presented by Jeff Chang of AgigaTech, will provide a general overview of this emerging technology and why the industry is starting to take notice.

You will learn what an NVDIMM is, how it works, where it fits and why every system architect should consider them for their next generation enterprise server and storage designs.

Join this summit with other smart people from around the world to participate in live events and ask your questions. Register at https://www.brighttalk.com/webcast/663/95329 to attend live or be notified of on-demand viewing.