Author: Mike Jochimsen

Kubernetes Links & Resources to Keep You in the Know

Mike Jochimsen
Our recent SNIA CSTI webcast, “Kubernetes in the Cloud (Part 3): (Almost) Everything You Need to Know about Stateful Workloads” offered a wealth of insight on how to address the challenges of running stateful workloads in Kubernetes. This webcast was the third installment of our Kubernetes in the Cloud webcast series and it is now available on-demand as are “Kubernetes in the Cloud (Part 1)” and “Kubernetes in the Cloud (Part 2).” Our expert presenters, Paul Burt and Ingo Fuchs, have provided additional resources to help keep you in the know on Kubernetes. Here they all are: Read More

Kubernetes Links & Resources to Keep You in the Know

Mike Jochimsen

Our recent SNIA CSTI webcast, “Kubernetes in the Cloud (Part 3): (Almost) Everything You Need to Know about Stateful Workloads” offered a wealth of insight on how to address the challenges of running stateful workloads in Kubernetes. This webcast was the third installment of our Kubernetes in the Cloud webcast series and it is now available on-demand as are “Kubernetes in the Cloud (Part 1)” and “Kubernetes in the Cloud (Part 2).”

Our expert presenters, Paul Burt and Ingo Fuchs, have provided additional resources to help keep you in the know on Kubernetes. Here they all are:

If you know of some Kubernetes resources to share, please comment on this blog and we’ll add them to our list.

Stateful Workloads on Kubernetes: (Almost) Everything You Need to Know

Mike Jochimsen
Kubernetes is great for running stateless workloads, like web servers. It’ll run health checks, restart containers when they crash, and do all sorts of other wonderful things. So, what about stateful workloads? Large implementers like Uber say to avoid it if you can [1], and gurus like Kelsey Hightower echo that sentiment [2]. It’s the topic we’ll address on August 20th at our live SNIA Cloud Storage Technologies Initiative webcast “Kubernetes in the Cloud (Part 3): Stateful Workloads.”  In this session, we’ll explore when it’s appropriate to run a stateful workload in cluster, or out. We’ll discuss the best options for running a workload like a database on the cloud, or in the cluster, and what’s needed to set that up. We’ll cover: Read More

Stateful Workloads on Kubernetes: (Almost) Everything You Need to Know

Mike Jochimsen

Kubernetes is great for running stateless workloads, like web servers. It’ll run health checks, restart containers when they crash, and do all sorts of other wonderful things. So, what about stateful workloads? Large implementers like Uber say to avoid it if you can [1], and gurus like Kelsey Hightower echo that sentiment [2].

It’s the topic we’ll address on August 20th at our live SNIA Cloud Storage Technologies Initiative webcast “Kubernetes in the Cloud (Part 3): Stateful Workloads.”  In this session, we’ll explore when it’s appropriate to run a stateful workload in cluster, or out. We’ll discuss the best options for running a workload like a database on the cloud, or in the cluster, and what’s needed to set that up.

We’ll cover:

  • Secrets management
  • Running a database on a VM and connecting it to Kubernetes as a service
  • Running a database in Kubernetes using a `stateful set`
  • Running a database in Kubernetes using an Operator
  • Running a database on a cloud managed service

Register today to save your place on August 20th. This is the 3rd installment of our Kubernetes in the Cloud webcast series. Kubernetes in the Cloud (Part 1) and Kubernetes in the Cloud (Part 2) are available on demand. I encourage you to check them out for great information and demonstrations on Kubernetes.

[1] https://eng.uber.com/dockerizing-mysql/

[2] https://twitter.com/kelseyhightower/status/963413508300812295

Understanding Kubernetes in the Cloud

Mike Jochimsen
Ever wonder why and where you would want to use Kubernetes? You’re not alone, that’s why the SNIA Cloud Storage Technologies Imitative is hosting a live webcast on May 2, 2019 “Kubernetes in the Cloud.” Kubernetes (k8s) is an open-source system for automating the deployment, scaling, and management of containerized applications. Kubernetes promises simplified management of cloud workloads at scale, whether on-premises, hybrid, or in a public cloud infrastructure, allowing effortless movement of workloads from cloud to cloud. By some reckonings, it is being deployed at a rate several times faster than virtualization. Read More

Understanding Kubernetes in the Cloud

Mike Jochimsen

Ever wonder why and where you would want to use Kubernetes? You’re not alone, that’s why the SNIA Cloud Storage Technologies Initiative is hosting a live webcast on May 2, 2019 “Kubernetes in the Cloud.”

Kubernetes (k8s) is an open-source system for automating the deployment, scaling, and management of containerized applications. Kubernetes promises simplified management of cloud workloads at scale, whether on-premises, hybrid, or in a public cloud infrastructure, allowing effortless movement of workloads from cloud to cloud. By some reckonings, it is being deployed at a rate several times faster than virtualization.

In this webcast, we’ll introduce Kubernetes and present use cases that make clear where and why you would want to use it in your IT environment. We’ll also focus on the enterprise requirements of orchestration and containerization, and specifically on storage aspects and best practices, discussing:

  • What is Kubernetes? Why would you want to use it?
  • How does Kubernetes help in a multi-cloud/private cloud environment?
  • How does Kubernetes orchestrate and manage storage?
  • Can Kubernetes use Docker?
  • How do we provide persistence and data protection?
  • Example use cases

We’re fortunate to have great experts for this session, Matt Baldwin, the founder and former CEO of Stackpoint Cloud and now with NetApp and Ingo Fuchs, Chief Technologist, Cloud and DevOps at NetApp.

I hope you will register today to join us on May 2nd. It’s live which means our expert presenters will be on-hand to answer your questions on the spot.

Benefits of RDMA in Accelerating Ethernet Storage Q&A

Mike Jochimsen

At our recent live Webcast “Benefits of RDMA in Accelerating Ethernet Storage Connectivity” experts from Emulex, Intel and Microsoft had an insightful discussion on the ways RDMA is having an impact on Ethernet storage. The live event was attended by nearly 200 people and feedback was overwhelming positive with several attendees thanking us for our vendor neutral presentation and one attendee commenting that it was, “Probably the most clearly comprehensible yet comprehensive webinar I’ve attended in some time.” If you missed the Webcast, it’s now available on demand. We did not have time to get to everyone’s questions, so as promised, below are answers to all of them. If you have additional questions, please ask them in the comments section in this blog and we’ll get back to you as soon as possible.

Q. Is RDMA over RoCEv2 in production?

A. The IBTA released the RoCEv2 Specification in September 2014.  In order to support that specification changes may be required across the RDMA stack, including firmware, drivers & operating systems.  Schedules for implementation of that specification will vary by operating system.  For example, the OpenFabrics Alliance (OFA) has not released an Open Fabrics Enterprise Distribution (OFED) version that implements that standard yet, although it is in process now.  Once OFA completes their OFED stack implementation, the Linux distribution vendors will then incorporate and support the updated OFED stack.  Implementations provided prior to full OFA and Distro vendor support would be preliminary, potentially incompatible with the OFED release, and require confirmation by the distro vendor with regard to the nature/level of support they would be providing

Q. I would have liked a list of Windows applications that take advantage of SMB Direct – both in a Hyper-V host or bare metal.

A. In Windows, any file-based application can make use of SMB3 and SMB Direct due to the native file-based programming interface support. No application changes are required. For certain enterprise applications such as Hyper-V and SQL Server, SMB3 is officially supported, and more information can be found in the product catalog at www.microsoft.com.

Q. Are there any particular benefits in using one network protocol over another for SMB Direct/RDMA (iWARP vs. RoCE vs. IB)?

A. There are no hard and fast rules; any adapter or protocol can be suitable for many scenarios. Of the Ethernet-based protocols we considered in today’s webcast

  • iWARP offers the benefit of operation over TCP with its reliability and routability, well-suited to a broad range of installed infrastructure.
  • RoCE offers a lightweight, efficient protocol when a DCB-enabled switched fabric is available. RoCE, however, is not routable.
  • RoCEv2 offers similar properties to RoCE, with the possibility to scale to larger routed and DCB-enabled fabrics.

Q. Who are the vendors offering iWARP capable RNICs?

A. Chelsio Communications has production iWARP adapters today, and both Intel and Qlogic have publicly committed to future iWARP controllers.

Q. How much testing has been done with SMB3, and in particular SMB direct, over WAN connections?

A. The SMB2 protocol was originally designed to adapt to WAN scenarios, and supports a credit-based management of large amounts of data to be outstanding, to make best use of WAN-type long pipes. The SMB3 protocol retains these design attributes, and the SMB Direct protocol also supports similar deep pipelining. The iWARP protocol, being layered on standard TCP, is well suited to such deployments, and RoCE WAN adapters are potentially available. Please contact the respective technology vendors for information on any available testing results.

Q. I love a future webcast for RDMA enabled distributed filesystems.

A. Thanks for the suggestion! We’re always looking for ideas for future webcasts and SNIA-ESF will consider this as a potential follow-on.

Q. Is Live Migration the scenario where “packet size” is 1MB?

A. All SMB Direct scenarios have workloads that range anywhere up to 8MB. For large file copies, most SMB3 clients request from 1MB to 8MB per operation, for Hyper-V live migration, transfers are typically similar, during the bulk transfer phase.

Q. SMB3 is being compared to FC for enterprise. If Ethernet based protocols are of interest, wouldn’t FCoE give the same performance as FC (same stack) vs. SMB3?

A. SMB3 with SMB Direct enables many workloads not possible with Fibre Channel over Ethernet, and performance comparisons are therefore difficult. Perhaps another SNIA webcast could investigate this!

Q. Regarding your SMB direct example with lots of small operations, how do you deal with the overhead of registering and unregistering buffers for the RDMA operations?

A. As answered later in the session, the registration and unregistration is not a protocol matter, but in the case of the Windows implementation, it is strictly performed for the specific buffers of each operation, which is critical for security, data integrity, and system protection. The standard “Fast Register Work Request” method is used, and careful implementation has shown that the overhead does not negatively impact performance, even for small I/O (4KB/operation). Check out Jose Barreto’s blog, which contains many benchmark results.

Q. But isn’t Live Migration done in 1MB “chunks”? So not “small” I/Os?

A. As answered later in the session, Hyper-V Live Migration is done in several phases, the first phase is the initial bulk copy of memory, done in large chunks, but immediately after it a second phase of copying individual pages which were dirtied by the live-running VM is performed. These operations are typically 4KB. Note: The faster the initial phase goes, the less work there is in this second phase, but in both phases, the faster, the better, and RDMA accelerates both.

Q. Are iSER and iWARP alternatives to one another?

A.  iWARP is an RDMA protocol, and iSER is a mapping of iSCSI to iWARP, as well as RoCE/InfiniBand.

Q. What’s Intel’s roadmap for RoCE and/or iWARP?

A. Intel is committed to iWARP and plans to incorporate it in future server chipsets and SOCs. See http://www.intel.com/content/www/us/en/ethernet-products/accelerating-ethernet-iwarp-video.html for more information.

Q. Is there any other Transport being used other than IB to create a reliable transport for RoceV2? Puristically it is possible?

A. RoCE was developed to leverage Infiniband as much as possible.  For that reason, the Infiniband transport was chosen when the RoCE standard was developed.  As the RoCEv2 standard was developed, the underlying Infiniband network protocol was replaced with IPv4 / IPv6 in order to provide the layer 3 routability and UDP to provide stateless encapsulation (and indication) of the Infiniband transport header that was retained.  While it may be possible to develop a reliable transport to replace Infiniband, the RoCE standards body has elected not to go that route as of this writing.

 

 

 

Ethernet Meets Enterprise Storage – Finally

Mike Jochimsen

Presumptuous, yes, because Ethernet has been a mainstay in enterprises since its early days over 40 years ago.  It initially grew to prominence as the local area network (LAN) connection in the enterprise. More recent advances have enabled Ethernet to become a standard for mission critical storage connectivity for block, file and object storage in many enterprises.

Block storage in large enterprises has long been focused on Fibre Channel due to its performance capabilities.   In order to bring the same performance benefits to Ethernet, the IEEE 802.1 Data Center Bridging Task Group proposed a number of new standards to enhance Ethernet reliability.  For example, 802.1Qbb Priority-based Flow Control (PFC) provides a link level flow control mechanism to ensure lossless transmission under congestion, 802.1Qaz Enhanced Transmission Selection (ETS) provides a management framework for prioritized bandwidth and Data Center Bridging Exchange Protocol (DCBX) enabled these features to be used between neighbors to ensure consistency on the network. Collectively, these and other enhancements have brought those enterprise-class storage networking features to the Ethernet platform.

In addition, the International Committee for Information Technology Services (INCITS) T11 Fibre Channel committee developed a specification for Fibre Channel over Ethernet (FCoE) in its FC-BB-5 standard in 2009, which allows the Fibre Channel protocol to run directly on top of Ethernet, eliminating the TCP/IP stack and allowing for efficient performance of the Fibre Channel protocol.  FCoE also depends on the Data Center Bridging standards from IEEE 802.1 in order to ensure the “losslessness” and flow control needed by Fibre Channel.

An alternative to FCoE, iSCSI, was designed to run over standard Ethernet with TCP/IP and was designed to tolerate the “lossy” aspects of Ethernet.  Its architecture and the additional layers of encapsulation involved can impact latency and performance. However, more recent innovations in iSCSI have enabled it to run over a DCB Ethernet network, which enables iSCSI to inherit some of the enterprise storage features which have always been inherent in Fibre Channel.  For more on this, read last year’s blog “How DCB Makes iSCSI Better ” from Allen Ordoubadian.

In 2013, INCITS submitted the FC-BB-6 standard for review which introduced, among other things, the VN2VN standard.  The VN2VN proposal will allow FCoE to work in a standard DCB switching environment without the presence of a Fibre Channel Forwarder (FCF).  An FCF allows for bridging between servers which are communicating with FCoE and storage devices which are communicating with traditional Fibre Channel.  As DCB switches and FCoE storage become more prevalent, the FC-BB-6 standard will allow for end-to-end FCoE connectivity in either a point to point (P2P) or DCB mesh environment. This will result in lower cost for FCoE environments. Products are beginning to appear which support VN2VN and over the next 18 months it is likely that all major vendors will support it. Check out our ESF Webcast “How VN2VN Will Help Accelerate Adoption of FCoE” for more details.

The availability of CNAs with processing capability allows for offloading storage protocol processing from the host processor, though some CNAs use host-based storage protocol initiators in system software and do selective stateless offloads in the data path.  Both FCoE and iSCSI require the storage protocol to be encapsulated in a frame to be sent across the Ethernet network.  In an enterprise environment, especially a virtual server environment, CPU utilization is tracked closely and target CPU thresholds are often set.  Anything which can minimize spikes in CPU utilization can allow for more workloads to be placed on servers and allows for predictable energy consumption.

For file storage, Ethernet has traditionally been the connectivity option of choice for file servers used as “shares” for centralized employee document storage. In the 21st century, usage of network attached storage (NAS) with the Network File System (NFS) has increased for enterprise databases and Hadoop clusters, especially with the availability of 10Gb Ethernet.  New features in NFS 4 and later introduced security and stateful protocol support after development of NFS was taken over by the Internet Engineering Task Force (IETF).

Object storage, has been around for nearly 20 years as a repository for storing data as objects which include not only the original file, but also a globally unique identifier and metadata which describes the object and various parameters about the object.  It has been used to store many forms of unstructured data, but found niches in certain areas, such as legal documents with retention policies and archiving photos and videos.  More recently, there seems to be a resurgence in object storage as the amount of unstructured data generated by enterprises continues to skyrocket.  Open source object storage in Ceph and OpenStack are also helping to drive the adoption. SNIA ESF is hosting a live Webcast on object storage on June 11, 2014, called “Object Storage 101.” I encourage you to register for this presentation for an unbiased look at the what, how and why of object storage technologies.

When combined with the advances in link speed, throughput capabilities, latency and input/output operations per second (IOPS) in modern 10Gb/s and 40Gb/s Ethernet, these existing and emerging Ethernet standards and storage architectures are having a profound effect on the ability of Ethernet as an enterprise class storage networking platform.  Vendors and customers are seeing the advantage in one wire, the Ethernet cable, carrying all LAN, WAN and storage traffic.