Michelle Tidwell, SNIA Board of Directors
Get Ready for Part 2 of Kubernetes in the Cloud
Category: Docker
Get Ready for Part 2 of Kubernetes in the Cloud
As enterprises move to a hybrid multi-cloud world, they are faced with many challenges. In addition to decisions surrounding what technologies to use, they are also seeing a transformation in traditional IT roles. Storage admins are asked to be more cloud savvy while new roles of cloud admins are emerging to handle the complexities of deploying simple and efficient clouds. Meanwhile, both these roles are asked to ensure a self-service environment is architected so that application developers can get resources needed to develop cutting edge apps not in week, days or hours but in minutes.
That’s why the SNIA Cloud Storage Technologies Initiative (CSTI) is hosting another live webcast on Kubernetes in the Cloud (Part 2) on July 17, 2019. In part one of this three part series, we covered the high level aspects of Kubernetes. This presentation will discus key capabilities IT vendors are creating based on open source technologies such as Docker and Kubernetes to build self-service infrastructure to support hybrid multi-cloud deployments. We’ll cover:
- Persistent storage and how to specify it
- Ensuring application portability between Private and Public Clouds
- Building a self-service infrastructure (Helm, Operators)
- Selecting Block, File, Object (Traditional Storage, SDS)
Register today to save your spot. And bring your questions, our experts will be on hand to answer them on the spot.
Understanding Kubernetes in the Cloud
Ever wonder why and where you would want to use Kubernetes? You’re not alone, that’s why the SNIA Cloud Storage Technologies Imitative is hosting a live webcast on May 2, 2019 “Kubernetes in the Cloud.”
Kubernetes (k8s) is an open-source system for automating the deployment, scaling, and management of containerized applications. Kubernetes promises simplified management of cloud workloads at scale, whether on-premises, hybrid, or in a public cloud infrastructure, allowing effortless movement of workloads from cloud to cloud. By some reckonings, it is being deployed at a rate several times faster than virtualization. Read More
Understanding Kubernetes in the Cloud
Ever wonder why and where you would want to use Kubernetes? You’re not alone, that’s why the SNIA Cloud Storage Technologies Initiative is hosting a live webcast on May 2, 2019 “Kubernetes in the Cloud.”
Kubernetes (k8s) is an open-source system for automating the deployment, scaling, and management of containerized applications. Kubernetes promises simplified management of cloud workloads at scale, whether on-premises, hybrid, or in a public cloud infrastructure, allowing effortless movement of workloads from cloud to cloud. By some reckonings, it is being deployed at a rate several times faster than virtualization.
In this webcast, we’ll introduce Kubernetes and present use cases that make clear where and why you would want to use it in your IT environment. We’ll also focus on the enterprise requirements of orchestration and containerization, and specifically on storage aspects and best practices, discussing:
- What is Kubernetes? Why would you want to use it?
- How does Kubernetes help in a multi-cloud/private cloud environment?
- How does Kubernetes orchestrate and manage storage?
- Can Kubernetes use Docker?
- How do we provide persistence and data protection?
- Example use cases
We’re fortunate to have great experts for this session, Matt Baldwin, the founder and former CEO of Stackpoint Cloud and now with NetApp and Ingo Fuchs, Chief Technologist, Cloud and DevOps at NetApp.
I hope you will register today to join us on May 2nd. It’s live which means our expert presenters will be on-hand to answer your questions on the spot.
Wondering What’s New in Container Storage?
The landscape of containers is moving fast and constantly changing, with new standards emerging every few months. If you wondering what’s new in container storage, you are not alone. That’s why the SNIA Cloud Storage Technologies Initiative is hosting a live webcast on February 26, 2019, “What’s New in Container Storage.”
In this webcast, Keith Hudgins of Docker joins us as a follow up to his earlier container webcast “Intro to Containers, Container Storage and Docker.” It’s our most popular webcast to date with thousands of views. If you missed it, it’s available on demand and will provide you with some great background information before our February 26th webcast.
I encourage you to register today for the February 26th session where you’ll learn: Read More
Does Your World Include Storage? Don’t Miss SDC!
Whether storage is already a main focus of your career or may be advancing toward you, you’ll definitely want to attend the flagship event for storage developers – and those involved in storage operations, decision making, and usage – SNIA’s 19th annual Storage Developer Conference (SDC), September 11-14, 2017 at the Hyatt Regency Santa Clara, California. Read More
Containers, Docker and Storage – An Expert Q&A
Containers continue to be a hot topic today as is evidenced by the more than 2,000 people who have already viewed our SNIA Cloud webcasts, “Intro to Containers, Container Storage and Docker“ and “Containers: Best Practices and Data Management Services.” In this blog, our experts, Keith Hudgins of Docker and Andrew Sullivan of NetApp, address questions from our most recent live event.
Q. What is the major challenge for storage in containerized environment?
A. Containers move fast. Users can spin up and spin down containers extremely quickly. The biggest challenge in production-bound container environments is simply keeping up with the movement of data.
Docker Engine does not delete base container images when the container is shut down. Likewise, Registry assumes you’ve got unlimited storage on hand. For containers that push frequent revisions (as would be the case in a continuous delivery environment), that leads to a lot of orphaned container images that can fill up all available storage if left unchecked.
There are some community-led scripts that will help to keep things in control. That’s the beauty of community-led technology.
Q. What about the speed of retrieving the data from storage?
A. That’s where being a solid storage architect comes in. Every storage system has different strengths and weaknesses, so it’s important to engineer your solution to fit your performance goals. Docker containers are running on the main kernel of the host system. IO is not constrained by abstraction, as in the case of virtual machines. Rather, it is constrained more by density – hundreds of containers on a host can push massive IOPS, so you want your pipes fat and data sources close to the host systems.
Q. Can you expand on moving Docker Volumes from On-Premise bare metal to Cloud Service Providers? Data Migration? Encryption?
A. None of these capabilities are built-in to Docker Engine. We rely on external storage systems to provide those features. Private-to-cloud replication is primarily a feature of software-based companies, like Portworx, Blockbridge, or Hedvig. Encryption and migration are both common features across other companies as well. Flocker from ClusterHQ is a service broker system that provides many bolt-on features for storage systems they support. You can also use community-supplied services like Ceph to get you there.
Q. Are you familiar with “Flocker” that apparently is able to copy persistent data to another container? Can share your thoughts?
A. Yes. ClusterHQ (makers of Flocker) provide an API broker that sits between storage engines and Docker (and other dynamic infrastructure providers, like OpenStack), and they also provide some bolt-on features like replication and encryption.
Q. Is there any sort of feature in the volume plugins that allows a persistent volume to re-connect to a container if the container is moved across multiple hosts?
A. There’s no feature in plugins to cover that specifically. The plugin API is very simple. In practice, what you would do is write your plugin to expose volumes to Docker Engine on every host that it’s possible to mount that volume. In your container specification, whether it’s a Compose file, DAB file, or what have you, specify the name of your volume. Wherever that unique name is encountered, it will be mounted and attached to the container when it’s re-launched.
If you have more questions on containers, Docker and storage, check out our first Q&A blog: Containers: No Shortage of Interest or Questions.
I also encourage you to join our Containers opt-in email list. It will be a good way to keep up with all the SNIA Cloud is doing on this important technology.
No Shortage of Container Storage Questions
We covered a lot of ground in out recent SNIA Ethernet Storage Forum webcast, “Current State of Storage in the Container World.” We had a technical discussion on why containers are so compelling, how Docker containers work, persistent shared storage and future considerations for container storage. We received some great questions during the live event, and as promised, here are answers to them all.
Q. Docker cannot be installed on bare metal and requires a base OS to operate upon right?
A. That is correct.
Q. Does the application code need to be changed so that it can “fit and operate” in a container?
A. No, the application code does not need to change. The challenge most people face when migrating an application to a container is how to maintain the application’s state. One of the motivations for this webcast was to explain how to allow applications within containers to persist data. Hopefully the Docker Volume construct will meet your needs.
Q. Seems like containers share one OS/kernel… That suggests that there is just one OS in the “containerized” server… And yet there is still mention of hypervisor (or at least Hyper-V)… Can you clarify? If the containers share an OS, is a hypervisor needed?
A. You are correct, containers are designed to share a single kernel; therefore a hypervisor is not required to run containers. Having said that, VMware and Microsoft both offer options that run a single container in its own virtual machine (running a minimal operating system).
Q. Can the Docker Hub be compared to something like the GitHub?
A. Yes, that is a great analogy. Docker Hub (hub.docker.com) is to container images as GitHub (github.com) is to source code.
Q. What are the differences between the base and the host image?
A. If you’re referring to the webcast slides; the box labeled “Base Image” is the first layer in an image. The box labeled “Host OS” is not a layer, but represents the hosting operating system (kernel) that is shared by the containers.
Q. So there is a separate root per container?
A. In most cases the image will provide a root, therefore each container will have a separate root. This is made possible by a kernel feature called namespaces. Alternatively, Docker does allow you to share a directory between the host operating system and any number of containers though.
Q. If Deduplication is enabled on the storage LUNs, won’t that affect the performance of the containers?
A. Well implemented data reduction features (compression and deduplication) should have little to no effect on performance and should provide significant benefit by reducing the space required to store containers.
Q. Can you please quickly review the concept of copy-on-write with one or two sentences to boil it down?
A. How the copy-on-write works depends on whether the driver is file or block based. For the sake of simplicity, let’s assume a file-based implementation. Since the image layers are read-only, we need an area to store the changes that the container has made. This area is the copy-on-write layer. When a process reads a file that has not been modified, the file is read from one of the read only layers. When that file is modified and needs to be written back to disk, the new file is written to the copy-on-write layer as is the metadata that describes the file. The next time this file is read, it is read from copy-on-write layer. The graph driver is responsible for this functionality and varies by implementation.
Q. Can network locations be used for /data? If yes, how does the Docker Engine manage network authentication for the driver?
A. Yes, network locations can be used. The best practice is to use the Local Volume Driver, where you can pass in the required authentication via the options (see slide 15). Alternatively, the network location can be mounted on the host operating system and exposed to containers (see slides 21 & 22).
Q. Is this where VAAI like primitives would get implemented?
A. VAAI defines several in-band primitives. The Docker Volume plug-in framework is completely out-of-band. There can be some overlap in features though. For example, the XCOPY primitive can be used to offload ‘copy jobs’ to an array. If the vendor chooses to do so, a ‘copy job’ can be offloaded through the Docker Volume plug-in as well. For example, a plug-in might implement a “clone” option that provides this service.
Q. Could you share some details about Kubernetes storage ? Persistent volumes and the difference from Docker volumes? Also, what is your perspective of Flocker?
A. Kubernetes has the concept of persistent storage. This abstraction is also called a volume. In addition, Kubernetes provides a plug-in option as well. The Kubernetes implementation predates the Docker Volume and is currently not compatible.
Q. Comment on mainframe: IBM runs Linux on zSeries, therefore can run Linux Docker containers.
A. Thanks, that’s good to know.
Q. How many operating systems changes on the x86 platform? How many on the mainframe platform? Can x86 architecture run the same code/OS from 40 years ago? Docker on mainframe?
A. The mainframe architecture has been very solid and consistent for many years.
Q. What is a big challenge for storage in container environment?
A. I don’t think storage has a challenge in the container environment. I think, with a properly implemented Docker Volume Plug-in, storage provides a solution to the persistent shared storage need in a container environment.
Q. Do you ever look into RexRay or VMDK storage drivers?
A. Yes, these are both examples of Docker Volume plug-in implementations.
The Next Step for Containers: Best Practices and Data Management Services
In our first SNIA Cloud webcast on containers, we provided a solid foundation on what containers are, container storage challenges and Docker. If you missed the live event, it’s now available on-demand. I encourage you to check it out, as well as our webcast Q&A blog.
So now that we have set the stage and you’ve become acquainted with basic container technologies and the associated storage challenges in supporting applications running within containers in production, we will be back on December 7th. This time we will take a deeper dive into what differentiates this technology from what you are used to with virtual machines. Containers can both complement virtual machines and also replace them, as they promise the ability to scale exponentially higher. They can easily be ported from one physical server to another or to one platform—such as on-premise—to another—such as public cloud providers like Amazon AWS.
At our December 7th webcast, “Containers: Best Practices and Data Management Services,” we’ll explore container best practices to address the various challenges around networking, security and logging. We’ll also look at what types of applications more easily lend themselves to a microservice architecture versus which applications may require additional investments to refactor/re-architect to take advantage of microservices.
On December 7th, we’ll be on hand to answer your questions on the spot. I encourage you to register today. We hope you can attend!
The Current State of Storage in the Container World
It seems like everyone is talking about containers these days, but not everyone is talking about storage – and they should be. The first wave of adoption of container technology was focused on micro services and ephemeral workloads. The next wave of adoption won’t be possible without persistent, shared storage. That’s why the SNIA Ethernet Storage Forum is hosting a live webcast on November 17th, “Current State of Storage in the Container World.” In this webcast, we will provide an overview of Docker containers and the inherent challenge of persistence when containerizing traditional enterprise applications. We will then examine the different storage solutions available for solving these challenges and provide the pros and cons of each. You’ll hear:
- An Overview of Containers
- Quick history, where we are now
- Virtual machines vs. Containers
- How Docker containers work
- Why containers are compelling for customers
- Challenges
- Storage
- Storage Options for Containers
- NAS vs. SAN
- Persistent and non-persistent
- Future Considerations
- Opportunities for future work
This webcast should appeal to those interested in understanding the basics of containers and how it relates to storage used with containers. I encourage you to register today! We hope you can make it on November 17th. And if you’re interested in keeping up with all that SNIA is doing with containers, please sign up for our Containers Opt-In Email list and we’ll be sure to keep you posted.