Addressing Cloud Security Threats with Standards

In a recent SNIA webinar, Cloud Standards: What They Are, Why You Should Care, the SNIA Cloud Storage Technologies Initiative (CSTI) highlighted some of the key cloud computing standards being developed and published by the ISO/IEC JTC 1/SC 38 (Cloud Computing and Distributed Platforms) and SC 27 (Information security, cybersecurity and privacy protection) standards committees. While ISO and IEC are not the only organizations producing cloud computing standards and specifications (e.g., ITU-T, OASIS, NIST, ENISA, SNIA, etc.), their standards, sometime developed jointly with ITU-T, can play a role in addressing WTO Agreement on Technical Barriers to Trade (TBT) issues. More importantly, they provide a baseline of cloud terminology, concepts, guidance/requirements, and expectations that are recognized internationally. Read More

Pay Attention to These Cloud Standards

What’s going on in the world of cloud standards? Since the initial publication of the National Institute of Standards and Technology (NIST) definition of cloud computing in NIST SP 800-145 in 2011, international standards development organizations (SDOs) have sought to refine and expand the cloud computing landscape. On February 13, 2020 at our next live SNIA Cloud Storage Technologies Initiative webcast “Cloud Standards: What They Are, Why You Should Care” we will dive into the cloud standards worth noting as Eric Hibbard, standards expert and ISO editor, will discuss: Read More

Hyperscalers Take on NVMe™ Cloud Storage Questions

Our recent webcast on how Hyperscalers, Facebook and Microsoft are working together to merge their SSD drive requirements generated a lot of interesting questions. If you missed “How Facebook & Microsoft Leverage NVMe Cloud Storage” you can watch it on-demand. As promised at our live event. Here are answers to the questions we received. Q. How does Facebook or Microsoft see Zoned Name Spaces being used? Read More

How Facebook & Microsoft Leverage NVMe Cloud Storage

What do Hyperscalers like Facebook and Microsoft have in common? Find out in our next SNIA Networking Storage Forum (NSF) webcast, How Facebook and Microsoft Leverage NVMe Cloud Storage, on November 19, 2019 where you’ll hear how these cloud market leaders are using NVMe SSDs in their architectures. Our expert presenters, Ross Stenfort, Hardware System Engineer at Facebook and Lee Prewitt, Principal Hardware Program Manager, Azure CSI at Microsoft, will provide a close up look into their application requirements and challenges, why they chose NVMe flash for storage, and how they are successfully deploying NVMe to fuel their businesses. You’ll learn: Read More

Answering Your Kubernetes Storage Questions

Our recent SNIA Cloud Storage Technologies Initiative (CSTI) Kubernetes in the Cloud series generated a lot of interest, but also more than a few questions. The interest is a great indicator of Kubernetes rising profile in the world of computing. Following the third episode in the series, we’ve chosen a few questions that might help to better explain (or bring additional context to) our presentation. This post is our answer to your very important questions. If you’re new to this webcast series about running Kubernetes in the cloud, you can catch the three parts here: The rest of this article includes your top questions from, and our answers to, Part 3: Q. What databases are best suited to run on Kubernetes? Read More

Answering Your Kubernetes Storage Questions

Our recent SNIA Cloud Storage Technologies Initiative (CSTI) Kubernetes in the Cloud series generated a lot of interest, but also more than a few questions. The interest is a great indicator of Kubernetes rising profile in the world of computing.

Following the third episode in the series, we’ve chosen a few questions that might help to better explain (or bring additional context to) our presentation. This post is our answer to your very important questions.

If you’re new to this webcast series about running Kubernetes in the cloud, you can catch the three parts here:

The rest of this article includes your top questions from, and our answers to, Part 3:

Q. What databases are best suited to run on Kubernetes?

A. The databases better suited are the ones which have leaned into the container revolution (which has been happening over the past five years). Typically, those databases have a native clustering capability. For example, CockroachDB has great documentation and examples available showing how to set it up on Kubernetes.

On the other hand, Vitess provides clustering capabilities on top of MariaDB or MySQL to better enable them to run on Kubernetes. It has been accepted as an incubation project by the Cloud Native Computing Foundation (CNCF), so there is a lot of development expertise behind it to ensure it runs smoothly on Kubernetes.

Traditional relational databases like Postgres or single-node databases like Neo4j are fine for Kubernetes. The big caveat is that these are not designed to easily scale. That means the responsibility is on you to understand the limits of that DB, and any services it might support. Scaling these pre-cloud solutions tends to require sharding, or other similarly tricky techniques.

As long as you maintain a comfortable distance from the point where you’d need to scale a pre-cloud DB, you should be fine.

Q. When is it appropriate to run Kubernetes containerized applications on-premises versus in the cloud?

A. In the cloud you tend to benefit most from the managed service or elasticity. All of the major cloud providers have database offerings which run in Kubernetes environments. For example, Amazon Aurora, Google Cloud SQL, and Microsoft Azure DB. These offerings can be appropriate if you are a smaller shop without a lot of database architects.

The decision to run on-premises is usually dictated by external factors. Regulatory requirements like GDPR, country requirements, or customer requirements may require you to run on-premises. This is often where the concept of data gravity comes into effect – it is easier to move your compute to your data, versus moving your data to your compute.

This is one of the reasons why Kubernetes is popular. Wherever your data lives, you’ll find you can bring your compute closer (with minimal modifications), thanks to the consistency provided by Kubernetes and containers.

Q. Why did the third presentation feature old quotes about the challenges of Kubernetes and databases?

A. Part of that is simply my (the presenter’s) personal bias, and I apologize. I was primarily interested in finding a quote by recognizable brands. Kelsey Hightower and Uber are both highly visible in the container community, so the immediate interest was in finding the right words from them.

That does raise a relevant question, though. Has something changed recently? Are containers and Kubernetes becoming a better fit for stateful workloads? The answers is yes and no.

Running stateful workloads on Kubernetes has remained about the same for a while now. As an example, a text search for “stateful” on the 1.16 changelog returns no results. If a change was made for StatefulSets, or “Stateful workloads” we would hope to see a result. Of course, free text search is never quite that easy. Additional searches for “persist” show only results for CSI and ingress related changes.

The point is that stateful workloads are running the same way now, as they were with the prior versions. Part of the reason for the lack of change, is that stateful workloads are difficult to generalize. In the language of Fred Brooks, we could say that stateful workloads are essentially complex. In other words, the difficulty of generalizing is unavoidable and it must be addressed with a complex solution.

Solutions like operators are tackling some of this complexity, and some changes to storage components indicates progress is occurring elsewhere. A good example is in the 1.14 release (March 2019), where local persistent volumes graduated to general availability (GA). That’s a nice resolution to this blog from 2016, which said,

“Currently, we recommend using StatefulSets with remote storage. Therefore, you must be ready to tolerate the performance implications of network attached storage. Even with storage optimized instances, you won’t likely realize the same performance as locally attached, solid state storage media.”

Local persistent volumes fix some of the performance concerns around running stateful workloads on Kubernetes. Has Kubernetes made progress? Undeniably. Are stateful workloads still difficult? Absolutely.

Transitioning databases and complex, or stateful work to Kubernetes still requires a steep learning curve. Running a database through your current setup requires no additional knowledge. Your current database works, and you and your team already know everything you need to keep that going.

Running stateful applications on Kubernetes requires you attain knowledge of init containers, persistent volumes (PVs), persistent volume claims (PVCs), storage classes, service accounts, services, pods, config maps, and more. This large learning requirement has remained one of the biggest challenges. That means Kelsey and Uber’s cautionary notes remain relevant.

It’s a big undertaking to run stateful workloads on Kubernetes. It can be very rewarding, but it also comes with a large cost.

Q. If Kubernetes introduces so much complexity, why should we use it?

A. One of the main points of part three of our Kubernetes in the Cloud series, is that you can pick and choose. Kubernetes is flexible, and if you find running projects outside of it easier, you still have that option. Kubernetes is designed so that it’s easy to mix and match with other solutions.

Aside from the steep learning curve, it can seem like there are a number of other major challenges that come with Kubernetes. These challenges are often due to other design choices, like a move towards microservices, infrastructure as code, or etc. These other philosophies are shifts in perspective, that change how we and our teams have to think about infrastructure.

As an example, James Lewis and Martin Fowler note how a shift towards microservices will complicate storage:

“As well as decentralizing decisions about conceptual models, microservices also decentralize data storage decisions. While monolithic applications prefer a single logical database for persistent data, enterprises often prefer a single database across a range of applications – many of these decisions driven through vendor’s commercial models around licensing. Microservices prefer letting each service manage its own database, either different instances of the same database technology, or entirely different database systems – an approach called Polyglot Persistence.”

Failing to move from a single enormous database to a unique datastore per service can lead to a distributed monolith. That is, an architecture which looks superficially like microservices, but furtively still contains many of the problems of a monolithic architecture.

Kubernetes and containers align well with newer cloud native philosophies like microservices. It’s no surprise then, that a project to move towards microservices will often involve Kubernetes. A lot of the apparent complexity of Kubernetes actually happens to be due to these accompanying philosophies. They’re often paired with Kubernetes, and anytime we stumble over a rough area, it can be easy to just blame Kubernetes for the issue.

Kubernetes and these newer philosophies are popular for a number of reasons. They’ve been proven to work at mega-scales, at companies like Google and Netflix (eg Borg and the microservice re-architecting on AWS). When done right, development teams also seem more productive.

If you are working at a larger scale and struggling, this probably sounds great. On the other hand, if you have yet to feel the pain of scaling, all of this Kubernetes and microservices stuff might seem a bit silly. There are a number of good reasons to avoid Kubernetes. There are also a number of great reasons to embrace it. We should be mindful that the value of Kubernetes and the associated philosophies is very dependent on where your business is on “feeling the pain of scaling.”

Q. Where can I learn more about Kubernetes?

A. Glad you asked! We provided more than 25 links to useful and informative resources during our live webcast. You can access all of them here.

Conclusion

Thanks for sending in your questions. In addition to the each of the webcasts being available on the SNIAVideo YouTube channel, you can also download PDFs of the presentations and check out the previous Q&A blogs from this series to learn more.

Kubernetes Links & Resources to Keep You in the Know

Our recent SNIA CSTI webcast, “Kubernetes in the Cloud (Part 3): (Almost) Everything You Need to Know about Stateful Workloads” offered a wealth of insight on how to address the challenges of running stateful workloads in Kubernetes. This webcast was the third installment of our Kubernetes in the Cloud webcast series and it is now available on-demand as are “Kubernetes in the Cloud (Part 1)” and “Kubernetes in the Cloud (Part 2).” Our expert presenters, Paul Burt and Ingo Fuchs, have provided additional resources to help keep you in the know on Kubernetes. Here they all are: Read More

Kubernetes Links & Resources to Keep You in the Know

Our recent SNIA CSTI webcast, “Kubernetes in the Cloud (Part 3): (Almost) Everything You Need to Know about Stateful Workloads” offered a wealth of insight on how to address the challenges of running stateful workloads in Kubernetes. This webcast was the third installment of our Kubernetes in the Cloud webcast series and it is now available on-demand as are “Kubernetes in the Cloud (Part 1)” and “Kubernetes in the Cloud (Part 2).”

Our expert presenters, Paul Burt and Ingo Fuchs, have provided additional resources to help keep you in the know on Kubernetes. Here they all are:

If you know of some Kubernetes resources to share, please comment on this blog and we’ll add them to our list.

Kubernetes in the Cloud Q&A

Kubernetes is a hot topic these days, generating lots of interest and questions. The goal of our SNIA Cloud Storage Technologies Initiative Kubernetes in the Cloud webcast series is to cut through the hype and provide a vendor neutral look at what Kubernetes is and how it is being used. Our most recent webcast, Kubernetes in the Cloud (Part 2), generated some interesting questions. Here are answers from our expert presenters. Q. If I’m running my Kubernetes infrastructure at a cloud service provider, do I need CSI support by the cloud provider? If this is not available, I will need a virtual storage array that provides CSI leveraging the underlying cloud storage. Do you know whether there are solutions on the market that I can deploy as a virtual machine at my cloud provider? A. Current solutions using the CSI interface for public cloud storage are not available at this point. It will be up to the cloud provider to decide whether to support those interfaces to their storage layers. Q. Does each pod run on one CPU core? I am trying to understand how to size the server configuration? A. Containers use Linux cgroups to limit the amount of CPU and memory a container can consume and this is exposed in Kubernetes as limits that you can set. Read More

Kubernetes in the Cloud Q&A

Kubernetes is a hot topic these days, generating lots of interest and questions. The goal of our SNIA Cloud Storage Technologies Initiative Kubernetes in the Cloud webcast series is to cut through the hype and provide a vendor neutral look at what Kubernetes is and how it is being used. Our most recent webcast, Kubernetes in the Cloud (Part 2), generated some interesting questions. Here are answers from our expert presenters.

Q. If I’m running my Kubernetes infrastructure at a cloud service provider, do I need CSI support by the cloud provider? If this is not available, I will need a virtual storage array that provides CSI leveraging the underlying cloud storage. Do you know whether there are solutions on the market that I can deploy as a virtual machine at my cloud provider?

A. Current solutions using the CSI interface for public cloud storage are not available at this point. It will be up to the cloud provider to decide whether to support those interfaces to their storage layers.

Q. Does each pod run on one CPU core? I am trying to understand how to size the server configuration?

A. Containers use Linux cgroups to limit the amount of CPU and memory a container can consume and this is exposed in Kubernetes as limits that you can set.

Q. In today’s environment for Kubernetes Flex storage, what is the suggested process to “backup” these stateful PVs or is that not necessary anymore?

A. Backups are still as important with containers as they are today with traditional applications. There are many different approaches available to backup containers: storage snapshots via native storage interfaces, deployment of backup clients in containers, application level backups, etc.

Q. You mentioned Kubernetes a lot – but what is the status of native Docker CSI support? Can I use CSI for usage within Docker without deploying Kubernetes? And if yes: Can I get rid of the need for docker volume drivers then?

A. Docker universal control point (UCP) support of CSI is currently in beta. Once UCP is generally available, we’ll be able to answer your question in more detail.

Interested in more Kubernetes in the Cloud information? Watch our first installment Kubernetes in the Cloud (Part 1) on-demand at your convenience and sign up for our next webcast, Kubernetes in the Cloud (Part 3): Stateful Workloads which will be live on August 20, 2019 and available on demand after that.