nCast: The Cloud Optimization Podcast from nOps – Details, episodes & analysis

Karpenter has achieved GA and is disrupted the autoscaling game, with data pointing to accelerated adoption. 

Today, Josh Cypher, DevOps leader at Sonos, joins us to talk about some unexpected byproducts of adopting Karpenter at Sonos. Josh dives into his favorite features and efficiency gains, from node consolidation to better disruption controls.

The public cloud bill is a massive operational expense for tech organizations, yet tracking the success of optimization efforts often frustrates engineers. Josh and James explore these challenges and how to address them effectively.

As big-time Spot adopters, Sonos has unlocked impressive savings (50%?!) by focusing on high-impact, low-overhead strategies. Josh explains how visibility brought them quick wins and paved the way for further optimization across Sonos’s infrastructure.

Plus, Josh and James preview what they’re looking forward to at KubeCon, including key conversations on Kubernetes, AI, and cloud sustainability.

Dr. Haoran Qiu, a fresh PhD from the University of Illinois Urbana-Champaign, joins our host James Wilson, VP of Engineering at nOps. They’re diving into multidimensional autoscaling, an area in which Haoran’s pioneering research is making waves in the Kubernetes community. 

Some workloads work better with Horizontal Pod Autoscaler (HPA), others with Vertical Pod Autoscaler (VPA).  Running them together can create conflicts, but using only one limits efficiency gains. A Multidimensional Pod Autoscaler solves this dilemma by combining the benefits of both VPA and HPA to dynamically adjust both the number and size of pods.

But is MPA poised to redefine resource optimization? What problems does it solve, and what fresh complexities are involved in its implementation?

Haoran and James dig into these questions while debating traditional heuristic versus Machine Learning approaches, industry versus academia, and other hot topics in Kubernetes.

Listen now to discover if MPA is the holy grail of cloud optimization as we discuss the evolution of autoscaling technologies and their impact on cost, sustainability, and developer experience.

Chapters:

0:00 - 2:20: Haoran Chu and the state of cloud resource management

2:20-6:00: Historical evolution of autoscaling

6:01 - 10:45: HPA, VPA and Multidimensional Autoscaling

10:46 - 18:50: Challenges of MPA: heuristics versus machine learning

18:51 - 24:20: How to quantify excess capacity?

24:21 - 32:16: The state of ML in autoscaling

32:16 - 37:37: Operationalizing ML in production environments
37:37 - 42:01: The near-term future of autoscaling

In today’s episode, we’re joined by Tim Cassell, VP of Product at nOps, to talk about the future of FinOps.  

What are the biggest obstacles that engineering teams encounter today? How are AI and ML going to influence cost allocation and forecasting, automation, and day-to-day operations? And what does the evolving Spot market mean for teams? 

We debate these questions and share hot takes on ChatGPT, ClickOps, shifting left in the development process, and other trends we see (or don’t see) on the horizon. What's the moonshot over the next few years? Some ideas may seem far-fetched today but could be the standard sooner than you think.

Karpenter is a powerful tool for optimizing EKS workloads and improving compute utilization in real-time. But what exactly is Karpenter, and how does it compare to the more traditional Cluster Autoscaler?

We explore the lay of the land and the limitations of the Cluster Autoscaler Spot support and discuss some of the primary problems that EKS users face today. These include custom configurations for different workload types, adopting Spot, optimizing EC2 usage through node packing and instance selection, and addressing latency issues.

Fortunately, Karpenter makes it possible for users to enjoy improved availability, lower compute costs, and minimized operational overhead.  But how easy is it to migrate to Karpenter? Our experts share their best practices and tips for a successful migration, including the use of Blueprints and EKS add-ons.

Don't miss out on this informative episode that's packed with insights and practical tips from our experts. Tune in now to learn more about Karpenter and how it can benefit your EKS workload optimization needs.

Engineers are good at building and innovating, but keeping track of AWS pricing plans is a whole different story. And should even they have to? We’re joined by JT Giri and Andrew Lewis to discuss why long-term spending commitments are so challenging for engineers.

Commitments can last 2-3 years, and they’re bucketed by the hour. We’ll dive into common misconceptions, pitfalls, and all the ways it can get complicated. Ever found yourself locked into openSearch when Databricks and Druid comes along? We’ll share advice for what you should do and how to avoid backing yourself into a corner. 

Plus, we’ll talk through what to do when the CFO’s yelling and management comes along to demands cuts. What tools do you have at your disposal to do that? We’ll share some battle stories of what we did when facing the heat, what went right, what went wrong, and what you can learn from it. 

And the essential question: how do you free engineers from the shackles of all this complexity so they can make great things? Spoiler alert: there’s a game-changer in the mix. 

We get into advanced tooling like RI automation, Spot automation, variable workloads and hacks for doing it easy.

Let's dive into everything Kubernetes with Andrew Lewis, Principal Cloud Optimization Architect, alongside nCast's host, James Wilson, VP of Engineering at nOps.

We’ll journey from the stone age of manual config mayhem with Packer, Chef, and Vagrant… until along came Docker and the dawn of the Container Orchestration Wars. 

Our speakers unravel the past, present and future of EKS cost optimization. When engineers are measured by the number of 99.99s that they deliver, how do you make cost optimization less punitive? Why is everyone so afraid of Spot? 

Find out which tools and strategies can come to the rescue, from Kubecost to binpacking to VPA hacks. But beware the hidden traps — like when the most optimal node sizing decision unwittingly explodes your Datadog bill.

Tune in to hear straight from the engineering leaders and luminaries out there in the field, in the mines, doing the work and innovating with EKS.

Tech thought leader and host of the Kubernetes Unpacked podcast Kristina Devochko joins nCast today to talk all things cloud cost optimization, Kubernetes and green tech. 

We start by talking about the fact that many companies aren’t even using HALF of their compute resources. But does slashing your AWS bill necessarily mean that you’re saving the plant? We delve into cost optimization and how it aligns (or not) with sustainability. 

Kristina shares her insights on measuring your cloud carbon footprint and the tools you need (KEDA, Karpenter, Kepler) to increase cloud sustainability. We discuss key practical ways to get started cutting unnecessary cloud waste, from eliminating orphaned resources to scheduling during off hours. 

Plus, we're revealing how nOps has managed to run our production on Spot instances — talk about recycling! 

0:00 - 1:09: Introduction

1:10 - 4:20: Sustainability at Kubecon Europe and other recent events

4:21 - 9:31: Is cost optimization the same as sustainability?

9:32 - 12:53: Green data centers and your carbon footprint

12:54 - 15:21: Portability and the downsides of over-committing to pricing plans

15:22 - 19:51: Measuring your organization’s cloud sustainability

19:53 - 26:51: KEDA, Karpenter, Kepler and the tools you need

26:52 - 31:12: Leveraging available Spot capacity and choosing instances

31:13 - 37:15: Running production environments on Spot 

37:15 - 44:46: Continual rightsizing and automated tools

44:47 - 48:18: Carbon-efficient Karpenter scaling

Show notes

• GitHub issue for proposal of carbon-efficient design to Karpenter that needs some community support

• Kepler project

• Carbon-aware KEDA operator

• Cloud Carbon Footprint open source tool

• BoaviztAPI open source API for environmental impacts of ICT

• APIs that provide electricity data, data on carbon emissions and electricity sources: https://app.electricitymaps.com and https://watttime.org

• CNCF TAG Environmental Sustainability

• Contact Kristina Devochko

• Kristina Devochko’s Tech blog
  

Today we’re joined by Wade Piehl, Senior FinOps Success Manager on the Optics Team at AWS, to discuss all things On Demand, Reserved, and Spot. 

Good purchasing decisions have an enormous impact on your bottom line — but how do you know what and how much to buy? We walk through the step-by-step decision-making process and the most common pitfalls that can cost you.

Wade shares all the practical advice you need to decide between Reserved Instances vs. Savings Plans. Find out how the equation changes from compute spend to database, storage and networking. We’ll also dig into strategies like layering purchase plans, leveraging Spot, and other methods of maximizing your savings.

Today we’re joined by Savanna Jensen, Senior FinOps Success Manager on the Optics Team at AWS, to discuss how to implement a Cloud FinOps automation strategy. The less amount of “people time” you can dedicate to cloud management and the more automation you can bake into the system, the easier it will be — but what are the right tools to use?

We start out by tapping Savanna’s insider knowledge on the latest and greatest AWS Cost Management tools. Get the latest on the shiny new updates to Cost Explorer that just launched. Plus, pro tips on the best filters and features to use for various use cases when it comes to the Cost and Usage Report (CUR), Cost Explorer, and QuickSight.

We dive into Engineering vs. FinOps perspectives and frustrations. How can you orchestrate a culture where cost optimization feels motivating rather than punitive to Engineers (recognition, career rewards, gamification…)? How do you troubleshoot if you’re a FinOps leader seeing zero traction on your initiatives?

Hear real-world battle stories about organizations tackling cost management challenges — and the takeaways for achieving true visibility and control over cloud costs. And stay tuned next week for Part 2 with the AWS Optics team.

In Episode 9, we’re joined by AI expert Marcos Heidemann to pull the curtain back on GenAI and whether the hype means we’re at the cusp of a massive transformation. Will it automate away our jobs in a dystopian future, or is it just glorified autocomplete?

Our panelists do a technical dive into new technologies like ChatGPT and LangChain. How does GenAI think, and what does that mean for building commercial products on it? 

We’ll give you a sneak peak into nOps’s firsthand experience using GenAI to solve real-world cost optimization problems in novel ways that have never been tried before — and what we’re currently in the process of rolling out. 

The conversation wraps up with hot takes on the current state of AI technology and where it’s going. What do ChatGPT 5, 6, 7, and their competitors look like? What’s the moonshot for GenAI over the next 2-5 years?

Listen now to find out.