Discover how CloudZero helps engineering and finance get on the same team — and unlock cloud cost intelligence to power cloud profitability

Learn more Arrow Arrow

Explore CloudZero

Discover the power of cloud cost intelligence

Why Change Icon
Why Change?

Give your team a better cost platform

Services Icon

Give engineering a cloud cost coach

About Icon

Learn more about CloudZero and who we are

Pricing Icon

Learn more about CloudZero's pricing

Tour Icon

Take a customized tour of CloudZero


Explore CloudZero by feature

Cost Anomaly Detection Icon
Cost Anomaly Detection

Build fast with cost guardrails

Budgeting Icon
Budgeting And Forecasting

Drive accountability and stay on budget

Discount Dashboard Icon
Discount Optimization Dashboard

Manage all your discounts in one place

Dimensions Icon
CloudZero Dimensions

Organize spend to match your business

By Use Case

Cost Per Customer
Cost Per Customer Analysis

Understand your cloud unit economics and measure cost per customer

Kubernetes Cost Analysis
Kubernetes Cost Analysis

Discover and monitor your real Kubernetes and container costs

Unit Cost Analysis
Unit Cost Analysis

Measure and monitor the unit metrics that matter most to your business

Cost Allocation
Tagging And Cost Allocation

Allocate cost and gain cost visibility even if your tagging isn’t perfect

SaaS COGS Measurement

Identify and measure your software COGS

Engineering Cost Awareness
Engineering Cost Awareness

Decentralize cost decisions to your engineering teams

Cloud Cost Optimization
Cloud Cost Optimization

Automatically identify wasted spend, then proactively build cost-effective infrastructure

By Role

All Your Cloud Spend, In One View

CloudZero ingests data from AWS, GCP, Azure, Snowflake, Kubernetes, and more

View all cost sources Arrow Arrow


Discover the best cloud cost intelligence resources

Resources Icon Resources

Browse webinars, ebooks, press releases, and other helpful resources

Blog Icon Blog

Discover the best cloud cost intelligence content

Case Study Icon Case Studies

Learn how we’ve helped happy customers like SeatGeek, Drift, Remitly, and more

Events Icon Events

Check out our best upcoming and past events

Cost Assessment Icon Free Cloud Cost Assessment

Gauge the health and maturity level of your cost management and optimization efforts


CloudZero Advisor

Compare pricing and get advice on AWS services including EC2, RDS, ElastiCache, and more

Learn more Arrow Arrow

How SeatGeek Measures Cost Per Customer

Discover how SeatGeek decoded its AWS bill and measures cost per customer

Read customer story orangearrow arrow-right

How Skyscanner Creates A Cost-Aware Culture

Learn how Skyscanner decentralized cloud cost to their engineering teams

Read customer story orangearrow arrow-right

How Malwarebytes Measures Cost Per Customer

Learn how Malwarebytes measures cloud cost per product

Read customer story orangearrow arrow-right

How Remitly Shifts Cloud Costs Left

Learn how Remitly built an engineering culture of cost autonomy

Read customer story orangearrow arrow-right

How Ninjacat Combines AWS And Snowflake Spend

Discover how Ninjacat uses cloud cost intelligence to inform business decisions

Read customer story orangearrow arrow-right

How Smartbear Uses Cloud Cost To Inform GTM Strategies

Learn Smartbear optimized engineering use and inform go-to-market strategies

Read customer story orangearrow arrow-right
arrow-left arrow-right
View all customer stories

Tracking Kubernetes Spend With CloudZero in 3 Easy Steps

If you are looking to account for your container costs and understand how your AWS Kubernetes spend maps to your workloads, this guide was made for you!

Is your current cloud cost tool giving you the cost intelligence you need?  Most tools are manual, clunky, and inexact. Discover how CloudZero takes a new  approach to organizing your cloud spend.Click here to learn more.

If you are looking to account for your container costs and understand how your AWS Kubernetes or EKS cluster spend maps onto your workloads, this guide was made for you! In this post we will walk through the steps to get up and running with CloudZero and Amazon Container Insights fast.

CloudZero has integrated with Amazon CloudWatch Container Insights for its strength as a secure and automated solution for sharing container metrics within AWS. In addition for the many customers who have already embraced Container Insights, CloudZero gained the added benefit of being able to detect and start reporting on your container costs immediately without any configuration.

"CloudZero and Amazon’s CloudWatch Container Insights made it possible for us to go further, and easily track cluster costs based on actual consumption — which let us identify the applications consuming the most resources, and prioritize our engineering resources where we could leverage them the most." Matthew Jackson, DevOps @ Drift

Step 1: Connect CloudZero to your AWS account

If you don’t already have CloudZero configured for your environment, start a free trial and create your account. Once connected we will guide you through the setup process. 

You will need access to your primary AWS payer account and an account that has a Kubernetes or EKS cluster deployed. If you are unsure about how best to get started, click on our in product chatbot and we can help you out. Once connected (via our fully automated CloudFormation template) and you have AWS billing data flowing, you are ready to install Contain Insights

Step 2: Deploy Container Insights to your Cluster

Apply Container Insights to your Cluster

The AWS getting started documentation to configure Container Insights for your Kubernetes or EKS cluster is a little verbose, here is the abridged version.

  1. Give your cluster the ability to write data to AWS CloudWatch by attaching the AWS Managed CloudWatchAgentServerPolicy to the IAM policy attached to your Kubernetes or EKS worker nodes. (more details on this and other considerations can be found here)
  2. From a terminal window that has kubectl installed and connected to your cluster, run the following command after replacing CLUSTER-NAME and CLUSTER-REGION with your actual cluster name and region:

    $ curl https://raw.githubusercontent.com/aws-samples/amazon-cloudwatch-container-insights/latest/k8s-deployment-manifest-templates/deployment-mode/daemonset/container-insights-monitoring/quickstart/cwagent-fluentd-quickstart.yaml \| sed "s//CLUSTER-NAME/;s//CLUSTER-REGION/" | kubectl apply -f -

You can also download, edit, and review this template before applying it.

This installs two Daemonsets, one for cloudwatch-agent and one for fluentd. This Daemonset is created in the amazon-cloudwatch namespace, which you can verify using the following terminal command:

$ kubectl get ds -n amazon-cloudwatch
cloudwatch-agent     2         2         2       2            2           <none>          91d
fluentd-cloudwatch   2         2         2       2            2           <none>          3d4h


Once configured, you can verify that Container Insights data is properly making its way to AWS by checking that the /aws/containerinsights/<cluster-name>/performance CloudWatch log group is present and has data, you should also start to see data populate the Container Insights dashboard.

To verify run the following AWS CLI command, which should produce a list of available performance logs:

$ aws logs describe-log-groups --log-group-name-prefix /aws/containerinsights/|grep performance

Once deployed, Container Insights gathers performance metric data (CPU/memory/network usage) and writes this information to the following CloudWatch Log groups:

CloudWatch Log Group




Performance Metrics


/aws/containerinsights/<cluster name>/application

Application Logs


/aws/containerinsights/<cluster name>/dataplane

Dataplane Logs, kubelet.service, kubeproxy.service, and docker.service


/aws/containerinsights/<cluster name>/host

Logs from Hosts/nodes.  var/log/dmesg, /var/log/secure, and /var/log/messages



Container Insights also uses the Embedded Metric Format (EMF) to create custom CloudWatch Metrics which also drive the Container Insights dashboards and are available for you to use in your own CloudWatch dashboards and alarms.

Step 3: Connecting CloudZero to Container Insights

This step is the easiest because it’s automatic and one of the reasons why this integration is so powerful. CloudZero automatically identifies when Container Insights is enabled and uses the data from the /aws/containerinsights/<cluster-name>/performance log to calculate and display your container costs with no additional configuration required on your part. CloudZero performs a very efficient CloudWatch Logs Insights query every hour to aggregate metrics with minimal impact on your AWS costs.

CloudZero will also automatically ingest past data as well so if you already have a month’s worth of Container Insights data, after connecting CloudZero to your account, you will be able to analyze a month’s worth of costs as well.


Note: Container Insight Cost Considerations 
Container Insights is an additional AWS service with its own monthly cost. You should review the pricing (see Example 7 — Container Insights for Amazon EKS and Kubernetes (k8s)) when considering its use. When using the Container Insights service you have several configuration options that we’ve included below which can lower Container Insights monthly costs if desired. Ask your CloudZero success team to help estimate those costs if you’re interested in that option.

Additional Configuration Considerations

Verify CloudZero’s IAM Permissions

Ensure CloudZero has the necessary IAM permissions to access your Container Insights CloudWatch Log stream. 

If you configured CloudZero automatically using our CloudFormation template, then the CloudZero Role has everything it needs and no further configuration is necessary. If however, you configured CloudZero manually, the easiest way to verify access is to ensure the CloudWatchLogsReadOnlyAccess policy is attached to the CloudZero cross-account access role (note: not to be confused with the CloudWatchReadOnlyAccess policy). You will need to configure the CloudZero role in each AWS account that has Container Insights enabled. 

You can also choose to restrict the CloudZero access role to only those CloudWatch Log Groups necessary using the following policy snippet:

  "Sid": "ContainerInsightsMonitoring",
"Effect": "Allow",
  "Action": [
  "Resource": "arn:aws:logs:*:*:log-group:/aws/containerinsights/*"

Note: You will need to configure the CloudZero cross-account access role if you manually connected your accounts or connected them using CloudFormation before June 1. 

You can reconfigure the CloudZero IAM Role manually by editing the CloudZero resource role or by removing the CloudZero CloudFormation stack and re-connecting your account via the CloudZero account connection page.

Tune Container Insights for your Environment

1: The log groups created by Container Insights are created without any expiration set. While the log storage costs are likely small, you should always consider how long you want to retain any data. To configure the log retention days you have two options:

  1. Set the expiration in fluentd.yml by adding retention_in_days and add "logs:PutRetentionPolicy" permissions to your IAM worker node policy (this permission is not included in the managed policy you attached earlier). 
  2. Manually change the retention settings on the log group itself

2. Container Insights creates custom cloudwatch metrics via EMF which depending on the size and scale of your clusters can be cost-prohibitive.

For example, a small cluster might look something like this:
24 cluster metrics + (5 nodes or EC2 instances * 8 node metrics) + (10 unique pod names * 9 pod metrics * 5 namespace) + (5 unique service names * 6 service metrics * 5 namespace) + (1 unique namespace * 6 namespace metrics) 

This gives us the equation:
 24 + (5 * 8) + (10 * 9 * 5) + (5 * 6 * 5) + (5 * 6) = 694 metrics

Which at $0.30 per metric for first 10,000 metrics, this comes to $208.20 per month

If the metrics will be too costly for your environment, you can turn these metrics off using a CloudZero provided configuration that disables the CloudWatch EMF metrics, more information on this can be found here

NOTE: Doing this will completely disable the Container Insights Dashboard and is only recommended if you find the custom metrics to be too costly. These metrics are not required for CloudZero to monitor your costs.

3: If you already have centralized logging solution configured for your cluster you can remove the application logging (fluentd-cloudwatch) component by running the following command: 

$ kubectl delete daemonset fluentd-cloudwatch -n amazon-cloudwatch


The AWS CloudWatch Agent DaemonSet is not loading

  • First check that you have enough capacity across your nodes, if either the CloudWatch or FluentD daemonsets are not deploying, you might be out of resources.
  • Next, check your taints and tolerations. Many organizations will taint their more ephemeral auto-scaling node groups (for example node groups that use Spot compute) to prevent kube-system and other similar components from running. If so you will need to configure a toleration to allow the CloudWatch agent to run, for example, if you have tainted your node group as follows:

$ kubectl taint nodes MyNodeGroup key=value:NoSystem

Then you will need to add the following toleration to allow the CloudWatch agent to run:

- key: "key"
  operator: "Exists"
  effect: "NoSystem"

No data or partial data is flowing to CloudWatch

This is almost always due to a misconfiguration of your node groups IAM policy. Ensure you have applied the CloudWatchAgentServerPolicy to your policies and ensure you have covered all of your node groups. It is typical to have multiple node groups, either managed by EKS, self-managed, or completely custom-configured. When in doubt get a list of your nodes using kubectl get nodes and verify each one has the right IAM policy attached.

Somethings not working, how can I start from scratch again?

Removing CloudZero: 

Delete the CloudZero CloudFormation Stack to remove CloudZero’s access from your environment. 

Note, CloudZero may have created the AWS Cost and Usage report (CUR) during setup which writes data to S3. Removing the CloudZero CloudFormation stack will not remove the CUR configuration or the S3 bucket.

Removing Container Insights: 

Run the same command as before, this time using delete instead of apply, don’t forget to replace CLUSTER-NAME and CLUSTER-REGION with your actual cluster name and region:

curl https://raw.githubusercontent.com/aws-samples/amazon-cloudwatch-container-insights/latest/k8s-deployment-manifest-templates/deployment-mode/daemonset/container-insights-monitoring/quickstart/cwagent-fluentd-quickstart.yaml | sed "s//CLUSTER-NAME/;s//CLUSTER-REGION/" | kubectl delete -f -

Wrapping up: Fully Automated Container Cost Allocation

By using CloudZero to combine the metrics from Container Insights and AWS Cost and Usage information customers can automatically allocate costs to the workloads being orchestrated by Kubernetes. CloudZero calculates the cost of each pod within the architecture and then re-aggregates them to attribute costs to other hierarchical Kubernetes concepts, like namespaces and clusters.

Container Cost Nirvana: Mapping your AWS and container costs to your business

Cost is calculated based on the cost of the EC2 instance - which represents the cost of a node in the Kubernetes system - combined with pod-level CPU and memory utilization. This allows us to assign a portion of the node’s total cost to the pod. This is handled automatically in the CloudZero platform; there is no need for manual allocation rules.

Generally speaking, this proportional algorithm works across a broad range of EC2 instance types, including those with SSD, NVMe SSD, GPU cores, GPU memory, and networking enhancements.

The final result is a new way to explore your container costs over time by cluster, pod or namespace using CloudZero. For example, we can use CloudZero to take a look at one of our clusters and see how it’s costs decrease as we scale down the cluster. 

We can also observe the cost to run Container Insights itself on the cluster, which in this example, is only about $0.01 a day.

A better view into EC2 and Kubernetes spend with CloudZero

For Drift, this has empowered them to go beyond their cost savings goals and start to think proactively about optimizing their cloud computing spend.

“Now, in addition to having hit our most recent cost goals, we're much more consistently able to identify when applications are consuming more of the shared cluster resources in advance, and can give timely feedback to their developers before it becomes a problem.”  

Matthew Jackson, DevOps @ Drift

Erik Peterson

Author: Erik Peterson

Erik Peterson is the co-founder and CTO of CloudZero, is a software startup veteran, and has been building and securing cloud software for over a decade. Erik is a frequent speaker on FinOps, cloud economics, DevOps, and security.


Join thousands of engineers who already receive the best AWS and cloud cost intelligence content.