AWS price and cost — analysing and monitoring

You are up and running on AWS, and the bills are trickling in (and money out). Now it is time to start being proactive about spending, as a bad design or bugs in your programs could lead to significant increase in spend. To be able to have a proactive approach to billing you need to monitor and analyse your usage.

Analysing your usage with Cost Explorer (CE)

The billing dashboard (which got a new design in June 2022) should be your first place to look for up to date information on spend. Here you will find an overview and access to details in your bill.

When you need to dive into more details, Cost Explorer is the place to go next. In Cost Explorer you have some pre-build views of your spend, and you can also create and save your own views.

Cost explorer is an interactive tool where you can drill down into extreme details, split cost into groups and include/exclude services and other fees.

When you are analysing your cost one of the important features is the “Group by” feature. Note however that you can only group by one dimension.

The filters on the right is a very good tool to exclude things you do not need to look into, or include the thing you like to investigate. There are some filters I recommend you add to avoid noise and clutter in your report if you are investigating regular service usage.

Have a look at the following graph — what is going on here?

The high “cost” on August 1st is actually tax, which is applied after the previous month has been calculated. This is irrelevant in most cases. The missing part in September is because we got some service-credits being subtracted from our usage so the total is 0 (zero). If you are using the cost explorer to investigate spend related to usage, use the filter and only include “Usage” to avoid random financial transactions. Other random transactions might be discounts, one-time-fees, up front one-time payments, etc.

If you are not using the group by function, you can get a forecast for the upcoming days or months. This forecast is based on previous usage and is also depending on your filter settings of course, so remember to remove irregular transactions.

Cost explorer also lets you save reports you have created, and it also has an API if you like to get data programmatically.

Analysing your invoice with Cost and Usage Report (CUR)

If you like to analyse your invoice in detail or understand all aspects and details about your spend and billing, you can get the underlying billing data delivered into an S3-bucket at regular intervals. Have in mind that this can be a huge amount of data as you can get one item (record) of information per resource per hour. You can get it in Apache Parquet format, or CSV (ZIP/gzip).

This feature is for more advanced users of AWS, but is sometimes nice to look into to really understand how billing is done if you want to dive deep into the details.

Monitoring and alarms with budgets

Another important aspect is to monitor for costs that you really do not need. This can be related to services you are not using, regions you are not using, or simply someone else obtaining access to your account and spinning up some workloads.

Budgets are a nice way to both get alerts about your current spending, as well as the forecasted total in the current month. There are different types of budget, and I will focus on “Cost budget” here. A budget can be daily, monthly, quarterly or annually. Personally I prefer monthly, as it aligns well with the AWS invoices and are easy to track and relate to.

When you set up a budget you can set up the same fixed amount for every period (in my examples I use monthly), you can add different numbers to your budget from month to month, or the budget can auto adjust based on previous period(s).

If you like, you can create a filter and only set up a budget for some services, regions, etc. Then under “advanced options” you can add/remove different charges. As mentioned earlier you would normally not create a budget looking at for example discount, taxes and credits as they will make the budgeting harder.

Next step is to add alarm(s). Then you would need to think about when and how to be notified. Add a trigger on actual or forecasted spend. In many cases I prefer forecasted to be notified as early as possible. AWS is making a forecast when the month starts and adjusts it continuously.

You can add multiple alarms, so if you also like to get notified when the actual spend is for example 100% of the budget, you can easily add that too. Each alarm can have a separate notification-list, and notifications can be sent via email, to SNS and to the Amazon Chatbot (Slack and Amazon Chime). The more advanced users can also stop an EC2 or RDS-instance when an alarm is triggered.

In the example above I made a budget to keep track of cost on our production workload, but you can also use budgets to monitor for usage you do not want to happen. In my case I am not using regions outside the US and EU, so I create an almost zero budget for the regions I am not using. In that way I can get a notification if someone is starting to use services in those regions, intentionally or not.

Cost Anomaly detection

When your usage grows with multiple services and perhaps in multiple regions it is very hard to spot changes that are relatively small compared to your current spend by using the budgets and other reports. This is when Cost Anomaly detection comes into play. This feature will spot unusual changes in spend compared to your historical spend and alert you when it happens.

You can find Cost Anomaly detection in the Cost Management Console where you also have a dashboard listing the activities found.

You must first create a monitor and there are four different options. Let us start with “AWS services” which is the easiest one. After that you set up where your alerts should go. Daily and weekly alerts are sent via email, while individual alerts are published on SNS. My suggestion is to start with daily alerts to see how it works.

It is also possible to set up SNS as a subscriber if you like to automate the actions yourself.

Summary

It is important to keep an eye on the cost in your AWS-account. In this part we looked at Cost explorer which is a good tool for drilling down into the cost. Budgets and Cost Anomaly detection are tools for getting alerted when you cross defined thresholds. When looking at cost, remember to filter out factors that are not directly related to your running services like tax, one-time-fees, AWS credits etc.

The content in this blog-post series “AWS price and cost” is based on the main topics I am facing at AWS customer sites and doing AWS training.

Next blogposts will cover more advanced topics and tools related to Price and Cost.