CloudWatch Custom Metrics

Standard/default CloudWatch metrics are ones that AWS makes available out-of-the-box: for EC2 instances, these are metrics related to your CPU and network utilization. However, to get better insight into your resources, you can enable custom metrics that will give you insight into, for example, items such as your EC2 instance?s memory utilization and memory used/available. Additionally, CloudCheckr can ingest all of these CloudWatch metrics and store them for you for the life of your account, maintaining a searchable record of all your CloudWatch metrics.

How Do I Get Custom Metric Data Into CloudCheckr?

For most custom metrics, you will need to turn on Detailed Monitoring within AWS CloudWatch and then set up CloudCheckr to look for the exact metrics you wish to have.

For custom metrics related to EC2 instance memory, you will need to install an agent on the EC2 instance to generate the memory metrics. Once your resource is generating the desired metric, the data is sent to CloudWatch and can next be ingested by CloudCheckr where it is used for various reporting and alerting features.

How Do I Add Default Custom Metrics from CloudWatch?

In addition to metrics that need a separate agent installed on a resource, AWS has a set of default custom metrics that offer extra insight with relatively little setup.

AWS has a list of custom metrics that can be used for EC2 instances. For our example, we will collect CloudWatch data for the custom metric called CPUCreditUsage.

  1. Turn on detailed monitoring for your specific instance within AWS Console.

  2. Make sure you have your CloudWatch permissions up-to-date within CloudCheckr. If not, see the Complete IAM Policy topic to update your account.

    If your CloudCheckr yellow warning notification icon has a number next to it, you'll know there are certain permissions your account needs to have updated.

  3. From the left navigation pane, choose Utilization > Alerts > Manage Custom Metrics > Manage Custom Metrics.
  4. Select the Custom Metrics tab.

  5. Add a Namespace, Metric, and Dimension.

    These entries must exactly match the entries in AWS. In our example, use the following:

    • Namespace: AWS/EC2
    • Metric: CPUCreditBalance
    • Dimension: *
      Using an asterisk allows CloudCheckr to input all dimensions for this metric.

  6. Click the Add Custom Metric button.

    Your Custom Metric will be added and you will see that its status switch is set to ON.

    Your CloudWatch Custom Metrics will populate within the next 24 hours, depending on when your last CloudCheckr report update occurred.

    Alternatively, you can manually trigger a report update by clicking on the Reports Updated notification button at the top right of your screen. Depending on the size of your account, the update will take a few hours.

How Do I Add EC2 Memory Metrics into CloudWatch?

To get custom CloudWatch memory metrics, you will need to install an agent/monitoring script on your EC2 instance.
The role of the agent is to specifically monitor memory—this is a functionality that is not available by default.
This agent will send send the data to CloudWatch, and once within CloudWatch, the data can be ingested by CloudCheckr.

The general workflow for custom metrics that need an installed agent is as follows:

The CloudWatch Custom Metrics that can report on instance memory are:
  • System/Linux MemoryAvailable
  • System/Linux MemoryUsed
  • System/Linux MemoryUtilization
  • System/Windows MemoryAvailable
  • System/Windows MemoryUsed
  • System/Windows MemoryUtilization

Part 1

Setup for Linux Instances: You will need to install monitoring scripts on your Linux EC2 instances, which will send the memory metrics to CloudWatch.
AWS has documentation on this setup.

Setup for Windows Instances: You will need to use the EC2Config service to export performance counter data to CloudWatch. You will want to send the available memory data.
AWS has documentation on this setup.

Part 2

  1. After you have configured your instances to export memory metrics to CloudWatch, go to the AWS Console and access the CloudWatch section.
  2. Choose EC2 and verify that you are seeing your new memory metrics.

Part 3

  1. From the left navigation pane, choose Utilization > Manage Custom Metrics.

    You will be taken to the default tab EC2 Memory Metrics.
  2. Turn on the Memory metrics that you have just enabled within AWS Console. Linux and Windows memory metrics each have three selections.
  3. CloudCheckr will begin to download the EC2 memory metrics from AWS and will populate various reports with the data.

CloudCheckr Reports and Features That Use Custom Metrics

In general, memory metrics will help inform any reports that alert you to under-utilized or idle resources, giving you a fuller understanding of your EC2 instances behavior so you are aware of CPU, network, and memory activity. This section lists some of the reports that use this data.

EC2 Instance Right Sizing Report

You will get data on your instances' 30-day peak and average memory usage, indicating whether it is doing memory-intensive operations—even if CPU or network usage is low—to give you a true understanding of the instance's utility.

CloudWatch Historical Export

While the AWS Console stores only 14 days worth of data, CloudCheckr stores all of your CloudWatch metrics for the life of your account, which you can access via the CloudWatch historical export.

How did we do?