AWS EC2 Windows CloudWatch Agent to push memory and disk space metrics

amazon ec2amazon-cloudwatchamazon-web-servicesaws-systems-manager

I am trying to configure custom metrics for windows server in EC2. I need memory utilization and free disk space metrics from perfmon. So, the document says we can use AWS SSM and cloudwatch agent to push these metrics. Have enabled SSM and attached role and everything on EC2. The config file I created in the path "C:\Program Files\Amazon\AmazonCloudWatchAgent\" is amazon-cloudwatch-agent.json. The contents of this file are:

{
"agent": {
    "metrics_collection_interval": 10,
    "logfile": "c:\\Program Files\\Amazon\\CloudWatchAgent\\Logs\\amazon-cloudwatch-agent.log"
  },
"metrics": {
"metrics_collected": {
  "statsd": {},
  "LogicalDisk": {
    "measurement": [
      {"name": "Free Megabytes", "unit": "String"},
      {"name": "LogicalDisk % Disk Read Time", "rename": "DISK_READ"},
      "% Disk Write Time"
    ],
    "resources": [
      "*"
    ]
  },
  "Memory": {
    "metrics_collection_interval": 5,
    "measurement": [
      "Available MBytes"
    ],
    "append_dimensions": {
      "d3": "win_bo"
    }
  },
"append_dimensions": {
  "InstanceId": "${aws:InstanceId}"
},
"aggregation_dimensions" : [["InstanceId"], ["d1"],[]]
}
}
}

Now, when I try to push this config file to cloudwatch agent using command line ".\amazon-cloudwatch-agent-ctl.ps1 -a fetch-config -m ec2 -c file:.\amazon-cloudwatch-agent.json", I get an error as attached here:

Can someone please guide me to achieve what I am looking for. I am not sure about this error, and stuck here. Any support would be highly appreciated.

Best Answer

Yes, the Cloudwatch Agent error messages are quite misleading.

I think the problem is with your "Free Megabytes" measurement - try "% Free Space" instead. The name should match the name of the metric in Performance Monitor. Possibly some of your other measurements are also incorrect.

Related Solutions

Fix Amazon CloudWatch Agent Crashes on Windows

This behaviour can occur if you haven't attached an IAM role to the instance that grants it permission to write metrics to CloudWatch as well as communicate with EC2 and Systems Manager.

Creation of the IAM role can be completed as follows:

Sign in to the AWS Management Console and open the IAM console at https://console.aws.amazon.com/iam/.

In the navigation pane on the left, choose Roles, Create role.

For Choose the service that will use this role, choose EC2 Allows EC2 instances to call AWS services on your behalf. Choose Next: Permissions.

In the list of policies, select the check box next to CloudWatchAgentServerPolicy. Use the search box to find the policy, if necessary.

To use SSM to install or configure the CloudWatch agent, select the check box next to AmazonEC2RoleforSSM. Use the search box to find the policy, if necessary. This policy is not necessary if you start and configure the agent only through the command line.

Choose Next: Review.

Confirm that CloudWatchAgentServerPolicy and optionally AmazonEC2RoleforSSM appear next to Policies. In Role name, type a name for the role, such as CloudWatchAgentServerRole. Optionally give it a description, and choose Create role.

The role is now created.

-- Source: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/create-iam-roles-for-cloudwatch-agent.html

Having created the IAM role you should then attach it to your EC2 instance, either at launch time or by right clicking the instance in EC2 and selecting Instance Settings > Attach/Replace IAM Role.

AWS Unified CloudWatch Agent and Beanstalk

You'll need to install it by providing a script in .ebextensions and following the steps outlined in the command line installation instructions for the unified agent: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/install-CloudWatch-Agent-commandline-fleet.html

    files:
      "/opt/elasticbeanstalk/hooks/appdeploy/post/install_cw_unified_agent.sh":
        mode: "000755"
        owner: root
        group: root
        content: |
          # download agent
          wget https://s3.us-east-1.amazonaws.com/amazoncloudwatch-agent-us-east-1/amazon_linux/amd64/latest/amazon-cloudwatch-agent.rpm

          # install CW agent
          sudo rpm -U ./amazon-cloudwatch-agent.rpm

          # run the agent
          sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c file:/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-config.json -s

You'll need a configuration file as well which you can find in the docs here: https://docs.aws.amazon.com/en_pv/AmazonCloudWatch/latest/monitoring/CloudWatch-Agent-Configuration-File-Details.html

Here's the sample config from the docs:

    {
      "agent": {
        "metrics_collection_interval": 10,
        "logfile": "/opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log"
      },
      "metrics": {
        "metrics_collected": {
          "cpu": {
            "resources": [
              "*"
            ],
            "measurement": [
              {"name": "cpu_usage_idle", "rename": "CPU_USAGE_IDLE", "unit": "Percent"},
              {"name": "cpu_usage_nice", "unit": "Percent"},
              "cpu_usage_guest"
            ],
            "totalcpu": false,
            "metrics_collection_interval": 10,
            "append_dimensions": {
              "customized_dimension_key_1": "customized_dimension_value_1",
              "customized_dimension_key_2": "customized_dimension_value_2"
            }
          },
          "disk": {
            "resources": [
              "/",
              "/tmp"
            ],
            "measurement": [
              {"name": "free", "rename": "DISK_FREE", "unit": "Gigabytes"},
              "total",
              "used"
            ],
             "ignore_file_system_types": [
              "sysfs", "devtmpfs"
            ],
            "metrics_collection_interval": 60,
            "append_dimensions": {
              "customized_dimension_key_3": "customized_dimension_value_3",
              "customized_dimension_key_4": "customized_dimension_value_4"
            }
          },
          "diskio": {
            "resources": [
              "*"
            ],
            "measurement": [
              "reads",
              "writes",
              "read_time",
              "write_time",
              "io_time"
            ],
            "metrics_collection_interval": 60
          },
          "swap": {
            "measurement": [
              "swap_used",
              "swap_free",
              "swap_used_percent"
            ]
          },
          "mem": {
            "measurement": [
              "mem_used",
              "mem_cached",
              "mem_total"
            ],
            "metrics_collection_interval": 1
          },
          "net": {
            "resources": [
              "eth0"
            ],
            "measurement": [
              "bytes_sent",
              "bytes_recv",
              "drop_in",
              "drop_out"
            ]
          },
          "netstat": {
            "measurement": [
              "tcp_established",
              "tcp_syn_sent",
              "tcp_close"
            ],
            "metrics_collection_interval": 60
          },
          "processes": {
            "measurement": [
              "running",
              "sleeping",
              "dead"
            ]
          }
        },
        "append_dimensions": {
          "ImageId": "${aws:ImageId}",
          "InstanceId": "${aws:InstanceId}",
          "InstanceType": "${aws:InstanceType}",
          "AutoScalingGroupName": "${aws:AutoScalingGroupName}"
        },
        "aggregation_dimensions" : [["ImageId"], ["InstanceId", "InstanceType"], ["d1"],[]],
        "force_flush_interval" : 30
      },
      "logs": {
        "logs_collected": {
          "files": {
            "collect_list": [
              {
                "file_path": "/opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log",
                "log_group_name": "amazon-cloudwatch-agent.log",
                "log_stream_name": "amazon-cloudwatch-agent.log",
                "timezone": "UTC"
              },
              {
                "file_path": "/opt/aws/amazon-cloudwatch-agent/logs/test.log",
                "log_group_name": "test.log",
                "log_stream_name": "test.log",
                "timezone": "Local"
              }
            ]
          }
        },
        "log_stream_name": "my_log_stream_name",
        "force_flush_interval" : 15
      }
    }

Best Answer

Related Solutions

Fix Amazon CloudWatch Agent Crashes on Windows

AWS Unified CloudWatch Agent and Beanstalk

Related Topic