How to monitor ephemeral storage usage in Fargate

How to monitor ephemeral storage usage in Fargate

Share Everywhere

Table of contents

Since there is no native CloudWatch metrics to monitor ephemeral storage on ECS, we think the only way is to use sidecar containers [1].

There is one propose for this feature, however it seems that AWS has not processed it yet. Refer to this link for more information:

https://github.com/aws/containers-roadmap/issues/1613

You can use the official docker image [amazon/cloudwatch-agent][2] to deploy a sidecar container:

Step 1

Create CloudWatch agent configuration file. You can refer to the sample settings as below:

{

  "metrics": {

    "metrics_collected": {

      "disk": {

        "measurement": [

          {

            "name": "free",

            "unit": "Gigabytes"

          },

          {

            "name": "total",

            "unit": "Gigabytes"

          },

          {

            "name": "used",

            "unit": "Gigabytes"

          },

          "used_percent",

          "inodes_free",

          "inodes_used",

          "inodes_total"

        ],

        "resources": [

          "*"

        ],

        "append_dimensions": {

          "ECScw": "ECSdisk",

          "cluster": "Default",

          "service": "Fargate"

        }

      }

    }

  }

}

You can change the metrics they want to collect by CloudWatch agent in the settings file. For details on metrics collected by CloudWatch agent, please refer to document [3]

Step 2

Upload the CloudWatch agent configuration setup file to AWS System Manager. You can refer to the “put-parameter” command for more details in the document [4]. Sample AWS CLI command to upload:

aws ssm put-parameter --name "cwagentconfigfile" --type "String" --value file://amazon-cloudwatch-agent.json --region <region_name>

Step 3

Create task definition for application container and sidecar container Set the “cwagentconfigfile” parameter for the container to transfer the CloudWatch agent configuration file created above to the sidecar container.

Sample task definition file:

{

    "containerDefinitions": [

        {

            "name": "demo",

            "image": "<your_image_uri",

            "cpu": 0,

            "portMappings": [],

            "essential": true,

            "environment": [],

            "mountPoints": [],

            "volumesFrom": [],

            "logConfiguration": {

                "logDriver": "awslogs",

                "options": {

                    "awslogs-group": "/ecs/yen-ephermeral-task",

                    "awslogs-region": "ap-northeast-1",

                    "awslogs-stream-prefix": "ecs"

                }

            }

        }

    ],

    "family": "yen-ephermeral-task",

    "taskRoleArn": "arn:aws:iam::<account_id:role/ecsTaskExecutionRole",

    "executionRoleArn": "arn:aws:iam::<accound_id>:role/ecsTaskExecutionRole",

    "networkMode": "awsvpc",

    "volumes": [],

    "placementConstraints": [],

    "requiresCompatibilities": [

        "FARGATE"

    ],

    "cpu": "256",

    "memory": "512",

    "ephemeralStorage": {"sizeInGiB": 100 }

}

As a result, CloudWatch metrics collected information about ECS Fargate’s disk usage.

How to monitor ephemeral storage usage in Fargate

Note:

  • Ensure that the ECS task role has the permissions for whatever your application needs as well as the CloudWatchAgentServerPolicy and the ECS task execution role has the policies AmazonSSMReadOnlyAccess, AmazonECSTaskExecutionRolePolicy, and CloudWatchAgentServerPolicy.
  • If you are not assigning the Fargate tasks to a public subnet, Ensure your service will have access to the CloudWatch and SSM services (you can set up NAT gateway or use VPC endpoints)
  • Confirm from the sidecar container logs that everything is working as expected and the CloudWatch agent is correctly processing the configurations as defined in the above steps. Note: If task runs and stops, ensure check to ensure CloudWatch permissions are correct or manually create a log group entry in CloudWatch where data will write to.
  • I have been suggested to use Fluentbit and cAdvisor besides aws cloudwatch agent as sidecar container. I have not tested this because lack of time. You should also tried them and I am very happy if you provide me some feedback.

References:

[1] https://docs.aws.amazon.com/AmazonECS/latest/developerguide/cloudwatch-metrics.html#available_cloudwatch_metrics

[2][amazon/cloudwatch-agent]

https://hub.docker.com/r/amazon/cloudwatch-agent

[3][CloudWatch ]

https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/metrics-collected-by-CloudWatch-agent.html

[4][put-parameters]

https://awscli.amazonaws.com/v2/documentation/api/latest/reference/ssm/put-parameter.html

Bạn thấy bài viết này như thế nào?
0 reactions

Add new comment

Restricted HTML

  • Allowed HTML tags: <a href hreflang> <em> <strong> <cite> <blockquote cite> <code> <ul type> <ol start type> <li> <dl> <dt> <dd> <h2 id> <h3 id> <h4 id> <h5 id> <h6 id>
  • Lines and paragraphs break automatically.
  • Web page addresses and email addresses turn into links automatically.

Bài viết liên quan

Khám phá FinOps

Khám phá FinOps - công nghệ đám mây

Nhiều doanh nghiệp ngày nay lựa chọn chuyển sang công nghệ đám mây với hi vọng đạt được lợi thế cạnh tranh so với đối thủ nhờ tiềm năng về hiệu quả cao và tiết kiệm chi phí hơn của công nghệ này.
Microservices Roadmap

Microservices Roadmap

- Kafka, RabbitMQ, Amazon SQS: Efficient and reliable message brokers for seamless communication between microservices.
The Data Analyst Roadmap

The Data Analyst Roadmap

**Database Knowledge**: Gain proficiency in working with databases like MySQL, PostgreSQL, or MongoDB.
Architectural patterns in software design

Architectural patterns in software design

Choose the architecture that aligns with your application's unique needs and goals. Each pattern offers a tailored approach to elevate your software system!
Exploring the Technological Marvel Behind Netflix

Exploring the Technological Marvel Behind Netflix

Ever wondered about the tech wizardry that powers your binge-watching adventures on Netflix?