Sử dụng AWS copilot CLI để đơn giản hóa việc tạo và quản lý ECS cluster
Share Everywhere
Table of contents
AWS Copilot CLI là công cụ có thể tạo ra “production ready containerized applications” một cách nhanh chóng, chỉ cần 1 file Dockerfile thôi, mọi chuyện còn lại có Copilot lo! Ngoài ra, Copilot cũng đơn giản hóa việc thực hiện ECS Exec (đặc biệt là với fargate).
Thông tin về copilot xem tại đây: https://aws.github.io/copilot-cli/
Tham khảo thêm về các command của Copilot ngoài phạm vi bài guide này tại đây: https://aws.github.io/copilot-cli/docs/commands/docs/
Cài đặt copilot CLI
https://aws.github.io/copilot-cli/docs/getting-started/install/
Note: Sử dụng windows thì download bản .exe tại đây https://github.com/aws/copilot-cli/releases/latest/download/copilot-windows.exe
Lưu ý rằng mặc dù có đuôi .exe nhưng lại không phải bản cài đặt mà chạy như 1 tool portable.
Note:
- Bạn cần phải configure AWS CLI trước khi sử dụng
- Để có thể sử dụng ECS Exec với Copilot, thì bạn cần phải cài đặt Session Manager plugin cho AWS CLI. Chi tiết tham khảo tại đây: https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager-working-with-install-plugin.html
Sử dụng
Sử dụng Dockerfile dưới đây cho mục đích testing:
FROM public.ecr.aws/ubuntu/ubuntu:18.04
CMD ["/bin/sleep", "10000"]
Lưu ý hãy copy tệp copilot-windows.exe download ở bên trên vào thư mục có chứa Dockerfile.
copilot-windows.exe init
Lúc này copilot sẽ hỏi bạn có muốn đặt tên cho app và service không, kiểu workload mà bạn muốn là gì, và Dockerfile. Bạn sử dụng các phím mũi tên trên bàn phím để di chuyển lựa chọn mong muốn.
Ở đây tôi tạo 1 app với workload type là Fargate và sử dụng Dockerfile tôi đã tạo bên trên
Note: It's best to run this command in the root of your Git repository.
Welcome to the Copilot CLI! We're going to walk you through some questions
to help you get set up with a containerized application on AWS. An application is a collection of
containerized services that operate together.
What would you like to name your application? [? for help] yen-demo-ecs-app
Application name: yen-demo-ecs-app
Workload type: Backend Service
What do you want to name this service? [? for help] yen-demo-service
Service name: yen-demo-service
Dockerfile: ./Dockerfile
parse EXPOSE: no EXPOSE statements in Dockerfile ./Dockerfile
Ok great, we'll set up a Backend Service named yen-demo-service in application yen-demo-ecs-app.
√ Created the infrastructure to manage services and jobs under application yen-demo-ecs-app.
√ The directory copilot will hold service manifests for application yen-demo-ecs-app.
√ Wrote the manifest for service yen-demo-service at copilot\\yen-demo-service\\manifest.yml
Your manifest contains configurations like your container size and port.
√ Created ECR repositories for service yen-demo-service.
All right, you're all set for local development.
Deploy: No
No problem, you can deploy your service later:
- Run `copilot env init --name test --profile default --app yen-demo-ecs-app` to create your staging environment.
- Run `copilot deploy` to deploy your service.
- Be a part of the Copilot community!
Ask or answer a question, submit a feature request...
Visit <https://aws.github.io/copilot-cli/community/get-involved/> to see how!
Khi copilot hỏi tôi có muốn deploy môi trường test không, tôi chọn no vì muốn customize chút xíu.
Copilot sẽ tạo ra 1 file tên là manifest.yml bên trong thư mục chứa copilot/<app_name> để cho phép bạn customize, file này có thể hiểu tương đương file task-definition.json
File manifest.yml của tôi như sau:
# The manifest for the "yen-demo-service" service.
# Read the full specification for the "Backend Service" type at:
# <https://aws.github.io/copilot-cli/docs/manifest/backend-service/>
# Your service name will be used in naming your resources like log groups, ECS services, etc.
name: yen-demo-service
type: Backend Service
# Your service does not allow any traffic.
# Configuration for your containers and service.
image:
# Docker build arguments. For additional overrides: <https://aws.github.io/copilot-cli/docs/manifest/backend-service/#image-build>
build: Dockerfile
cpu: 256 # Number of CPU units for the task.
memory: 512 # Amount of memory in MiB used by the task.
count: 1 # Number of tasks that should be running in your service.
exec: true # Enable running commands in your container.
# Optional fields for more advanced use-cases.
#
#variables: # Pass environment variables as key value pairs.
# LOG_LEVEL: info
#secrets: # Pass secrets from AWS Systems Manager (SSM) Parameter Store.
# GITHUB_TOKEN: GITHUB_TOKEN # The key is the name of the environment variable, the value is the name of the SSM parameter.
# You can override any of the values defined above by environment.
#environments:
# test:
# count: 2 # Number of tasks to run for the "test" environment.
# deployment: # The deployment strategy for the "test" environment.
# rolling: 'recreate' # Stops existing tasks before new ones are started for faster deployments.
Bây giờ, giả sử tôi muốn thêm ephermeral storage vào, tôi sẽ add thêm đoạn sau vào file:
storage:
ephemeral: 200
Sau đó tôi thực hiện tạo staging environment như hướng dẫn của copilot:
copilot-windows.exe env init --name test --profile default --app yen-demo-ecs-app
Lúc này, copilot sẽ hỏi tôi về các configure đối với environment. Vì tôi thực hiện mục đích test, nên tôi sẽ chọn default, các bạn có thể thực hiện customize theo ý muốn
Lưu ý: option profile ở đây, chính là profile của aws cli!
Sau đó, việc của bạn là đi pha 1 tách cafe và ngồi đợi Copilot hoàn thành công việc của mình!
D:\\PoC\\fargate-ephemeral\\demo-cli-tools\\copilot-cli\\copilot\\yen-demo-service>dir
Volume in drive D has no label.
Volume Serial Number is E408-B720
Directory of D:\\PoC\\fargate-ephemeral\\demo-cli-tools\\copilot-cli\\copilot\\yen-demo-service
06/09/2022 03:38 PM <DIR> .
06/09/2022 03:38 PM <DIR> ..
06/09/2022 03:38 PM 1,547 manifest.yml
1 File(s) 1,547 bytes
2 Dir(s) 990,423,146,496 bytes free
D:\\PoC\\fargate-ephemeral\\demo-cli-tools\\copilot-cli\\copilot\\yen-demo-service>cd ..
D:\\PoC\\fargate-ephemeral\\demo-cli-tools\\copilot-cli\\copilot>cd ..
D:\\PoC\\fargate-ephemeral\\demo-cli-tools\\copilot-cli>copilot env init --name test --profile default --app yen-demo-ecs-app
'copilot' is not recognized as an internal or external command,
operable program or batch file.
D:\\PoC\\fargate-ephemeral\\demo-cli-tools\\copilot-cli>copilot-windows.exe env init --name test --profile default --app yen-demo-ecs-app
Default environment configuration? Yes, use default.
√ Linked account <account_id> and region ap-northeast-1 to application yen-demo-ecs-app.
←[92m√←[0m Proposing infrastructure changes for the yen-demo-ecs-app-test environment.
- Creating the infrastructure for the yen-demo-ecs-app-test environment. ←[92m[create complete]←[0m ←[2m[128.0s]←[0m
- An IAM Role for AWS CloudFormation to manage resources ←[92m[create complete]←[0m ←[2m[36.3s]←[0m
- An ECS cluster to group your services ←[92m[create complete]←[0m ←[2m[11.1s]←[0m
- An IAM Role to describe resources in your environment ←[92m[create complete]←[0m ←[2m[37.4s]←[0m
- A security group to allow your containers to talk to each other ←[92m[create complete]←[0m ←[2m[7.3s]←[0m
- An Internet Gateway to connect to the public internet ←[92m[create complete]←[0m ←[2m[14.0s]←[0m
- Private subnet 1 for resources with no internet access ←[92m[create complete]←[0m ←[2m[7.3s]←[0m
- Private subnet 2 for resources with no internet access ←[92m[create complete]←[0m ←[2m[7.3s]←[0m
- A custom route table that directs network traffic for the public subnets ←[92m[create complete]←[0m ←[2m[14.2s]←[0m
- Public subnet 1 for resources that can access the internet ←[92m[create complete]←[0m ←[2m[7.3s]←[0m
- Public subnet 2 for resources that can access the internet ←[92m[create complete]←[0m ←[2m[7.3s]←[0m
- A private DNS namespace for discovering services within the environment ←[92m[create complete]←[0m ←[2m[48.8s]←[0m
- A Virtual Private Cloud to control networking of your AWS resources ←[92m[create complete]←[0m ←[2m[33.6s]←[0m
√ Created environment test in region ap-northeast-1 under application yen-demo-ecs-app.
Bây giờ, đến bước deploy! Nếu bạn có nhiều workload đang chờ deploy, thì Copilot sẽ hỏi bạn deploy workload nào.
D:\\PoC\\fargate-ephemeral\\demo-cli-tools\\copilot-cli>copilot-windows.exe deploy
Only found one workload, defaulting to: yen-demo-service
Only found one environment, defaulting to: test
Environment test is already on the latest version v1.9.0, skip upgrade.
failed to get console mode for stdin: The handle is invalid.
[+] Building 11.7s (5/5) FINISHED
=> [internal] load build definition from Dockerfile 0.1s
=> => transferring dockerfile: 107B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for public.ecr.aws/ubuntu/ubuntu:18.04 4.7s
=> [1/1] FROM public.ecr.aws/ubuntu/ubuntu:18.04@sha256:989d0c264519c70756c09284693b51285d05ac3dd0bc0657b02fae3200ec26c1 6.9s
=> => resolve public.ecr.aws/ubuntu/ubuntu:18.04@sha256:989d0c264519c70756c09284693b51285d05ac3dd0bc0657b02fae3200ec26c1 0.0s
=> => sha256:989d0c264519c70756c09284693b51285d05ac3dd0bc0657b02fae3200ec26c1 1.46kB / 1.46kB 0.0s
=> => sha256:8e23bcd2e0b02256462e475fe6da7a8735f6d648a268702f7965d97fee3fe1c6 446B / 446B 0.0s
=> => sha256:4c03d945ff2e00c8e42ed6d0b2954eda088415a2da888489405fef0e8787690d 1.60kB / 1.60kB 0.0s
=> => sha256:3da98577fc3b62ae1ba0dcbb2c8560fa3a3a75255eba85877a9ad508f8dd4973 25.69MB / 25.69MB 5.2s
=> => extracting sha256:3da98577fc3b62ae1ba0dcbb2c8560fa3a3a75255eba85877a9ad508f8dd4973 1.3s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:ad7097ef8559adf37f826b93c4e1758f504bcd8a04205bb7fc4115ecb7bdb94a 0.0s
=> => naming to <account_id>.dkr.ecr.ap-northeast-1.amazonaws.com/yen-demo-ecs-app/yen-demo-service 0.0s
Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
Login Succeeded
Using default tag: latest
The push refers to repository <account_id>.dkr.ecr.ap-northeast-1.amazonaws.com/yen-demo-ecs-app/yen-demo-service]
08a8962d5c49: Pushed
latest: digest: sha256:a5e8cb5f6ee99ef1b33d25a9b2c1edc968be31024afed4b6dd07087f7f336b43 size: 528
√ Proposing infrastructure changes for stack yen-demo-ecs-app-test-yen-demo-service
- Creating the infrastructure for stack yen-demo-ecs-app-test-yen-demo-service [create complete] [102.1s]
- Service discovery for your services to communicate within the VPC [create complete] [1.6s]
- Update your environment's shared resources [create complete] [4.6s]
- An IAM role to update your environment stack [create complete] [35.7s]
- An IAM Role for the Fargate agent to make AWS API calls on your behalf [create complete] [35.7s]
- A CloudWatch log group to hold your service logs [create complete] [1.6s]
- An ECS service to run and maintain your tasks in the environment cluster [create complete] [38.0s]
Deployments
Revision Rollout Desired Running Failed Pending
PRIMARY 1 [completed] 1 1 0 0
- An ECS task definition to group your containers and run them on ECS [create complete] [4.0s]
- An IAM role to control permissions for the containers in your tasks [create complete] [35.7s]
√ Deployed service yen-demo-service.
Giờ thử exec vào môi trường đã deploy xem thế nào:
D:\\PoC\\fargate-ephemeral\\demo-cli-tools\\copilot-cli>copilot-windows.exe svc exec
Found only one deployed service yen-demo-service in environment test
Execute `/bin/sh` in container yen-demo-service in task 9ec9674a69ca467e8eb72d62ffef4700.
Starting session with SessionId: ecs-execute-command-04ebd91e1f409e3e5
# df -h
Filesystem Size Used Avail Use% Mounted on
overlay 206G 8.3G 189G 5% /
tmpfs 64M 0 64M 0% /dev
shm 1.9G 0 1.9G 0% /dev/shm
tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup
/dev/nvme1n1 206G 8.3G 189G 5% /etc/hosts
/dev/nvme0n1p1 4.9G 1.6G 3.3G 32% /managed-agents/execute-command
tmpfs 1.9G 0 1.9G 0% /proc/acpi
tmpfs 1.9G 0 1.9G 0% /sys/firmware
#
Bạn đã exec vào ECS Fargate thành công!
Lên AWS Console xem:
Task definition của cluster trên sẽ như sau:
{
"ipcMode": null,
"executionRoleArn": "arn:aws:iam::<account-id>:role/yen-demo-ecs-app-test-yen-demo-servi-ExecutionRole-16Y1CE68OJEY5",
"containerDefinitions": [
{
"dnsSearchDomains": [],
"environmentFiles": [],
"logConfiguration": {
"logDriver": "awslogs",
"secretOptions": [],
"options": {
"awslogs-group": "/copilot/yen-demo-ecs-app-test-yen-demo-service",
"awslogs-region": "ap-northeast-1",
"awslogs-stream-prefix": "copilot"
}
},
"entryPoint": [],
"portMappings": [],
"command": [],
"linuxParameters": null,
"cpu": 0,
"environment": [
{
"name": "COPILOT_APPLICATION_NAME",
"value": "yen-demo-ecs-app"
},
{
"name": "COPILOT_ENVIRONMENT_NAME",
"value": "test"
},
{
"name": "COPILOT_SERVICE_DISCOVERY_ENDPOINT",
"value": "test.yen-demo-ecs-app.local"
},
{
"name": "COPILOT_SERVICE_NAME",
"value": "yen-demo-service"
}
],
"resourceRequirements": null,
"ulimits": [],
"dnsServers": [],
"mountPoints": [],
"workingDirectory": null,
"secrets": [],
"dockerSecurityOptions": [],
"memory": null,
"memoryReservation": null,
"volumesFrom": [],
"stopTimeout": null,
"image": "<account-id>.dkr.ecr.ap-northeast-1.amazonaws.com/yen-demo-ecs-app/yen-demo-service@sha256:a5e8cb5f6ee99ef1b33d25a9b2c1edc968be31024afed4b6dd07087f7f336b43",
"startTimeout": null,
"firelensConfiguration": null,
"dependsOn": null,
"disableNetworking": null,
"interactive": null,
"healthCheck": null,
"essential": true,
"links": [],
"hostname": null,
"extraHosts": [],
"pseudoTerminal": null,
"user": null,
"readonlyRootFilesystem": null,
"dockerLabels": {},
"systemControls": [],
"privileged": null,
"name": "yen-demo-service"
}
],
"placementConstraints": [],
"memory": "512",
"taskRoleArn": "arn:aws:iam::<account-id>:role/yen-demo-ecs-app-test-yen-demo-service-TaskRole-IHN4AT56KS08",
"compatibilities": [
"EC2",
"FARGATE"
],
"taskDefinitionArn": "arn:aws:ecs:ap-northeast-1:<account-id>:task-definition/yen-demo-ecs-app-test-yen-demo-service:1",
"family": "yen-demo-ecs-app-test-yen-demo-service",
"requiresAttributes": [
{
"targetId": null,
"targetType": null,
"value": null,
"name": "com.amazonaws.ecs.capability.logging-driver.awslogs"
},
{
"targetId": null,
"targetType": null,
"value": null,
"name": "ecs.capability.execution-role-awslogs"
},
{
"targetId": null,
"targetType": null,
"value": null,
"name": "com.amazonaws.ecs.capability.ecr-auth"
},
{
"targetId": null,
"targetType": null,
"value": null,
"name": "com.amazonaws.ecs.capability.docker-remote-api.1.19"
},
{
"targetId": null,
"targetType": null,
"value": null,
"name": "com.amazonaws.ecs.capability.docker-remote-api.1.17"
},
{
"targetId": null,
"targetType": null,
"value": null,
"name": "com.amazonaws.ecs.capability.docker-remote-api.1.21"
},
{
"targetId": null,
"targetType": null,
"value": null,
"name": "com.amazonaws.ecs.capability.task-iam-role"
},
{
"targetId": null,
"targetType": null,
"value": null,
"name": "ecs.capability.execution-role-ecr-pull"
},
{
"targetId": null,
"targetType": null,
"value": null,
"name": "ecs.capability.extensible-ephemeral-storage"
},
{
"targetId": null,
"targetType": null,
"value": null,
"name": "com.amazonaws.ecs.capability.docker-remote-api.1.18"
},
{
"targetId": null,
"targetType": null,
"value": null,
"name": "ecs.capability.task-eni"
}
],
"pidMode": null,
"requiresCompatibilities": [
"FARGATE"
],
"networkMode": "awsvpc",
"runtimePlatform": null,
"cpu": "256",
"revision": 1,
"status": "ACTIVE",
"inferenceAccelerators": null,
"proxyConfiguration": null,
"volumes": []
}
Bây giờ thực hiện clean môi trường:
D:\\PoC\\fargate-ephemeral\\demo-cli-tools\\copilot-cli>copilot-windows.exe svc delete
Only found one service, defaulting to: demoa
Sure? Yes
√ Deleted service demoa from environment test.
√ Deleted resources of service demoa from application demo.
√ Deleted service demoa from application demo.
Recommended follow-up action:
- Run `copilot pipeline deploy` to update the corresponding pipeline if it exists.
D:\\PoC\\fargate-ephemeral\\demo-cli-tools\\copilot-cli>copilot-windows.exe env delete
Only found one environment, defaulting to: test
Sure? Yes
√ Deleted environment test from application demo.
D:\\PoC\\fargate-ephemeral\\demo-cli-tools\\copilot-cli>copilot-windows.exe app delete
Sure? Yes
√ Cleaned up deployment resources.
√ Deleted application resources.
√ Deleted application configuration.
√ Deleted local .workspace file.
Lưu ý:
copilot env delete deletes an environment from your application. If there are running applications in your environment, you need to first run [copilot svc delete](<https://aws.github.io/copilot-cli/docs/commands/svc-delete/>)
Tham khảo thêm: https://aws.amazon.com/blogs/containers/amazon-ecs-on-aws-fargate-now-enables-customers-to-configure-ephemeral-storage-up-to-200gib/
Bạn thấy bài viết này như thế nào?
