Automatize a instalação e configuração do CloudWatch Agent com Systems Manager e EventBridge

Nesta automação, vamos usar um template do AWS CloudFormation para provisionar os recursos que executam as seguintes etapas:

1. Criar uma role do IAM para o AWS Systems Manager executar o documento de automação — o Systems Manager precisa de uma role de execução para rodar o runbook em seu nome nas instâncias EC2 de destino e conceder permissões a essas instâncias para gravar métricas no CloudWatch.
2. Enviar o arquivo de configuração do CloudWatch agent para o Parameter Store do Systems Manager — o template do CloudFormation envia o arquivo de configuração customizado do CloudWatch para o Parameter Store do Systems Manager. As instâncias buscam esse arquivo para configurar o CloudWatch agent.
3. Criar o documento de automação do Systems Manager — com o CloudFormation, vamos montar um runbook customizado para instalar e configurar o CloudWatch agent.
4. Criar uma role do IAM para a regra do EventBridge invocar a automação do Systems Manager — o EventBridge precisa de uma role de invocação para disparar o fluxo de automação do Systems Manager que instala e configura o CloudWatch agent.
5. Criar a regra do EventBridge — o CloudFormation cria uma regra do EventBridge que filtra eventos de instâncias EC2 no estado "running" e dispara a automação do SSM.
6. Opcional (criar uma SQS Dead Letter Queue) — uma fila SQS padrão atuando como Dead Letter Queue da regra do EventBridge, para capturar detalhes de eventuais erros nos disparos.

AWSTemplateFormatVersion: '2010-09-09'
Description: A template to create an AWS Systems Manager Automation Document that installs Amazon CloudWatch agent, sets up necessary permissions, and configures CloudWatch agent to publish memory metrics to CloudWatch and trigger SSM automation with eventbridge rule
Resources:
  SsmAutomationRole:
    Type: AWS::IAM::Role
    Properties:
      Description: IAM role for AWS Systems Manager to execute automation document
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service: ssm.amazonaws.com
            Action: sts:AssumeRole
            Condition:
              StringEquals:
                aws:SourceAccount: !Sub ${AWS::AccountId}
              ArnLike:
                aws:SourceArn: !Sub arn:${AWS::Partition}:ssm:*:${AWS::AccountId}:automation-execution/*
      ManagedPolicyArns:
        - !Sub arn:${AWS::Partition}:iam::aws:policy/service-role/AmazonSSMAutomationRole
      Path: /
      Policies:
        - PolicyName: SsmMemMetricIamPolicy
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: Allow
                Action:
                  - iam:GetRole
                  - iam:GetInstanceProfile
                  - iam:GetPolicy
                  - iam:AttachRolePolicy
                  - iam:ListInstanceProfiles
                  - ec2:DescribeInstances
                Resource: '*'
  CloudWatchAgentConfigFile:
    Type: AWS::SSM::Parameter
    Properties:
      Name: CloudwatchAgentConfigForMemoryMetricsLinux.json
      Description: Store CloudWatch Agent configuration file as AWS Systems Manager Parameter
      Type: String
      Value: |
        {
            "agent": {
                    "metrics_collection_interval": 60,
                    "run_as_user": "cwagent"
            },
            "metrics": {
                    "append_dimensions": {
                        "InstanceId": "${aws:InstanceId}"
                    },
                    "metrics_collected": {
                    "mem": {
                            "measurement": [\
                                "mem_used_percent"\
                            ],
                            "metrics_collection_interval": 60
                    }
                }
            }
        }
  SystemManagersMemoryMetricsRunbook:
    Type: AWS::SSM::Document
    Properties:
      DocumentFormat: YAML
      DocumentType: Automation
      Name: CloudwatchAgent_Install_Configure_MemoryMetrics_OnEC2Linux
      Content:
        description: Install CloudWatch Agent, Add permissions to target instances and configure CloudWatch agent to publish metrics
        schemaVersion: '0.3'
        assumeRole: !Sub ${SsmAutomationRole.Arn}
        parameters:
          InstanceId:
            type: String
            description: Select instances
        mainSteps:
          - name: Attach_CloudWatchAgent_ServerPolicy_to_instance_role
            action: aws:executeScript
            onFailure: Abort
            isCritical: true
            timeoutSeconds: 600
            description: |
              ## Find the attached role, attach CloudWatchAgentServer managed policy to the role
            inputs:
              Runtime: python3.8
              Handler: attach_cloudwatch_agent_managed_policy
              InputPayload:
                InstanceIds: '{{InstanceId}}'
              Script: |
                import boto3
                ec2_client = boto3.client('ec2')
                iam_client = boto3.client('iam')
                current_session = boto3.session.Session()
                current_region = current_session.region_name
                partition = current_session.get_partition_for_region(current_region)
                cloudwatchagent_policy_arn = f'arn:{partition}:iam::aws:policy/CloudWatchAgentServerPolicy'
                def attach_cloudwatch_agent_managed_policy(event,context):
                  instance_id = event['InstanceIds']
                  response = ec2_client.describe_instances(InstanceIds=[instance_id])
                  iam_role = response['Reservations'][0]['Instances'][0]['IamInstanceProfile']['Arn']
                  role_name=iam_role.split('/')[-1]
                  iam_client.attach_role_policy(RoleName=role_name, PolicyArn=cloudwatchagent_policy_arn)
          - name: Install_cloudWatchAgent_onTargetInstance
            action: aws:runCommand
            onFailure: Abort
            inputs:
              Parameters:
                action:
                  - Install
                installationType:
                  - Uninstall and reinstall
                name:
                  - AmazonCloudWatchAgent
              DocumentName: AWS-ConfigureAWSPackage
              InstanceIds:
                - '{{InstanceId}}'
          - name: Configure_CloudWatchAgen_onTargetInstance
            action: aws:runCommand
            inputs:
              DocumentName: AmazonCloudWatch-ManageAgent
              InstanceIds:
                - '{{InstanceId}}'
              Parameters:
                action: configure
                mode: ec2
                optionalConfigurationSource: ssm
                optionalConfigurationLocation: CloudwatchAgentConfigForMemoryMetricsLinux.json
                optionalRestart: 'yes'
  AutomationIAMRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: Amazon_EventBridge_Invoke_Start_Automation_Execution
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service: events.amazonaws.com
            Action: sts:AssumeRole
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AmazonSSMAutomationRole  # AmazonSSMAutomationRole managed policy
  EventRule:
     Type: AWS::Events::Rule
     Properties:
       EventBusName: default
       EventPattern:
         source:
           - aws.ec2
         detail-type:
           - EC2 Instance State-change Notification
         detail:
           state:
             - running
       Name: cloudwatch-install-configure-rule
       State: ENABLED
       Targets:
         - Id: cloudwatch-ssm-document
           #Arn: >-
            # arn:aws:ssm:us-east-1:XXXXXXXXXXX:automation-definition/CloudwatchAgent_Install_Configure_MemoryMetrics_OnEC2Linux
            #arn:${Partition}:ssm:${Region}:${Account}:document/${DocumentName}
           Arn: !Sub arn:${AWS::Partition}:ssm:${AWS::Region}:${AWS::AccountId}:automation-definition/CloudwatchAgent_Install_Configure_MemoryMetrics_OnEC2Linux
           RoleArn: !Sub ${AutomationIAMRole.Arn}
           InputTransformer:
             InputPathsMap:
               instance: $.detail.instance-id
             InputTemplate: '{"InstanceId":[<instance>]}'
           DeadLetterConfig:
             Arn: !Sub ${SQSQueue.Arn}
  SQSQueue:
    Type: AWS::SQS::Queue
    Properties:
      MaximumMessageSize: 262144 #integer value from 1,024 bytes (1 KiB) to 262,144 bytes (256 KiB). The default value is 262,144 (256 KiB).
      MessageRetentionPeriod: 345600 # 4 Days
      QueueName: DLQ-for-event-bridge

Outputs:
  SsmAutomationRoleName:
    Description: Name of the SSM Automation IAM Role
    Value: !Ref SsmAutomationRole
  IAMRoleArn:
    Description: ARN of the created IAM role
    Value: !GetAtt AutomationIAMRole.Arn
  IAMRoleName:
    Description: Name of the created IAM role
    Value: !Ref AutomationIAMRole
  AutomationIAMRoleArn:
    Description: EventBridge Invokation role name to Start Automation Execution
    Value: !GetAtt AutomationIAMRole.Arn
  SQSQueueArn:
    Description: SQS Queue to capture EventBridge execution errors
    Value: SQSQueue.Arn

Passo a passo:

1. Crie a stack do CloudFormation acima pela CLI ou pelo console:

Passo 1

2. Confira o documento de automação do Systems Manager que foi implantado. Esse documento customizado tem 3 etapas:

1. Identificar a role anexada à instância de destino e anexar a managed policy CloudWatchAgentServer a ela.
2. Instalar o CloudWatch Agent na instância de destino.
3. Configurar o CloudWatch Agent para enviar métricas ao CloudWatch.

Passo 2

3. Confira a regra do Amazon EventBridge:

Passo 3

Padrão de evento — o padrão de evento abaixo filtra eventos de status "running" à medida que ocorrem.

Event
{
  "detail-type": ["EC2 Instance State-change Notification"],
  "source": ["aws.ec2"],
  "detail": {
    "state": ["running"]
  }
}

E a transformação de entrada do EventBridge usa o Input Transformer abaixo para extrair o ID da instância de destino.

Input transformer

Input path
 {
   "instance": "$.detail.instance-id"
 }

Input template
 {
   "InstanceId":[<instance>]
 }

4. Crie uma nova instância:

- A instância EC2 de teste precisa ter o AWS Systems Manager Agent (SSM Agent) instalado para que o Systems Manager consiga se comunicar com ela. Além disso, as instâncias EC2 precisam de conectividade de rede com os endpoints públicos do serviço Systems Manager ou com os endpoints AWS PrivateLink VPC para Systems Manager. (Observação: usei a AMI Amazon Linux 2023 e fiz o launch na VPC default.)

- Crie a role "EC2ssmCoreRole" com a managed policy "AmazonSSMManagedInstanceCore" da AWS para permitir que o Systems Manager gerencie a instância EC2. Crie um instance profile e associe a role a ele. Por fim, suba uma instância de teste (t2.micro). Usei o template do CloudFormation abaixo para subir a instância.

AWSTemplateFormatVersion: 2010-09-09
Description: CloudFormation template to create EC2 instance with SSM role

Parameters:
  LatestAmiId:
    Type: 'AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>'
    Default: '/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2'

Resources:
  EC2Role:
    Type: AWS::IAM::Role
    Properties:
      RoleName: EC2ssmCoreRole
      Description: IAM role to allow Systems Manager to manage the EC2 instance
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service: ec2.amazonaws.com
            Action: sts:AssumeRole
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore

  EC2InstanceProfile:
    Type: AWS::IAM::InstanceProfile
    Properties:
      Roles:
        - !Ref EC2Role
      InstanceProfileName: EC2ssmCoreRole

  EC2Instance:
    Type: AWS::EC2::Instance
    Properties:
      ImageId: !Ref LatestAmiId
      InstanceType: t2.micro
      IamInstanceProfile: !Ref EC2InstanceProfile
      Tags:
        - Key: Name
          Value: "Test-instance"
        - Key: Env
          Value: "test-environment"

Outputs:
  EC2ssmCoreRoleName:
    Description: Name of the SSM Automation IAM Role
    Value: !Ref EC2Role

  EC2ssmCoreRoleArn:
    Description: ARN of the created IAM role
    Value: !GetAtt EC2Role.Arn

  EC2InstanceId:
    Description: Instance Id
    Value: !Ref EC2Instance

5. Confirme que a regra do EventBridge foi disparada quando a instância passou para o estado "running" e que o Systems Manager executou a automação, instalando e configurando o CloudWatch agent na instância EC2 para enviar métricas de memória.

6. Verifique a execução da automação do Systems Manager — assim que a instância entra no estado "running", a regra do EventBridge é disparada e executa o documento de automação do Systems Manager.

Passo 6

7. Acompanhe o status e o progresso da automação — são 3 etapas:

- Anexar a policy CloudWatch Agent Server à role da instância.

- Instalar o CloudWatch Agent na instância de destino.

- Configurar o CloudWatch Agent buscando o arquivo de configuração no Parameter Store do Systems Manager para enviar métricas de memória customizadas.

Passo 7

Passo 7

8. Verifique no CloudWatch se o CloudWatch Agent foi instalado e configurado para enviar métricas de memória customizadas.

Passo 8

Conclusão — vimos como automatizar de ponta a ponta a instalação e a configuração do CloudWatch Agent e enviar métricas de memória customizadas para o CloudWatch usando o Systems Manager Automation e o AWS EventBridge, sem nenhuma etapa manual, em instâncias EC2 recém-criadas.