代码之家 › 专栏 › 技术社区 › Vandalism

通过Eventbridge从S3向ECS任务fargate传递触发器元数据

amazon-cloudtrail aws-event-bridge aws-fargate boto3 amazon-s3

0

Vandalism · 技术社区 · 2 年前

我收到的bucketname和key是一个空字符串,导致脚本失败。我正试图从S3中检索导致事件桥中触发的文件名,但无法获取。


aws events put-targets --rule s3-to-rds --targets '{
    "Id": "",
    "Arn": "ecs cluster ARN",
    "RoleArn": "eventbridge role ARN with permissions",
    "EcsParameters": {
        "TaskDefinitionArn": "task ARN",
        "TaskCount": 1,
        "LaunchType": "FARGATE",
        "NetworkConfiguration": {
            "awsvpcConfiguration": {
                "Subnets": [
                    ""
                ],
                "AssignPublicIp": "DISABLED"
            }
        },
        "Group": "default"
    },
    "InputTransformer": {
        "InputPathsMap": {
            "s3-bucket": "$.detail.requestParameters.bucketName",
            "s3-key": "$.detail.requestParameters.key"
        },
        "InputTemplate": "{
            \"containerOverrides\": [
                {
                    \"name\": \"containername\",
                    \"environment\": [
                        {
                            \"name\": \"S3_BUCKET_NAME\",
                            \"value\": \"<s3-bucket>\"
                        },
                        {
                            \"name\": \"S3_OBJECT_KEY\",
                            \"value\": \"<s3-key>\"
                        }
                    ]
                }
            ]
        }"
    }
}'

用于检索env变量的python代码的一部分

S3_BUCKET_NAME = os.environ.get('S3_BUCKET_NAME')
logging.info(f"S3_BUCKET_NAME: {S3_BUCKET_NAME}")
S3_OBJECT_KEY = os.environ.get('S3_OBJECT_KEY')
logging.info(f"S3_BUCKET_NAME: {S3_OBJECT_KEY}")

s3 = boto3.client('s3')
response = s3.get_object(Bucket=S3_BUCKET_NAME, Key=S3_OBJECT_KEY)

这是我用于检索环境变量和动态检索的示例。

这是cloudtrail日志中的输出:

August 30, 2023 at 17:03 (UTC+2:00) response = handler(**kwargs)    tess
August 30, 2023 at 17:03 (UTC+2:00) ^^^^^^^^^^^^^^^^^   tess
August 30, 2023 at 17:03 (UTC+2:00) File "/usr/local/lib/python3.11/site-packages/botocore/handlers.py", line 284, in validate_bucket_name  tess
August 30, 2023 at 17:03 (UTC+2:00) raise ParamValidationError(report=error_msg)    tess
August 30, 2023 at 17:03 (UTC+2:00) botocore.exceptions.ParamValidationError: Parameter validation failed:  tess
August 30, 2023 at 17:03 (UTC+2:00) Invalid bucket name "<$s3-bucket>": Bucket name must match the regex "^[a-zA-Z0-9.\-_]{1,255}$" or be an ARN matching the regex "^arn:(aws).*:(s3|s3-object-lambda):[a-z\-0-9]*:[0-9]{12}:accesspoint[/:][a-zA-Z0-9\-.]{1,63}$|^arn:(aws).*:s3-outposts:[a-z\-0-9]+:[0-9]{12}:outpost[/:][a-zA-Z0-9\-]{1,63}[/:]accesspoint[/:][a-zA-Z0-9\-]{1,63}$"    tess
August 30, 2023 at 17:03 (UTC+2:00) api_params = self._emit_api_params( tess
August 30, 2023 at 17:03 (UTC+2:00) ^^^^^^^^^^^^^^^^^^^^^^  tess
August 30, 2023 at 17:03 (UTC+2:00) File "/usr/local/lib/python3.11/site-packages/botocore/client.py", line 1043, in _emit_api_params   tess
August 30, 2023 at 17:03 (UTC+2:00) self.meta.events.emit(  tess
August 30, 2023 at 17:03 (UTC+2:00) File "/usr/local/lib/python3.11/site-packages/botocore/hooks.py", line 412, in emit tess
August 30, 2023 at 17:03 (UTC+2:00) return self._emitter.emit(aliased_event_name, **kwargs) tess
August 30, 2023 at 17:03 (UTC+2:00) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^    tess
August 30, 2023 at 17:03 (UTC+2:00) File "/usr/local/lib/python3.11/site-packages/botocore/hooks.py", line 256, in emit tess
August 30, 2023 at 17:03 (UTC+2:00) return self._emit(event_name, kwargs)   tess
August 30, 2023 at 17:03 (UTC+2:00) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  tess
August 30, 2023 at 17:03 (UTC+2:00) File "/usr/local/lib/python3.11/site-packages/botocore/hooks.py", line 239, in _emit    tess
August 30, 2023 at 17:03 (UTC+2:00) Traceback (most recent call last):  tess
August 30, 2023 at 17:03 (UTC+2:00) File "/app/main.py", line 97, in <module>   tess
August 30, 2023 at 17:03 (UTC+2:00) response = s3.get_object(Bucket=S3_BUCKET_NAME, Key=S3_OBJECT_KEY)  tess
August 30, 2023 at 17:03 (UTC+2:00) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ tess
August 30, 2023 at 17:03 (UTC+2:00) File "/usr/local/lib/python3.11/site-packages/botocore/client.py", line 535, in _api_call   tess
August 30, 2023 at 17:03 (UTC+2:00) return self._make_api_call(operation_name, kwargs)  tess
August 30, 2023 at 17:03 (UTC+2:00) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ tess
August 30, 2023 at 17:03 (UTC+2:00) File "/usr/local/lib/python3.11/site-packages/botocore/client.py", line 928, in _make_api_call  tess
August 30, 2023 at 17:03 (UTC+2:00) 2023-08-30 15:03:52,840 - INFO - S3_BUCKET_NAME:    tess
August 30, 2023 at 17:03 (UTC+2:00) 2023-08-30 15:03:52,840 - INFO - S3_BUCKET_NAME:    tess

通过替换这些键的值修复了此问题 “s3 bucket”:“$.detail.requestParameters.backetName”, “s3 key”:“$.detail.requestParameters.key” 到 “s3 bucket”:“$.detail.backet.name”, “s3键”:“$.detail.object.key”

0 回复 | 直到 2 年前