Add support for dynamic attributes in S3 prefix for AWS S3 Exporter #29872
Description
Component(s)
exporter/awss3
Is your feature request related to a problem? Please describe.
We are currently unable to dynamically include pod annotations in the s3_prefix when using the AWS S3 Exporter with the OpenTelemetry Collector. We want to organize our S3 object paths with pod-specific information to easily identify and categorize logs from different pods.
Describe the solution you'd like
We would like the AWS S3 Exporter to be enhanced to support dynamic attributes in the s3_prefix. This would allow us to construct S3 object paths that include values from specific attributes, such as pod annotations. For example, we could use the logging/prefix annotation from a pod's metadata to structure the path like so:
pod_log/cluster_name=${cluster_name}/namespace_name=${namespace_name}/prefix=${logging/prefix}/year=XXXX/month=XX/day=XX/hour=XX/
Describe alternatives you've considered
As an alternative, we are considering using Fluent Bit, which supports dynamic values in the S3 object key configuration. This would allow us to include pod-specific annotations in the S3 paths. However, we would prefer to use OpenTelemetry Collector if it supported dynamic attributes in the S3 prefix, as it would keep our observability stack consistent and leverage OpenTelemetry's advanced processing capabilities.
Additional context
No response