Skip to content

Commit bffcf2c

Browse files
rogercollChrsMark
rogercoll
and
ChrsMark
authored
feat: add the Elastic Collector distribution (#31)
* add the Elastic Collector distribution * feat: add Elastic distro to Helm deployment * remove attributes processors for k8s config * chore: bump elastic-agent snapshot version * docs: add k8s configuration deployment command * chore: remove otel header from Elastic config * use Bearer for apm authorization header * Update kubernetes/elastic-helm/values.yaml Co-authored-by: Chris Mark <[email protected]> * Update kubernetes/elastic-helm/configmap-elastic.yaml Co-authored-by: Chris Mark <[email protected]> --------- Co-authored-by: Chris Mark <[email protected]>
1 parent af241f7 commit bffcf2c

File tree

9 files changed

+152
-59
lines changed

9 files changed

+152
-59
lines changed

‎.env.override‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,3 +10,10 @@ IMAGE_NAME=ghcr.io/elastic/opentelemetry-demo
1010
# *********************
1111
AD_SERVICE_DOCKERFILE=./src/adservice/Dockerfile.elastic
1212
FRAUD_SERVICE_DOCKERFILE=./src/frauddetectionservice/Dockerfile.elastic
13+
14+
# *********************
15+
# Elastic Collector
16+
# *********************
17+
COLLECTOR_CONTRIB_IMAGE=docker.elastic.co/beats/elastic-agent:8.15.0-7b611e39-SNAPSHOT
18+
OTEL_COLLECTOR_CONFIG=./src/otelcollector/otelcol-elastic-config.yaml
19+
OTEL_COLLECTOR_CONFIG_EXTRAS=./src/otelcollector/otelcol-elastic-config-extras.yaml

‎.github/README.md‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,15 +7,17 @@ The following guide describes how to setup the OpenTelemetry demo with Elastic O
77
- The .NET agent within the [Cart service](../src/cartservice/src/Directory.Build.props) has been replaced with the Elastic distribution of the OpenTelemetry .NET Agent. You can find more information about the Elastic distribution in [this blog post](https://www.elastic.co/observability-labs/blog/elastic-opentelemetry-distribution-dotnet-applications).
88
- The Elastic distribution of the OpenTelemetry Node.js Agent has replaced the OpenTelemetry Node.js agent in the [Payment service](../src/paymentservice/package.json). Additional details about the Elastic distribution are available in [this blog post](https://www.elastic.co/observability-labs/blog/elastic-opentelemetry-distribution-node-js).
99

10+
Additionally, the OpenTelemetry Contrib collector has also been changed to the [Elastic OpenTelemetry Collector distribution](https://github.com/elastic/elastic-agent/blob/main/internal/pkg/otel/README.md). This ensures a more integrated and optimized experience with Elastic Observability.
11+
1012
## Docker compose
1113

1214
1. Start a free trial on [Elastic Cloud](https://cloud.elastic.co/) and copy the `endpoint` and `secretToken` from the Elastic APM setup instructions in your Kibana.
13-
1. Open the file `src/otelcollector/otelcol-config-extras.yml` in an editor and replace the following two placeholders:
15+
1. Open the file `src/otelcollector/otelcol-elastic-config-extras.yaml` in an editor and replace the following two placeholders:
1416
- `YOUR_APM_ENDPOINT_WITHOUT_HTTPS_PREFIX`: your Elastic APM endpoint (*without* `https://` prefix) that *must* also include the port (example: `1234567.apm.us-west2.gcp.elastic-cloud.com:443`).
1517
- `YOUR_APM_SECRET_TOKEN`: your Elastic APM secret token.
1618
1. Start the demo with the following command from the repository's root directory:
1719
```
18-
docker-compose up -d
20+
make start
1921
```
2022

2123
## Kubernetes
@@ -46,6 +48,9 @@ The following guide describes how to setup the OpenTelemetry demo with Elastic O
4648
# !(when an older helm open-telemetry repo exists) update the open-telemetry helm repo
4749
helm repo update open-telemetry
4850
51+
# deploy the configuration for the Elastic OpenTelemetry collector distribution
52+
kubectl apply -f configmap-elastic.yaml
53+
4954
# deploy the demo through helm install
5055
helm install -f values.yaml my-otel-demo open-telemetry/opentelemetry-demo
5156
```

‎docker-compose.minimal.yml‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -582,7 +582,8 @@ services:
582582
limits:
583583
memory: 200M
584584
restart: unless-stopped
585-
command: [ "--config=/etc/otelcol-config.yml", "--config=/etc/otelcol-config-extras.yml" ]
585+
command: ["otel", "--config", "/etc/otelcol-config.yml", "--config", "/etc/otelcol-config-extras.yml" ]
586+
entrypoint: [ /usr/share/elastic-agent/elastic-agent ]
586587
user: 0:0
587588
volumes:
588589
- ${DOCKER_SOCK}:/var/run/docker.sock:ro

‎docker-compose.yml‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -692,7 +692,8 @@ services:
692692
limits:
693693
memory: 200M
694694
restart: unless-stopped
695-
command: [ "--config=/etc/otelcol-config.yml", "--config=/etc/otelcol-config-extras.yml" ]
695+
command: ["otel", "--config", "/etc/otelcol-config.yml", "--config", "/etc/otelcol-config-extras.yml" ]
696+
entrypoint: [ /usr/share/elastic-agent/elastic-agent ]
696697
user: 0:0
697698
volumes:
698699
- ${DOCKER_SOCK}:/var/run/docker.sock:ro

‎kubernetes/elastic-helm/configmap-elastic.yaml‎

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
---
2+
apiVersion: v1
3+
kind: ConfigMap
4+
metadata:
5+
name: elastic-otelcol-agent
6+
namespace: default
7+
labels:
8+
app.kubernetes.io/name: otelcol
9+
10+
data:
11+
relay: |
12+
connectors:
13+
spanmetrics: {}
14+
exporters:
15+
debug: {}
16+
otlp/elastic:
17+
endpoint: ${env:ELASTIC_APM_ENDPOINT}
18+
compression: none
19+
headers:
20+
Authorization: Bearer ${ELASTIC_APM_SECRET_TOKEN}
21+
extensions:
22+
processors:
23+
batch: {}
24+
receivers:
25+
httpcheck/frontendproxy:
26+
targets:
27+
- endpoint: http://example-frontendproxy:8080
28+
otlp:
29+
protocols:
30+
grpc:
31+
endpoint: ${env:MY_POD_IP}:4317
32+
http:
33+
cors:
34+
allowed_origins:
35+
- http://*
36+
- https://*
37+
endpoint: ${env:MY_POD_IP}:4318
38+
service:
39+
extensions:
40+
pipelines:
41+
logs:
42+
exporters:
43+
- debug
44+
- otlp/elastic
45+
processors:
46+
- batch
47+
receivers:
48+
- otlp
49+
metrics:
50+
exporters:
51+
- otlp/elastic
52+
- debug
53+
processors:
54+
- batch
55+
receivers:
56+
- httpcheck/frontendproxy
57+
- otlp
58+
- spanmetrics
59+
traces:
60+
exporters:
61+
- otlp/elastic
62+
- debug
63+
- spanmetrics
64+
processors:
65+
- batch
66+
receivers:
67+
- otlp
68+
telemetry:
69+
metrics:
70+
address: ${env:MY_POD_IP}:8888
71+

‎kubernetes/elastic-helm/values.yaml‎

Lines changed: 23 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,12 @@ default:
44
tag: 1.10.1
55

66
opentelemetry-collector:
7+
image:
8+
repository: docker.elastic.co/beats/elastic-agent
9+
tag: 8.15.0-SNAPSHOT
10+
command:
11+
name: "/usr/share/elastic-agent/elastic-agent"
12+
extraArgs: ["otel", "-c", "/etc/elastic-agent/otel.yaml"]
713
mode: "deployment"
814
presets:
915
kubernetesAttributes:
@@ -24,38 +30,23 @@ opentelemetry-collector:
2430
secretKeyRef:
2531
name: elastic-secret
2632
key: elastic_apm_secret_token
27-
config:
28-
exporters:
29-
otlp/elastic:
30-
endpoint: ${ELASTIC_APM_ENDPOINT}
31-
compression: none
32-
headers:
33-
Authorization: Bearer ${ELASTIC_APM_SECRET_TOKEN}
34-
receivers:
35-
otlp:
36-
protocols:
37-
grpc:
38-
endpoint: ${MY_POD_IP}:4317
39-
http:
40-
cors:
41-
allowed_origins:
42-
- http://*
43-
- https://*
44-
endpoint: ${MY_POD_IP}:4318
45-
service:
46-
pipelines:
47-
traces:
48-
receivers: [otlp]
49-
processors: [batch]
50-
exporters: [otlp/elastic]
51-
metrics:
52-
receivers: [otlp]
53-
processors: [batch]
54-
exporters: [otlp/elastic]
55-
logs:
56-
receivers: [otlp]
57-
processors: [batch]
58-
exporters: [otlp/elastic]
33+
34+
configMap:
35+
create: false
36+
existingName: elastic-otelcol-agent
37+
38+
livenessProbe:
39+
httpGet:
40+
port: 8888
41+
path: metrics
42+
readinessProbe:
43+
httpGet:
44+
port: 8888
45+
path: metrics
46+
47+
opensearch:
48+
enabled: false
49+
5950
grafana:
6051
enabled: false
6152

‎src/otelcollector/otelcol-config-extras.yml‎

Lines changed: 0 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -16,26 +16,3 @@
1616
# pipelines:
1717
# traces:
1818
# exporters: [spanmetrics, otlphttp/example]
19-
20-
exporters:
21-
otlp/elastic:
22-
# !!! Elastic APM https endpoint WITHOUT the "https://" prefix
23-
endpoint: "YOUR_APM_ENDPOINT_WITHOUT_HTTPS_PREFIX"
24-
compression: none
25-
headers:
26-
Authorization: "Bearer YOUR_APM_SECRET_TOKEN"
27-
28-
service:
29-
pipelines:
30-
traces:
31-
receivers: [otlp]
32-
processors: [batch]
33-
exporters: [spanmetrics, otlp/elastic]
34-
metrics:
35-
receivers: [otlp, spanmetrics]
36-
processors: [batch]
37-
exporters: [otlp/elastic]
38-
logs:
39-
receivers: [otlp]
40-
processors: [batch]
41-
exporters: [otlp/elastic]

‎src/otelcollector/otelcol-elastic-config-extras.yaml‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
exporters:
2+
otlp/elastic:
3+
# !!! Elastic APM https endpoint WITHOUT the "https://" prefix
4+
endpoint: "YOUR_APM_ENDPOINT_WITHOUT_HTTPS_PREFIX"
5+
compression: none
6+
headers:
7+
Authorization: "Bearer YOUR_APM_SECRET_TOKEN"
8+
9+
service:
10+
pipelines:
11+
traces:
12+
receivers: [otlp]
13+
processors: [batch]
14+
exporters: [spanmetrics, otlp/elastic]
15+
metrics:
16+
receivers: [otlp, spanmetrics]
17+
processors: [batch]
18+
exporters: [otlp/elastic]
19+
logs:
20+
receivers: [otlp]
21+
processors: [batch]
22+
exporters: [otlp/elastic]

‎src/otelcollector/otelcol-elastic-config.yaml‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
receivers:
2+
otlp:
3+
protocols:
4+
grpc:
5+
http:
6+
cors:
7+
allowed_origins:
8+
- "http://*"
9+
- "https://*"
10+
11+
exporters:
12+
debug:
13+
14+
processors:
15+
batch:
16+
17+
connectors:
18+
spanmetrics:

0 commit comments

Comments
 (0)