valuesFrom does not work for PostgreSQL datasource password field

[piotrblasiak]

When configuring a PostgreSQL datasource using GrafanaDatasource CRD with valuesFrom to inject the password from a Kubernetes secret, the password is not properly applied to the datasource. This results in authentication failures with error: pq: password authentication failed for user "report_reader".

The same password works correctly when hardcoded directly in secureJsonData.password, confirming this is an issue with the valuesFrom mechanism specifically.

Environment

• Grafana Operator version: v5.19.0
• Kubernetes version: v1.33.5-eks-113cf36 (EKS)
• Grafana version: 12.1.0

Current Configuration (Not Working)

apiVersion: grafana.integreatly.org/v1beta1
kind: GrafanaDatasource
metadata:
  name: postgresql
  namespace: monitoring
spec:
  instanceSelector:
    matchLabels:
      app: grafana
  datasource:
    name: PostgreSQL
    type: postgres
    uid: postgresql
    url: mydb.eu-west-1.rds.amazonaws.com:5432
    access: proxy
    database: mydb
    user: report_reader
    jsonData:
      sslmode: require
  valuesFrom:
    - targetPath: secureJsonData.password
      valueFrom:
        secretKeyRef:
          name: grafana
          key: postgresql-password

Expected Behavior

The password from the secret should be injected into the datasource's secureJsonData.password field, allowing successful authentication to the PostgreSQL database.

Actual Behavior

The datasource is created but authentication fails with:

db query error: pq: password authentication failed for user "report_reader"

What Works (Workaround)

Hardcoding the password directly in the datasource configuration works:

datasource:
  # ... other fields ...
  secureJsonData:
    password: "actual-password-here"

Additional Information

Operator Logs

The operator logs show the datasource is being reconciled successfully:

2025-11-02T19:03:38Z INFO GrafanaDatasourceReconciler found matching Grafana instances for datasource

Status

The GrafanaDatasource status shows as successfully applied:

status:
  conditions:
  - lastTransitionTime: "2025-11-02T15:43:37Z"
    message: Datasource was successfully applied to 1 instances
    observedGeneration: 1
    reason: ApplySuccessful
    status: "True"
    type: DatasourceSynchronized

Analysis

1. The secret exists and contains the correct password (verified by base64 decoding)
2. The operator is processing the datasource (seen in logs)
3. The datasource status shows as "successfully applied"
4. However, the password is not actually being injected into Grafana

This appears to be similar to previously reported issues (#1482, #1485) with secureJsonData in the operator, but those were supposedly fixed in versions after 5.8.1.

Reproducible Test Case

1. Create a secret with a PostgreSQL password
2. Create a GrafanaDatasource of type postgres using valuesFrom to reference the secret
3. Try to query the datasource in Grafana
4. Observe authentication failure

Impact

This prevents secure password management for PostgreSQL datasources, forcing users to either:

• Hardcode passwords in the datasource configuration (security risk)
• Find alternative methods for password injection

Related Issues

• [Bug] secureJsonData don't work in 5.8.1 #1482 - secureJsonData don't work in 5.8.1
• [Bug] Unable to set secureJsonData.basicAuthPassword in GrafanaDatasource #1485 - Unable to set secureJsonData.basicAuthPassword in GrafanaDatasource

[Baarsgaard]

Does it work if you do:

spec:
  datasource:
    # ... other fields ...
    secureJsonData:
      password: "${postgresql-password}"

[piotrblasiak]

Does it work if you do:

spec:
datasource:
# ... other fields ...
secureJsonData:
password: "${postgresql-password}"

No - using environment variables also didn´t work unfortunately.

[theSuess]

I was unable to reproduce this using the following YAML:

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: postgres-repro
spec:
  replicas: 1
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
      - name: postgres
        image: docker.io/library/postgres:18
        env:
        - name: POSTGRES_PASSWORD
          value: test-from-secret
        ports:
        - containerPort: 5432
---
apiVersion: v1
kind: Service
metadata:
  name: postgres
spec:
  ports:
  - name: psql
    port: 5432
    protocol: TCP
    targetPort: 5432
  selector:
    app: postgres
  type: ClusterIP
---
apiVersion: v1
kind: Secret
metadata:
  name: repro-credentials
stringData:
  postgresql-password: test-from-secret
---
apiVersion: grafana.integreatly.org/v1beta1
kind: GrafanaDatasource
metadata:
  name: postgresql
spec:
  instanceSelector:
    matchLabels:
      instance: repro
  datasource:
    name: PostgreSQL
    type: postgres
    uid: postgresql
    url: postgres:5432
    access: proxy
    database: postgres
    user: postgres
    jsonData:
      sslmode: disable
    secureJsonData:
      password: '${postgresql-password}'
  valuesFrom:
    - targetPath: secureJsonData.password
      valueFrom:
        secretKeyRef:
          name: repro-credentials
          key: postgresql-password

The API for valuesFrom in datasources is a bit awkward to use so make sure you both:

• reference the field using valuesFrom
• reference the name property in the value of the field

[piotrblasiak]

I was unable to reproduce this using the following YAML:

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: postgres-repro
spec:
  replicas: 1
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
      - name: postgres
        image: docker.io/library/postgres:18
        env:
        - name: POSTGRES_PASSWORD
          value: test-from-secret
        ports:
        - containerPort: 5432
---
apiVersion: v1
kind: Service
metadata:
  name: postgres
spec:
  ports:
  - name: psql
    port: 5432
    protocol: TCP
    targetPort: 5432
  selector:
    app: postgres
  type: ClusterIP
---
apiVersion: v1
kind: Secret
metadata:
  name: repro-credentials
stringData:
  postgresql-password: test-from-secret
---
apiVersion: grafana.integreatly.org/v1beta1
kind: GrafanaDatasource
metadata:
  name: postgresql
spec:
  instanceSelector:
    matchLabels:
      instance: repro
  datasource:
    name: PostgreSQL
    type: postgres
    uid: postgresql
    url: postgres:5432
    access: proxy
    database: postgres
    user: postgres
    jsonData:
      sslmode: disable
    secureJsonData:
      password: '${postgresql-password}'
  valuesFrom:
    - targetPath: secureJsonData.password
      valueFrom:
        secretKeyRef:
          name: repro-credentials
          key: postgresql-password

The API for valuesFrom in datasources is a bit awkward to use so make sure you both:

• reference the field using valuesFrom
• reference the name property in the value of the field

Oh wow yes that did work - so is this a bug (meaning I should monitor this for a fix) or the expected behavior? I don´t think it works like this with other CRDs I´ve seen?

[piotrblasiak]

Does it work if you do:
spec:
datasource:

... other fields ...

secureJsonData:
password: "${postgresql-password}"

No - using environment variables also didn´t work unfortunately.

Sorry you were right this did indeed work - I thought you meant I should also load env vars from the secret first. But like @theSuess pointed out, this is needed when doing it from secrets as well which is weird.

[Baarsgaard]

Don't worry, I was being quite terse.
But we have seen people confused about this before and are talking about addressing in a way that would make your initial attempts valid and be backwards compatible 😄
I'm glad that it was figured it out in the end!