Skip to content

[BUG]: Maximum number of active requests reached: payload is discarded #6159

@PauloRenato98

Description

@PauloRenato98

Tracer Version(s)

v5.46.0

Node.js Version(s)

18

Bug Report

Description

We are facing the following error when using Datadog APM with dd-trace in our Node.js application:

Maximum number of active requests reached: payload is discarded.

We have several Node.js applications running in Kubernetes. Some of them experience this behavior, while others do not. When the application starts, it sends traces for about one minute, then stops and starts throwing the error: Maximum number of active requests reached: payload is discarded.
Note: not all applications are affected — some continue working as expected.

Image

Expected Behavior

The tracer should send payloads normally to the Datadog Agent, even under moderate or high load, without discarding traces.


Current Behavior

The tracer reaches an internal limit of active requests and begins discarding payloads with the error message above. As a result, APM data does not appear in the Datadog dashboard.


Troubleshooting Steps

  • We verified that the Datadog Agent is running and listening on port 8126.

  • We attempted to increase buffer and payload limits with the following configuration:

    DD_TRACE_WRITER_MAX_PAYLOADS=30
    DD_TRACE_WRITER_BUFFER_SIZE_BYTES=10000000
    DD_TRACE_SAMPLE_RATE=0.2
  • The issue persists, appearing to be related to an internal limit within the tracer.


Question

Is there any way to increase the number of active requests allowed by the tracer, or configure the flush behavior to avoid losing payloads?


Original Error

Maximum number of active requests reached: payload is discarded.

I found the code in github with the log

log.debug('Maximum number of active requests reached: payload is discarded.')

Environment

env: Kubernetes GCP with datadog operator
Node.js version: v18
Tracer version: v5.46.0
Agent version: 7.64.1

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions