Create configure_tests convention plugin
#9859
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sarahchen6
added
type: enhancement
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 54 metrics, 11 unstable metrics. Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.55.0-SNAPSHOT~f654867f52, baseline=1.55.0-SNAPSHOT~a632fe70e1
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.027 s) : 0, 1026814
Total [baseline] (8.644 s) : 0, 8643720
Agent [candidate] (1.027 s) : 0, 1027251
Total [candidate] (8.684 s) : 0, 8684453
section iast
Agent [baseline] (1.166 s) : 0, 1165893
Total [baseline] (9.359 s) : 0, 9359281
Agent [candidate] (1.157 s) : 0, 1156606
Total [candidate] (9.329 s) : 0, 9328763
gantt
title insecure-bank - break down per module: candidate=1.55.0-SNAPSHOT~f654867f52, baseline=1.55.0-SNAPSHOT~a632fe70e1
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.457 ms) : 0, 1457
crashtracking [candidate] (1.465 ms) : 0, 1465
BytebuddyAgent [baseline] (699.737 ms) : 0, 699737
BytebuddyAgent [candidate] (700.844 ms) : 0, 700844
GlobalTracer [baseline] (244.298 ms) : 0, 244298
GlobalTracer [candidate] (244.544 ms) : 0, 244544
AppSec [baseline] (32.244 ms) : 0, 32244
AppSec [candidate] (32.278 ms) : 0, 32278
Debugger [baseline] (6.378 ms) : 0, 6378
Debugger [candidate] (6.362 ms) : 0, 6362
Remote Config [baseline] (679.713 µs) : 0, 680
Remote Config [candidate] (675.394 µs) : 0, 675
Telemetry [baseline] (12.965 ms) : 0, 12965
Telemetry [candidate] (15.071 ms) : 0, 15071
Flare Poller [baseline] (8.035 ms) : 0, 8035
Flare Poller [candidate] (4.979 ms) : 0, 4979
section iast
crashtracking [baseline] (1.478 ms) : 0, 1478
crashtracking [candidate] (1.45 ms) : 0, 1450
BytebuddyAgent [baseline] (828.552 ms) : 0, 828552
BytebuddyAgent [candidate] (821.252 ms) : 0, 821252
GlobalTracer [baseline] (233.027 ms) : 0, 233027
GlobalTracer [candidate] (233.158 ms) : 0, 233158
AppSec [baseline] (29.267 ms) : 0, 29267
AppSec [candidate] (27.778 ms) : 0, 27778
Debugger [baseline] (6.177 ms) : 0, 6177
Debugger [candidate] (6.06 ms) : 0, 6060
Remote Config [baseline] (604.372 µs) : 0, 604
Remote Config [candidate] (591.751 µs) : 0, 592
Telemetry [baseline] (8.493 ms) : 0, 8493
Telemetry [candidate] (8.301 ms) : 0, 8301
Flare Poller [baseline] (4.186 ms) : 0, 4186
Flare Poller [candidate] (4.074 ms) : 0, 4074
IAST [baseline] (32.474 ms) : 0, 32474
IAST [candidate] (32.785 ms) : 0, 32785
Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.55.0-SNAPSHOT~f654867f52, baseline=1.55.0-SNAPSHOT~a632fe70e1
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.026 s) : 0, 1026299
Total [baseline] (10.805 s) : 0, 10805338
Agent [candidate] (1.029 s) : 0, 1028965
Total [candidate] (10.801 s) : 0, 10801338
section appsec
Agent [baseline] (1.207 s) : 0, 1206987
Total [baseline] (10.912 s) : 0, 10912064
Agent [candidate] (1.199 s) : 0, 1198640
Total [candidate] (10.865 s) : 0, 10865421
section iast
Agent [baseline] (1.159 s) : 0, 1159266
Total [baseline] (11.114 s) : 0, 11113674
Agent [candidate] (1.169 s) : 0, 1168929
Total [candidate] (11.144 s) : 0, 11144256
section profiling
Agent [baseline] (1.172 s) : 0, 1171987
Total [baseline] (10.883 s) : 0, 10883293
Agent [candidate] (1.181 s) : 0, 1181401
Total [candidate] (10.972 s) : 0, 10971966
gantt
title petclinic - break down per module: candidate=1.55.0-SNAPSHOT~f654867f52, baseline=1.55.0-SNAPSHOT~a632fe70e1
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.451 ms) : 0, 1451
crashtracking [candidate] (1.449 ms) : 0, 1449
BytebuddyAgent [baseline] (699.963 ms) : 0, 699963
BytebuddyAgent [candidate] (701.696 ms) : 0, 701696
GlobalTracer [baseline] (243.898 ms) : 0, 243898
GlobalTracer [candidate] (244.333 ms) : 0, 244333
AppSec [baseline] (32.104 ms) : 0, 32104
AppSec [candidate] (32.403 ms) : 0, 32403
Debugger [baseline] (6.309 ms) : 0, 6309
Debugger [candidate] (6.386 ms) : 0, 6386
Remote Config [baseline] (678.284 µs) : 0, 678
Remote Config [candidate] (674.696 µs) : 0, 675
Telemetry [baseline] (13.725 ms) : 0, 13725
Telemetry [candidate] (15.196 ms) : 0, 15196
Flare Poller [baseline] (7.086 ms) : 0, 7086
Flare Poller [candidate] (5.706 ms) : 0, 5706
section appsec
crashtracking [baseline] (1.467 ms) : 0, 1467
crashtracking [candidate] (1.459 ms) : 0, 1459
BytebuddyAgent [baseline] (729.107 ms) : 0, 729107
BytebuddyAgent [candidate] (723.147 ms) : 0, 723147
GlobalTracer [baseline] (236.844 ms) : 0, 236844
GlobalTracer [candidate] (235.262 ms) : 0, 235262
AppSec [baseline] (174.45 ms) : 0, 174450
AppSec [candidate] (173.82 ms) : 0, 173820
Debugger [baseline] (5.903 ms) : 0, 5903
Debugger [candidate] (5.935 ms) : 0, 5935
Remote Config [baseline] (624.545 µs) : 0, 625
Remote Config [candidate] (612.194 µs) : 0, 612
Telemetry [baseline] (8.441 ms) : 0, 8441
Telemetry [candidate] (8.419 ms) : 0, 8419
Flare Poller [baseline] (3.867 ms) : 0, 3867
Flare Poller [candidate] (3.89 ms) : 0, 3890
IAST [baseline] (25.087 ms) : 0, 25087
IAST [candidate] (25.025 ms) : 0, 25025
section iast
crashtracking [baseline] (1.461 ms) : 0, 1461
crashtracking [candidate] (1.461 ms) : 0, 1461
BytebuddyAgent [baseline] (821.606 ms) : 0, 821606
BytebuddyAgent [candidate] (829.825 ms) : 0, 829825
GlobalTracer [baseline] (233.569 ms) : 0, 233569
GlobalTracer [candidate] (234.765 ms) : 0, 234765
AppSec [baseline] (32.538 ms) : 0, 32538
AppSec [candidate] (29.919 ms) : 0, 29919
Debugger [baseline] (6.111 ms) : 0, 6111
Debugger [candidate] (6.13 ms) : 0, 6130
Remote Config [baseline] (594.86 µs) : 0, 595
Remote Config [candidate] (596.546 µs) : 0, 597
Telemetry [baseline] (8.599 ms) : 0, 8599
Telemetry [candidate] (8.538 ms) : 0, 8538
Flare Poller [baseline] (4.27 ms) : 0, 4270
Flare Poller [candidate] (4.258 ms) : 0, 4258
IAST [baseline] (29.194 ms) : 0, 29194
IAST [candidate] (32.022 ms) : 0, 32022
section profiling
crashtracking [baseline] (1.46 ms) : 0, 1460
crashtracking [candidate] (1.483 ms) : 0, 1483
BytebuddyAgent [baseline] (725.864 ms) : 0, 725864
BytebuddyAgent [candidate] (731.898 ms) : 0, 731898
GlobalTracer [baseline] (220.242 ms) : 0, 220242
GlobalTracer [candidate] (221.647 ms) : 0, 221647
AppSec [baseline] (32.314 ms) : 0, 32314
AppSec [candidate] (32.649 ms) : 0, 32649
Debugger [baseline] (11.389 ms) : 0, 11389
Debugger [candidate] (13.066 ms) : 0, 13066
Remote Config [baseline] (732.386 µs) : 0, 732
Remote Config [candidate] (703.685 µs) : 0, 704
Telemetry [baseline] (11.228 ms) : 0, 11228
Telemetry [candidate] (9.864 ms) : 0, 9864
Flare Poller [baseline] (4.174 ms) : 0, 4174
Flare Poller [candidate] (4.167 ms) : 0, 4167
ProfilingAgent [baseline] (109.249 ms) : 0, 109249
ProfilingAgent [candidate] (110.289 ms) : 0, 110289
Profiling [baseline] (109.876 ms) : 0, 109876
Profiling [candidate] (110.921 ms) : 0, 110921
LoadParameters
See matching parameters
SummaryFound 1 performance improvements and 1 performance regressions! Performance is the same for 10 metrics, 12 unstable metrics.
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.55.0-SNAPSHOT~f654867f52, baseline=1.55.0-SNAPSHOT~a632fe70e1
dateFormat X
axisFormat %s
section baseline
no_agent (4.314 ms) : 4262, 4365
. : milestone, 4314,
iast (9.466 ms) : 9308, 9625
. : milestone, 9466,
iast_FULL (14.503 ms) : 14213, 14793
. : milestone, 14503,
iast_GLOBAL (11.338 ms) : 11131, 11545
. : milestone, 11338,
profiling (8.771 ms) : 8632, 8910
. : milestone, 8771,
tracing (8.024 ms) : 7901, 8147
. : milestone, 8024,
section candidate
no_agent (4.225 ms) : 4172, 4277
. : milestone, 4225,
iast (9.48 ms) : 9320, 9640
. : milestone, 9480,
iast_FULL (14.249 ms) : 13967, 14530
. : milestone, 14249,
iast_GLOBAL (10.368 ms) : 10174, 10561
. : milestone, 10368,
profiling (8.964 ms) : 8824, 9105
. : milestone, 8964,
tracing (7.772 ms) : 7662, 7882
. : milestone, 7772,
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.55.0-SNAPSHOT~f654867f52, baseline=1.55.0-SNAPSHOT~a632fe70e1
dateFormat X
axisFormat %s
section baseline
no_agent (36.676 ms) : 36375, 36976
. : milestone, 36676,
appsec (47.652 ms) : 47233, 48071
. : milestone, 47652,
code_origins (43.751 ms) : 43374, 44128
. : milestone, 43751,
iast (44.443 ms) : 44064, 44822
. : milestone, 44443,
profiling (48.957 ms) : 48465, 49449
. : milestone, 48957,
tracing (42.874 ms) : 42507, 43242
. : milestone, 42874,
section candidate
no_agent (37.492 ms) : 37190, 37795
. : milestone, 37492,
appsec (47.29 ms) : 46864, 47716
. : milestone, 47290,
code_origins (44.154 ms) : 43772, 44535
. : milestone, 44154,
iast (44.418 ms) : 44037, 44799
. : milestone, 44418,
profiling (49.821 ms) : 49340, 50302
. : milestone, 49821,
tracing (44.265 ms) : 43896, 44635
. : milestone, 44265,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics. Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.55.0-SNAPSHOT~f654867f52, baseline=1.55.0-SNAPSHOT~a632fe70e1
dateFormat X
axisFormat %s
section baseline
no_agent (1.474 ms) : 1463, 1486
. : milestone, 1474,
appsec (2.459 ms) : 2407, 2510
. : milestone, 2459,
iast (2.217 ms) : 2153, 2281
. : milestone, 2217,
iast_GLOBAL (2.246 ms) : 2183, 2310
. : milestone, 2246,
profiling (2.069 ms) : 2016, 2122
. : milestone, 2069,
tracing (2.015 ms) : 1966, 2065
. : milestone, 2015,
section candidate
no_agent (1.475 ms) : 1464, 1487
. : milestone, 1475,
appsec (3.635 ms) : 3422, 3847
. : milestone, 3635,
iast (2.21 ms) : 2146, 2274
. : milestone, 2210,
iast_GLOBAL (2.251 ms) : 2187, 2315
. : milestone, 2251,
profiling (2.058 ms) : 2007, 2110
. : milestone, 2058,
tracing (2.035 ms) : 1985, 2086
. : milestone, 2035,
Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.55.0-SNAPSHOT~f654867f52, baseline=1.55.0-SNAPSHOT~a632fe70e1
dateFormat X
axisFormat %s
section baseline
no_agent (14.907 s) : 14907000, 14907000
. : milestone, 14907000,
appsec (15.106 s) : 15106000, 15106000
. : milestone, 15106000,
iast (18.746 s) : 18746000, 18746000
. : milestone, 18746000,
iast_GLOBAL (18.029 s) : 18029000, 18029000
. : milestone, 18029000,
profiling (15.001 s) : 15001000, 15001000
. : milestone, 15001000,
tracing (15.178 s) : 15178000, 15178000
. : milestone, 15178000,
section candidate
no_agent (14.793 s) : 14793000, 14793000
. : milestone, 14793000,
appsec (15.013 s) : 15013000, 15013000
. : milestone, 15013000,
iast (18.535 s) : 18535000, 18535000
. : milestone, 18535000,
iast_GLOBAL (18.356 s) : 18356000, 18356000
. : milestone, 18356000,
profiling (14.788 s) : 14788000, 14788000
. : milestone, 14788000,
tracing (15.024 s) : 15024000, 15024000
. : milestone, 15024000,
|
|
🎯 Code Coverage 🔗 Commit SHA: f654867 | Docs | Was this helpful? Give us feedback! |
Kafka / producer-benchmarkParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics. See unchanged results
|
Kafka / consumer-benchmarkParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics. See unchanged results
|
sarahchen6
commented
Oct 27, 2025
|
|
||
| apply from: "$rootDir/gradle/configure_tests.gradle" | ||
|
|
||
| tasks.named("latestDepTest", Test) { |
There was a problem hiding this comment.
These files should already be applying this plugin via gradle/java_no_deps.gradle.
sarahchen6
commented
Oct 27, 2025
| // maxRetries = 3 | ||
| // } | ||
| // } | ||
| // } |
There was a problem hiding this comment.
I'm not sure how to convert this part to Kotlin... Any help or recommendations would be appreciated!
Since CI is passing, I wonder if we can remove this section, but perhaps these CI runs did not need to leverage test retries whereas other runs will need to.
There was a problem hiding this comment.
This should be kept, the issue is likely to be that you need to acces the develocity types. Usually this involves referring to the plugin without applying it. But since this is develocity, I'm not sure.
sarahchen6
force-pushed
the
sarahchen6/configure-tests-convention-plugin
branch
from
24e44ed to
be41722
Compare
AlexeyKuznetsov-DD
requested changes
Oct 27, 2025
Comment on lines
100
to
118
| tasks.register("allTests") { | ||
| dependsOn(providers.provider { | ||
| tasks.withType<Test>().filter { testTask -> | ||
| !testTask.name.contains("latest", ignoreCase = true) && testTask.name != "traceAgentTest" | ||
| } | ||
| }) | ||
| } | ||
|
|
||
| tasks.register("allLatestDepTests") { | ||
| dependsOn(providers.provider { | ||
| tasks.withType<Test>().filter { testTask -> | ||
| testTask.name.contains("latest", ignoreCase = true) | ||
| } | ||
| }) | ||
| } | ||
|
|
||
| tasks.named("check") { | ||
| dependsOn(tasks.withType<Test>()) | ||
| } |
There was a problem hiding this comment.
Would be cool to comment each register block for future refernce
There was a problem hiding this comment.
Like this: afa8ed9?
bric3
reviewed
Oct 28, 2025
Comment on lines
+10
to
+28
| fun isTestingInstrumentation(project: Project): Boolean { | ||
| return listOf( | ||
| "cucumber", | ||
| "cucumber-junit-4", | ||
| "cucumber-junit-5", | ||
| "junit-4.10", | ||
| "junit-4.13", | ||
| "junit-5.3", | ||
| "junit-5.8", | ||
| "karate", | ||
| "munit-junit-4", | ||
| "scalatest", | ||
| "selenium", | ||
| "spock-junit-5", | ||
| "testng-6", | ||
| "testng-7", | ||
| "weaver" | ||
| ).contains(project.name) | ||
| } |
There was a problem hiding this comment.
thought: I would rather avoid having existing project in a convention plugin. Anything in buildSrc shouldn't be aware of dd-trace-java project, but some projects could provide answer to this plugin. E.g. with an project or task extension (...thinking out loudly).
Note, it was there before. But since the goal is to make it a convention plugin, I think this should be tackled sooner in this case.
Comment on lines
+51
to
+57
| val activePartitionProvider = providers.provider { | ||
| project.extra.properties["activePartition"] as? Boolean ?: true | ||
| } | ||
|
|
||
| // Go through the Test tasks and configure them | ||
| tasks.withType(Test::class.java).configureEach { | ||
| enabled = activePartitionProvider.get() |
There was a problem hiding this comment.
thought: I wonder if this "trait" or "feature" should be moved over the ci convention plugin ?
| // maxRetries = 3 | ||
| // } | ||
| // } | ||
| // } |
There was a problem hiding this comment.
This should be kept, the issue is likely to be that you need to acces the develocity types. Usually this involves referring to the plugin without applying it. But since this is develocity, I'm not sure.
|
|
||
| apply from: "$rootDir/gradle/configure_tests.gradle" | ||
|
|
||
| tasks.named("latestDepTest", Test) { |
| } | ||
|
|
||
| apply from: "$rootDir/gradle/configure_tests.gradle" | ||
| apply plugin: 'datadog.configure-tests' |
There was a problem hiding this comment.
suggestion: Does it still work when it's moved at the top ?
Comment on lines
+47
to
+53
| val skipTestsProvider = rootProject.providers.gradleProperty("skipTests") | ||
| val skipForkedTestsProvider = rootProject.providers.gradleProperty("skipForkedTests") | ||
| val skipFlakyTestsProvider = rootProject.providers.gradleProperty("skipFlakyTests") | ||
| val runFlakyTestsProvider = rootProject.providers.gradleProperty("runFlakyTests") | ||
| val activePartitionProvider = providers.provider { | ||
| project.extra.properties["activePartition"] as? Boolean ?: true | ||
| } |
There was a problem hiding this comment.
note: Maybe, these could end-up in the extension I mentioned earlier. I'm kind if undecided there.
Comment on lines
+105
to
+109
| dependsOn(providers.provider { | ||
| tasks.withType<Test>().filter { testTask -> | ||
| !testTask.name.contains("latest", ignoreCase = true) && testTask.name != "traceAgentTest" | ||
| } | ||
| }) |
There was a problem hiding this comment.
suggestion: I think the provider is not needed there, but filter here is a faux-ami in this case as it works against the collection (which means the tasks have to be "realized"). Instead to filter lazily a TaskCollection (and any gradle *DomainObject*), the matching method is more adapted to the situation.
Suggested change
| dependsOn(providers.provider { | |
| tasks.withType<Test>().filter { testTask -> | |
| !testTask.name.contains("latest", ignoreCase = true) && testTask.name != "traceAgentTest" | |
| } | |
| }) | |
| dependsOn(tasks.withType<Test>().matching { testTask -> | |
| !testTask.name.contains("latest", ignoreCase = true) && testTask.name != "traceAgentTest" | |
| }) |
| // Register a task "allLatestDepTests" that depends on all Test tasks whose names include 'latest'. | ||
| // This is used when we want to run tests against the latest dependency versions. | ||
| tasks.register("allLatestDepTests") { | ||
| dependsOn(providers.provider { |
| dependsOn(tasks.withType<Test>()) | ||
| } | ||
|
|
||
| tasks.withType(Test::class.java).configureEach { |
There was a problem hiding this comment.
nitpick:
Suggested change
| tasks.withType(Test::class.java).configureEach { | |
| tasks.withType<Test>().configureEach { |
| } | ||
|
|
||
| // Go through the Test tasks and configure them | ||
| tasks.withType(Test::class.java).configureEach { |
There was a problem hiding this comment.
nitpick:
Suggested change
| tasks.withType(Test::class.java).configureEach { | |
| tasks.withType<Test>().configureEach { |
|
Cool, I have been eying to convert this script. Looking further down the road, I believe this could be split in two, as some tests config might only make sense for integrations, not so much for other components. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What Does This Do
Turn
configure_tests.gradlefile into a convention plugin. Make some lazy API improvements along the way as well.Motivation
Improve our build with convention plugins and lazy API!
Additional Notes
Contributor Checklist
type:and (comp:orinst:) labels in addition to any useful labelsclose,fixor any linking keywords when referencing an issue.Use
solvesinstead, and assign the PR milestone to the issueJira ticket: [PROJ-IDENT]