[SqlClient] Add benchmarks for SqlProcessor and implement initial quick wins

[stevejgordon]

Contributes to #2238
Follow up from #3023

Changes

Implement some initial quick wins to avoid SqlProcessor overhead.

• Avoid boxing using Dictionary rather than Hashtable
• Set initial StringBuilder capacity
• Avoid SubString on NET targets

These changes approximately half the time required when a statement is cached (moving to Dictionary). For non-cached statements they save approx 30% if the allocations and reduce execution time by approx 36%.

Before (with cache disabled)

BenchmarkDotNet v0.13.12, Windows 11 (10.0.26100.4946)
Unknown processor
.NET SDK 9.0.304
  [Host]     : .NET 9.0.8 (9.0.825.36511), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
  DefaultJob : .NET 9.0.8 (9.0.825.36511), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI


| Method | Sql                  | Iterations | Mean       | Error    | StdDev    | Gen0   | Allocated |
|------- |--------------------- |----------- |-----------:|---------:|----------:|-------:|----------:|
| Simple | SELEC(...)ls od[39]  | 1          |   445.2 ns |  4.32 ns |   3.83 ns | 0.0687 |     864 B |
| Simple | SELE(...)_id) [101]  | 1          | 1,349.6 ns | 46.81 ns | 136.55 ns | 0.0935 |    1184 B |

After (with cache disabled)

BenchmarkDotNet v0.13.12, Windows 11 (10.0.26100.4946)
Unknown processor
.NET SDK 9.0.304
  [Host]     : .NET 9.0.8 (9.0.825.36511), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
  DefaultJob : .NET 9.0.8 (9.0.825.36511), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI


| Method | Sql                  | Iterations | Mean     | Error    | StdDev   | Gen0   | Allocated |
|------- |--------------------- |----------- |---------:|---------:|---------:|-------:|----------:|
| Simple | SELEC(...)ls od [39] | 1          | 355.9 ns |  3.49 ns |  2.92 ns | 0.0420 |     528 B |
| Simple | SELE(...)_id) [101]  | 1          | 804.1 ns | 14.63 ns | 12.97 ns | 0.0725 |     912 B |

I've left the summary benchmark results in SqlProcessorBenchmarks for comparison.

We can save several hundred bytes per statement (exact saving depends on the input) by avoid the StringBuilders. This is a larger change to switch to rented memory + span processing, at least for NET targets. I would expect execution time to reduce also. This is something I can investigate if we think the gain offsets the effort? I'd also look to remove the SqlProcessorState to save another 40B per uncached statement.

Merge requirement checklist

• CONTRIBUTING guidelines followed (license requirements, nullable enabled, static analysis, etc.)
• Unit tests added/updated
• Appropriate CHANGELOG.md files updated for non-trivial changes
• Changes in public API reviewed (if applicable)

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 69.71%. Comparing base (c9913e8) to head (b2e5283).
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3031      +/-   ##
==========================================
- Coverage   69.81%   69.71%   -0.11%     
==========================================
  Files         403      413      +10     
  Lines       16184    16240      +56     
==========================================
+ Hits        11299    11321      +22     
- Misses       4885     4919      +34     

Flag	Coverage Δ
unittests-Contrib.Shared.Tests	87.50% <100.00%> (+0.13%)	⬆️
unittests-Exporter.Geneva	53.47% <ø> (+0.13%)	⬆️
unittests-Exporter.InfluxDB	95.14% <ø> (ø)
unittests-Exporter.Instana	74.86% <ø> (ø)
unittests-Exporter.OneCollector	94.61% <ø> (ø)
unittests-Extensions	91.12% <ø> (ø)
unittests-Extensions.Enrichment	100.00% <ø> (ø)
unittests-Instrumentation.AWS	83.80% <ø> (ø)
unittests-Instrumentation.AspNet	74.93% <ø> (-0.25%)	⬇️
unittests-Instrumentation.AspNetCore	70.76% <ø> (ø)
unittests-Instrumentation.Cassandra	23.52% <ø> (?)
unittests-Instrumentation.ConfluentKafka	14.10% <ø> (ø)
unittests-Instrumentation.ElasticsearchClient	80.12% <ø> (ø)
unittests-Instrumentation.EntityFrameworkCore	80.83% <ø> (ø)
unittests-Instrumentation.EventCounters	76.36% <ø> (ø)
unittests-Instrumentation.GrpcCore	91.42% <ø> (ø)
unittests-Instrumentation.GrpcNetClient	79.61% <ø> (ø)
unittests-Instrumentation.Hangfire	84.61% <ø> (ø)
unittests-Instrumentation.Http	74.18% <ø> (ø)
unittests-Instrumentation.Owin	88.62% <ø> (ø)
unittests-Instrumentation.Process	100.00% <ø> (ø)
unittests-Instrumentation.Quartz	78.76% <ø> (ø)
unittests-Instrumentation.Runtime	100.00% <ø> (ø)
unittests-Instrumentation.ServiceFabricRemoting	34.54% <ø> (ø)
unittests-Instrumentation.SqlClient	87.16% <ø> (ø)
unittests-Instrumentation.StackExchangeRedis	70.81% <ø> (ø)
unittests-Instrumentation.Wcf	78.95% <ø> (ø)
unittests-OpAmp.Client	61.08% <ø> (ø)
unittests-PersistentStorage	65.55% <ø> (-0.34%)	⬇️
unittests-Resources.AWS	75.00% <ø> (ø)
unittests-Resources.Azure	84.56% <ø> (ø)
unittests-Resources.Container	67.34% <ø> (ø)
unittests-Resources.Gcp	71.42% <ø> (ø)
unittests-Resources.Host	73.91% <ø> (ø)
unittests-Resources.OperatingSystem	76.98% <ø> (ø)
unittests-Resources.Process	100.00% <ø> (ø)
unittests-Resources.ProcessRuntime	79.59% <ø> (ø)
unittests-Sampler.AWS	88.25% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
src/Shared/SqlProcessor.cs	97.80% <100.00%> (+0.06%)	⬆️

... and 14 files with indirect coverage changes

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[martincostello]

Nice.

It's much easier to grok the improvement by looking at this 👍

[Kielek]

Only some infra stuff comments.

I know that you are adding benchmarks to the Shared folder, but it is done mostly for SqlClient.

There are 2 options to fix it.

1. Move your code to SqlClient specific test project, as defined under

   opentelemetry-dotnet-contrib/build/Projects/Component.proj

   Lines 14 to 16 in c9913e8

   	<SolutionProjects
   	Condition="Exists('$(RepoRoot)\test\$(BUILD_COMPONENT).Benchmarks\$(BUILD_COMPONENT).Benchmarks.csproj')"
   	Include="$(RepoRoot)\test\$(BUILD_COMPONENT).Benchmarks\$(BUILD_COMPONENT).Benchmarks.csproj" />

   . No more infrastructure related stuff is needed.
2. Keep it as Shared benchmarks, it is also fine, but requires some additional changes.

• need to update detection changes:

  opentelemetry-dotnet-contrib/.github/workflows/ci.yml

  Line 33 in c9913e8

  contrib-shared-tests: ['test/OpenTelemetry.Contrib.Shared.Tests/**']

• Extend this project definiotion https://github.com/open-telemetry/opentelemetry-dotnet-contrib/blob/c9913e8e36c741f85b22f3b1ce8377389fe26d1d/build/Projects/OpenTelemetry.Contrib.Shared.Tests.proj by benchmarks, similarly to

  opentelemetry-dotnet-contrib/build/Projects/Component.proj

  Lines 14 to 16 in c9913e8

  	<SolutionProjects
  	Condition="Exists('$(RepoRoot)\test\$(BUILD_COMPONENT).Benchmarks\$(BUILD_COMPONENT).Benchmarks.csproj')"
  	Include="$(RepoRoot)\test\$(BUILD_COMPONENT).Benchmarks\$(BUILD_COMPONENT).Benchmarks.csproj" />

I have resolved conflicts with the main during review.

EDIT: The second option is probably better from the infrastructure perspective, the first one, for the SqlClient developers.