Decrease binary and module cache size #2590
Description
This issue is to group information related to sizes.
We are using official API clients to reduce the maintenance: it's easier to update a dependency than maintain an internal implementation based on API documentation (the quality of the API documentation is very often not good, and documentation changes can be complex to follow).
The different changes related to API clients, between v4.23.1 and now, produced a reduction in sizes:
- Binary size:
| v4.23.1 | v4.25.0 | Δ | |
|---|---|---|---|
| without stripping | 158 MB | 85 MB | -71 MB / -46% |
| with stripping * | 109 MB | 60 MB | -49 MB / -45% |
*: This is the way we build the official binaries.
- Cache size:
| v4.23.1 | v4.25.0 | Δ | |
|---|---|---|---|
| Download | 1.724 GiB 147971 files 14955 directories |
1.407 GiB 89658 files 13933 directories |
-317 MiB -58313 files -1022 directories |
| Tidy | 1.762 GiB 151108 files 15672 directories |
1.442 GiB 92567 files 14580 directories |
-320 MiB -58541 files -1092 directories |
oraclecloud
Replacement of the official API client by a fork based on modules.
| Version | Size inside the binary | Module size (GOCACHE) |
|---|---|---|
| official | 1.6 MB | 109.86 MiB |
| fork | 1.6 MB | 1.6 MiB |
- oraclecloud: replace oci-go-sdk by a modular fork #2556
- https://github.com/nrdcg/oci-go-sdk
cloudflare
Instead of bumping to v4, the replacement of the official API client by an internal implementation.
| Version | Size inside the binary | Module size (GOCACHE) |
|---|---|---|
| v0.115 | 2.2 MB | 3.166 MiB |
| v4.5.1 | 5.6 MB | 29.393 MiB |
| internal | 10 kB | - |
- chore: replace official Cloudflare API client by an internal API client #2583
- Client size cloudflare/cloudflare-go#4171
Civo
New versions of the official API client depend on Kubernetes.
Replacement of the API client by an internal implementation.
| Version | Size inside the binary | Module size (GOCACHE) |
|---|---|---|
| official | 229 kB | 734.251 KiB |
| internal | 8.2 kB | - |
alidns
Updates the official API client to v4 and replaces it with a fork.
| Version | Size inside the binary | Module size (GOCACHE) |
|---|---|---|
| v1.63.100 | 11 MB | 88.654 MiB |
| v4.5.10 | 4.6 MB | 2.04 MiB |
| fork | 112 kB | 2.04 MiB |
- alidns: migrate to SDK v2 #2558
- alidns: replace alidns-20150109 with a fork #2589
- https://github.com/go-acme/alidns-20150109
tencentcloud/dnspod
Replacement of the official API client by a fork.
| Version | Size inside the binary | Module size (GOCACHE) |
|---|---|---|
| official | 3.9 MB | 1.1 MiB |
| fork | 183 kB | 1.1 MiB |
- tencentcloud: replace tencentcloud-sdk-go with a fork #2588
- https://github.com/go-acme/tencentclouddnspod
huaweicloud
Replacement of the structure DnsClient with an internal implementation
| Version | Size inside the binary | Module size (GOCACHE) |
|---|---|---|
| official | 3.9 MB | 65.5 MiB |
| internal | 259 kB | 65.5 MiB |
yandex
Usage of API client v2 (modules)
| Version | Size inside the binary | Module size (GOCACHE) |
|---|---|---|
| v1 | ~31 MB | ~40 MiB |
| v2 | ~505 kB | ~31 MiB |
Sakura Cloud
Uses a custom API client constructor.
We could hope for other changes to remove about 2.24 MB of the binary size.
| Version | Size inside the binary | Module size (GOCACHE) |
|---|---|---|
| official | 3.8 MB | 3.9 MiB |
| constructor | 2.3 MB | 3.9 MiB |
Azure
This client is now the biggest element inside the module cache.
The azure provider is deprecated inside lego since 2023, and it is replaced by azuredns provider.
I planned to remove the old SDK (module cache size: >497.4 MiB) in v5 of lego.
AWS
This client is now the biggest element inside the binary, but the client is complex and cannot be replaced easily.
Module cache size
| Name | Module cache size |
|---|---|
| github.com/aws/aws-sdk-go-v2/service/[email protected] | 3.972 MiB |
| github.com/aws/aws-sdk-go-v2/service/[email protected] | 3.280 MiB |
| github.com/aws/[email protected] | 2.304 MiB |
| github.com/aws/aws-sdk-go-v2/service/[email protected] | 1.840 MiB |
| github.com/aws/[email protected] | 732.014 KiB |
| github.com/aws/aws-sdk-go-v2/service/[email protected] | 494.559 KiB |
| github.com/aws/aws-sdk-go-v2/[email protected] | 407.385 KiB |
| github.com/aws/aws-sdk-go-v2/service/[email protected] | 287.202 KiB |
| github.com/aws/aws-sdk-go-v2/service/[email protected] | 245.332 KiB |
| github.com/aws/aws-sdk-go-v2/[email protected] | 184.670 KiB |
| github.com/aws/aws-sdk-go-v2/service/internal/[email protected] | 169.932 KiB |
| github.com/aws/aws-sdk-go-v2/feature/ec2/[email protected] | 118.485 KiB |
| github.com/aws/aws-sdk-go-v2/internal/[email protected] | 101.403 KiB |
| github.com/aws/aws-sdk-go-v2/service/internal/[email protected] | 77.191 KiB |
| github.com/aws/aws-sdk-go-v2/internal/endpoints/[email protected] | 66.960 KiB |
| github.com/aws/aws-sdk-go-v2/aws/protocol/[email protected] | 64.454 KiB |
| github.com/aws/aws-sdk-go-v2/internal/[email protected] | 41.693 KiB |
| github.com/aws/aws-sdk-go-v2/service/internal/[email protected] | 32.621 KiB |
| github.com/aws/aws-sdk-go-v2/service/internal/[email protected] | 26.220 KiB |
| github.com/aws/aws-sdk-go-v2/internal/[email protected] | 25.744 KiB |
Size inside the binary
| Percent | Name | Size inside the binary |
|---|---|---|
| 15.98% | github.com/aws/aws-sdk-go-v2 | 14 MB |
| 0.31% | github.com/aws/smithy-go | 258 kB |
Details of github.com/aws/aws-sdk-go-v2:
| Name | Size inside the binary |
|---|---|
| github.com/aws/aws-sdk-go-v2/service/lightsail | 4.8 MB |
| github.com/aws/aws-sdk-go-v2/service/s3 | 4.1 MB |
| github.com/aws/aws-sdk-go-v2/service/route53 | 2.7 MB |
| github.com/aws/aws-sdk-go-v2/service/sts | 496 kB |
| github.com/aws/aws-sdk-go-v2/aws | 381 kB |
| github.com/aws/aws-sdk-go-v2/service/ssooidc | 290 kB |
| github.com/aws/aws-sdk-go-v2/config | 257 kB |
| github.com/aws/aws-sdk-go-v2/service/sso | 225 kB |
| github.com/aws/aws-sdk-go-v2/credentials | 78 kB |
| github.com/aws/aws-sdk-go-v2/feature/ec2/imds | 53 kB |
| github.com/aws/aws-sdk-go-v2/service/internal/checksum | 50 kB |
| github.com/aws/aws-sdk-go-v2/internal/v4a | 48 kB |
| github.com/aws/aws-sdk-go-v2/service/internal/s3shared | 39 kB |
| github.com/aws/aws-sdk-go-v2/internal/ini | 16 kB |
| github.com/aws/aws-sdk-go-v2/internal/endpoints | 8.5 kB |
| github.com/aws/aws-sdk-go-v2/internal/sync/singleflight | 7.8 kB |
| github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding | 6.1 kB |
| github.com/aws/aws-sdk-go-v2/internal/auth | 5.1 kB |
| github.com/aws/aws-sdk-go-v2/internal/configsources | 3.0 kB |
| github.com/aws/aws-sdk-go-v2/internal/middleware | 2.1 kB |
| github.com/aws/aws-sdk-go-v2/internal/sdk | 1.6 kB |
| github.com/aws/aws-sdk-go-v2/internal/rand | 1.4 kB |
| github.com/aws/aws-sdk-go-v2/internal/shareddefaults | 296 B |
| github.com/aws/aws-sdk-go-v2/service/internal/presigned-url | 0 B |
Google Cloud
This client is the second biggest element inside the module cache, but the client is complex and cannot be replaced easily.
Module cache size
| Name | Module cache size |
|---|---|
| google.golang.org/[email protected] | 314.986 MiB |
| google.golang.org/[email protected] | 12.995 MiB |
| google.golang.org/[email protected] | 9.204 MiB |
| github.com/google/[email protected] | 1.276 MiB |
| google.golang.org/genproto/googleapis/[email protected] | 1014.847 KiB |
| cloud.google.com/go/[email protected] | 801.912 KiB |
| github.com/google/[email protected] | 460.904 KiB |
| github.com/googleapis/gax-go/[email protected] | 182.272 KiB |
| google.golang.org/genproto/googleapis/[email protected] | 164.189 KiB |
| github.com/googleapis/[email protected] | 159.428 KiB |
| github.com/google/[email protected] | 77.172 KiB |
| cloud.google.com/go/compute/[email protected] | 72.847 KiB |
| cloud.google.com/go/auth/[email protected] | 34.354 KiB |
Size inside the binary
| Percent | Name | Size inside the binary |
|---|---|---|
| 2.23% | google.golang.org/protobuf | 1.9 MB |
| 1.32% | google.golang.org/grpc | 1.1 MB |
| 0.59% | google.golang.org/api | 498 kB |
| 0.28% | github.com/google/s2a-go | 234 kB |
| 0.23% | cloud.google.com/go/auth | 193 kB |
| 0.15% | google.golang.org/genproto/googleapis/api | 124 kB |
| 0.06% | github.com/googleapis/gax-go/v2 | 50 kB |
| 0.06% | google.golang.org/genproto/googleapis/rpc | 47 kB |
| 0.04% | cloud.google.com/go/compute/metadata | 36 kB |
| 0.01% | github.com/googleapis/enterprise-certificate-proxy | 12 kB |
| 0.01% | github.com/google/uuid | 6.6 kB |
More
For now, I won't trim the other modules (like AWS) because it isn't worth the cost (complexity, small gain, regressions, maintenance, etc.)