@aws-sdk/client-s3 (@aws-sdk/signature-v4-crt) 3.29.0 introduces aws-crt and deprecated dependencies

[yo1dog]

Describe the bug

When updating to 3.29.0, @aws-sdk/client-s3 and @aws-sdk/s3-request-presigner now have @aws-sdk/signature-v4-crt as a dependency, which has aws-crt as a dependency, which is both very large (over 200MB) and has several deprecated dependencies. Notably: har-validator@5.1.5, uuid@3.4.0, and request@2.88.2. The total number of dependencies for @aws-sdk/client-s3 also more than triples from 85 to 273.

Your environment

SDK version number

@aws-sdk/client-s3@3.29.0
@aws-sdk/s3-request-presigner@3.29.0
@aws-sdk/signature-v4-crt@3.29.0

Is the issue in the browser/Node.js/ReactNative?

Node.js

Details of the browser/Node.js/ReactNative version

Node v14.17.6

Steps to reproduce

% npm i @aws-sdk/client-s3@3.29.0
npm WARN deprecated har-validator@5.1.5: this library is no longer supported
npm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142

added 273 packages, and audited 274 packages in 9s

found 0 vulnerabilities

% du -sh node_modules
244M    node_modules

% npm why har-validator
har-validator@5.1.5
node_modules/har-validator
  har-validator@"~5.1.3" from request@2.88.2
  node_modules/request
    request@"^2.54.0" from cmake-js@6.1.0
    node_modules/cmake-js
      cmake-js@"6.1.0" from aws-crt@1.9.0
      node_modules/aws-crt
        aws-crt@"^1.9.0" from @aws-sdk/signature-v4-crt@3.29.0
        node_modules/@aws-sdk/signature-v4-crt
          @aws-sdk/signature-v4-crt@"3.29.0" from @aws-sdk/middleware-sdk-s3@3.29.0
          node_modules/@aws-sdk/middleware-sdk-s3
            @aws-sdk/middleware-sdk-s3@"3.29.0" from @aws-sdk/client-s3@3.29.0
            node_modules/@aws-sdk/client-s3
              @aws-sdk/client-s3@"^3.29.0" from the root project

% npm i @aws-sdk/client-s3@3.28.0

added 85 packages, and audited 86 packages in 4s

found 0 vulnerabilities

% du -sh node_modules
28M     node_modules

Observed behavior

Warnings about deprecated packages are printed when updating. Very large node_modules directory.

Expected behavior

No deprecated or overly large packages are installed when updating.

[jvlch]

I am also having issues with the new release. My codebuild projects running a serverless package/deploy are unable to correctly trace and package lambdas.

Serverless: [serverless-jetpack] .serverless/examplelambdaname.zip dependency package tracing misses:
--
574 | - node_modules/aws-crt/dist/native/binding.js [55:18]: require(path)

relevant snippet from above file:

const bin_path = path.resolve(source_root, 'bin');
const search_paths = [
    path.join(bin_path, 'native', binary_name),
    path.join(bin_path, platformDir, binary_name),
];
let binding;
for (const path of search_paths) {
    if (fs_1.existsSync(path + '.node')) {
        binding = require(path);
        break;
    }
}

[darrinholst]

I'm having issues as well on alpine...

/app # node
Welcome to Node.js v14.17.6.
Type ".help" for more information.
> require('aws-crt')
Segmentation fault
/app # 

and

    Error loading shared library ld-linux-x86-64.so.2: No such file or directory (needed by /builds/node_modules/aws-crt/dist/bin/linux-x64/aws-crt-nodejs.node)
      at Runtime._loadModule (node_modules/jest-runtime/build/index.js:991:29)
      at Object.<anonymous> (node_modules/aws-crt/lib/native/binding.js:40:19)

[tschleu]

We ran into the size issue of the dependencies:

"Unzipped size must be smaller than 201542732 bytes (Service: Lambda, Status Code: 400, Request ID: 569bd 46f-a948-4fe5-b218-d08ee5c607bd, Extended Request ID: null)"

Even with a minimal dependency like

  "dependencies": {
    "@aws-sdk/client-s3": "3.29.0"
  },

The node_modules folder is 20750416 in size which is already too large to be deployed.

[lukasa1993]

this makes it unusable how this lib breaks at every update ?

[tripodsan]

we also run into problems when trying to bundle and deploy on lambda:

2021-09-04T18:17:33.745Z	34da863b-c195-438e-b323-9b03b4d13ed9	ERROR	error while invoking function Error: AWS CRT binary not present in any of the following locations:
	/var/task/node_modules/aws-crt/bin/native/aws-crt-nodejs
	/var/task/node_modules/aws-crt/bin/linux-x64/aws-crt-nodejs
    at Object../node_modules/aws-crt/dist/native/binding.js (/var/task/index.js:163137:11)
    at __webpack_require__ (/var/task/index.js:322288:42)
    at Object../node_modules/aws-crt/dist/native/crt.js (/var/task/index.js:163172:35)
    at __webpack_require__ (/var/task/index.js:322288:42)
    at Object../node_modules/aws-crt/dist/index.js (/var/task/index.js:162591:26)
    at __webpack_require__ (/var/task/index.js:322288:42)
    at Object../node_modules/@aws-sdk/signature-v4-crt/dist/cjs/CrtSignerV4.js (/var/task/index.js:125828:19)
    at __webpack_require__ (/var/task/index.js:322288:42)
    at Object../node_modules/@aws-sdk/signature-v4-crt/dist/cjs/index.js (/var/task/index.js:126098:27)
    at __webpack_require__ (/var/task/index.js:322288:42)

[bluepeter]

Ugh. Same here :(

[mtalbert]

I am having the same issue. The aws-crt dependency is forcing the filesize of an unpacked Lambda Layer to be larger than the 250MB hard limit. Does anyone know a version of @aws-sdk/client-s3 that does not carry this heavy dependency? I would prefer to use this than revert back to the aws-sdk dependency.

[utsav-techwondoe]

I kept on having this issue

    Error loading shared library ld-linux-x86-64.so.2: No such file or directory (needed by /builds/node_modules/aws-crt/dist/bin/linux-x64/aws-crt-nodejs.node)
      at Runtime._loadModule (node_modules/jest-runtime/build/index.js:991:29)
      at Object.<anonymous> (node_modules/aws-crt/lib/native/binding.js:40:19)

For temp. resolving it, I reverted my s3-client lib to 3.26.0

[pnutmath]

I have also issue with this version,

Server terminations signals getting after this update

[rtrvrtg]

To get my Serverless deployments working after this update, I had to blow away the following files before running sls deploy:

• node_modules/aws-crt/crt/aws-lc/crypto/hpke/test-vectors.json
• node_modules/aws-crt/crt/aws-lc/generated-src/*.cc
• node_modules/aws-crt/crt/aws-lc/third_party/wycheproof_testvectors
• node_modules/aws-crt/crt/aws-lc/ssl/test

And while it's not related to aws-crt, since it's still related to the AWS SDK:

• node_modules/aws-sdk/dist/*-react-native.js

This is a pretty bad hack, and won't help everyone, but it might help make sure to keep your Lambda bundles free of a lot of the bulky source or test data that won't be used in your PROD environments anyway. It also won't do much to help people who use React Native or need to do testing stuff in their Lambda envs.

[hiepxanh]

I like your action ❤️ blow away , BTW you blow away our block today, thank you so much, it such a heavy stone on our way