Cannot determine payload type, datacontenttype is undefined

[EthanSK]

[REQUIRED] Environment info

firebase-tools:
14.14.0

Platform:
macOS (latest)

[REQUIRED] Test case

I made a repro project https://github.com/EthanSK/onobjectfinalize-indirect

Detailed (but ai generated) intsructions are in the readme

It was hard to reliably get it to break at the same point every time, hence the multiple iterations to try and trigger the issue.

[REQUIRED] Steps to reproduce

make sure to do this in the inner node_modules under functions/, not the project root node modules. sorry about that:

add this log to line 186 of firestore.js , at the start of createBeforeSnapshot in functions/node_modules/firebase-functions/lib/v2/providers/firestore.js

console.log("create before snapshot: ", event)

, put a debug breakpoint on this line

    logger.error(`Cannot determine payload type, datacontenttype is ${event.datacontenttype}, failing out.`);

in one terminal, run npm run serve , then once thats booted up , in another terminal run ./test-race-conditions.sh

make sure the debugger is running

result:
after some iterations, the breakpoint should be activated. If you inspect the event data, you'll see it's a storage event, but this method seems only equipped to handle firestore events. This causes the function to error out and break for the remaining function invocations, rendering local development useless.

This is a serious dev breaking bug as my whole function instance shuts down after the error occurs.

this only works with debugger / inspect-functions

i know this might seem like an extreme overload and spam of the poor emulator, but in my actual app the overload conditions are much less and it happens consistently, but i couldnt quite boil down exactly the right cause, so i included all the possible reasons in the repro, and after a lot of wrangling got it to finally work consistently enough.

[REQUIRED] Expected behavior

It should not error, storage events should not be going through createBeforeSnapshot maybe? and not error out.

A hint of what might be happening may be to do with some race condition when storage triggers start overlapping, in a context that was iniated from firestore triggers. When the sleep between invocations is increased, this issue becomes harder and harde to repro.

[REQUIRED] Actual behavior

Storage events are going through firestore's createBeforeSnapshot in the emulators (sometimes, it is not predictable when this happens but its very likely to happen after some iterations in the repro project)

This only seems to happen under more intense load that overwhelms the storage emulator? or some emulator i dont know exactly what it could be (although I inspected the source code and had extensive conversations with AI)

>  }
>  {"severity":"ERROR","message":"Error: Cannot determine payload type, datacontenttype is undefined, failing out.\n    at entryFromArgs (/Users/ethansarif-kattan/Projects/aimvs repros/onobjectfinalize-indirect/functions/node_modules/firebase-functions/lib/logger/index.js:134:19)\n    at Object.error (/Users/ethansarif-kattan/Projects/aimvs repros/onobjectfinalize-indirect/functions/node_modules/firebase-functions/lib/logger/index.js:121:11)\n    at createBeforeSnapshot (/Users/ethansarif-kattan/Projects/aimvs repros/onobjectfinalize-indirect/functions/node_modules/firebase-functions/lib/v2/providers/firestore.js:194:16)\n    at makeChangedFirestoreEvent (/Users/ethansarif-kattan/Projects/aimvs repros/onobjectfinalize-indirect/functions/node_modules/firebase-functions/lib/v2/providers/firestore.js:236:37)\n    at func (/Users/ethansarif-kattan/Projects/aimvs repros/onobjectfinalize-indirect/functions/node_modules/firebase-functions/lib/v2/providers/firestore.js:317:32)\n    at /Users/ethansarif-kattan/Projects/aimvs repros/onobjectfinalize-indirect/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:524:16\n    at runFunction (/Users/ethansarif-kattan/Projects/aimvs repros/onobjectfinalize-indirect/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:506:15)\n    at runCloudEvent (/Users/ethansarif-kattan/Projects/aimvs repros/onobjectfinalize-indirect/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:523:11)\n    at processBackground (/Users/ethansarif-kattan/Projects/aimvs repros/onobjectfinalize-indirect/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:490:16)\n    at /Users/ethansarif-kattan/Projects/aimvs repros/onobjectfinalize-indirect/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:690:27"}
⚠  functions: Error: Error: Cannot parse event payload.
    at createBeforeSnapshot (/Users/ethansarif-kattan/Projects/aimvs repros/onobjectfinalize-indirect/functions/node_modules/firebase-functions/lib/v2/providers/firestore.js:195:15)
    at makeChangedFirestoreEvent (/Users/ethansarif-kattan/Projects/aimvs repros/onobjectfinalize-indirect/functions/node_modules/firebase-functions/lib/v2/providers/firestore.js:236:37)
    at func (/Users/ethansarif-kattan/Projects/aimvs repros/onobjectfinalize-indirect/functions/node_modules/firebase-functions/lib/v2/providers/firestore.js:317:32)
    at /Users/ethansarif-kattan/Projects/aimvs repros/onobjectfinalize-indirect/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:524:16
    at runFunction (/Users/ethansarif-kattan/Projects/aimvs repros/onobjectfinalize-indirect/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:506:15)
    at runCloudEvent (/Users/ethansarif-kattan/Projects/aimvs repros/onobjectfinalize-indirect/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:523:11)
    at processBackground (/Users/ethansarif-kattan/Projects/aimvs repros/onobjectfinalize-indirect/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:490:16)
    at /Users/ethansarif-kattan/Projects/aimvs repros/onobjectfinalize-indirect/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:690:27
    at Layer.handle [as handle_request] (/Users/ethansarif-kattan/Projects/aimvs repros/onobjectfinalize-indirect/node_modules/express/lib/router/layer.js:95:5)
    at next (/Users/ethansarif-kattan/Projects/aimvs repros/onobjectfinalize-indirect/node_modules/express/lib/router/route.js:149:13)
⚠  Your function was killed because it raised an unhandled error.
i  functions: Finished "us-central1-onUploadFileFinalize" in 16603.434ms