[FEATURE]: Allow LLMObs to start / end spans

[yunfeng-enrich]

Package Name

No response

Package Version(s)

No response

Describe the feature you'd like

Hi, currently llmobs spans can only wrap around a function. This does not work for streaming calls where the end is likely not in the same function. Can you help add functionality to allow start / end of spans? Thank you!

Is your feature request related to a problem?

No response

Describe alternatives you've considered

No response

Additional context

No response

[sabrenner]

Hi @yunfeng-enrich! Thanks for opening this FR 😄 This is something we've thought about adding. But, i'm curious about your use case first, as I believe it should be possible to wrap the entirety of the streamed call in a block, with something like

let input
await llmobs.trace({ kind: 'llm', name: 'streamedCall' }), async () => {
  const response = // make response to get async iterator streamed response
  let result
  for await (const chunk of response) {
    result += // handle coalescing chunks
  }

  llmobs.annotate({ inputData: input, outputData: result })
})

Let me know if this provides an alternative for you, or if your use case is a bit different - I'm happy to provide a path forward or look into adding manual start/end for LLM Observability spans!

[yunfeng-enrich]

@sabrenner good point. yes it would work for my scenario although I had to rearrange things a bit differently since I have a control flow in my code, it looks like:
while () {
const response = make_call()
if (response is error) {
...
continue
}
for all chunks {
...
}
}
I need to deal with "continue" separately but it'll still work

[sabrenner]

@yunfeng-enrich awesome, feel free to update in this thread if that solution works out or if you run into any other issues 😄