feat(sr): Support image capture in Session Replay #813
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Current support for Android image processing is very slow, but confined to a background thread on Android. However, because we halt collection until individual images are processed, this will negatively affect replays with a large number of images. We'll need to look for an async method of processing images or speed it up significantly on Android refs: RUM-11125 # Conflicts: # packages/datadog_session_replay/example/lib/app.dart # packages/datadog_session_replay/example/lib/routes.dart # packages/datadog_session_replay/example/lib/screens/main_screen.dart
fuzzybinary
force-pushed
the
jward/RUM-11125-image-capture
branch
from
1b893f4 to
7e8c260
Compare
fuzzybinary
force-pushed
the
jward/RUM-11125-image-capture
branch
from
5a35f05 to
efdc738
Compare
jonathanmos
reviewed
Aug 27, 2025
| val resourceWriter: ResourceWriter, | ||
| val bitmapHandler: BitmapHandler = DefaultBitmapHandler(internalLogger) | ||
| ) : ResourceResolver { | ||
| private val resourceKeyMap: MutableMap<Int, ResourceResolver.ResourceEntry> = mutableMapOf() |
There was a problem hiding this comment.
It looks like items are added to resourceKeyMap, but I don't see where they are cleared from it. Also, should we limit the size of the map? Same questions about knownResources.
There was a problem hiding this comment.
Right now we're not clearing them, except at the end of the session. I'd have to think about under what circumstances we wanted to clear them. What criteria does the Android SDK use? I didn't see it.
There was a problem hiding this comment.
I'll probably do that as a separate task. Created RUM-11500 to track
...lay/android/src/main/kotlin/com/datadoghq/flutter/sessionreplay/resource/ResourceResolver.kt
Show resolved
Hide resolved
...ndroid/src/test/kotlin/com/datadoghq/flutter/sessionreplay/FlutterSessionReplayBridgeTest.kt
Outdated
Show resolved
Hide resolved
...ndroid/src/test/kotlin/com/datadoghq/flutter/sessionreplay/FlutterSessionReplayBridgeTest.kt
Outdated
Show resolved
Hide resolved
...android/src/test/kotlin/com/datadoghq/flutter/sessionreplay/resource/ResourceResolverTest.kt
Outdated
Show resolved
Hide resolved
mariedm
previously approved these changes
Aug 27, 2025
packages/datadog_session_replay/lib/src/ios/datadog_session_replay_platform_ios.dart
Outdated
Show resolved
Hide resolved
...tadog_session_replay/ios/datadog_session_replay/Sources/Swift/Resource/ResouceResolver.swift
Outdated
Show resolved
Hide resolved
...tadog_session_replay/ios/datadog_session_replay/Sources/Swift/Resource/ResourcesWriter.swift
Outdated
Show resolved
Hide resolved
...ession_replay/ios/datadog_session_replay/Sources/Swift/Resource/ResourceRequestBuilder.swift
Show resolved
Hide resolved
| internal var mimeType: String | ||
| internal var context: Context | ||
|
|
||
| internal init( |
There was a problem hiding this comment.
nit: I don't think this init is needed; since EnrichedResource is a struct, you get memberwise initializer for free.
There was a problem hiding this comment.
I don't know why this is needed, but it is apparently?
jonathanmos
previously approved these changes
Aug 28, 2025
...lay/android/src/main/kotlin/com/datadoghq/flutter/sessionreplay/resource/ResourceResolver.kt
Outdated
Show resolved
Hide resolved
packages/datadog_session_replay/lib/src/capture/element_recorders/image_recorder.dart
Outdated
Show resolved
Hide resolved
...on_replay/ios/datadog_session_replay/Sources/Swift/Feature/FlutterSeasionReplayFeature.swift
Show resolved
Hide resolved
fuzzybinary
force-pushed
the
jward/RUM-11125-image-capture
branch
from
b50bab8 to
5f4505f
Compare
fuzzybinary
force-pushed
the
jward/RUM-11125-image-capture
branch
from
5f4505f to
3d2e603
Compare
jonathanmos
approved these changes
Aug 31, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What and why?
Allow Session Replay tree traversal to capture images from Session Replay.
This works by using
RawImage'stoByteData(format: ImageByteFormat.rawRgba)method, which gives a raw, RGBA image with premultiplied alpha. This is then copied directly to the native layer for processing.On Android, we transfer the data using a
ByteBuffer, and then use that buffer to create aBitmapfor compression and hashing. Because Android's raw bitmap format is also RGBA premultiplied (despite what the constant says), we can simply pass in the wrapped buffer to create the requiredBitmapOn iOS, we create
Data/NSDatawhich copies the data directly from the buffer pointer. Doing this relatively safely requires we bypass theobjective_cpackage, as usingaddressin Dart requires calling an FFI leaf function (one that can be guaranteed not to call back into Dart) directly.Because of the memory constraints of the copy, we are limiting copies to 640,000 pixels (an 800x800 image), which would be a raw size of 2.4 MB.
Any images larger than that are captured as "Large Image", shown below.
This still needs to support caching which images we've sent between sessions.
Review checklist