feat(mouse): enhance mouse events with grid coordinates #110

junkdog · 2025-06-30T13:42:26Z

Mouse events now report grid coordinates (col, row) instead of pixel coordinates, and event registration has been unified through the terminal API with proper lifecycle management.

Key Changes

Grid coordinates: MouseEvent now has col/row fields for terminal cell positions
Unified API: Event registration moved from backend options to terminal.on_mouse_event() / terminal.on_key_event()
Lifecycle management: Event callbacks are properly cleaned up and replaced to prevent memory leaks
Callback replacement: Registering a new callback automatically clears the previous one

Usage

// mut due to later holding onto the callbacks/closures
let mut terminal = MultiBackendBuilder::with_fallback(BackendType::Dom)
  .build_terminal()?;

terminal.on_mouse_event(|mouse_event| {
  println!("{:?} at cell: ({}, {})", mouse_event.kind, mouse_event.col, mouse_event.row);
}).ok(); // WebGL2 doesn't support mouse events yet

// This replaces any previous key event callback
terminal.on_key_event(|key_event| {
  println!("Key: {:?}", key_event.code);
}).ok();

Mouse Event Types

Available MouseEventKind variants:

Moved - mouse move event
ButtonDown(MouseButton) - button pressed
ButtonUp(MouseButton) - button released
Entered - mouse entered terminal area
Exited - mouse left terminal area
SingleClick(MouseButton) - single click (down + up)
DoubleClick(MouseButton), - single click x2 in quick succession
Wheel { delta_x, delta_y, delta_z } - scroll in delta cols, rows (should remove z)

Breaking Changes

Mouse coordinates changed from pixels to grid cells
Event registration: backend_options.mouse_event_handler() → terminal.on_mouse_event()
Terminal must be mut for event registration
Both mouse and key callbacks are now lifecycle managed (prevents leaks, replaces previous callbacks)

Backend Support

✅ DomBackend & CanvasBackend: Full support
✅ All backends: Key events with lifecycle management
❌ WebGL2Backend mouse events: Not yet supported (req feat(backend): update WebGl2Backend integration #108)

orhun

Nice, pretty clean and works!

Had some comments

src/backend/canvas.rs

orhun · 2025-07-13T18:39:18Z

src/backend/utils.rs

+    let width = if width > 0 { width } else { 10 };
+    let height = if height > 0 { height } else { 19 };


We defined these constants somewhere, probably those can be used.

added MouseEventHandler - a simple wrapper around the boxed fn.

orhun · 2025-07-13T18:40:01Z

src/backend/utils.rs

+///
+/// This function creates a span with the same styling as terminal cells,
+/// measures its dimensions, and then removes it from the DOM.
+pub(super) fn measure_dom_cell_size(


I'm curious about a couple of things:

Is this being affected by the font being used by the app? (e.g. defined in index.html)

We should probably avoid calling this frequently, because manipulating DOM takes time. Is that right?

i'll have to confirm - but iirc it's sparsely called (at init/resize maybe).

We should probably avoid calling this frequently, because manipulating DOM takes time. Is that right?

this is at least 1 or 2 PR:s ;) the DomBackend frequently does a layout pass on everything, at least that what i gathered from a brief look at the profiler. there's also a lack of memoization around all the CSS style strings, but in the grand scheme of things - it's not the current bottleneck (it marginally improved perf with the current impl).

only called during init; i wonder if it handles browser zoom levels correctly. (i'll check tomorrow)

i wonder if it handles browser zoom levels correctly.

it does

src/event.rs

orhun · 2025-07-13T18:42:35Z

src/backend/dom.rs

+                if let Ok(element) = target.dyn_into::<web_sys::Element>() {
+                    let rect = element.get_bounding_client_rect();
+                    let grid_rect = (rect.left(), rect.top(), rect.width(), rect.height());
+                    handler(MouseEvent::new_relative(event, cell_size_px, grid_rect));
+                } else {
+                    // Fallback to viewport-relative coordinates if we can't get the element
+                    handler(MouseEvent::new(event, cell_size_px));
+                }


I'm not sure what's going on here. What's new vs new_relative?

hmm, i don't recall exactly - if it's hypothetical or encountered, but the new_relative is the mouse coordinates relative to the canvas (and not the window)

so, during DOM rebuild, i think mouse events can still fire. i'm sure i can't replicate the behavior though - should we skip the fallback and only process the mouse event when we're sure that the cursor is hovering a cell. yeah... makes sense when i write it.

replaced it with debug_assert() - better no event than a wrong event

::new_relative has been removed in the latest series of commits

orhun · 2025-07-13T18:44:19Z

src/render.rs

-    /// Handles mouse events.
-    ///
-    /// This method takes a closure that will be called on every `mousemove`, 'mousedown', and `mouseup`
-    /// event.
-    fn on_mouse_event<F>(&self, mut callback: F)
-    where
-        F: FnMut(MouseEvent) + 'static,
-    {
-        let closure = Closure::<dyn FnMut(_)>::new(move |event: web_sys::MouseEvent| {
-            callback(event.into());
-        });
-        let window = window().unwrap();
-        let document = window.document().unwrap();
-        document
-            .add_event_listener_with_callback("mousemove", closure.as_ref().unchecked_ref())
-            .unwrap();
-        document
-            .add_event_listener_with_callback("mousedown", closure.as_ref().unchecked_ref())
-            .unwrap();
-        document
-            .add_event_listener_with_callback("mouseup", closure.as_ref().unchecked_ref())
-            .unwrap();
-        closure.forget();
-    }
-


It makes sense to do this in the Backend options, but I think it causes an inconsistency. We should follow the same style for all the events and callbacks imo.

are you suggesting we move it into Terminal (or the other way round)?

For it to work as expected, we'd need some sort of lifecycle management on the closure (and not closure.forget()), so that closures can be added and removed. adding a second event callback is UB territory without deregistering the old callback. I assume this applies to the current key event callback registration too, but it didn't occur to me until now.

So, either we move it back to Terminal and fix the issues with double callback registration or we move the key event registration to backend init. For the former to work, I guess we need to figure out the flow too (e.g., should a new callback replace the old one or do we want to support multiple callbacks per type).

(let me confirm this is actually the issue and not a misinterpretation of previous results)

updated description, but it's been moved back into WebRenderer/WebEventHandler.

junkdog · 2025-07-21T18:31:25Z

current issue: dom backend pos are offset, canvas coordinates are likely inverted in relation to browser zoom level

junkdog · 2025-07-23T21:18:04Z

updated PR description to be more comprehensive

junkdog · 2025-07-23T22:31:13Z

i'll take another look through the code tomorrow, but i think this addresses all mentioned issues + fixes stuff i didn't think about previously (support mouse wheel events, exposing all mouse events leaving the choice of low-level or high-level mouse events to the user, offset bug fix etc),

the minimal example has been updated to show the current cursor position and event data. maybe we should copy it to minimal_mouse_input and restore the old minimal example?

junkdog · 2025-07-24T17:41:53Z

i guess that's it; callback lifecycle management it a bit more managed. i've also restored the old "minimal" example and copied the other minimal to a new example: "mouse".

0x01d · 2025-07-25T16:21:32Z

Thx dog, this is exactly what I needed, right when I needed it. Going to give it a whirl :b

orhun · 2025-08-02T17:59:20Z

@0x01d have you tried this yet?

Man... I need to re-review this sometime

0x01d · 2025-08-09T11:14:43Z

@orhun Yes it worked like a charm. I use it on my website rbn.dev.

edit: All is fine error on my end..

orhun · 2025-08-09T16:42:48Z

Damn, your website looks pretty cool! I should also rebuild my blog using ratzilla sometime.

Do you mind adding it to the docs here? 😇

0x01d · 2025-08-12T11:14:30Z

@orhun Would love that, thx.The whole codebase is open source on my profile. Would love some feedback, not immediately though as it still needs a massive refactor -mvp state right now.

orhun · 2025-09-21T08:14:47Z

Added in #126 :)

junkdog mentioned this pull request Jul 11, 2025

feat(Mouse): Move and Click events by cell accuracy #111

Closed

orhun reviewed Jul 13, 2025

View reviewed changes

junkdog added 4 commits July 21, 2025 20:19

mouse events with grid coordinates

876afa5

add MouseEventHandler for deriving Debug

53e5d00

put pub fns first

0fa93e5

DomBackend::setup_mouse_event_handler with debug_assserts

28c09f8

junkdog force-pushed the mouse-grid-coords branch from 0541f05 to 28c09f8 Compare July 21, 2025 18:20

highlight hovered cell

b88d5a2

junkdog added 7 commits July 22, 2025 01:02

lifecycle manage js callbacks (wip)

3c4a77a

consolidate

b37bf45

DRY and update examples

a1a0312

register additional mouse event callbacks

3c1bb0d

fix mouse coordinate offset problem

f49d867

fix DomBackend mouse cursor calc

2c685fb

formatting

f26d8ce

junkdog added 2 commits July 23, 2025 23:20

fix doc test

447b97e

improve wheel scroll behavior

640ae46

junkdog added 7 commits July 24, 2025 00:42

better names

a9e7a4d

rename fields

621614a

EventCallback

27e1070

formatting

5972218

prep webgl2 backend

326ff93

clean up

762e21e

MouseConfig and fix offset calc

8c2f781

junkdog force-pushed the mouse-grid-coords branch from 0a9ee70 to 8c2f781 Compare July 24, 2025 17:29

junkdog added 2 commits July 24, 2025 19:38

rename minam example to 'mouse' and restore old minimal

9eecd5d

derive Debug

a6777cb

		let width = if width > 0 { width } else { 10 };
		let height = if height > 0 { height } else { 19 };

Uh oh!

feat(mouse): enhance mouse events with grid coordinates #110

Are you sure you want to change the base?

feat(mouse): enhance mouse events with grid coordinates #110

Uh oh!

Conversation

junkdog commented Jun 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Key Changes

Usage

Mouse Event Types

Breaking Changes

Backend Support

Uh oh!

orhun left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

junkdog Jul 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

junkdog commented Jul 21, 2025

Uh oh!

junkdog commented Jul 23, 2025

Uh oh!

junkdog commented Jul 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

junkdog commented Jul 24, 2025

Uh oh!

0x01d commented Jul 25, 2025

Uh oh!

orhun commented Aug 2, 2025

Uh oh!

0x01d commented Aug 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

orhun commented Aug 9, 2025

Uh oh!

0x01d commented Aug 12, 2025

Uh oh!

orhun commented Sep 21, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

junkdog commented Jun 30, 2025 •

edited

Loading

junkdog Jul 14, 2025 •

edited

Loading

junkdog commented Jul 23, 2025 •

edited

Loading

0x01d commented Aug 9, 2025 •

edited

Loading