Explain relationship between UiGlobalTransform, entity-local coordinates, and window coordinates

[viridia]

How can Bevy's documentation be improved?

In previous versions of Bevy, there was a method that allowed you to get the logical rectangle for a UI node. This has been replaced by various properties on ComputedNode which interact in complex ways.

There are a several different variables in play here:

• The translation field on UiGlobalTransform.
• The inverse_scale_factor in ComputedNode.
• The UiScale resource.
• The coordinates in picking events.
• The distance value in picking events.

It's not always clear, in the documentation, how to combine these:

• Is the transform field on UiGlobalTransform in physical or logical coordinates?
• Are window coordinates logical or physical? By "window coordinates" I mean whatever numbers are used for absolute positioning.
• How can I compute the rectangle of a node in window coordinates?
• How can I compute the rectangle of the window in the same coordinates?
• What's the coordinate origin of a UI node? (Turns out it's the center, but the docs don't say this.)
• A picking coordinates logical or physical?
• Do I need to ever care about camera / world space when working with UI?

So far I have managed to get the right calculations, but this is mostly by accident or by using recipes supplied to me by other people on Discord.

What I would like to see is an overview or reference guide that explains all these different coordinates and how they relate to each other.

@alice-i-cecile @ickshonpe

[alice-i-cecile]

So far I have managed to get the right calculations, but this is mostly by accident or by using recipes supplied to me by other people on Discord

My preference is actually to primarily supply this via helper methods, rather than documentation. We do the same thing for projection matrices, e.g. view_to_world, with a lot of success.

[ickshonpe]

Is the transform field on UiGlobalTransform in physical or logical coordinates?

Physical

Are window coordinates logical or physical? By "window coordinates" I mean whatever numbers are used for absolute positioning.

There are three coordinate systems: logical coordinates, window coordinates and physical coordinates.
Multiply logical coordinates by UiScale to get window coordinates.
Multiply window coordinates by the window scale factor to get physical coordinates.

How can I compute the rectangle of a node in window coordinates?

Node coordinates are object centered. To calculate the coordinates in window coordinates:

let half_size = node.size() * 0.5;
let pos_top_left     = Vec2::new(-half_size.x,  -half_size.y);
let pos_top_right    = Vec2::new( half_size.x,  -half_size.y);
let pos_bottom_left  = Vec2::new(-half_size.x, half_size.y);
let pos_bottom_right = Vec2::new( half_size.x, half_size.y);

let pos_top_left_window = ui_global_transform.transform_point2(pos_top_left) * node.inverse_scale_factor() * ui_scale.0;

Are picking coordinates logical or physical?

Picking uses "Window" coordinates. That's why we had to divide them by the UiScale (to get logical coordinates) to fix the core widget bugs.

Do I need to ever care about camera / world space when working with UI?

No, not unless you are doing world UI or rendering to texture.

(Really sleepy and haven't checked any of this, so I might have got some details wrong.)

[ickshonpe]

The biggest annoyance atm is that bevy_picking doesn't know about UiScale and there doesn't seem to be any trivial way to get it to use it. Node hits are resolved by the UI's picking_backend module so they work okay but drag event offsets are reported in window coordinates and need to be divided by the UiScale to get logical UI coords.

[viridia]

That's one annoyance, but not the only one. What motivated me writing this ticket was something else: trying to decide where I needed to multiply by inverse_scale_factor and where I didn't.