[TEC] Mouse input

[radcapricorn]

For some time now, I was wondering if I am able to do some modding for SH5. Since currently my time capacity (nevermind any artistic talent, or rather lack of it) does not allow me to work on something of considerable size, I've decided to start small, with tools.

In one of his posts in NewUIs thread, TheDarkWraith stated that mouse input in SH5 is a mess, and I totally agree with that statement. The whole UI scripting is cumbersome and documentation is, well, lacking. Nevertheless, I always wanted to have in-game tools such as attack disc or RAOBF behave just as they did in, say, SH3: i.e. so that I could click my mouse and rotate those rings to my hearts content. And to be honest, my middle finger just got tired of rotating the mouse wheel when I'm using NewUIs Not to mention that my setup periodically lags, and from what I can tell (or at least suspect), this causes TDW's mouse polling thread to go 'out of sync' with what I actually see on screen. This brought me some problems in, for example, placing supermarks on the nav map.

Interestingly enough, API provided for UI elements allows adding drag/resize handles for items, but these are just not enough for situations when I would have a complex object, such as RAOBF. Here, I want to share what I was able to find out.

So, I started to dig around and see how exactly point-click-drag can be implemented. Well, basics are obvious: I just need to periodically poll mouse state with that 'animation' timer whilch I'm sure is familiar to most modders. Thus I can emulate mouse motion events and handle dragging. So, I've quickly coded a small class (in C#) that can help with this task. It's called InputCanvas, and it's only purpose is to emulate mouse motion events for FocusMenuItemWrapper, the main UI element class that can receive mouse input (non-focusable elements receive only mouse-in and mouse-out events). Incidentally, it also forwards mouse press/release/click events, just for ease of coding Python scripts.

But then things get trickier. Say I want to implement front side of the attack disc, with all 4 elements draggable by mouse. This means that I'll have 4 overlaying images (not counting outer relative bearing scale, which is static), for which I need to implement rotation via dragging. And of course these images are not just rectangles, they are mostly circles with customary pointers attached to them. But by default, game treats the whole image as an input area, therefore I need some way to determine which image I actually click: in case of attack disc, images are positioned 'on top' of one another, and though some areas of top-level image are transparent, they would still receive mouse input (though I visually click on the image 'below'). Well, everybody saw the attack disc . I knew custom shapes for UI elements could be done, since even stock game provides round and triangle-shaped buttons (though I must say, for a while I was afraid they are hard-coded), I just needed to find out how.

While I was familiarizing myself with API, for several times I encountered a mention of 'touchpads'. As it turned out, you can specify the shape of input area for MenuItemWrapper by adding those touchpads. Once at least one touchpad is added, the game stops treating UI element as a rectangle, and processes input only in allowed area.
There are two main types of touchpads. One allows input (MenuItemWrapper.InsideTestModes.Inside) and one denies it (MenuItemWrapper.InsideTestModes.NotInside). As far as I can tell, you can add any number of touchpads to the item, and game will combine them all to determine if mouse input is actually allowed for a particular area of the item.
There are three possible 'shapes' for a touchpad: an ellipse (ideal for circular dials and buttons), a triangle (e.g. TAI maximize/minimize button) and an arbitrary polygon (which I think game just splits up to multiple triangles). I made a quick test with ellipses (attached one to each of attack disc parts), and my attack disc suddenly started to respond to dragging its elements. It was first 'hurray' for me. Though before that, I had a small issue in determining what do parameters for AddTouchPadEllipse() actually mean, but more on that later. Anyway, some more quick tests quickly showed me that I can actually define any custom shape for the input area: it's only a matter of determining the coordinates for touchpads.

And so I patiently calculated all the coordinates for each pointer on the attack disc, added a corresponding touchpad polygon, started my game and... well, was surprised. For some reason, game refused to react on my attempts to drag a pointer. And the Attack Course pointer started to behave very strange when I rotated it (when it was horizontal I had a hard time finding a place which allowed me to drag it). Something was definetly going on. Luckily, I found out what rather quickly.

Documentation states that changing Rotation property for MenuItemWrapper (i.e. rotating it) also rotates all touchpads. What it doesn't say is that rotations are confined to item's bounding box. If it is rectangular, polygons start to shrink or extend when rotated. And several images from my attack disc (I use the one from TheDarkWraith's NewUIs) are indeed rectangular. Well, I thought OK, how about I put each disc in a separate control group, make these groups square and rotate them instead. Unfortunately, the game refused me this attempt, stating that it cannot rotate items without a texture (and control groups are just that). So I had to resize all the textures to make them square, recalculate my touchpads and try again. And it worked: all the discs rotated correctly. This was the second 'hurray'. The only issue I have with this is that I had to make images larger, therefore now they consume more VRAM. Though I don't know if it can become a significant problem, considering the overall ratio of UI to game world texture usage (and anyway, square images are only needed if you intend to rotate them).

I promised to tell about parameters for funcions that add touchpads. Here goes:

For all functions, IfInside parameter sets the type of the touchpad: including (allows input) or excluding (denies input)

AddTouchPadEllipse( IfInside, CenterXPercent, CenterYPercent, WidthPercent, HeightPercent )

Coordinates are in the range [0,1] (hence the 'Percent') suffix. Width and height are ellipse radiuses. But there is a quirk with them. Initially, my thought process was like this: if the full range of coordinates is 1, then to make an incircle for the square image, I'd have to place center at (0.5, 0.5) and make 'width' and 'height' equal to half image width, thus, also (0.5, 0.5). Well, I was wrong. It turns out, that the game considers maximum values for width and height parameters as halves of actual image width and height. Thus, for 200 pixels-wide image (incircle should have a radius of 100 pixels), WidthPercent 0.5 would mean 0.5*0.5*200 = 50! Therefore, to make an actual incircle, parameters should be:

CenterXPercent = 0.5
CenterYPercent = 0.5
WidthPercent = 1.0
HeightPercent = 1.0

This is important to remember when calculating actual size for this type of touchpad.

AddTouchPadTriangle( IfInside, x0, y0, x1, y1, x2, y2 )
AddTouchPadPolygon( IfInside, Single[] coords )

These two are pretty straightforward: you just specify a list of 2D vertices (x,y pairs), where each coordinate is in range [0,1]. One thing to note is that 0 for x is left, 0 for y is top, no matter what offsets you specified while placing an image on page (i.e. these are local coordinates).

One last thing to note about touchpads: AFAICT, they can only be added, but not removed, and they are not removed when page's script is unloaded. This is a bit unfortunate, but I personally can live with that.

To summarize: while somewhat problematic, it's actually not that hard to implement custom-shaped widgets. With taking several quirks I mentioned into account, it becomes only a matter of calculating actual touchpad areas. This is my second step, as I'm now starting to work on application that will allow to manually "draw" touchpad areas for textures and save them into text file, so that it can be read inside script during initialization. Another approach would be to use some form of edge detection and vectorization to calculate the input areas automagically, but that's beyond my abilities for now.

If anyone is interested, I've uploaded my InputCanvas class along with sample Python module. Maybe it'll help someone in creating another awesome mod. Simply place the dll into SH5 installation folder and use it like this:

import clr
clr.AddReference("RadCapTools")
from RadCapTools import InputCanvas

The documentation for InputCanvas is included in the archive.

Download here

I've recorded a short video to illustrate what all the above rant means.
. I hope whoever would watch would also excuse my English


Cheers