[TEC] Mouse input

[radcapricorn]

Okay, for all UI modders who want to utilize the touchpads functionality right now, here's the how-to. Beware it's a bit technical!
Also note, the how-to does not use the sample python script provided in the download, for that wait for a further example.

1. Not really a step, just a note. Be prepared to do some calculations (as you don't want to wait till I finish my calculator application )
2. If you're creating/editing a rotatable control (attack disc, RAOBF, 3 bearings tool, etc.), make sure that it's texture is square. Having non-square textures will screw up your touchpads when you rotate it. Example:
   
   
   
   
   On the left is the original rectangular image, on the right is the proper square one. Notice how I left the image itself centered.
3. Make sure that you synchronize your changes to image sizes with SH5 pages configuration (i.e. open up menu editor and enter proper values)
4. For each image, define a set of areas you want (or don't want) to click on. Remember there are three basic shapes to choose from (ellipse/circle, triangle and polygon).
   
   Let's take the "Vorhalt" (lead angle) pointer from the attack disc. In this example, I'll mark the areas that I want to handle input with green, and those I don't want to handle input with red.
   Note that while I use the same image that I used when created sample code, the numbers I provide here may differ slightly since I redid the whole procedure again.
   
   This is the area I want to 'listen' to my mouse:
   
   
   
   
   I need to split up this area into simple shapes. I immediately see a circle:
   
   
   
   
   Yes, I know that it covers the space where mouse should not be handled, but bear with me on that.
   Now, I look at the pointer. It's a trapezoid:
   
   
   
   
   Ok, so now I have defined two areas where I want my mouse input. Now for the areas where I don't want it. I want to keep everything as simple as possible, so:
   
   I don't want my mouse handled in the lower half of the image:
   
   
   
   
   I don't want my mouse handled in these two areas:
   
   
   
   
   Note that I actually cheat, since there's also a small circle in center which also should not receive mouse input:
   
   
   
   
   But I won't bother with it, since I know that when I combine all images, it will always be covered by another object (the attack course pointer).
   
   Using your image editor (I use GIMP) or any other application that allows you to see mouse coordinates, measure all these areas. For circles, we'll need center and radius (two radiuses for ellipses).
   For anything else, we'll need corner points (vertices).
   
   The whole image is 556x556 pixels. For the circle, it's center will be in the middle of image (278x278), and I measured it's radius to be ~151-152 pixels (I used GIMP's measure tool).
   For pointer, I just found the coordinates of it's corners (yep, pointed my mouse at them and took GIMP's feedback). Here's an illustration of what I got:
   
   
   
   
   Next, I traversed this polygon and wrote down the coordinates of these points as I encountered them:
   
   
   
   
   Next, I did the same measurements for the rectangle an the triangles. Here's what I got in the end:
   
   Circle: center = 278,278, radius = 152
   Pointer: 250,123 -> 270,0 -> 286,0 -> 305,123
   Rectangle: 0,278 -> 556,278 -> 556,556 -> 0,556
   Left Triangle: 278,278 -> 110,278 -> 137,212
   Right Triangle: 278,278 -> 454,278 -> 418,212
   
   Okay, you've wrote down all those coordinates. Now you need to convert them into relative coordinates in the range [0,1]. Those familiar with vector graphics know what I mean, for everyone else here's what to do:
   take each point, and divide X coordinate by image width and Y coordinate by image height. For my pointer, I get the following:
   
   250,123 -> 270,0 -> 278,0 -> 305,123 divide each X by image width (556) and each Y by image height (also 556):
   
   0.449,0.221 -> 0.485,0 -> 0.514,0 -> 0.548,0.221
   
   You don't need to take all the significant digits, in fact 2 or 3 will be quite enough. Repeat this for each polygon you've got.
   
   Now, with circles... stop. If you haven't read my original post, you don't know that circles are funky. For center, divide the coordinates as usual by image size. But for radius, divide by half image size!
   E.g. for my circle I will divide center coordinates by 556, but radius by 278. If you've got ellipse and not a circle, you'll need to divide X radius by half width, and Y radius by half height.
   
   My numbers after the calculations:
   Circle: center = 0.5,0.5, radius = 0.546
   Pointer: 0.449,0.221 -> 0.485,0 -> 0.514,0 -> 0.548,0.221
   Rectangle: 0,0.5 -> 1,0.5 -> 1,1 -> 0,1
   Left Triangle: 0.5,0.5 -> 0.197,0.5 -> 0.246,0.381
   Right Triangle: 0.5,0.5 -> 0.816,0.5 -> 0.751,0.381
   
   Tedious? Well, it's a lot more in writing than in doing. And anyway, wait for my calculator app. You'll just draw all those areas instead!
   
   I must say it would have been a LOT easier if we could just create a mask texture that would show exactly what parts of image are processing input (and I have an itching suspicion that stock game dials do just that).
5. Code
   
   In your page script, import the InputCanvas class:
   
   

   import clr
   clr.AddReference("RadCapTools")
   from RadCapTools import InputCanvas

   Add a global for InputCanvas so you can access it from different functions:
   
   

   BearingLeadPointer = None

   In your InitializeScript() function, add the touchpad areas to your items:
   
   

   def InitializeScript():
       if Page_Test1_AttackDisc_Front_BearingLeadAngle.Touchpads.Length == 0:
           # Adding areas that process mouse input (MenuItemWrapper.InsideTestModes.Inside)
           # Circle:
           Page_Test1_AttackDisc_Front_BearingLeadAngle.AddTouchPadEllipse( MenuItemWrapper.InsideTestModes.Inside, 0.5, 0.5, 0.546, 0.546 )
           # Pointer:
           Page_Test1_AttackDisc_Front_BearingLeadAngle.AddTouchPadPolygon( MenuItemWrapper.InsideTestModes.Inside, 0.449,0.221, 0.485,0.0, 0.514,0.0, 0.548,0.221 )
           # Adding areas that don't process mouse input (MenuItemWrapper.InsideTestModes.NotInside)
           # Rectangle:
           Page_Test1_AttackDisc_Front_BearingLeadAngle.AddTouchPadPolygon( MenuItemWrapper.InsideTestModes.NotInside, 0.0,0.5, 1.0,0.5, 1.0,1.0, 0.0,1.0 )
           # Left Triangle:
           Page_Test1_AttackDisc_Front_BearingLeadAngle.AddTouchPadTriangle( MenuItemWrapper.InsideTestModes.NotInside, 0.5,0.5, 0.197,0.5, 0.246,0.381 )
           # Right Triangle:
           Page_Test1_AttackDisc_Front_BearingLeadAngle.AddTouchPadTriangle( MenuItemWrapper.InsideTestModes.NotInside, 0.5,0.5, 0.816,0.5, 0.751,0.381 )

   Here, one at a time, I add all the areas that I've previously calculated, specifying whether I want mouse input in them or not.
   
   Why is that Page_Test1_AttackDisc_Front_BearingLeadAngle.Touch pads.Length == 0 check there? Well, the touchpads may only be added, not removed. At least I haven't found a way to do that. But InitializeScript() function will be called more
   than once during gameplay. This means that (a) we need some check to only add touchpads once and (b) if you make changes to your touchpads during testing, you'll have to reload the game. Oh well.
   
   Now, in that same function, create an InputCanvas and attach some event handlers to it:
   
   

       # InitializeScript() continued
       global BearingLeadPointer
       BearingLeadPointer = InputCanvas(Page_Test1_AttackDisc_Front_BearingLeadAngle)
       #
       BearingLeadPointer.LostFocus += This_LostFocus
       BearingLeadPointer.MouseLeftButtonDown += This_MouseLeftButtonDown
       BearingLeadPointer.MouseLeftButtonUp += This_MouseLeftButtonUp
       BearingLeadPointer.MouseMoved += This_MouseMoved

   Here, I've basically created my mouse polling object and told it what functions it should call when certain events happen. Now for those who haven't seen the video I've posted, I'll repeat that you can track mouse position for the control even when mouse is actually outside it. If TheDarkWraith is reading this, I'm sure he's laughing at this point, because programmatically, there's no restriction whatsoever if the mouse is inside or not, we can just poll it at any given time. What I mean is for our purposes, we need some way to determine whether mouse position is still relevant for our UI item. That's why there are two separate types of events for stock API objects: mouse in/out events and focus events. An item gains focus whenever mouse is moved over it (so MouseIn and GotFocus event arrive in pair). Now if you press and hold mouse item inside an item and drag mouse out of it, it'll still have mouse focus, though it already got a MouseOut event. That's why I add a LostFocus handler here: I want to know when to actually stop waiting for mouse input (i.e stop dragging). The InputCanvas class basically forwards events form FocusMenuItemWrappers it handles, it's usefulness is in that MouseMoved event which is not present for the stock API objects.
   
   In UnloadScript() function, add code to dispose of InputCanvas:
   
   

   def UnloadScript():
       global BearingLeadPointer
       if BearingLeadPointer:
           BearingLeadPointer.Dispose()
       BearingLeadPointer = None

   The only thing left is to define those event handler functions. Here's an example with comments:
   
   

   Dragging = False # This will tell us whether we drag our pointer
   
   def This_LostFocus( sender ):
       global Dragging
       Dragging = False # When our pointer lost input focus, stop dragging
       
   def This_MouseLeftButtonDown( sender, point ):
       global Dragging
       Dragging = True # When mouse button is pressed, start dragging
       
   def This_MouseLeftButtonUp( sender, point ):
       global Dragging
       Dragging = False # When mouse button is released, stop dragging
       
   def This_MouseMoved( sender, newPos, oldPos ):
       global Dragging
       if not Dragging:
           return # we're not dragging, so we don't want to do anything
       
       # Below is simple code to apply rotation based on mouse motion
       # It's the same code found in sample python module
       # Note that it uses mouse displacement to calculate rotation angle
       # You can easily use just current mouse position and calculate
       # absolute rotation angle
   
       # We need to convert coordinates since InputCanvas gives them in screen space    
       
       # previous mouse position relative to center
       v0 = sender.screenToClient( oldPos, MenuItemWrapper.LocationPresets.Center )
       # current mouse position relative to center
       v1 = sender.screenToClient( newPos, MenuItemWrapper.LocationPresets.Center )
       
       # rotate one of the discs
       delta = atan2(v1.Y, v1.X) - atan2(v0.Y, v0.X)
       # atan2 gives proper sign, so just add the delta to current rotation
       sender.Rotation = sender.Rotation + delta

   That's pretty much it.

Now you may ask, then what the hell this AttackDisc.py in sample downloads is for? Well, this how-to I built from scratch. And that script I extracted from my actual testing sandbox. It's basically just a more
sophisticated version of code presented in this how-to, and it handles the full attack disc. If you look carefully at that code, you'll find that it's pretty similar to the one presented here, it only differs in function signatures (since AttackDisc is a class there) and in amount of InputCanvas objects.

I put in the sample just what I exctracted into one neat class so that I don't clutter my own functions with it. In fact,
I already figured out how to actually use it as a separate script file (i.e. scripts/menu/radcapricorn/AttackDisk.py) without creating any dummy pages, but that's another topic.
If I get permission from TheDarkWraith to post changes to his scripts, I'll show you how you can use that sample python module to actually control attack disc in TDW's UI. I've actually already tested that code with his UI (since I already have touchpad areas calculated for that attack disc) and I must say it works perfectly.

For any other objects (RAOBFs, 3-bearing tools and whatnot) you'll have to wait until I've finished my calculator app: I'm done with manual calculations

As a side note, you don't need to use my InputCanvas if you just want to create shaped buttons. Touchpads are enough for that. See how triangle buttons are done in stock Page Default Hud.py, for example.
You can use the same approach to make round buttons, hexagonal ones, you name it. There are limitations, of course, but I'll post them once I've tracked down at least all major ones.

Wow, that took a lot more time than I expected. But that's not all. Stay tuned

Have fun!

[reaper7]

This is amazing, well done - this is one thing I solely missed making my UI .
Wii add this to my UI Ui-boat V5.

Ps The RAOBF are from my Ui-Boat V5 and also HAHD U-Boot_HAHD
Nice to see it working the way I always dreamed it to work

[radcapricorn]

Thanks, reaper7! I was beginning to think nobody's interested

I still have to insist that this RAOBF I took from CSP MaGui by dr. Jones I've taken it to make that example video just for my own sake: it's basically two images (there's more in total, but actual tool is just that), so there wasn't a lot input areas to calculate.

See the outer ring: it has this metallic rod thing at 8 o'clock, and a plastic 'cover' with red mark at 1 o'clock. These actually (I guess) should block the input to the inner ring (they are a part of the outer ring after all), but as of now such fancy stuff is not possible to do without adding (dummy) items, as I encountered some issues with overlapping input areas inside one UI item (I'll investigate a bit more and update the first post accordingly). IIRC, in your mod the plastic markers are in a separate image, so this should not pose any problems at all.

PS.
I beleive I've also answered your PM, but right now my 'sent items' box is empty. I only guess it's due to ongoing maintenance, so I'll wait and see if it pops up.

[Trevally.]

Hi radcapricorn

I have just watched your vid for the mouse control and it looks great

With the download, if I am using TDWs UI and install the .dll in Sh5 main dir and the .py script in the script/menu file - will i be able to control the RAOBF and attack disc as you did in the vid

[radcapricorn]

Thanks, Trevally!

Originally Posted by Trevally

With the download, if I am using TDWs UI and install the .dll in Sh5 main dir and the .py script in the script/menu file - will i be able to control the RAOBF and attack disc as you did in the vid

Not per se, no. This is not that automated The sample .py script is just that - sample (though the touchpad areas defined for the attack disc there are for the attack disc from TDW's UI). I've extracted it from my UI test page script. You'll need to actually add some code to TDW's scripts to use the functionality. One way is to copy/paste that code into TDW's Page TDC.py (IIRC the Attack Disc in TDW's UI is located at TDC page) and add proper initialization/cleanup to InitializeScript()/UnloadScript() functions.
For example, here's how I do it in my test page (simplified code off the top of my head):

# File scripts/menu/pages/Page_Test1.py

# The following is the contents of the sample script

import clr
clr.AddReference("RadCapTools")
from RadCapTools import InputCanvas

class AttackDisc( object ):
# The definition of the attack disc from the sample script follows
# ...

# The contents of sample script end here

# global variable to store the attack disc input handler
g_AttackDisc = None

def InitializeScript():

    compassRose = Page_Test1_AttackDisc_Front_CompassRose
    bearingLead = Page_Test1_AttackDisc_Front_BearingLeadAngle
    targetDisc = Page_Test1_AttackDisc_Front_TargetDisc
    attackCourse = Page_Test1_AttackDisc_Front_AttackCourse

    # Create the attack disc input handler
    global g_AttackDisc
    g_AttackDisc = AttackDisc(compassRose, bearingLead, targetDisc, attackCourse)

# Some more code skipped...

def UnloadScript():
    # Destroy the attack disc input handler
    global g_AttackDisc
    if g_AttackDisc:
        g_AttackDisc.dispose()
    g_AttackDisc = None

That'd roughly be the modification to TDW's scripts/menu/pages/Page TDC.py.
Now, those Page_Test1_AttackDisc_Front_* names are for my UI page. For TDW's, you'll have to look at data/menu/pages/Page TDC.ini to figure out proper names.

I think I may have to write a more detailed instructions on what to do, but that'll be later once I'm at my SH5 rig and have an actual UI mod files to use for illustration.

Oh, yeah, and a follow-up:
In the first post, I mentioned the issues with rotation if the textures are not square. Some original attack disc textures from TDW's UI are not square so I had to edit them accordingly (this required resizing them in GIMP and making changes in SH5 Menu Editor). Anyway, excpect an update with proper instructions in an hour or so.

[Trevally.]

Thanks radcapricorn

[radcapricorn]

Post #2 is updated with a small how-to.