[REL] sh4speech - voice command for SH4

[panthercules]

Wow! Looks like you've been real busy while I've been gone on vacation. Had no Internet access for the last week or so, and just got back to see all this activity. Just D/L'ed the newest version (1.55) and trying to figure out the best way to migrate my customized 1.0 install to this new one. Before I start playing around with the .csv files, I had a basci installation question. I've renamed my old 1.0 sh4speech folder to keep as a backup, and unzipped the new 1.55 sh4speech folder to my c:\ root to replace it. I still have all the other prerequisite applications installed (perl, python, SDK 5.1, etc.) that I was using for version 1.0.

So - here's the question: Do I need to run the "install.bat" again, or should I be able to just start running the program like I did the previous version?

[edit] Also, do I understand the heirarchy of the file structure correctly, such that the voice_commands.csv is really the key/controlling file, in that if I don't want or need a bunch of the new button-bar or TM-related commands I can leave them in the other .csv files and they'll just be ignored if I don't put any triggering voice commands into the voice_commands.csv file that use or refer to them (as long as I don't create any conflicts by what I put into the voice_commands.csv file)? I've already customized my voice_commands.csv file so heavily from version 1.0 that I'm thinking I might be better off just pasting in a few of the cool new things (like the tubes stuff) into my old file rather than start over customizing the new one (but I'm still poking around in the new files checking out all the cool new stuff, so that might change).

[minsc_tdp]

Wow! Looks like you've been real busy while I've been gone on vacation.

Welcome back! Where'd you go?

I was using version 1.0. Do I need to run the "install.bat" again, or should I be able to just start running the program like I did the previous version?

It should just run without reinstalling. The install.bat hasn't changed much, other than to help people get it installed who were having trouble. Worst case you could always run install.bat again, it won't hurt anything (including your CSVs) to repeat the install.

Since there have been so many changes, I'd advise you to read the readme all the way through again, I've injected lots of little changes throughout.

Also, do I understand the heirarchy of the file structure correctly, such that the voice_commands.csv is really the key/controlling file, in that if I don't want or need a bunch of the new button-bar or TM-related commands I can leave them in the other .csv files and they'll just be ignored if I don't put any triggering voice commands into the voice_commands.csv file that use or refer to them (as long as I don't create any conflicts by what I put into the voice_commands.csv file)? I've already customized my voice_commands.csv file so heavily from version 1.0 that I'm thinking I might be better off just pasting in a few of the cool new things (like the tubes stuff) into my old file rather than start over customizing the new one (but I'm still poking around in the new files checking out all the cool new stuff, so that might change).

If there's no entry in voice commands with an ID that matches one of the other files, it is essentially ignored. It is still loaded into memory, but we're talking such small amounts here that it doesn't matter.

The upcoming version moves the various types of commands into many different CSV files: Keys, SetVar, Tubes, Commandbar, Dials. They're all basically the same format but split up just to help organization.

It'll be a bit of a pain to merge your changes, so to make it easy, you should go through your changes to the files and make a note of which rows have changed or are new. All columns to the right of the last one I use are ignored so you can add notes like the "Trigger Maru" notes you see I put in. Then you can sort by this column, and it'll group all of your customizations together for easy copypasting.

I'm working pretty hard on the Dials stuff to get precise depth, heading, and speed control in. If I get it working, it should work very well, and be very easy to add definitions to cover other resolutions. I think it will be a few days at best before I get this released, though I'm going to push for tonight.

[panthercules]

Just spent the last week or so sitting on various beaches, riding the waves and looking out to sea comparing the view to what I remembered of the screenshot teasers for leo's LBO mod

I'm intrigued with all the changes you made to this to incorporate the command bar buttons, but I've already gotten pretty far (using Setkeys) toward assigning keystrokes to most of the button bar commands I wanted to use, so I'm not sure whether I'll need/use those parts of this latest version or not (I'm puzzled how this would work with all the changes that RFB seems to have made with the command bar - how does the program deal with the fact that certain commands are not always present on the bar in the same place or on the same screens? I see you've had to do some special stuff to make this work with the way TM messed with the command bar, so I'm guessing you'd need to do something similar if you wanted to accommodate RFB as well). I think the command bar stuff is probably a really good addition to help make this more "mainstream" for folks who don't want to go to the trouble of messing with SetKeys to assign keys to these commands (or haven't already done so), but for me I'm thinking my key-based approach will probably work fine since I've already done most of that work.

However, the tube selection work you've added seems fantastic - can't wait to try that out. And of course, I'm really looking forward to your getting the incremental depth and course selection mechanism working. And the recognition speed/efficiency improvements and ability to add comments to the .csv files are quite good as well. As I said, it looks like you've been quite busy while I've been gone - maybe I'll have to go on vacation more often

[minsc_tdp]

Originally Posted by panthercules

so I'm not sure whether I'll need/use those command bar parts of this latest version or not

Well you'll have to check it and see if I'm supporting anything you haven't mapped to a key yet. Use whatever works for you. I think keypressing is the best method overall since there's no chance of the mousecursor being "invisible" and it yanking your periscope view while reaching for a button. So, you can either choose to spend a bunch of time mapping keys and avoiding the command bar, or just use it as is and get the hang of how the mouse commands works.

with all the changes that RFB seems to have made with the command bar - how does the program deal with the fact that certain commands are not always present on the bar in the same place or on the same screens?

It doesn't deal with this at all. It is set up for SH4 Defaults + Trigger Maru (TM is only additional buttons, not changes.) Each command bar button is defined as an absolute x/y coordinate for each resolution. It has to be right or it'll press a different button - it doesn't know any better. If this RFB thing you speak of moves buttons around, you'll have to update the CSV.

It's really easy to update the mouse coordinates - look at the coordinates already in there - it's the same ones over and over since there's only a few button positions. The Y position is always the same and the difference between each X position is constant, like +60 pixels or so to move to the next button. So if a mod swaps the position of two buttons, then swap the coordinates of those two buttons with each other in the CSV and you're good, and if there's a new button on the right of a bar that never had a button there, it shouldn't be hard to figure out the coordinate without even having to pinpoint it on a screenshot. As I've mentioned, The Gimp is a free photoshop-like program that is perfect for getting the coordinates - you just move the cursor and it updates the x/y coordinates in the status bar at all times.

I'm considering adding real-time file reading support. This way, you could edit CSVs (like from a 2nd laptop PC) while running the game and try out your changes in realtime, without having to restart anything. This would make experimentation and tuning a lot easier.

[panthercules]

Originally Posted by minsc_tdp

I'm considering adding real-time file reading support. This way, you could edit CSVs (like from a 2nd laptop PC) while running the game and try out your changes in realtime, without having to restart anything. This would make experimentation and tuning a lot easier.

Oooh - this does sound useful - is there any reason this wouldn't work to let you just alt-tab out of the game and edit the .csv files on the same PC? It has been kinda slow having to shut down and restart the game and sh4speech, though since a lot of what I was doing required changing the commands.cfg file to map new keys I would have had to do that anyway. Might be different though (and more help) if playing around with mouse positions and stuff for this new version.

[minsc_tdp]

Originally Posted by minsc_tdp

I'm considering [ability to] edit CSVs (like from a 2nd laptop PC) while running the game and try out your changes in realtime, without having to restart anything.

Originally Posted by panthercules

is there any reason this wouldn't work to let you just alt-tab out of the game and edit the .csv files on the same PC?

That would work just fine too. I don't like doing that on my PC though since sometimes the display gets corrupted, resolution mode-switching takes forever on my display, and a laptop would be a very elegant and fast solution - I wouldn't even have to pause the game. Tweak some numbers, hit save, restate the voice command and see what happens.

It has been kinda slow having to shut down and restart the game and sh4speech, though since a lot of what I was doing required changing the commands.cfg file to map new keys I would have had to do that anyway. Might be different though (and more help) if playing around with mouse positions and stuff for this new version.

You know, you can just pause the game, alt-tab, stop sh4speech only, change some CSVs, restart sh4speech, and flip back to the game and it will work just fine. sh4speech is not "connected" to the game in any way. It just works the keyboard and mouse pretty much the same way a human does, and if a game happens to be in the foreground, all the better.

[Snakeeyes]

Hey Minsc,

I'm new to VAC. How do I install your SHIV profile?

[minsc_tdp]

Originally Posted by Snakeeyes

Hey Minsc,I'm new to VAC. How do I install your SHIV profile?

I'm new to VAC too, since I don't know what VAC means.

What profile? If you're talking about keyboard configuration and command bar setup, this sh4speech is designed for an unmodified SH4 default configuration.

[panthercules]

Well, just running the new version (without using the install.bat since I still had all the version 1.0 ancillary programs already installed) seems to have worked just fine. The single-window, more verbose test mode seemed to work well, though perhaps it would be nice to be able to have the verbose mode be a selectable/switch type thing so you would only get all that extra stuff if you're having issues and want to debug them - it was a bit hard/time consuming to wade through all the extra output just to confirm that the commands were being recognized correctly when I was just trying to do a quick run through of all my commands to make sure they worked.

Haven't had a chance to try this out in game yet, but the tube selection/door opening and closing mechanism worked very well in test mode. One thing that threw me briefly was the lack of any output in test mode in response to the mouse/command bar voice commands I tried - it finally dawned on me that I probably wasn't going to get results in test mode from asking for the fuel, CO2 and battery status since I wasn't in game, but it would have been helpful if there was some sort of output that would at least acknowledge that the command had been recognized even though the output wasn't a key stroke. Small thing - just something to think about.

Also - I noticed that one of the command bar items (sets of commands) was labeled "oxygen" and had associated voice commands about running out of air to breathe - IIRC however, shouldn't that actually be referring to the compressed air level indicator, and not the breathable air levels (which is really a function of the CO2 indicator in an inverse sense)?

[minsc_tdp]

Originally Posted by panthercules

it would be nice to be able to have the verbose mode be a selectable/switch type thing so you would only get all that extra stuff if you're having issues and want to debug them - it was a bit hard/time consuming to wade through all the extra output

True, it's a lot, but I try to indent everything that's not an actual spoken command so they show up on the left. But I probably should add a debug mode/logger or something for all that extra.

One thing that threw me briefly was the lack of any output in test mode in response to the mouse/command bar voice commands I tried - it finally dawned on me that I probably wasn't going to get results in test mode from asking for the fuel, CO2 [...] but it would have been helpful if there was some sort of output

In test mode, you should see it move the mouse, but not click the mouse button. That's the test feedback, I just avoid clicking since you're probably in windows and I don't want to accidentally delete all your porn or upload your credit card number or something.

Also - I noticed that one of the command bar items (sets of commands) was labeled "oxygen" and had associated voice commands about running out of air to breathe - IIRC however, shouldn't that actually be referring to the compressed air level indicator, and not the breathable air levels (which is really a function of the CO2 indicator in an inverse sense)?

That was a joke more than anything I thought O2 was breathable but you're probably right, CO2 is probably the only thing related to crew breathing and O2 should be renamed to compressed air in all the voice commands.

[panthercules]

Originally Posted by minsc_tdp

In test mode, you should see it move the mouse, but not click the mouse button. That's the test feedback, I just avoid clicking since you're probably in windows and I don't want to accidentally delete all your porn or upload your credit card number or something.

LOL - never mind - apparently, somewhere along the way in merging my old version 1.0 .csv files with the new ones, I must have failed to save the version that had the fuel/CO2/battery button bar commands, which explains why I wasn't getting any feedback in test mode. Added them back in to my voice_commands.csv and it worked like a charm in test mode.

Originally Posted by minsc_tdp

True, it's a lot, but I try to indent everything that's not an actual spoken command so they show up on the left. But I probably should add a debug mode/logger or something for all that extra.

I think the reason the indentation doesn't help as much as it could is that for some reason I can't seem to maximize/resize the command prompt window that the test mode is running in to make it large enough to keep some of the longer text from wrapping around to the left margin again, which clutters the screen a bit during the test mode. Obviously, not worth spending any real time on, since it does seem to be working in any event.

Now to get out to sea to try this out for real - keep up the great work

[minsc_tdp]

I think the reason the indentation doesn't help as much as it could is that for some reason I can't seem to maximize/resize the command prompt window that the test mode is running in to make it large enough to keep some of the longer text from wrapping around to the left margin again, which clutters the screen a bit during the test mode.

Just make a shortcut to the two BAT files, then right-click Properties on them. Then use the Font and Layout tabs to format it better and make the window larger.

[minsc_tdp]

I've made some excellent progress on the dials. There will be precise 360 degree level control over all of the dials, or less in the cases where it makes sense (like the rudder which is 1-38 degrees).

If you look at the dials in terms of geometry, it gets hairy: For some, the clickable dial area is smaller than others within the same dial. The voice command "right" goes two different directions for rudder versus heading (rudder travels left along the circle to go right!) Rudder has "top dead center" at the bottom and the depth dials have "top dead center" at an even weirder location. Some have unusable areas of the dial. Some are not in degrees but in other units, like depth in feet or speed in knots. Not to mention you have to make sure the dial is actually up with the toggle before you try to click on it.

NONETHELESS, I have everything in place to support all of these quirks. And amazingly, it's only a few definitions in the CSVs that have to be added for each resolution to make the dials work. The items are just dial center point x/y coordinate, radius (in pixels), multiplier (heading uses 0, rudder uses 4), the max setting of the dial, how far the start point is rotated from top dead center, and whether it uses reverse direction (like the rudder where right = left.) When you say how far you want it to go, it only needs this data and a few equations to determine the exact pixel, and I've tested it out successfully. Now it's just a matter of getting it in the code and adding the other tracker stuff (like making sure the right dial is visible) and then typing in all the possible voice commands for each degree. So after this, it shouldn't be much of a stretch to add support for the various torpedo dials (range, speed, AOB, gyro, etc!) So this release with the heading/depth/rudder controls will be 1.6 and the torpedo control version will probably be 1.7.

UPDATE:
Check out this preliminary debug output!

heading right ninety degrees voice command detected at +0.00 seconds
heading right ninety degrees (voice_command.csv id = 705)
Type is HEADINGDIAL0, data is 804,673
Dial 705 needs to be off
Dial info: radius=60, multi=1, max=360, rot=0, rev=1
Clicking coordinates 864,673 to set desired angle 90 on dial
Phrase processed. Waiting for a new phrase...

[panthercules]

RL has gotten in the way of my getting out to sea to try this, but it sounds like you're making some real progress with the course and depth stuff - great news

[minsc_tdp]

1.6 released with full speed, heading, rudder, and depth control, with perfect precision matching the dials (1-24 knots, 1-38 degrees rudder, 0-360 heading, 15-450 feet depth). Over 900 voice commands added to support this feature. Enjoy.

I'm tired so I haven't tested anything but 1024x768 mode so far. 1920x1080 mode is defined but the dial clicking will be off a bit if the coordinates aren't perfect - I had to tune them a bit for 1024x768 but the 1920x1080 ones are still my original figures. Since that's the mode I play in, I'll be tuning that soon. I'm too tired to do any more tonight. I work my butt off for you guys! If only Ubi would release updates this often and with this much information along with it (see the readme to see what I'm talking about!)