[REL] sh4speech - voice command for SH4
 

shSpeech - Voice Command for Silent Hunter 4 by minsc_tdp

sh4speech is a voice command program for Silent Hunter 4, but could be adapted to other games (and has been adapted to SH3 now!) It currently supports everything the game can do, allowing you to issue all orders completely hands-free, which frees up your mouse to only manipulate the cameras and periscope, map, and other things that traditionally did not involve spoken orders.

It is extremely quick, responsive, reliable, and the recognition is great if you follow the readme and spend a few minutes training your speech system and if you adjust your microphone often. The first time you hear someone bark "Yes sir!" in response to an order you barked yourself at your crew, your whole SH experience will change forever! :)

Downloads
v2.55 ***LATEST GOOD VERSION*** aka "DON'T STOP BELIEVIN' RELEASE
Download page: http://knepfler.com/shSpeech

Silent Hunter 3
TomcatTwo modded this to work with Silent Hunter 3. To be honest, he put my work to shame! He added a lot of commands, even in German, and his documentation and immersion level is outstanding! Check it out here. I'm glad to have contributed the core technology to this awesome project. Congratulations TomcatTwo on an awesome release!

Latest Changes
June 29, 2016 (2.55 aka "DON'T STOP BELIEVIN RELEASE") - Added tmo 2.5 files by razark
Mar 01 2008 (2.54) - Fixed resolution scaling problems for TomcatTwo's SH3 mod (dials should work better in this one vs. 2.53)
Mar 01 2008 (2.53) - Fixed resolution scaling problems for TomcatTwo's SH3 mod
Feb 23 2008 (2.52) - Added "Pause Recognition" and "Resume Recognition" commands with underlying code support
Feb 19 2008 (2.51) - Added additional launch BAT files for left-handed users with underlying code support. Untested.
Feb 13 2008 (2.50) - Important adjustments made to location of voice_commands.csv and press_to_talk.csv files. If you have customized your CSV files, you must read the ChangeLog! Other minor adjustments nobody will really notice.
Feb 9 2008 (2.25) - Fixes a problem where extra spaces at the end of voice commands would cause them to not work
Feb 5 2008 (2.24) - Adds support for multiple "hold keys", so you can do CTRL-SHIFT-ALT-X instead of being limited to only one, where you could only do CTRL-X or SHIFT-X or ALT-X. See the ChangeLog for full explanation.
See the full changelog for a complete history of changes.

Docs
Readme
ChangeLog
Voice Command Reference Card

Videos
http://knepfler.com/shSpeech

URLs
Older Thread: http://www.subsim.com/radioroom/showthread.php?t=119039

I take my hat of for you m8,its late now but will test this tommorow..Well done :up:

I've been playing with it for more than an hour or so. DAMN is it fun! Hasn't misinterpreted a single command. Did a convoy kamikaze mission to stress it out. Only a very small number of commands went unheard, such as "hard to port" for some reason (probably due to the rough T sounds), but usually only when stuff on my boat was exploding, which I find very realistic! The recognition and response is very quick. I can't wait to set up shooting solutions with it. I'm playing with a nice mic about 1.5 feet from my mouth on the desk, and it works great even with 5.1 sound blaring throughout the room and my brother's stereo cranked in the next room.

Man - that was quick work - I can't wait to plow into this. I got my new PC built and so far so good on it (doubled fps on my SH4 benchmark :D ), but have run into a little trouble on one of the other 3 I'm upgrading so it will still probably be tomorrow night or later before I can get back and test this out. Early reports from the posts above are certainly encouraging.

Thanks for hanging with this :up:

I've updated some critical install information to help you get things running. Let me know if there's any problems! God I'm so drunk right now I can't believe I'm posting this. The world is a total blur.

UPDATE: Now sober, I've tweaked the install instructions again and matched up the readme.txt instructions in the download with the instructions in the thread here.

Got some time while one of the other PCs is installing windows and was going to start playing around with this, but I'm confused by the install instructions. The first post above states:

2. Start > run > \sh4speech\install.bat. This will start the install for:
ActivePerl
...

I've downloaded ActivePerl and Active Python from the links you provided, but I haven't installed them yet (I'd already downloaded and installed the MS SDK thing a few days ago). Does the statement above mean that your bat file installs ActivePerl and/or ActivePython, or do I need to have installed ActivePerl and Active Python from the downloads first before I start running your install.bat file?

Sorry, no you have to install ActivePerl & ActivePython yourself. The install.bat was going to do it but it made the zip too big so I dumped them. Normal installs on both is fine. Same for the Microsoft SAPI 5 SDK. After installing all those, run install.bat.

Nice to have this option :yep: Great work here :up:


RDP

Oh, boy! New toy to play with. :up: Thanks!

My wife will be wondering who I'm yelling at soon.

Although I'm pretty sure I followed all the directions for the install, I've got a weird error problem trying to run this - when I try to do the test mode, I get the following message:

"Can't locate Win32/GuiTest.pm in @INC <@INC contains: C:/Perl/site/lib .> at go.pl line 9.
BEGIN failed--compilation aborted at go.pl line 9."

When I searched for GuiTest.pm, I found it in C:\sh4speech\libraries\Win32-GuiTest

There we an awful lot of questions during the ActivePerl and/or Active Python install processes - I just picked the default answers (except where I had to answer "yes" to the license questions), but maybe something went wrong anyway - any idea what this error is all about? Is there some trick to installing those Active things that maybe I missed?

[edit] - I re-ran the installs of ActivePerl and ActivePython (picked "repair" option in the latter) - everything seemed to go just fine. However, when I run either the test mode batch file or the main start batch file, I still get the same error message noted above. BTW - I did also do the step no. 3 in the notes on the first post (with the PythonWin editor) and all the other training/calibration steps before the first time I tried to run the batch files. Any ideas?

[edit] well, I also re-ran the sh4speech install.bat, just in case - everything appeared to go fine with that as well, but again I still get the same error message when I run the test mode :(

Any ideas welcome - would really like to give this a whirl

Look inside the install.bat and you'll see a line that looks like this:

ppm install \sh4speech\libraries\Win32-GuiTest\Win32-GuiTest.ppd

Put that into a command prompt and see what happens. It should install that module which is not installed on your system for some reason, see why when you run it manually. The results should look like this:

C:\Documents and Settings\you>ppm install \sh4speech\libraries\Win32-GuiTest\Win32-GuiTest.ppd
Unpacking Win32-GuiTest-1.50.5...done
Generating HTML for Win32-GuiTest-1.50.5...done
Updating files in site area...done
16 files installed

You might want to copy everything from C:\sh4speech\libraries\Win32-GuiTest into c:\temp and then try

ppm install \temp\Win32-Guitest.ppd

I've noticed it didn't install once from a deep folder but from there it was OK.

If all else fails, add this to the install.bat and run it again:

md \perl\site\lib\win32
md \perl\site\lib\win32\guitest
copy \sh4speech\libraries\Win32-GuiTest\GuiTest.pm \perl\site\lib
copy \sh4speech\libraries\Win32-GuiTest\lib\win32\guitest\*.* \perl\site\lib\win32\guitest

If it's STILL not workin do this on a command prompt and tell me the results:

perl -e "print join(',', @INC);"

Mine returns this, as should yours:
C:/Perl/site/lib,C:/Perl/lib,.

snakebit I guess - did all the steps you suggested above (thanks for the detailed instructions, BTW).

Every step along the way resulted in exactly the results you indicated should happen.

BUT, when I try to run the test mode batch file, I now get the following error:

"Can't locate Win32/API.pm in @INC <@INC contains: C:/Perl/site/lib .> at go.pl line 12.
BEGIN failed--compilation aborted at go.pl line 12"

So close, and yet so far :cry:

Any other suggestions?

That's good, you've made progress, you got Win32-GuiTest to install and work. Line 12 invokes Win32-Api which the install.bat also should have installed.

What happens at a dos prompt when you type:

ppm install Win32-API

It should download the module from the 'net automatically and install it. I'm thinking maybe ppm isn't running for some reason. PPM is the Perl Package Manager that installs such libraries from an online repository called CPAN. ppm is completely self-sufficient and knows how and where to download stuff from and how to install it.

Worst case, you could download http://knepfler.com/Win32-API.zip and unzip it to \perl\site\lib\win32

Chat online with me if you're still stuck, I'll be around for a little bit. Early night though, 6 AM flight.

aim: erikdotla
icq: 1352055
yahoo: minsc_tdp

I hope nobody is discouraged from the install troubles some are having. I've never had trouble getting things to install with PPM on dozens of systems and I didn't expect any trouble here. I might consider an alternate install-manual.bat which manually puts the files where they need to be instead of relying on PPM. While there's no simple one-click EXE install for this, this is really not very painful compared to most applications out there, and it's worth the work so I hope you all give it a shot.

Seeing how this is really one of the very few applications that I've written, properly documented, and released to the public, I'm very proud of it even though it is relatively simplistic. I intend to support it indefinitely and I'll personally help anyone bring this online who is having trouble. Just post here!

I've also just updated the install.bat in the ZIP to hopefully take care of some of these problems, by trying every module install twice, once by calling PPM, and once by calling PPM specifically with the path where it should be (\perl\bin). Hopefully you've installed Perl to c:\perl or \perl (root of your windows drive.)

AWESOME - thanks for babying me through this - I tried just D/L'ing your program again to try the new install.bat file, but that didn't work.

then I tried the part where you said:

"What happens at a dos prompt when you type:

ppm install Win32-API"

That did the trick - it did what you said it should, and now running the test mode script calls up the window and starts recognizing the voice commands - unfortunately it's too late to stay up and play with it, but hopefully the hard part is over and it'll be fun time tomorrow night (darn RL work gonna get in the way first :cry: )

Thanks again - can't wait to take her for a spin :up:

[edit] Couldn't resist checking out the voice command file for a bit to see how easy it is to insert your own commands - turns out it's remarkably easy. This is set up really well - kudos :up:

I added several of my preferred ways of saying some commands, sometimes replacing some of the stock ones and sometimes adding some more, and they all worked flawlessly in the test mode - well done :rock:

Now, only one more request to bother you with before I head out to sea with this tomorrow - I tried to figure out how to change the key for "fire" from "Enter" to "Ctrl+Enter", since I don't use "Enter" to fire torps (a little lesson learned from accidental firings in SH3), but I wasn't sure about the interplay between the two key-related .csv files (may just be late and I wasn't thinking straight) and I didn't want to screw something up now that I finally got the test mode thing working. Any chance you could post an explanation on how to change the keystroke outputs for this? If not, I'll try reading it all again tomorrow night when I've gotten some sleep - it's probably in there already and I'm just missing it.

Thanks again for this - gonna be great fun, especially if/when you get the mouse/course and depth stuff working :up:

Pretty easy:

add this to key_codes.csv
CTRL-ENTER,0xA2+0x0D,0x00

As you'll notice from other lines near the bottom of that file, 0xA2 is obviously CTRL and ENTER is already defined up above as 0x0D. You can combine up to two keys using + in this way. Codes for SHIFT and ALT and anything else I forgot can be found here: http://delphi.about.com/od/objectpascalide/l/blvkc.htm (just preface all with 0x)

Next, in key_commands.csv, change the line:

12,Key,Fire,ENTER
to
12,Key,Fire,CTRL-ENTER

Since CTRL-ENTER matches the entry in key_codes.csv, that's it, you're done!

These are separate from voice_command.csv so that you can add a dozen voice commands for "fire" without worrying about putting in the right key. It's all documented at the bottom of the readme (though I realize I forgot to include the URL for the key codes... I've updated the readme.txt with this and also included links in a "docs" subfolder in the zip.)

AHA! I'm 99% sure I've fixed the installer problems. There were two: First, one command was prompting an invisible question which would stick the bat to prevent you from reaching the next step. Second, since I'm running PPM which is actually a bat file from within another bat file, it's best to use the "call" prefix. So I'm quite sure these installer problems will go away for anyone else.

Great - thanks - I had the first part figured out, but I wasn't sure what the last column (0x00 above) should be (wasn't sure what a scan code was or how to figure out what the scan code should be).

Had to help someone do something tonight, so didn't get any sub time in tonight :(

Will have to try this out on patrol tomorrow - Now that I got my RFB install tweaked up I'm really looking forward to testing it out with this voice thing. Thanks again!

Just did some more testing (just in test mode) to check the key change for firing - looks like it's going to work great.
This is going to be very cool - I really like being able to have multiple voice comands assigned to the same key/action, especially for things like diving (I've added "Clear the bridge" and "Lookouts below") and firing each tube. I know you still have to work a bit on the tube selection aspect, but I went ahead and set it up so "Fire one", Fire two", Fire three", etc. up to "Fire ten" all work, so I can now order the crew to fire the correct tube (assuming I've selected the right tube before I say it :D )

I hope there's a way to map a key/combo to open a specific tube door or doors - I'd love to be able to say "make ready all bow tubes", or "open outer doors on tubes 1 and 2" and have it actually work. I never really played using the keyboard much unless I had to (tried to use the mouse on everything possible) - with this though I'm going to have to rethink that, and check into all the possibilities of assigning keys to things I never would have messed with before.

Is there a list somewhere of every action that folks have figured out can be mapped to keys? That would be really handy for working with this speech thing (for example, can you map the weather report function to a key, or just to a button on the orders bar? - it would be great to be able to ask your watch officer for a weather report by voice command).

Yes all of that is possible and more. Have a look in the Commands.cfg file. There are a ton of commands that are functional but never mapped to a key. I have'nt fully customized my SHIV file yet, but in SHIII I had nearly 3 full keyboards of commands (using shift/control), just so I could issue orders verbally using Shoot. I am just the opposite of you when it comes to the mouse. Mouse clicking kills immersion for me. I try to only use the mouse to look around. I use a number of multiple commands for one phrase also. Here are a few from SHIII:
"Lower all masts" - both scopes & snorkel are lowered.
"Prepare to dive" - all masts are lowered & tubes closed & dive ordered.
"Silent running" - silent running is ordered & gramophone turned off.
"At ease men" - secure from silent running & the gramophone switched back on.
"Emergency Surface" - blow ballast-go to flank speed-surface-blow ballast.

I even mapped timed inputs for the arrow keys to walk the Free camera from one station to the next, climbing ladders & going through hatches and so forth based on a verbal command. Below are a few snippets from Commands.cfg


[Cmd315]
Name=Select_tube_1
Ctxt=1

[Cmd316]
Name=Select_tube_2
Ctxt=1

[Cmd317]
Name=Select_tube_3
Ctxt=1

[Cmd318]
Name=Select_tube_4
Ctxt=1

[Cmd319]
Name=Select_tube_5
Ctxt=1

[Cmd320]
Name=Select_tube_6
Ctxt=1



;NAVIGATION OFFICER
[Cmd250]
Name=Navigation_officer
Ctxt=1
GoBack=Navigator_view

[Cmd251]
Name=Choose_destination_point
Ctxt=1

[Cmd252]
Name=Plot_course
Ctxt=1,11
MnID=0x3F060001

[Cmd253]
Name=Return_to_course
Ctxt=1
Key0=0x56,C,"V"
MnID=0x3F060004

[Cmd254]
Name=Search_pattern_1
Ctxt=1
MnID=0x3F240002

[Cmd255]
Name=Search_pattern_2
Ctxt=1
MnID=0x3F240003

[Cmd256]
Name=Search_pattern_3
Ctxt=1
MnID=0x3F240004

[Cmd257]
Name=Report_time_to_turn
Ctxt=1
Key0=0x4e,Cc,"Ctl+N"
MnID=0x3F250002

[Cmd258]
Name=Report_time_to_course_end
Ctxt=1
MnID=0x3F250003

[Cmd259]
Name=Report_range_to_course_end
Ctxt=1
Key0=0x42,Cc,"Ctl+B"
MnID=0x3F250004

[Cmd260]
Name=Report_depth_under_keel
Ctxt=1
Key0=0x42,C,"B"

[Cmd261]
Name=Report_weather
Ctxt=1
Key0=0x56,Cc,"Ctl+V"
MnID=0x3F250006

Like I said I'm just getting started customizing my SHIV.