[REL] Voice Recognition for SH5

Ok, I resolved my problem in which SHSpeechEngineTest couldn't connect to the SHSpeech server, so I thought I'd post it in case it helps someone else.

For my setup (in Windows 7 Ultimate), I use the mic on my logitech 9000 webcam (which works great) and my speakers. The webcam mic didn't have a port listed (such as usb, front mic, etc.), so after seeing your post to Gusteno8, I tried to set the headphones mic as default, to no avail. I tried to disable the headphones mic, to no avail. I then "overruled it from the speechrecognition settings" as you suggested in that post, and made my headphones mic my preferred device. That got the SHSpeechEngineTest to connect to the Speech Engine Server and allowed me to issue commands. I then reset my logitech 9000 webcam to be my preferred device, reloaded SHSpeechEngineTest, and it still worked. To make sure it wouldn't revert back to not connecting to the server, I restarted my pc and tried again. All was good. I tested it in game, and it works fine.

I loved SHOOT in the old days until SH3Speech came out, which I loved, because it was easy to modify and worked great. With SH3Speech, SH3 was fun to play - I barely touched the keyboard. I can't even imagine playing again without voice commands (and wouldn't if I didn't have it). I used to use german voices, and spoke all the commands in German, despite not speaking (much) German. For SH5, I edited SHSpeech (that was for SH4) by completely redoing the IDs into groups, and removing the dials and clicks info, so that it would only take keyboard commands from commands.cfg. That was great. I'm not sure if I even need permission, because I haven't modded minsc_tdp's program, but, as a courtesy, I asked him for permission to see if he won't mind me uploading the csv files for others to use, since many people said they wanted to use voice (or at least get over the missing commands). It may help the XP users at least. I haven't heard back from him, though.

But now that I've tried this program, I think I'll be using it if I can add my own phrases. Using the "name=" lines from commands.cfg, rather than the key binds, makes it universal enough that it doesn't matter what anyone's keys are set up to. I think this is the approach that should have been used from the get go. Good on ya, skwasjer!!

My question now is "how do we add our own stuff?" For starters, I want to add navigation map and time compression. For time compression, I use key words, such as "riding" for 64,
"coasting" for 128, "cruising" for 256 "speeding" for 512, "flying" for 1024, "max time" for 2048, "warp speed" for 4096, and "real time" for time compression 1, etc., because they are
less cumbersome to say. In SHSpeech, you would have to set it up to reset to TC 1, then repeat the TC key to the amount that relates to that number. For example, in ShSpeech's
key_commands file, TC 4 is set up as Enter&Numpad +&Numpad +. For your VR, is this something that needs to be set up by you in the Page Voice Recognition file to simulate the multiple
presses, or is it something simple that we can do? For navigation map, I tried to add a new section in SHSpeech_Commands.xml, but it doesn't recognize my commands. I added a section as
follows:

<!-- Maps -->
<order>
<phrase>navigation map</phrase>
<phrase>map</phrase>
<phrase>close map</phrase>
<commandText>Navigation_map</commandText>
</order>

Is that incorrect, or is there more that has to be done?

For all those people who hate the fact that many commands are missing, they just have to modify the commands.cfg to add the keys, but if they use your VR, no keys need to be added, because everything seems to be taken from (stock) command.cfg - it's easy as pie!

Keep up the good work, man! I love it.

[skwasjer]

Yea you are on the right track. The commands.xml file maps the recognized text to an ingame order (optionally with default or spoken numbers in the param1/param2 fields). To actually let the speech engine recognize a certain word or phrase, you must also edit the grammar.xml file, because the engine only 'listens' to what is defined in there. The grammar-file uses a W3C specification:

http://www.w3.org/TR/speech-grammar/

This specification defines the structure of that file. The most basic way to add a new grammar-line is to add a new item to the root:

<item>navigation map</item>

And you can add more for the same command:

<item>map</item>

You can see I've used more detailed definitions:

<item repeat="3">dive</item>

This means the word dive must be said 3 times in a row to be recognized as 'dive dive dive'.

There's also 'optional' words (repeat="0-1"), or even 'choice' (the one-of element)

Each accepted phrase (item) is looked up in the commands.xml file and then passed on to the .py script file, which finally passes the command on to the game... This is how it basically works.

Basically, you could define your map command like so:

<item>
   <item repeat="0-1">
       <one-of>
            <item>navigation</item>
            <item>close</item>
       </one-of>
   </item>
   map
</item>

You can also add each line using a seperate <item></item> line, but the above method is better optimized for the speech engine to use less resources or find close matches.

PLEASE NOTE: We are working on improved commands and phrases, for both English and German. Also note, not all game commands work (yet). So stay tuned for an 'official' update.

NOTE2: all game commands can be found in a file in the Documentation folder of this mod.

[skwasjer]

Time compression is a special one, there is no game command for that. However, it can be set via script. I suggest you wait for the next version of this mod, which implements that. It takes a little bit too much time to explain how it is done, but you will see later (and can change it to your liking). The speed/depth/rudder/course changes which take a spoken number are done in a similar way btw. Be advised, I will change the grammar definitions still, to support extra languages etc. so keep track of your own changes because you will have to redo them later...

[edit]The speech engine is set up so that it automatically reloads changes to both XML-files, so you can have the game running (or the test app), while editting the files and immediately test the new commands. (in the odd case, it may CTD the game though, in case of file locks but this is rare)

hey skwasjer,

Thanks for that info. It seems simple and logical enough!

So, basically, with the "one of", it takes any of the first terms and adds it to the last term. I enjoyed that stuff (maybe I should have been a programmer, because it looks like fun .. hehehe). I gave it a try and it worked nicely. I added a time compression section, putting in "normal time" and "increase tc" as well, so that I have them in for now, until you have them in the official update. I'm going to try to add things I need for now until that update. I'll definitely keep track of my changes as per your advice.

I'm glad you mentioned that triple repeat for dive, just by chance, because I actually want to use just one dive command for that, so now I know how to change it.

From what I see, users will be able to easily add their own preferred command phrases once they see how it's done. Once all the commands are in the final version of the SHSpeech_Grammar.xml file, I imagine that it should be as simple as modifying the SHSpeech_Commands.xml file after that.

In regards to time compression, Time_compr_reduce, Time_compr_increase, and Time_compr_normal are there, but for things like setting tc 1024, etc., I can see how scripts will be required like you said. I look forward to those coming in later on. For now, with my additions based on your help, at least I can do the basics using voice.

I don't know if there's anything I can do to help, because you have things under control and know what you're doing, but if there is anything, let me know. As long as I can find time, I'll try to help.

Have a good one!

[roadrage]

Great Mod. 1.3 is working good for me so far. I'm looking forward to more commands in future releases.

[skwasjer]

Originally Posted by 7Infanterie19

I don't know if there's anything I can do to help, because you have things under control and know what you're doing, but if there is anything, let me know. As long as I can find time, I'll try to help.

Have a good one!

Hey man, any suggestions that you add yourself are welcome, or if you have ideas on new commands for which no game command exists but for which you think that they could perhaps be 'scripted' let me know too.

For example:
'decks awash' is implemented as 'set depth to 8 meters' and
'man all guns' is implemented as a dual combo firing two commands to man both the deck gun and the flak gun(s).

[RSColonel_131st]

Don't own SH5 yet, but this gotta be the most impressive mod ever.

I mean, there are tons of programs that will convert a spoken word into a key command, but to actually enter numerical values into the sim, totally embedded system, that is awesome.

Will it work for setting target data while looking trough the periscope?

[skwasjer]

Thanks, that's exactly the strength of it

There's issues with setting up a torpedo solution via voice, it will take me some time to work this out. I am not 100% confident it can be solved, because this is an area that the game has some stuff hardcoded (and thus can not be set via script/commands). But hey, this mod is just out a week and I really haven't gotten to the advanced stuff yet, but there's quite alot possible

I'm not even missing all the extra UI knobs and dials that people complain about, when playing with speech

[edit] It just struck me Voice Recognition isn't really the right name for the mod, Speech Recognition is. As such I will rename the mod accordingly...

[Noren]

Im awed that this could work! It could really add to the immersion!

But Im afraid that I would have to shout or repeat over and over again,
or perhaps go flank when Im running silent. Now can this be 100% avoided?

As long as the surrounding noises your microphone catches are low enough, and you trained your voice recognition to a decent level, you really don't need to shout or repeat something. I play SH3 with SHSpeech, and i can say that in really nearly all cases (>>90%) the orders are recognized correctly. Playing with speakers here, so my missing % are mostly related to issueing an order when a loud explosion or something is going on, which can be eliminated by setting up correct speaker setups in terms of loudness, or by simply don't command when something loud is happening.

I suggest you just try it after training your voice recognition, and you will see for yourself how well this works.

Originally Posted by skwasjer

Hey man, any suggestions that you add yourself are welcome, or if you have ideas on new commands for which no game command exists but for which you think that they could perhaps be 'scripted' let me know too.

For example:
'decks awash' is implemented as 'set depth to 8 meters' and
'man all guns' is implemented as a dual combo firing two commands to man both the deck gun and the flak gun(s).

Sounds good! One I can think of already is the TAI map. I use the less annoying mod (merged with the dials and clock). I don't see anything in the commands.cfg other than a multiplayer thing that TAI was set up with. Actually, another thing I noticed is when setting new direction, it bumps the speed up to ahead standard. That got me in trouble trying to sneak away from destroyers. If that can be worked around, it would save lives. If there's anything else, I'll let you know, but just keep doing what you're doing 'cause it's good!

Originally Posted by Noren

Im awed that this could work! It could really add to the immersion!

But Im afraid that I would have to shout or repeat over and over again,
or perhaps go flank when Im running silent. Now can this be 100% avoided?

In regards to having to shout or repeat things, I don't see the need to shout if your mic is set up correctly. I use a webcam mic two feet from me and it works fine. Just use your regular voice level and you should be fine. Dont' be afraid, just go try it - you'll like it! From time to time, you may get something that appears to not work, but just repeat your command. I noticed that (even with SHSpeech) if you are in the nav map, sometimes a command won't take, or if you do an ID target, the command takes, but it doesn't seem to register, but I think that means the target needs to get into clearer range.

There's less patience required (if any) with this than SHSpeech or Shoot. I'm extemely happy that I get into the game and can already issue commands. I often forgot to load Shoot or SHSpeech, then had to tab out to do it, or if something was buggered (especially with Shoot), I had to tab out and reload the program. This is seamless in that aspect. It just becomes natural to get in and start talking - "Map" "New heading north" "New depth 20 metres" "Ahead slow".

I haven't trained speech recognition either, by the way, so it works without it, but I imagine that if you trained sr, then it can only get more accurate (if you feel that it's needed).

Enjoy.

and skwasjer, thanks again!

[skwasjer]

Originally Posted by 7Infanterie19

Sounds good! One I can think of already is the TAI map. I use the less annoying mod (merged with the dials and clock). I don't see anything in the commands.cfg other than a multiplayer thing that TAI was set up with.

What specifically do you want, the TAI map opened?

Originally Posted by 7Infanterie19

Actually, another thing I noticed is when setting new direction, it bumps the speed up to ahead standard. That got me in trouble trying to sneak away from destroyers.

Yea I noticed this too. The problem is ahead standard seems to be tied to a course change whenever you have engines stopped, regardless of whether you are at silent running, under attack or whatever. I have not really looked at it more closely though. Perhaps there's a way around it.

For now, I'm almost done implementing all English commands, and after that I can start on the German version (I've already received translations from Karlkoch)

[minsc_tdp]

Great work skwasjer! Haven't tried yet as I don't even have sh5 yet, very little time anymore.

Very cool that you managed to hook more directly into the program instead of emulating keyboard/mouse commands.

Thanks for doing this, I was worried I might get increased pressure to do the sh5speech and feared I would never get around to it.

Someone has PM'd me however, who may have working CSVs for sh5 and I told him I would post them to the web page. So you guys might want to bookmark that and keep watching, it's in my sig, if you're having trouble with this mod.

[Swordfishy]

Hey bud, this mod literally turned me into a buyer so Ubi should give you some of the cash ^_^. Anyway, I am a bit new to the sub sim scene, however I am crafty with 3D studio max and if I get enough enjoyment out of it Im willing to work with some people to edit model meshes and graphics etc. That aside, I cant get the mod to work, speech recognition works, running English windows 7 64bit here, and the test app works flawlessly, however when I load the game, do I have to tell it to start litsening or does it automatically recognize it? Just curious if Im missing something? Would bring the game to a playable level (after buying it I discovered the forum and realized how it was released.)

(its installed with JSGME and the other mods work fine) Thank you for your help in advance.

It looks like the community is fairly large and thats what i love the most about simulation games.

-sword

[skwasjer]

What confidence level worked for you in the test app?

See this:
http://www.subsim.com/radioroom/show...8&postcount=71