Platypus

Open Source Cross-Platform Voice Macros & Dictation
Copyright © 2013 Henry Kroll III  www.thenerdshow.com
Version 0.78

WHAT IS IT?

Platypus takes spoken text from speech recognition programs like Dragon NaturallySpeaking® (that run in the Wine Windows® emulator) and relays it as keystrokes, so it can be used anywhere, including Directx games, Virtualbox, VMWare and Qemu sessions, VNC, other users' desktops, remote terminals or anywhere else dictation didn't work before.

Platypus also serves as a basic text editor, a global "Paste anywhere" and voice-enabled macro utility to automate certain tasks. It can type user configurable keyboard commands, such as 'Alt_L F4' (file close) and 'Control_L s' (file save) by voice command. It can even launch Windows or Linux programs and scripts, e.g. "start terminal". See customization hints below.

Announcement

There is now an open source alternative to Platypus and DragonNaturally Speaking! See my FreeSpeech project.

CHANGELOG

Wed Oct 20 2010 0.78 Made a configure file, win64 support, fix warnings.

Sun Apr 04 2010 0.77 Added German characters, ÄäÖöÜüβ. Thanks, Torsten Ziegler.

Sun Mar 21 2010 0.76 actions-->Paste anywhere. Fix mingw build.

Sat Mar 20 2010 0.75 Add accelerator hotkeys to the menu bar. Fixed the underlying Wine "Edit" control, but the fix has not been included into wine yet.

Fri Mar 19 2010 0.74 Platypus is now also a text editor. Added file and editing functions, bugfixes.

Wed Mar 17 2010 0.73 Re-wrote the cursor handling routines to use half as many keystrokes. It should no longer wrap the buffer or delete the wrong text. Fixed an endless loop when issuing "scratch that" too many times.

INSTALLATION

1. Install Wine. (Follow link to official website or use your operating system's update tool.)
2. Create a separate wine prefix named ".dictation" (What is this?)
   1. Open a terminal.
   2. Make sure you are not the root user. (Why?)
   3. Type or paste the following command:

   export WINEPREFIX=$HOME/.dictation

   Important: Keep this terminal open. The WINEPREFIX environment variable has been set to "$HOME/.dictation" so that Wine software will install to that location.

3. Use the above terminal to install Dragon NaturallySpeaking®. (Installation instructions for different versions.). Fedora users: For best results, install alsa-plugins-pulseaudio.i686, run winecfg and choose "alsa" under the audio drivers tab.
4. Make sure that dictation into DragonPad is working.
5. Some libraries will be needed to compile Platypus. Download them using your operating systems update tool or package manager to pull in any dependenocies. Keep in mind that if you are planning to cross-compile to 32 bit on 64 bit hosts, you will need 32 bit (?86) versions of these libraries:

       wine-devel
       xorg-x11-proto-devel
       libX11-devel
       libXtst-devel
   

6. Download Platypus. Unzip the archive and install to $HOME/.dictation. Again, do not use root.

       export WINEPREFIX=$HOME/.dictation
       ./configure
       make
       make install
   

Ubuntu User Notes

I've used now Platypus with Ubuntu 11.04 , but needs some changes to install successfully:

1. Where you say to do "export WINEPREFIX=$HOME/.dictation" I'0ve understood that it can be replaced by the location of an existing Wine prefix. For example:

export WINEPREFIX=$HOME/.PlayOnLinux/wineprefix/dragon

2. Packages to be installed before, in Ubuntu are named:

wine1.3-dev libx11-dev libxtst-dev x11proto-core-dev

(in case of Wine v1.3)

3. libX11.so is not located where you expect; the following instructions are a workaround in Ubuntu 11.04 (Natty):

if [ -f /usr/lib/i386-linux-gnu/libX11.so ] ; then
    sudo ln -s /usr/lib/i386-linux-gnu/libX11.so /usr/lib/libX11.so
else
    sudo ln -s /usr/lib/x86_64-linux-gnu/libX11.so  /usr/lib/libX11.so
fi

4. Only english language users have the desktop folder in ~/Desktop ; french, spanish or other locale environments have another name for the desktop folder, but your install.sh assumes that $HOME/Desktop/platypus.desktop can be created because that path exists (then fails the setup). Is better to use a tool like desktop-file-install

USAGE

1. Select which microphone to use:

       alsamixer -c0 -V capture
       (or make a custom amixer script)
   

2. Start Dragon NaturallySpeaking (Applications--Wine--Programs)
3. Turn on Dragon's microphone
4. Start Platypus from the desktop icon.
5. Open any Linux application and start dictating.

CUSTOMIZATION

Edit the file, "platypus_commands"

Spoken text:            What to do:

"start Firefox"        "firefox&"
"start terminal"       "xterm&"
"file close"           "Alt_L F4"
"file save"            "Control_L s"
"file print"           "Control_L p"
"banana radio"         "T y p e space s o m e t h i n g ."

Commands ending with the ampersand (&) character, like "firefox&" are executed directly on the host system. The rest are interpreted as keystrokes separated by spaces. Be aware that "make install" copies this file to the program directory, so don't edit that one or it will be overwritten if you install again.

Special characters, like "space" and "Control_L", use the quaint X-style syntax. A full list of those special key codes (with "KX_" prepended to them) is defined in /usr/include/X11/keysymdef.h (but selection is limited to what can actually be typed on the keyboard.)

A $DISPLAY variable can be passed as an optional argument. Keyboard commands will then go to that particular Xserver display. This could give multiple users access to voice recognition. Authorization may be required.

platypus.exe :0.0
platypus.exe $DISPLAY
platypus.exe :2.0

FAQ

Will this work on mac?

We have not heard of anyone using this on a Mac yet. It should compile and run on any system that has wine and X11 development libraries installed, however. Note: The configure script has only been tested on Fedora so far. It may need editing to work with your particular system.

Does not compile

Requires wine-devel. If cross-compiling Platypus as 32 bit on a 64 bit host, make sure to install glibc-devel.i686 (Redhat) or libc6-dev-i386 (Debian). The configure file only checks a few standard places for libraries. If your libraries are somewhere else you may have to set the path manually with ./configure --libdir option. ./configure --help for options. If all else fails, edit configure and send me the updated file so I can include changes in the next version.

Platypus is not working.

Newer versions of wine may install the 64 bit version of Dragon NaturallySpeaking by default. This may have problems interacting with Platypus compiled as 32 bit (though that could be a wine bug). The interim solution is to either uninstall NaturallySpeaking and re-install it as 32 bit (which could use up a license) or compile Platypus as 64 bit (--with-64bit configure option). The author does not have a 64 bit Dragon NaturallySpeaking to test with, but it should work...

If you play video games on Wine, launch windows applications, or mess with Dragon's settings, Platypus will lose the focus and you will have to click on it to bring the window on top of other windows again. It also helps to run the dictation software and Platypus together within a separate $WINEPREFIX from the one your game or application is on. This is the recommended setup.

But Platypus refuses to stay on top!

Running Platypus in a Wine desktop window is how we get around this problem. The included registry file automatically configures this during installation. The Wine desktop can be minimized, hiding it from view.

Text appears twice.

It is normal for text to appear twice when dictating into Platypus as it converts the "Windows messages" from Dragon NaturallySpeaking into Linux keystrokes. Click on a Linux application to bring that into the foreground and you will be able to dictate text into it.

The cursor is in the wrong place, messing up.

Sometimes the dictation buffer can fill up or go out of sync and it may be necessary to clear it. This can be done through the application menu, (Alt_L a r) or by voice command, "reset dictation". Another voice command hard-coded into the source is "undo all that" which undoes recent editing (as if you had pressed Control_L z in both windows).

Text is all jagged.

Use winetricks to get smooth, anti-aliased fonts.

TECHNICAL

Platypus is a strange creature that lives in the swampy marshland between Windows and Linux, taking over the X display, sending keystrokes and making Linux system() calls from within a Windows application inside the Wine desktop environment. This might scare some security-minded people. News flash: Wine is not a sandbox.

Running make install with a different $WINEPREFIX exported in the terminal is supported.

Platypus can be made into a Windows-only application by linking with Windows' msvcrt -mno-cygwin compiler option with winegcc or MingW "make mingw". But then it will need the full pathname to launch anything other than Windows applications. And since there is no X server to send keys to, it will output spoken text to the terminal.

The file, platypus_keys can be edited to include keysims for your particular keyboard. Later definitions overwrite earlier ones and only the last 2 hexidecimal digits are meaningful (bitmask 0x00ff). Beware because of that, if you enter a character like 0xff7a Muhenkan, it will replace 0x007a z and z will be replaced with Muhenkan in everything you dictate. Keys that require multiple key combinations to type can be programmed internally by taking any unused keycode and modifying the source so as to press those extra keys whenever that keycode is encountered. For example, 0x0014 Left is a made-up shifted version of the left arrow key in our implementation.

If you have an X server installed in Windows along with the necessary development libraries (cygwin?) you could experiment with the -D__WINE__ commandline option to MingW and see if it compiles that way. Then the fake keypresses might actually be generated in Windows. I don't know. Or you could try linking it with the Windows version of wine, equally strange, but I'd like to hear about it if it works.

Should you experience trouble with missing characters as you type, there is a Delay setting in xkeys.h that you can set and re-compile. The default is 0. Try something like 5, 10 or 50.

Windows® is a registered trademark of The Microsoft Corporation. Linux® is a registered trademark of Linus Torvalds in the US and other countries. Mac® is a trademark of Apple Inc., registered in the U.S. and other countries. Any other trademarks mentioned belong to their rightful holders. The owner of this site disclaims any affiliation with the provided links.

This documentation is Copyright © 2010 Henry Kroll III. You are free to share, adapt and re-distribute this work with attribution under the terms of the Creative Commons SA license inasmuch as it agrees with the terms of the LGPL. Any of the above conditions can be waived if you get permission from the copyright holder.

Tweet