Okay, the first line should probably read "part of the Astromobile project" but I'm too excited to consider small details like that. :)
While our project partner, the ARTS lab of the Scuola Superiore Sant'Anna, has extended their navigation and localization part another couple of weeks to really finish it, the voice and touchscreen interaction and with it the part of Simon Listens has been developed, deployed and tested successfully on the robot prototype.
Have a look at the video below and see how Simon, Simone, Simontouch and even a bit of ownCloud fit together.
Part of the project was a voice- and touchscreen controlled kiosk software running on the robot.
Initially we were thinking about continuing our XBMC based solution, but soon decided to start from scratch.
XBMC is a great media center but it didn't fit very well with the rest of our solution.
So more out of necessity instead of huge aspirations, we decided to write a small, purpose built software called Simontouch that should - among other features - combine simple multimedia playback with communication features (phone and email).
Simontouch (to be found in the simon-tools repository) uses a QML user interface, Phonon powered video and audio playback, voice and video calling provided by Skype and a simple email client powered by Akonadi and Nepomuk.
Meanwhile, our colleagues at the Scuola Superiore Sant'Anna have been working on top-notch localization and navigation as well as a great design for the robot:
Our next trip to Pisa is scheduled for the middle of March and we're planning to bring all this technology together for a state of the art assistive robot - powered by KDE.
By the way: We are planning to take part in GSoC again this year. If you have any cool ideas regarding Simon or KDE Accessibility in general, check out the ideas page!
After using Sourceforge for the last couple of years, simon finally joined the kool kids on the KDE infrastructure!
As part of the move, we also united the Sourceforge- and github repositories - they were only separate for organizational reasons.
We then re-organized the codebase into two projects: simon (containing the simon application suite) and simon-tools (consisting of smaller tools that we created for various projects like a small command line utility to control Skype, a tiny calendar, and even a touch-friendly media center).
So far (about one week in) I can only say that I'm already thouroughly amazed about how incredibly active and helpful the KDE community really is - but more about that in a later post!
We, the non profit research organization simon listens e.V. are looking for qualified C++ / Qt / KDE hackers to join our team!
Initially, we would be looking to fill part time positions but they can be extended to full time afterwards.
While our projects mostly focus on speech recognition using our own, KDE based solution called simon, you do not need to know anything about speech recognition to join!
Interested? Contact me for more information or send me your resume right away: grasch at simon-listens dot org
I'm happy to report that since August, I can now officially call myself a Qt Ambassador!
As an Ambassador, I had the opportunity to apply for a loaned Nokia N950 to develop / port applications to MeeGo/Harmattan. I took Nokia up on their offer and the result is simone - a trimmed down, mobile version of simon. In other words: "simon embedded" or "simone".
The client features push to talk or automatic voice activity detection (configurable) and because of simons client / server architecture uses little power on the device itself. Even with voice activity detection running you should get many hours of continuous speech recognition out of a single charge.
On the one hand, simone can be used to replace the headset of a "full" simon installation but also includes a couple of default actions on the device. For example, you can use a voice controlled quick dial feature or start / stop a turn-by-turn navigation.
.
For more information and a live demo, have a look at the youtube demonstration:
Over the last couple of days I have again been working on what I started during this years Desktop Summit: simons AT-SPI 2 integration.
What started as a GSoC project idea back in April is now beginning to take shape.
The basic idea is still the same: First, integrate sequitur in simon to be able to transcribe arbitrary words automatically. To facilitate this, sequitur first needs to learn the transcription rules from a large dictionary. So I integrated a feature that let's user turn their shadow dictionary (which already supports many different formats) into a regular sequitur model.
After this sequitur model generation process, the system is used to transcribe words for the ATSPI plugin but also for adding new words manually.
Thanks to sequitur, simon can now transcribe words automatically that are definitely not in the shadow dictionary:
With this as the basic foundation and some help from Frederik and Joanie I created a plugin that would analyze the UI of currently active window, create vocabulary and grammar for it and associate commands with the user interface elements.
It's still in an early development stage (as is the support for ATSPI-2 of GTK and Qt) but the basic stuff already works. To check it out, either build and install the current development version of simon from Git (atspi branch) or have a look at the demonstration video below.
I just arrived back home (I flew home after the talks) after this years Desktop Summit and it was awesome! In retrospect I kinda regret not staying the whole week... Next year... :)
Anyways, I met tons of interesting people and had a lot of productive meetings and discussions. It's amazing what can get done in just a few minutes if the right people are sitting together.
If we (the KDE accessibility team) can implement even half of what was discussed in the last couple of days, I'm sure we're looking at a big step towards a truly accessible free desktop.
Oh and Martin: I'm looking forward to all those KWin effects for simon :P
