simon 0.3.0 has been released about two weeks ago and this means that even tough temperatures outside disagree it's once again summer in trunk!
Our newest addition is actually one that has been in the works for quite some time (in a separate branch) and presents itself to the user as a new plugin: The Dialog plugin.
While simon 0.3.0 is ideally suited to silently execute the commands you tell him, the next version of simon will talk back.
The basic idea is quite simple: The dialog system lets the user define an arbitrary amount of states which each has some transitions to move on to other states of the dialog. Each transition can also execute other simon commands if configured to do so.
An example use case could look like this:
Every day at 10 am the system displays a dialog that asks the user if he has already taken his medication. Yes -> "Great!"; No -> "Do you need help?" -> etc.
But you could also create quite complex menus like this:
You: "Computer!"
Computer: "Hi! What do you want to do? Say any of the following options: Read e-Mails; Browse the web; Check calendar; Close"
You: "Check calendar"
Computer: "Alright. These are your upcoming events: Birthday at Susies Place in Graz"
You: "Where am I?"
Computer: "You are in Graz, Austria."
You: "Thanks"
Sounds ridiculous, right? Well, let's look at this example in a little bit more detail...
States, Options and Transitions
The above dialog is a quite simple state based dialog. You can see three states: "Welcome", "Calendar" and "Location". We can define them in the dialog configuration.
You can see that dialog options to continue to other states can be added there as well. The text of the state actually goes through a templating engine so you can define paramters for that in the "Template options" page.
But more interesting is probably the "Bound values" page. There you can define variables and bind them to values. Those values can be static, determined at runtime through QtScript (Javascript) or values from plasma dataengines.
For example you can bind $currentTime$ to the Local/Time of the date and time plasma data engine. And because there already lots of great plasma data engines this means that the dialog system is already quite powerful.
TTS
Remember that I said that simon will talk back to you in simon 0.4? Yes we also have an all new TTS layer but I'll cover this in a separate blog post as this one is already too long :)
Demo
So to show off the current state of development, I created a very short demo video displaying the dialog above. While the code is not production ready, I didn't cheat: both the upcoming events and the location is determined dynamically, at runtime, through plasma data engines (upcoming events use the calendar data engine to get data from your Akonadi calendar).
For RRS readers here a direct link.
Mittwoch, 29. September 2010
Sonntag, 19. September 2010
simon at the AAL Forum 2010: Again
The whole simon listens team attended this years AAL forum in Odense, Denmark.
The AAL Forum is a plattform for projects of the ambient assited living joint program of the European Union and related projects. After having attended the Akademy this year it was quite interesting to see the other side of software development with almost all the projects there being quite well funded :).
Despite the very steep attendance fee (€ 450) the exhibition was quite active. In just three days we collected more than 30 business cards of interested people - many of them looking for project partners for their next projects.
All in all it was very interesting so to see related projects, discovering similarities and potential synergies. Cooperations across projects - even in the same call - are still far too seldom in my opinion.
Quite some people were surprised to find out that simon is open source and completely free ("Where are the hidden costs?") so we also got to introduce some people to the concept of free software.
On Thursday we we then got to see Dj Ruth Flowers at the networking dinner. And it was just awesome seeing so many suits dance :P.
I really wouldn't have thought that such a formal event could be turned around into a wild party just with a good DJ. Suffices to say: The booths were quite empty the following day.
We then spent the last evening in Copenhagen before flying home which as it turns out is a great city - and they have great cocktail bars as well :)
The AAL Forum is a plattform for projects of the ambient assited living joint program of the European Union and related projects. After having attended the Akademy this year it was quite interesting to see the other side of software development with almost all the projects there being quite well funded :).
Despite the very steep attendance fee (€ 450) the exhibition was quite active. In just three days we collected more than 30 business cards of interested people - many of them looking for project partners for their next projects.
All in all it was very interesting so to see related projects, discovering similarities and potential synergies. Cooperations across projects - even in the same call - are still far too seldom in my opinion.
Quite some people were surprised to find out that simon is open source and completely free ("Where are the hidden costs?") so we also got to introduce some people to the concept of free software.
On Thursday we we then got to see Dj Ruth Flowers at the networking dinner. And it was just awesome seeing so many suits dance :P.
We then spent the last evening in Copenhagen before flying home which as it turns out is a great city - and they have great cocktail bars as well :)
Montag, 13. September 2010
Application centric speech recognition for your desktop: simon 0.3.0 released
The new version 0.3.0 of the open source speech recognition simon has been released and boasts the all new scenario system allowing you to build your own customized speech recognition system with just a few mouseclicks.
With simon you can control your computer with your voice. You can open programs, URLs, type configurable text snippets, simulate shortcuts, control the mouse and keyboard and much more.
Because of simons architecture, it is not bound to a specific language and can be used with any dialect. It is also specifically designed to handle speech impairments which makes simon a viable alternative to conventional input methods especially for physically disabled people and senior citizens.
simon is based off the open source large vocabulary continuous speech recognition engine Julius.
Besides the scenario system the new version also provides the user not only with the possibility of creating his own model through training but also to use an existing acoustic model (base model) to get started even quicker - entirely without training.If the user wants more control or would like to improve recognition accuracy, personalized training is possible through the optional HTK (not included in simon due to license restrictions). simon then offers to adapt the used base model to your own voice or to create a new model entirely from scratch.
Additionally, we have been working hard to make simon even easier to use. Some of the more notable results of these efforts are the new introductory wizard that guides you through the initial setup as well as the speech model generation adapter that automatically fix a vast variety of common beginners mistakes for you.
Furthermore simon 0.3 introduces three new applications to the suite. Sam, an acoustic modeling tool is geared towards professionals who want to tinker with their speech model and get the best recognition out of it. It is also a great tool to create and test large models which can then be distributed as base models for other simon users. To create base models you also need a lot of speech data which can be easily collected through the newly introduced combo of ssc and sscd. ssc stands for simon sample collector and is the client to the sscd server. Together they provide a powerful, cross platform tool to collect samples from lots of different speakers - even allowing you to record with multiple microphones and / or sound cards simultaneously.
Readers of the RSS feed: Watch it on Youtube
With simon you can control your computer with your voice. You can open programs, URLs, type configurable text snippets, simulate shortcuts, control the mouse and keyboard and much more.
Because of simons architecture, it is not bound to a specific language and can be used with any dialect. It is also specifically designed to handle speech impairments which makes simon a viable alternative to conventional input methods especially for physically disabled people and senior citizens.
simon is based off the open source large vocabulary continuous speech recognition engine Julius.
New in simon 0.3
simon 0.3 introduces an application centric approach to speech recognition by using packaged use cases of the speech recognition called "scenarios". Scenarios contain the complete configuration for one specific task like controlling Firefox or using the voice controlled on screen keyboard. These scenarios can then be shared with other simon users and are collected in a central online repository which can be accessed directly from within the application.Besides the scenario system the new version also provides the user not only with the possibility of creating his own model through training but also to use an existing acoustic model (base model) to get started even quicker - entirely without training.If the user wants more control or would like to improve recognition accuracy, personalized training is possible through the optional HTK (not included in simon due to license restrictions). simon then offers to adapt the used base model to your own voice or to create a new model entirely from scratch.
Additionally, we have been working hard to make simon even easier to use. Some of the more notable results of these efforts are the new introductory wizard that guides you through the initial setup as well as the speech model generation adapter that automatically fix a vast variety of common beginners mistakes for you.
Furthermore simon 0.3 introduces three new applications to the suite. Sam, an acoustic modeling tool is geared towards professionals who want to tinker with their speech model and get the best recognition out of it. It is also a great tool to create and test large models which can then be distributed as base models for other simon users. To create base models you also need a lot of speech data which can be easily collected through the newly introduced combo of ssc and sscd. ssc stands for simon sample collector and is the client to the sscd server. Together they provide a powerful, cross platform tool to collect samples from lots of different speakers - even allowing you to record with multiple microphones and / or sound cards simultaneously.
Demonstration
Readers of the RSS feed: Watch it on Youtube
Download
You can download simon 0.3 as source archive but there are also packages available for Windows, OpenSUSE and Ubuntu on our Sourceforge page. Up to date installation instructions are available on the simon listens wiki.Sonntag, 12. September 2010
simon at the AAL Forum 2010
I'm happy to announce that the simon listens team will attend this years ambient assisted living forum in Odense, Denmark!
The AAL Forum (15th-17th September) is an annual event as part of the Ambient Assisted Living Joint Program of the European Union. The main goal of this program is to improve everyday live for healthy seniors.
While this might not sound as the most exciting topic at first, this is a fast moving, exciting field of research that covers everything from home automation to assistive robotics.
We will be represented through a booth in the exhibition hall and Franz Stieger, our chairman, will give both a short introductory talk simon listens and another one about the project in the context of robotics enabled assisted living.
It isn't all work and no play, tough. In the spirit of the conference an internationally proclaimed 69 year old DJ called Ruth flowers will apparently rock the the social event of the conference. I can't wait to see that :)
The AAL Forum (15th-17th September) is an annual event as part of the Ambient Assisted Living Joint Program of the European Union. The main goal of this program is to improve everyday live for healthy seniors.
While this might not sound as the most exciting topic at first, this is a fast moving, exciting field of research that covers everything from home automation to assistive robotics.
We will be represented through a booth in the exhibition hall and Franz Stieger, our chairman, will give both a short introductory talk simon listens and another one about the project in the context of robotics enabled assisted living.
It isn't all work and no play, tough. In the spirit of the conference an internationally proclaimed 69 year old DJ called Ruth flowers will apparently rock the the social event of the conference. I can't wait to see that :)
Samstag, 17. Juli 2010
Astromobile Kickoff Meeting or Creating a Terminator powered by KDE
It's a very busy time for simon :). On Saturday I came back from a weeks worth of hacking and socializing on this years Akademy in Finland and just a couple of days later the whole simon team set off to the our next adventure: From Tuesday to Thursday we were attending the kick off meeting of our Astromobile project at the facilities of our project partner: The ARTS Lab of the Scuola Superiore Sant' Anna in Pisa.
In the Astromobile project we are trying to help seniors who are well able to take care of their daily routine but would benefit from a little assistance here and there.
Often these people don't want to have a full time care giver staying with them both for financial reasons but and to preserve their autonomy.
The Astromobile project tries to address this issue and bridge the gap between living autonomously and full on assisted living by creating a special robot that is able to for example remind seniors to take their medication, provide a way to call for help in case of an emergency as well as using modern technologies like video chats to set up a communication link with family and friends.
In this project, simon listens is responsible for the touch screen and the voice interface to the robot using our KDE4 based simon system.
Just after the introduction of the Scuola Superiore Sant' Anna by one of their professors our robot plattform - the SCITOS G5 from a german company called MetraLabs - arrived.
After coordinating the most important steps we received a short introduction to the internal workings of the robot plattform and were pleasently surprised when we booted it up for the first time.
Yes you are seeing the KDM login screen of Fedora 12 :)
(Full disclosure: The robot contains both GNOME and KDE and actually logs into GNOME by default - but still :)
We then got a quick walk through on how to talk to the many sensors, how to use the integrated path planning, mapping features and the script language, AngelScript, that is used to communicate to the robot platform.
During our stay we also got to see the living lab of the ARTS Lab which is basically a 200 square meter appartment containing lots of smart home prototypes. This is also the place where we will be testing our solution with elder people.
While we were in the area we also visited a pilot project which deployed automatic garbage disposal robots called "Dustbots" which were also developed by the Scuola Superiore Sant'Anna.
They are running Ubuntu, btw :)
In the Astromobile project we are trying to help seniors who are well able to take care of their daily routine but would benefit from a little assistance here and there.
Often these people don't want to have a full time care giver staying with them both for financial reasons but and to preserve their autonomy.
The Astromobile project tries to address this issue and bridge the gap between living autonomously and full on assisted living by creating a special robot that is able to for example remind seniors to take their medication, provide a way to call for help in case of an emergency as well as using modern technologies like video chats to set up a communication link with family and friends.
In this project, simon listens is responsible for the touch screen and the voice interface to the robot using our KDE4 based simon system.
Just after the introduction of the Scuola Superiore Sant' Anna by one of their professors our robot plattform - the SCITOS G5 from a german company called MetraLabs - arrived.
After coordinating the most important steps we received a short introduction to the internal workings of the robot plattform and were pleasently surprised when we booted it up for the first time.
Yes you are seeing the KDM login screen of Fedora 12 :)
(Full disclosure: The robot contains both GNOME and KDE and actually logs into GNOME by default - but still :)
We then got a quick walk through on how to talk to the many sensors, how to use the integrated path planning, mapping features and the script language, AngelScript, that is used to communicate to the robot platform.
During our stay we also got to see the living lab of the ARTS Lab which is basically a 200 square meter appartment containing lots of smart home prototypes. This is also the place where we will be testing our solution with elder people.
While we were in the area we also visited a pilot project which deployed automatic garbage disposal robots called "Dustbots" which were also developed by the Scuola Superiore Sant'Anna.
They are running Ubuntu, btw :)
simon says: Hello Planet KDE!
This is the first blog post that is going to be aggregated to the planet and as such I feel that a short introduction is in order.
My name is Peter Grasch and for the past couple of years I have been working on an open source speech recognition software called simon.
With simon you can control your computer with voice commands.
simon uses the KDE 4 libraries, Julius and the HTK and is developed under the GPL license. You can find more information on our sourceforge page and in an interview on the dot.
Together with Franz Stieger, Mathias Stieger and Alexander Breznik I am also chairing the non profit research organization called "Simon Listens" which uses the simon software and other open source projects to research speech recognition and its applications through multiple research projects funded by the Austrian Benefit project and the European Union.
Through one of this projects I have also been lucky enough to be able to attend this years Akademy where I gave a talk and held a workshop about simon.
In Tampere I got the opportunity to, among many other interesting people, meet some of the KDE accessibility guys: Jeremy Whiting and Gunnar Schmidt. We discussed the current state of accessibility in KDE SC, the most pressing problems and how they could be tackled.
There is definitely a lot of work laying ahead of us but there are also some highly motivated people (yes Jeremy, I am looking at you :) working on this.
So lets join forces, buckle down and try to make KDE SC accessible to everybody!
My name is Peter Grasch and for the past couple of years I have been working on an open source speech recognition software called simon.
With simon you can control your computer with voice commands.
simon uses the KDE 4 libraries, Julius and the HTK and is developed under the GPL license. You can find more information on our sourceforge page and in an interview on the dot.
Together with Franz Stieger, Mathias Stieger and Alexander Breznik I am also chairing the non profit research organization called "Simon Listens" which uses the simon software and other open source projects to research speech recognition and its applications through multiple research projects funded by the Austrian Benefit project and the European Union.
Through one of this projects I have also been lucky enough to be able to attend this years Akademy where I gave a talk and held a workshop about simon.
In Tampere I got the opportunity to, among many other interesting people, meet some of the KDE accessibility guys: Jeremy Whiting and Gunnar Schmidt. We discussed the current state of accessibility in KDE SC, the most pressing problems and how they could be tackled.
There is definitely a lot of work laying ahead of us but there are also some highly motivated people (yes Jeremy, I am looking at you :) working on this.
So lets join forces, buckle down and try to make KDE SC accessible to everybody!
Dienstag, 6. Juli 2010
Day 4 at Akademy
Well the conference part of Akademy already ended on Sunday so for the past two days I've been hacking at Demola.
So far:
And of course I am already working on new features in the simon suite (ssc at the moment, actually).
Also, my presentation is already online.
To everyone at Akademy or somewhere near Tampere: The simon Workshop is tomorrow from 9:30 to 11:00 in Area 2. Hopefully I'll see you there!
So far:
- I met Jeremy Whiting, a KDE accessibility guy and had a very interesting discussion about kttsd and speech dispatcher.
- I attended the KDE Accessibility BoF which was sadly more like a list of things KDE desparately needs to do. To all KDE developers out there: We need to do more!
- I became a "Nokia Certified Qt Developer"! Yay :)
- I broke my entire desktop because after seeing all those cool features in the current KDE trunk I just had to try it :P
And of course I am already working on new features in the simon suite (ssc at the moment, actually).
Also, my presentation is already online.
To everyone at Akademy or somewhere near Tampere: The simon Workshop is tomorrow from 9:30 to 11:00 in Area 2. Hopefully I'll see you there!
Abonnieren
Posts (Atom)








