Abundance, and Five Years of Blogging
Tower of Babel: Machine shall speak unto Machine

The Tower of Babel: Nation shall speak Peace unto Nation

Appropriately enough, the BBC picks up on a New Scientist story about a "Tower of Babel" device that gives users the illusion of being bilingual.

My colleagues teaching Modern Languages are going to hate this but Douglas Adams would have loved it...

Users simply have to silently mouth a word in their own language for it to be translated and read out in another.

The researchers said the effect was like watching a television programme that had been dubbed.

The system, detailed in New Scientist, is not yet fully accurate, but experts said it showed the technology was "within reach".

The translation systems that are currently in use work by using voice recognition software. But this requires people to speak out loud and then wait for the translation to be read out, making conversations difficult.

But the new device, being created by researchers at Carnegie Mellon University, Pittsburgh, is different.

Electrodes are attached to the neck and face to detect the movements that occur as the person silently mouths words and phrases.

Using this data, a computer can work out the sounds being formed and then build these sounds up into words.

The system is then able to translate the words into another language which is read out by a synthetic voice.

The team currently has two prototypes: one that can translate Chinese into English and another that can translate English into Spanish or German.

The original story from New Scientist magazine, (issue 2575, 26 Oct 2006) offers some further insight into how the system works:

The team has developed a system that can recognise a potentially limitless lexicon. Their secret is to detect not just words but also the phonemes that form the building blocks of words. The system then uses these to reconstruct the word. To translate from English to another language, the user only has to train the system on the 45 phonemes used in spoken English.

The researchers use software that has been taught to recognise which phonemes are most likely to appear next to each other and in what order. When it encounters a string of phonemes it is unfamiliar with or has only partially heard, it uses this knowledge to come up with a range of sequences that make sense given the surrounding phonemes and words, assigns a probability to each one, and then picks the one with the highest probability.

It is worth noting that there is some related work being done here in England by Cambridge University's Machine Intelligence Laboratory.

Follow me on Twitter: @IanYorston

Comments