Words Plus

‘Words Plus’ was my final year Innovation Design Engineering MA (RCA) solo project, and resulted in a dissertation and the design of two interfaces: Volume (a digital audio book), and Lips-in-sync (a speaking and listening tool). Both interfaces use visualisations to help convey sound information such as words stress, rhythm, pronunciation and intonation. The project won a Dyson Bursary Award and a Professor’s Commendation.

Volume is a digital audio book designed to be displayed on a touch screen. It plays on the intuitive action of running your finger along the page as you read aloud. Animations are triggered in conjunction with syllabic sounds, and can be set to convey a range of phonic characteristics.

Lips-in-sync is a speaking and listening tool that uses words and lip reading to teach language. It works on the basis that each sound (phoneme) has a particular facial position (viseme). The user is displayed on one half of the screen and can practice recitation while trying to match the timing, pronunciation and therefore facial position of the tutor’s video which is shown on the other side of the screen. The tool gives the user real time visual feedback to aid his or her sound production, and is implemented by way of a webcam and a video display.

Context & Rationale

The implications of designing a more holistic communication medium, and integrating symbols, pictures, sound, vibration and animation elements into text has benefits of engaging individuals with different learning styles. Visual learners make up 65% of the population. They relate most effectively to written information, notes, diagrams and pictures. Auditory learners make up approximately 30% of the population. They relate most effectively to the spoken word. Information makes most sense when it has been heard, so reading aloud may help. Finally Kinaesthetic & tactile learners make up around 5% of the population. They learn best through touch movement and space, and learn skills by imitation and practice.

Teaching language also necessitates a lot of repetition, so having a resource where the user can practice time and time again helps alleviate teacher boredom. Both interfaces were designed primarily for adult literacy education, as a novel way to reengage individuals who perhaps dropped out of school because of a social situation or because a learning difficulties had not been correctly diagnosed. 

'Words Plus' is designed to enhance and sensitivity to component sounds in language, which leads to better progress in reading and spelling.