Introduction

My University recently hosted a 24-hour hackathon which was sponsored by ARM. I joined a team and entered with four other people across a couple of different academic disciplines (Electrical and Electronic Engineering as well as Computer Science). We decided that we wanted to make an interactive device with a useful application. Over the course of the week we developed several different ideas but decided to create an interactive glove. This idea was developed and we decided to create a sign language interpreter glove that could be used to translate sign language letters to an audio output.

The glove consisted of five push-button sensors on each finger of the glove. As the fingers were pressed down in different hand positions they signified different letters of the sign language alphabet (we used ASL (American sign language) since all of the letters could be made with only one hand). After a hand gesture was held for several seconds the mbed LPC1768 device determined which fingers were being pressed and matched that combination against all of the letters that had been programmed in.

Each hand gesture was made so that a unique set of push buttons were pressed each time as to distinguish between each letter. To do this we used five different bits (corresponding to the different push buttons) and made sure that each letter only occupied one of the 32 possible states. Unfortunately, several of the letters clashed in their combinations and we had to add a sixth state as to distinguish between letters further. To do this we added an accelerometer (that was controlled by an Arduino since the mbed device struggled with the connections to it) so that when certain hand gestures were made, the current angle of accelerometer would determine a different letter. As a result, we programmed in a degree of variability for each letter so that each letter could be formed faster and with less effort.

We then programmed the mbed device to string letters together with a corresponding wait gesture in between each letter so that a word could be recognised. We then connected the word to some audio software so that it was spoken aloud. Overall the project was successful in demonstrating this language conversion. The project also highlighted a wider range of other practical capabilities that an interactive glove might have such as playing games and displaying data.

Project Build

The two main components that we needed to collect were the gloves and the mbed LPC1768 device.

Each ASL hand letter was mapped out using five different bits to represent the five push buttons and a sixth bit for the accelerometer.

The mbed device was tested with the operation of the push buttons.

The push buttons were then soldered to bell wire that would lace the outer side of the glove to the mbed.

The push buttons were attached to the glove and an additional piece of cardboard was stuck onto the button as it was hard to press some of the buttons down when the fingers were stretched.

The accelerometer was attached to the glove and it was connected to the Arduino and mbed and then strapped to the wrist of the user.

After 24-hours all of the projects were showcased!

Our team won the 3rd place prize for: Most Commendable Project.