There are two roles in the team-work distribution; the physical makers and the program coders. We worked together to brainstorm, paper prototype and create the basic set up. After that, we distributed the tasks according to our roles. Jamie and I being the coders, dived right into coding after we had all agreed on the basic interactions of the tangible interface.
Physical prototyper: Nathalia, Heather
Digital prototyper: Jieying, Jamie
Korean words are made up of vowels and consonants, so we put them in cards and tested our concept with Heather, the Korean native speaker in our team. We started with pens and paper to test the learning experience.
In this video, we are testing whether this collaborative experience would help non-native speakers learn better. As you saw in the video, we ended up learning from each other, as well as having fun. You wouldn't be able to experience this if you were learning by yourself through a screen.
The set up consisted of:
-A round table that is translucent in the center.
-A projector used to reflect the image from the computer to the table.
-Plastic tiles with a Korean vowel/consonant at the top, and fiducials underneath (that are placed on the table), seen below.
-A mirror underneath the table, which was used to reflect the projector, so the computer that's attached to the projector can project the accompanying visuals.
-An IR camera and light that reads the fiducials.
Programming (Digital prototype)
Because we wanted this experience to be as realistic as possible, we decided to code the entire game experience. None of us were engineers or programmers, but Jamie and I were both excited to take on the challenge.
We sketched out how the task flow would look like for this game in order to help ourselves modulate code components.
It may seem quite easy to understand the task flow just by looking at the image, but it is hard for a visual learner to be in a situation where you have a mental framework constrained by the input and output. For time’s sake, we mostly focused on the gaming phase. The pre-gaming and ending phases will just be a pre-recorded video demo.
This chart shows what input and output I need in order to let the game going.
reacTIVision is an open source, cross-platform computer vision framework for the fast and robust tracking of fiducial markers attached onto physical objects, as well as for multi-touch finger tracking. We used reacTIVision as our main software for the game.