October 2014 came fast and I was ready for another HackManchester after having a blast the previous year. But this time having to work +60 hours in a week made me take the decision of doing something much simpler so I could get some sleep.
At this point I was starting to experiment with an idea for what later will become Apnea, my first ever commercial/experimental game (still in the making… but more on this on another post). One of the key features of Apnea was the detection of the user steps using the HMD’s accelerometer and another one was the detection of the user breath with the microphone. Soon I realized I had a problem: every time the user walked very strange signals appeared in the breath detector, quite odd! I fixed those problems much later, but by that time I decided what if I make a small interactive game out of this odd behaviour?
The idea was named Hack ‘n Slash (thanks Tom!) and it basically is a fencing game using sticks and smart-watches!
In order to play you will need:
- 2 Android smart-watches (I used LG ones)
- 2 Android smartphones (I used a Moto G and a Nexus 7)
- 2 players
- 2 swords (anything from wooden stick to foam sword)
The rules themselves are pretty simple, each player as 5 lives and every time they are hit by the other player in the body they will lose one. When a successful hit is landed the players will have 2 seconds to go back into the initial position before starting fencing again.
But how does it work?
Hit landing
Detecting that a sword is hitting an object is not a trivial task, a first naive approach would use the smart-watch accelerometer to detect that the user arm has suddenly stopped/changed course , but this would generate a lot of false positives as the swords are moving really fast in the air causing a lot of noise in the acceleration signal. Here is where the Apnea problem comes into play.
When the sword actually hits something it will micro-vibrate and this vibration will get translated to the user hand and wrist. If only we had a reliable way to read vibrations… but we do! The microphone in the smart-watch is an air-vibration detector that we can “easily” convert into a “wrist-microvibration detector”.
- Easy – Tape the microphone hole and maybe put some blu-tack on top of the tape so no air can come in at all.
- Easier – Wear the smart-watch, preferably on top of the big bone of your wrist.
- Not so easy – now we need to read the microphone data and identify the microvibrations of the wrist. If the sword hits something the bones will vibrate and the isolated microphone will catch up some noise. Since the bones are solid we are interested just in low frequency vibration, using the Fourier transform over the noise signal and reading just the lower values (using an empirical threshold) we can determine that a hit has landed! This can be further improved by mixing this data with the accelerometer peaks.
Hit synchronization
This was pretty much my second try with the Android Wear APIs, and I do hope they have improved them since. In order to detect if the sword is hitting the meat I needed 1 smartphone paired to each watch, O hope that by now you can use one phone to connect several watches or, even better do pairing watch 2 watch.
When player 2 hits something successfully the watch will inform his paired phone using the Google APIs and this phone will tell, using sockets, the player 1 phone (the server). Player 1 watch obviously communicates directly with the server-phone.
The server will then calculate if both players have registered a hit roughly at the same time (around 100ms difference) this will indicate either a draw: both player hit each other at the same time, or a clash: one sword hit the other. If none of these situations happened then the player that did not send a hit on time will lose a live and the system will close the communications for 2 seconds so the players can go back to the initial position.
Socket communication was fast enough and was adding just a 10ms delay… but the Google system to communicate watch2phone was incredibly slow (around 300 ms). How could I detect simultaneous hits?
The solution for this in the end was quite simple: after successfully pairing watches and phones both players will clash their swords to start the match. This, apart from looking cool as some sort of fighting ritual, will allow the program to measure the starting time difference between the two players. Then when a hit is send they will add a time-stamp taking in count the time difference measured at the beginning. The result was great: A sword clashing will look like 2 simultaneous hits just 5ms apart or so!
The code
The code for Hack ‘n Slash can be found in my GitHub. Please keep in mind this was done in 2014 when Android Wear was quite young so I am not sure if it will still stand.