BALLROOM DANCE MOVEMENT RECOGNITION USING A SMART WATCH AND REPRESENTATION LEARNING

Abstract

Smart watches are being increasingly used to detect human gestures and movements. Using a single smart watch, whole body movement recognition remains a hard problem because movements may not be adequately captured by the sensors in the watch. In this paper, we present a whole body movement detection study using a single smart watch in the context of ballroom dancing. Deep learning representations are used to classify well-defined sequences of movements, called figures. Those representations are found to outperform ensembles of decision trees and hidden Markov models. The classification accuracy of 85.95% was improved to 92.31% by modeling a dance as a first-order Markov chain of figures.

1. INTRODUCTION

Recent work has used low-cost smart watches to track the movement of human body parts. ArmTrak tracks arm movement, assuming that the body and torso are stationary (Shen et al., 2016) . In this paper, we perform whole body movement recognition using a single smart watch, which is a hard problem given that body movements need to be inferred using readings taken from a single location on the body (the wrist). The movements in the study are from ballroom dancing, which engages tens of thousands of competitors in the U.S. and other countries. Competitors dance at different skill levels and each level is associated with an internationally recognized syllabus, set by the World Dance Sport Federation. The syllabus breaks each dance into smaller segments with well-defined body movements. Those segments are called figures. In the waltz, for example, each figure has a length of one measure of the waltz song being danced to; the entire dance is a sequence of 40 to 60 figures (depending on the length of the song). The sequence is random, but the figures themselves are well-defined. The sequence is illustrated in Fig. 1 . The International Standard ballroom dances are a subset of ballroom dances danced around the world, and they include the waltz, tango, foxtrot, quickstep and Viennese waltz. A unique characteristic of all these dances is that the couple is always in a closed-hold, meaning they never separate. Also, both dancers in the couple maintain a rigid frame, meaning the arms and torso move together as one unit. The head and the lower body, however, move independently of that arms-torso unit. Our hypothesis in this paper is that the figures in each of these dances can be recognized with high accuracy using deep learning representations of data obtained from a single smart watch worn by the lead in the couple. That is possible because the rigid frame makes it unnecessary to separately instrument the arms and torso, and because most figures are characterized by distinct movements (translations and rotations in space) of the arms and torso. We refer the interested reader to the website www.ballroomguide.com for free videos and details on the various syllabus figures in all the International Standard ballroom dance styles. In this paper, we validate our hypothesis on the quintessential ballroom dance-the waltz. We chose 16 waltz figures that are most commonly danced by amateurs. The full names of the figures are included in Appendix A. Our goal is to accurately classify those figures in real-time using data from a smart watch. That data can be pushed to mobile devices in the hands of spectators at ballroom competitions, providing them with real-time commentary on the moves that they will have just watched being performed. That is an augmented-reality platform serving laymen in the audience who want to become more engaged with the nuances of the dance that they are watching. The main beneficiary of the analysis of dance movements would be the dancers themselves. The analysis will help them identify whether or not they are dancing the figures correctly. If a figure 1 

