In my continuing mission to explore strange new datasets, I’ve come across a little article analyzing data which came from… Samba dancing! The way the researchers turned Samba dancing into data was by capturing the motion of the dancers with Motion Capture (MoCap) technology, as often seen in “Behind the Scenes” segments of Andy Serkis films. On top of that, I found this huge repository of MoCap files at the Carnegie Mellon University Graphics Lab, containing all kinds of motions, from actors imitating animals to Indian dance moves, documented in special MoCap files called ASF, AMC, BVH or C3D. Great, I thought, and since there’s a package in R for just about everything, soon I’ll be playing with these datasets, making beautiful analyses of ballet vs. hip hop dancing. But no! To the amazement of not just me, there was absolutely nothing in the R-verse that could “talk to” these weird, weird files. Apparently the MoCap community is speaking Matlab and C++ only. The die has been cast, I had decided: I shall make the mocap package!

And I did.

But let me tell you, this was one bumpy ride. So bumpy I thought at times to just give up. You see there’s actually a lot of math here from the Computer Graphics (CG) world, in which I am foreigner. A lot of Linear Algebra, Trigonometry, Tree-Traversal, Data Structures, Recursion - I was a bit over my head, to be honest. If it wasn’t for this excellent Python code by Vanush Vaswani and this dissertation by Tido Röder, I couldn’t have done it. Credit is due also to Jernej Barbic for this Matlab code for converting the AMC file into a matrix.

So, if you know all about MoCap and just want to analyze it in R, go straight to the package.

If you want to see some cool gifs go straight to the “War: What Is It Good For?” section.

If you have no idea what I’m talking about enjoy this short intro to MoCap.

Capture This

So, an actor wears a MoCap bodysuit with various sensors attached to it. It looks something like this:

The actor is moving in 3D space, doing whatever the researcher tells him to do. The sensors locations can then be translated into the coordinates of a “skeleton” for each frame of its motion. Usually each sensor is attached to the bodysuit in a very strategic location such as the joint between two bones, and we get a skeleton similar to this:

Now you would expect the MoCap fancy machinery to output a simple file giving for each sensor or joint1, a simple matrix of dimensions nFrames x 3 axes (X, Y and Z). That we could work with. Alas, this is not the case. There are various files formats, each MoCap company invented their own. We’ll stick to the ASF/AMC file format which is the most human-readable in my opinion2.

The ASF/AMC Disaster

If you really want to know and use these files, I suggest reading this. I’ll summarise it for you.

Each motion is encoded into a pair of ASCII-based files: ASF (see example), a short file describing the general characteristics of the skeleton, and AMC (see example), a long file detailing the skeleton’s whereabouts for each in every frame, usually hundreds to thousands.

The ASF file has 4 main components:

• an intro: general documentation and parameters such as the angle type used (degrees or radians)

• root: the initial location of the “root” bone which is at the center of gravity of the skeleton

• bonedata: the necessary data for each and every bone. This includes its name, its length, its direction in space relative to its parent bone, its “Degrees of Freedom” (DOF), e.g. the “ltoes” bone can only move in the X direction, its angle limits in every axis it is allowed to move in, and its “angle axis” - an initial angle to axis the bone is in, for every axis.

• hierarchy: a list of “children” bone for every “parent” bone.

You can start seeing the complexity in this. For example, if I want to simply know where the left hand bone (“lhand”) is located in space, i.e. what are the coordinates of this sensor, I need to know where its parent bone (“lwrist”) is located. For that I need “lwrist”’s parent (“lradius”) location. For that I need “lradius”’s parent (“lhumerus”) location. Etc, all the way up to the root! Recall from highschool that a vector is basically Length or Size multiplied by Direction, so the location of the “lhand” sensor would be:

$$lhand = length_{lhand} * direction_{lhand} + length_{lwrist} * direction_{lwrist} + ... + length_{root} * direction_{root}$$

And that’s not even the hard part.

The hard part is translating the content of the AMC file into coordinates. Again, the AMC file doesn’t contain for each bone, for each frame its location in 3D space. That would have been too easy. Instead it contains its relative motion shift in Euler angles. Euler Angles are used to describe the rotation of the bone in space, relative to its parent bone. A local rotation can be achieved by multiplying the bone’s vector by three Rotation matrices, one for each axis. E.g. to get “lhand” location at frame $$t$$ we do:

$$lhand_{t} = lhand_{t - 1} * R_x(\alpha) * R_y(\beta) * R_z(\gamma)$$

Where $$\alpha$$, $$\beta$$ and $$\gamma$$ are the Euler angles at frame $$t$$, $$R_x$$, $$R_y$$ and $$R_z$$ are the Rotation matrices3. But again, this is relative to its parent bone, so this entire expression needs to be multiplied by the same expression for the parent “lradius”, which is multiplied by the same expression for the parent “lhumerus”, etc.

And I haven’t talked about the entire skeleton “translation” or movement in space and the “angle axes”. You see, it is impossible to convey to you a full course in CG in this small post. I just wanted you to get a feel of the complexity of this work. If you’re really interested take a look at this great course and of course my source code. It will keep you busy for days.

Basic Mocap

Let’s have a look at what the mocap package can do. We’ll parse in the ASF/AMC pair of files4:

library(mocap)

asfFilePath <- system.file("extdata", "lambada.asf", package = "mocap")

amcFilePath <- system.file("extdata", "lambada.amc", package = "mocap")
amc <- readAMC(amcFilePath, asf)

Let’s see what we got there:

str(asf)
## List of 5
##  $skeleton :'data.frame': 31 obs. of 16 variables: ## ..$ bone  : chr [1:31] "root" "lhipjoint" "lfemur" "ltibia" ...
##   ..$dirx : num [1:31] 0 0.6433 0.342 0.342 0.0435 ... ## ..$ diry  : num [1:31] 0 -0.707 -0.94 -0.94 -0.12 ...
##   ..$dirz : num [1:31] 0 0.295 0 0 0.992 ... ## ..$ length: num [1:31] 0 2.59 7.07 7.94 1.84 ...
##   ..$angx : num [1:31] 0 0 0 0 -90 -90 0 0 0 -90 ... ## ..$ angy  : num [1:31] 0.00 0.00 0.00 0.00 7.63e-16 ...
##   ..$angz : num [1:31] 0 0 20 20 20 20 0 -20 -20 -20 ... ## ..$ dofx  : num [1:31] 1 0 1 1 1 1 0 1 1 1 ...
##   ..$dofy : num [1:31] 1 0 1 0 0 0 0 1 0 0 ... ## ..$ dofz  : num [1:31] 1 0 1 0 1 0 0 1 0 1 ...
##   ..$child : num [1:31] 1 2 3 4 5 6 7 8 9 10 ... ## ..$ parent: num [1:31] 0 1 2 3 4 5 1 7 8 9 ...
##   ..$x : num [1:31] 0 9.401 13.645 15.331 0.452 ... ## ..$ y     : num [1:31] 0 -10.33 -37.49 -42.12 -1.24 ...
##   ..$z : num [1:31] 0 4.31 0 0 10.3 ... ##$ childs   :List of 31
##   ..$root : chr [1:3] "lhipjoint" "rhipjoint" "lowerback" ## ..$ lhipjoint: chr "lfemur"
##   ..$lfemur : chr "ltibia" ## ..$ ltibia   : chr "lfoot"
##   ..$lfoot : chr "ltoes" ## ..$ ltoes    : NULL
##   ..$rhipjoint: chr "rfemur" ## ..$ rfemur   : chr "rtibia"
##   ..$rtibia : chr "rfoot" ## ..$ rfoot    : chr "rtoes"
##   ..$rtoes : NULL ## ..$ lowerback: chr "upperback"
##   ..$upperback: chr "thorax" ## ..$ thorax   : chr [1:3] "lowerneck" "lclavicle" "rclavicle"
##   ..$lowerneck: chr "upperneck" ## ..$ upperneck: chr "head"
##   ..$head : NULL ## ..$ lclavicle: chr "lhumerus"
##   ..$lhumerus : chr "lradius" ## ..$ lradius  : chr "lwrist"
##   ..$lwrist : chr [1:2] "lhand" "lthumb" ## ..$ lhand    : chr "lfingers"
##   ..$lfingers : NULL ## ..$ lthumb   : NULL
##   ..$rclavicle: chr "rhumerus" ## ..$ rhumerus : chr "rradius"
##   ..$rradius : chr "rwrist" ## ..$ rwrist   : chr [1:2] "rhand" "rthumb"
##   ..$rhand : chr "rfingers" ## ..$ rfingers : NULL
##   ..$rthumb : NULL ##$ CMatList :List of 31
##   ..$root :List of 2 ## .. ..$ C   : num [1:3, 1:3] 1 0 0 0 1 0 0 0 1
##   .. ..$Cinv: num [1:3, 1:3] 1 0 0 0 1 0 0 0 1 ## ..$ lhipjoint:List of 2
##   .. ..$C : num [1:3, 1:3] 1 0 0 0 1 0 0 0 1 ## .. ..$ Cinv: num [1:3, 1:3] 1 0 0 0 1 0 0 0 1
##   ..$lfemur :List of 2 ## .. ..$ C   : num [1:3, 1:3] 0.94 -0.342 0 0.342 0.94 ...
##   .. ..$Cinv: num [1:3, 1:3] 0.94 0.342 0 -0.342 0.94 ... ## ..$ ltibia   :List of 2
##   .. ..$C : num [1:3, 1:3] 0.94 -0.342 0 0.342 0.94 ... ## .. ..$ Cinv: num [1:3, 1:3] 0.94 0.342 0 -0.342 0.94 ...
##   ..$lfoot :List of 2 ## .. ..$ C   : num [1:3, 1:3] 9.40e-01 -3.35e-17 -3.42e-01 3.42e-01 5.30e-17 ...
##   .. ..$Cinv: num [1:3, 1:3] 9.40e-01 3.42e-01 -1.33e-17 -3.35e-17 5.30e-17 ... ## ..$ ltoes    :List of 2
##   .. ..$C : num [1:3, 1:3] 9.40e-01 -3.35e-17 -3.42e-01 3.42e-01 5.30e-17 ... ## .. ..$ Cinv: num [1:3, 1:3] 9.40e-01 3.42e-01 -1.33e-17 -3.35e-17 5.30e-17 ...
##   ..$rhipjoint:List of 2 ## .. ..$ C   : num [1:3, 1:3] 1 0 0 0 1 0 0 0 1
##   .. ..$Cinv: num [1:3, 1:3] 1 0 0 0 1 0 0 0 1 ## ..$ rfemur   :List of 2
##   .. ..$C : num [1:3, 1:3] 0.94 0.342 0 -0.342 0.94 ... ## .. ..$ Cinv: num [1:3, 1:3] 0.94 -0.342 0 0.342 0.94 ...
##   ..$rtibia :List of 2 ## .. ..$ C   : num [1:3, 1:3] 0.94 0.342 0 -0.342 0.94 ...
##   .. ..$Cinv: num [1:3, 1:3] 0.94 -0.342 0 0.342 0.94 ... ## ..$ rfoot    :List of 2
##   .. ..$C : num [1:3, 1:3] 9.40e-01 3.35e-17 3.42e-01 -3.42e-01 5.30e-17 ... ## .. ..$ Cinv: num [1:3, 1:3] 9.40e-01 -3.42e-01 1.33e-17 3.35e-17 5.30e-17 ...
##   ..$rtoes :List of 2 ## .. ..$ C   : num [1:3, 1:3] 9.40e-01 3.35e-17 3.42e-01 -3.42e-01 5.30e-17 ...
##   .. ..$Cinv: num [1:3, 1:3] 9.40e-01 -3.42e-01 1.33e-17 3.35e-17 5.30e-17 ... ## ..$ lowerback:List of 2
##   .. ..$C : num [1:3, 1:3] 1 0 0 0 1 0 0 0 1 ## .. ..$ Cinv: num [1:3, 1:3] 1 0 0 0 1 0 0 0 1
##   ..$upperback:List of 2 ## .. ..$ C   : num [1:3, 1:3] 1 0 0 0 1 0 0 0 1
##   .. ..$Cinv: num [1:3, 1:3] 1 0 0 0 1 0 0 0 1 ## ..$ thorax   :List of 2
##   .. ..$C : num [1:3, 1:3] 1 0 0 0 1 0 0 0 1 ## .. ..$ Cinv: num [1:3, 1:3] 1 0 0 0 1 0 0 0 1
##   ..$lowerneck:List of 2 ## .. ..$ C   : num [1:3, 1:3] 1 0 0 0 1 0 0 0 1
##   .. ..$Cinv: num [1:3, 1:3] 1 0 0 0 1 0 0 0 1 ## ..$ upperneck:List of 2
##   .. ..$C : num [1:3, 1:3] 1 0 0 0 1 0 0 0 1 ## .. ..$ Cinv: num [1:3, 1:3] 1 0 0 0 1 0 0 0 1
##   ..$head :List of 2 ## .. ..$ C   : num [1:3, 1:3] 1 0 0 0 1 0 0 0 1
##   .. ..$Cinv: num [1:3, 1:3] 1 0 0 0 1 0 0 0 1 ## ..$ lclavicle:List of 2
##   .. ..$C : num [1:3, 1:3] 1 0 0 0 1 0 0 0 1 ## .. ..$ Cinv: num [1:3, 1:3] 1 0 0 0 1 0 0 0 1
##   ..$lhumerus :List of 2 ## .. ..$ C   : num [1:3, 1:3] 5.30e-17 -1.00 -9.18e-17 -8.66e-01 -1.23e-32 ...
##   .. ..$Cinv: num [1:3, 1:3] 5.30e-17 -8.66e-01 5.00e-01 -1.00 -7.12e-33 ... ## ..$ lradius  :List of 2
##   .. ..$C : num [1:3, 1:3] 5.30e-17 -1.00 -9.18e-17 -8.66e-01 -1.23e-32 ... ## .. ..$ Cinv: num [1:3, 1:3] 5.30e-17 -8.66e-01 5.00e-01 -1.00 -7.12e-33 ...
##   ..$lwrist :List of 2 ## .. ..$ C   : num [1:3, 1:3] 3.75e-33 -1.00 -3.90e-17 6.12e-17 -3.90e-17 ...
##   .. ..$Cinv: num [1:3, 1:3] 3.75e-33 6.12e-17 -1.00 -1.00 -3.90e-17 ... ## ..$ lhand    :List of 2
##   .. ..$C : num [1:3, 1:3] 3.75e-33 -1.00 -1.96e-16 6.12e-17 -1.96e-16 ... ## .. ..$ Cinv: num [1:3, 1:3] 3.75e-33 6.12e-17 -1.00 -1.00 -1.96e-16 ...
##   ..$lfingers :List of 2 ## .. ..$ C   : num [1:3, 1:3] 3.75e-33 -1.00 -4.54e-16 6.12e-17 -4.54e-16 ...
##   .. ..$Cinv: num [1:3, 1:3] 3.75e-33 6.12e-17 -1.00 -1.00 -4.54e-16 ... ## ..$ lthumb   :List of 2
##   .. ..$C : num [1:3, 1:3] 7.07e-01 -7.07e-01 1.60e-16 -8.25e-17 1.44e-16 ... ## .. ..$ Cinv: num [1:3, 1:3] 7.07e-01 -8.25e-17 -7.07e-01 -7.07e-01 1.44e-16 ...
##   ..$rclavicle:List of 2 ## .. ..$ C   : num [1:3, 1:3] 1 0 0 0 1 0 0 0 1
##   .. ..$Cinv: num [1:3, 1:3] 1 0 0 0 1 0 0 0 1 ## ..$ rhumerus :List of 2
##   .. ..$C : num [1:3, 1:3] 5.30e-17 1.00 9.18e-17 8.66e-01 -1.23e-32 ... ## .. ..$ Cinv: num [1:3, 1:3] 5.30e-17 8.66e-01 -5.00e-01 1.00 -7.12e-33 ...
##   ..$rradius :List of 2 ## .. ..$ C   : num [1:3, 1:3] 5.30e-17 1.00 9.18e-17 8.66e-01 -1.23e-32 ...
##   .. ..$Cinv: num [1:3, 1:3] 5.30e-17 8.66e-01 -5.00e-01 1.00 -7.12e-33 ... ## ..$ rwrist   :List of 2
##   .. ..$C : num [1:3, 1:3] 3.75e-33 1.00 3.90e-17 -6.12e-17 -3.90e-17 ... ## .. ..$ Cinv: num [1:3, 1:3] 3.75e-33 -6.12e-17 1.00 1.00 -3.90e-17 ...
##   ..$rhand :List of 2 ## .. ..$ C   : num [1:3, 1:3] 3.75e-33 1.00 1.96e-16 -6.12e-17 -1.96e-16 ...
##   .. ..$Cinv: num [1:3, 1:3] 3.75e-33 -6.12e-17 1.00 1.00 -1.96e-16 ... ## ..$ rfingers :List of 2
##   .. ..$C : num [1:3, 1:3] 3.75e-33 1.00 4.54e-16 -6.12e-17 -4.54e-16 ... ## .. ..$ Cinv: num [1:3, 1:3] 3.75e-33 -6.12e-17 1.00 1.00 -4.54e-16 ...
##   ..$rthumb :List of 2 ## .. ..$ C   : num [1:3, 1:3] 7.07e-01 7.07e-01 -1.60e-16 8.25e-17 1.44e-16 ...
##   .. ..$Cinv: num [1:3, 1:3] 7.07e-01 8.25e-17 7.07e-01 7.07e-01 1.44e-16 ... ##$ len      : num 0.45
##  $angleType: chr "deg" str(amc) ## List of 4 ##$ D       : num [1:62, 1:2180] 16.71 97.4 -152.8 2.51 -3.14 ...
##   ..- attr(*, "dimnames")=List of 2
##   .. ..$: NULL ## .. ..$ : chr [1:2180] "row" "row" "row" "row" ...
##  $degrees :List of 31 ## ..$ root     : num [1:2180, 1:3] 2.51 2.27 2.16 2.24 2.11 ...
##   ..$lowerback: num [1:2180, 1:3] 3.85 3.85 3.69 3.73 3.66 ... ## ..$ upperback: num [1:2180, 1:3] -2.63 -2.59 -2.54 -2.63 -2.51 ...
##   ..$thorax : num [1:2180, 1:3] -4.8 -4.78 -4.65 -4.77 -4.61 ... ## ..$ lowerneck: num [1:2180, 1:3] 6.15 7.01 6.8 5.88 5.58 ...
##   ..$upperneck: num [1:2180, 1:3] -14.4 -15.3 -15 -13.8 -13.7 ... ## ..$ head     : num [1:2180, 1:3] -4.09 -4.51 -4.4 -3.91 -3.83 ...
##   ..$rclavicle: num [1:2180, 1:3] 0 0 0 0 0 0 0 0 0 0 ... ## ..$ rhumerus : num [1:2180, 1:3] -2.98 -3.13 -3.41 -3.88 -4.08 ...
##   ..$rradius : num [1:2180, 1:3] 89.7 89.8 89.9 90 90.2 ... ## ..$ rwrist   : num [1:2180, 1:3] 0 0 0 0 0 0 0 0 0 0 ...
##   ..$rhand : num [1:2180, 1:3] -17 -17.1 -17 -16.7 -16.9 ... ## ..$ rfingers : num [1:2180, 1:3] 7.13 7.13 7.13 7.13 7.13 ...
##   ..$rthumb : num [1:2180, 1:3] 9.22 9.16 9.26 9.51 9.37 ... ## ..$ lclavicle: num [1:2180, 1:3] 0 0 0 0 0 0 0 0 0 0 ...
##   ..$lhumerus : num [1:2180, 1:3] 15.9 15.6 15.5 15.8 16.2 ... ## ..$ lradius  : num [1:2180, 1:3] 72.1 72.4 72.7 73.1 73.6 ...
##   ..$lwrist : num [1:2180, 1:3] 0 0 0 0 0 0 0 0 0 0 ... ## ..$ lhand    : num [1:2180, 1:3] -49.6 -49.6 -49.2 -48.3 -47.6 ...
##   ..$lfingers : num [1:2180, 1:3] 7.13 7.13 7.13 7.13 7.13 ... ## ..$ lthumb   : num [1:2180, 1:3] -22.2 -22.2 -21.8 -21 -20.3 ...
##   ..$rfemur : num [1:2180, 1:3] -4.28 -4.5 -4.95 -5.76 -6.31 ... ## ..$ rtibia   : num [1:2180, 1:3] 26.8 27.7 29 30.4 31.9 ...
##   ..$rfoot : num [1:2180, 1:3] -24.3 -25.2 -26 -26.4 -26.9 ... ## ..$ rtoes    : num [1:2180, 1:3] 35.7 33.4 32 32.1 31.6 ...
##   ..$lfemur : num [1:2180, 1:3] -20.4 -20 -19.7 -19.7 -19.3 ... ## ..$ ltibia   : num [1:2180, 1:3] 43.3 42.9 42.5 42 41.4 ...
##   ..$lfoot : num [1:2180, 1:3] -23.1 -23.3 -23.3 -23 -22.8 ... ## ..$ ltoes    : num [1:2180, 1:3] -24.3 -24.5 -23.6 -23.4 -24.1 ...
##   ..$lhipjoint: num [1:2180, 1:3] 0 0 0 0 0 0 0 0 0 0 ... ## ..$ rhipjoint: num [1:2180, 1:3] 0 0 0 0 0 0 0 0 0 0 ...
##  $nFrames : int 2180 ##$ skeleton:'data.frame':    31 obs. of  16 variables:
##   ..$bone : chr [1:31] "root" "lowerback" "upperback" "thorax" ... ## ..$ dirx  : num [1:31] 0 0.00142 0.01251 0.01364 0.07668 ...
##   ..$diry : num [1:31] 0 0.998 1 0.999 0.997 ... ## ..$ dirz  : num [1:31] 0 -0.05648 0.00361 0.02951 0.00422 ...
##   ..$length: num [1:31] 0 2.3 2.31 2.31 1.69 ... ## ..$ angx  : num [1:31] 0 0 0 0 0 0 0 0 180 180 ...
##   ..$angy : num [1:31] 0 0 0 0 0 0 0 0 30 30 ... ## ..$ angz  : num [1:31] 0 0 0 0 0 0 0 0 90 90 ...
##   ..$dofx : num [1:31] 1 1 1 1 1 1 1 0 1 1 ... ## ..$ dofy  : num [1:31] 1 1 1 1 1 1 1 1 1 0 ...
##   ..$dofz : num [1:31] 1 1 1 1 1 1 1 1 1 0 ... ## ..$ child : num [1:31] 1 12 13 14 15 16 17 25 26 27 ...
##   ..$parent: num [1:31] 0 1 12 13 14 15 16 14 25 26 ... ## ..$ x     : num [1:31] 0 0.0184 0.1629 0.1779 0.7307 ...
##   ..$y : num [1:31] 0 13 13 13 9.5 ... ## ..$ z     : num [1:31] 0 -0.7347 0.047 0.3848 0.0402 ...

OK, we got a lot. The main element in the asf object is the childs list, detailing for each “parent” bone its “children” bones. Other elements are either out of scope for this post, or generally should not be used5.

The main elements in the amc object are the skeleton data frame, describing various characteristics for each bone, and the degrees list, giving the (nFrames x 3 axes) matrix of Euler Angles for each bone.

Now to get the actual motion 3D coordinates, use the getMotionData function with the asf and amc objects:

xyz <- getMotionData(asf, amc)

The xyz object contains the bottom-line. It is a list of (nFrames x 3 axes) coordinates for each and every bone in the skeleton. These are the data you would use for plotting and for research (see section: “War: What Else Is It Good For?”).

And speaking of plotting, if you have the scatterplot3d and animation packages installed, you can make a movie out of this motion (either a .gif or a .mp4, see the manual):

makeMotionMovie(asf, amc, xyz, skipNFrames = 4)

Notice that for making the gif/mp4 files, the (wonderful!) animation package will need you to install ImageMagick and ffmpeg.

Now let’s do a backflip! (so you can it all at once)

asf <- readASF(system.file("extdata", "backflip.asf", package = "mocap"))
amc <- readAMC(system.file("extdata", "backflip.amc", package = "mocap"), asf)
xyz <- getMotionData(asf, amc)
makeMotionMovie(asf, amc, xyz, skipNFrames = 4)

In the CMU Graphics Lab Database there are thousands of these. Just download a ASF/AMC pair of files and make a movie out of them yourself. And here is the plce to make a BIG disclaimer: only ASF/AMC files from the CMU Graphics Lab Database were tested! It’s highly likely that other ASF/AMC files could fail. I’ll change this if there is demand.

War: What Is It Good For?

So what is this good for? First of all its fun!

You can plot a few replications of the same motion, or what I like to call…

asf <- readASF(system.file("extdata", "zombie.asf", package = "mocap"))
amc <- readAMC(system.file("extdata", "zombie.amc", package = "mocap"), asf)
xyz <- getMotionData(asf, amc)
makeMotionMovie(asf, amc, xyz, skipNFrames = 3, nSkeletons = 10, sdExtraSkeleton = 100)

You can plot two skeletons at the same time (there a few nice examples of such interaction in the CMU Database):

asf <- readASF(system.file("extdata", "charleston.asf", package = "mocap"))
amc1 <- readAMC(system.file("extdata", "charleston1.amc", package = "mocap"), asf)
amc2 <- readAMC(system.file("extdata", "charleston2.amc", package = "mocap"), asf)
xyz1 <- getMotionData(asf, amc1)
xyz2 <- getMotionData(asf, amc2)
makeMotionMovie(asf, amc1, xyz1, skipNFrames = 2,
twoSkeletons = TRUE, amc2 = amc2, xyz2 = xyz2, viewAngle = 20)

You can rotate a skeleton (not sure why I wanted this but the parameter is there…):

makeMotionMovie(asf, amc1, xyz1, skipNFrames = 2,
twoSkeletons = TRUE, amc2 = amc2, xyz2 = xyz2, viewAngle = 0,
rotateMotion = TRUE, axis = FALSE, grid = FALSE)

I have even dabbled with actually “dressing” the skeleton with a real person, but…

Yes… You can’t blame me for trying.

War: What Else Is It Good For?

Jokes aside, some really interesting research is done with these data. Going back to the little article6 which ignited my curiousity, the researchers used the motion data from a Samba teacher and two students in order to identify the student who had more difficulty in following the teacher’s steps. They simply calculated the “distance” between the teacher’s bones movement and the students’ bones movement and saw which student was “further away”. A similar use case could be having Roger Federer recorded playing tennis via MoCap, then having every other player recorded and seeing where do these other players deviate from the “ideal” which is Federer. They could probably get a few good pointers even Federer cannot put into words.

Tido Röder’s dissertation is another great source to see what is being done with MoCap data. I’ll leave you to it.

Ending Captions

I’ll admit it, the mocap package is far from perfect. Go ahead, test it on CMU Graphics Lab ASF/AMC files, tell me where it breaks. If there is demand it could grow into a “real” MoCap package. Now dance I said.

1. we actually call this element a bone

2. In fact the mocap package can currently parse this file format only. If there’s demand we’ll move on to BVH, C3D etc.

3. See Wikipedia, I’m talking 3x3 matrices with some $$cos$$ and $$sin$$

4. I have renamed these files so I can identify what they do. All files that come with the package are from the CMU Graphics Database, in which they have different names

5. I’m talking about the skeleton element - use the skeleton element which is returned by the amc object!

6. Chavoshi SH, De Baets B, Neutens T, De Tré G, Van de Weghe N (2015) Exploring Dance Movement Data Using Sequence Alignment Methods. PLoS ONE 10(7): e0132452. https://doi.org/10.1371/journal.pone.0132452