EyeSeeCam SCI

Description

The EyeSeeCam SCI is a combined eye tracker and head movement tracker. The head tracking is done with an IMU and the eye tracking is done with cameras and software that tracks the pupils of both eyes.

Construction of a gaze trace

From the EyeSeeCam data we can construct a head movement trace and an eye movement trace. These can be combined to create a gaze trace.

Defining a temporary lab coordinate system

We will define the starting direction as the direction the subject looks at the beginning of each trial. The recording of the data should start when the subject is looking in this direction.
For the purpose of the interpretation of the EyeSeeCam data we define the lab coordinates as identical to the EyeSeeCam coordinates when looking in the starting direction (starting gaze):

X = forward
Y = left
Z = upwards

In the corresponding matlab function we will use these temporary lab coordinates to calculate the rotation of the gaze direction but convert it in the end to double polar coordinates.

Head movement

For head movements rotations around X, Y and Z are defined with respect to the EyeSeeCam, which is moving.

The EyeSeeCam data is thus not in the lab frame!

The EyeSeeCam gives angle velocities for rotations around X, Y and Z axis. In order to create a head movement trace we have to calculate rotation matrices (delta_Rx, delta_Ry and delta_Rz) in EyeSeeCam coordinate for every time step.

Since the time steps are about 2 ms the rotations for each time step are small. Therefore the order in which delta_Rx, delta_Ry and delta_Rz are multiplied are not important and we calculate delta_R(t) in EyeSeeCam coordinates by

delta_R(t) = delta_Rx * delta_Ry * delta_Rz.

To get R(t) in lab coordinates we have

R(t) = R(t-delta_t) * delta_R(t).

Eye movement

The eye movements are given by rotations in the oculomotor coordinate system (ocs).

X = anterior (torsion of the pupil in the gaze direction)
Y = superior
Z = right

The torsion is not important for the gaze direction and is there excluded from the calculation.

Since the eye gaze is given as a rotation angle in OCS coordinates and the eye and head tracker coordinate are fixed relative to each other, we can easily transform OCS coordinates to the EyeSeeCam coordinates. We can then create rotation matrices in the EyeSeeCam coordinates for the azimuth and elevation of the gaze. Sine the torsion is excluded the rotation matrices for azimuth and elevation can be multiplied in any order to get the eye rotation matrix in EyeSeeCam coordinates.

eye_Rz(t) = Rz(eye_azimuth(t))
eye_Ry(t) = Ry(-eye_elevation(t))
eye_R(t)  = eye_Ry(t) * eye_Rz(t)

Gaze movement

The gaze movement is the combined head and eye movement. To create the total rotation matrix that represents the 'gaze' rotation in lab coordinates, you need to multiply the two rotation matrices you have for head and eye movement in the correct order. In your case, you have:

A rotation matrix for the head in lab coordinates. A rotation matrix for the eyes in head coordinates. The correct order of multiplication is as follows:

First, multiply the rotation matrix of the head in lab coordinates by the rotation matrix of the eyes in head coordinates.

R_gaze(t) = head_R(t) * eye_R(t)

Lastly you can multiply the R_gaze(t) with the starting gaze vector to get the gaze (in lab coordinates) at time t. We assume that the starting gaze is in the forward direction, which is in the X direction in the lab coordinates.

startingGaze = [1; 0; 0]
gaze(t) = R_gaze(t) * startingGaze

Matlab programming

Functions for converting EyeSeeCam data to double polar data

There is one function for head movement, one for eye movement and one for gaze movement. The gaze movement combines head and eye movement.

The functions take head rotation speeds and eye rotations (in degrees) as input and give objects containing the traces (head, eye and gaze respectivily) in double polar coordinates as output.

headTrace_DP = EyeSeeCamSciHeadRotationData2Trace_DP(Vx_deg, Vy_deg, Vz_deg)
eyeTrace_DP = EyeSeeCamSciEyeRotationData2Trace_DP(Zocs_deg, Yocs_deg)
gazeTrace_DP = EyeSeeCamSciHeadAndEyeRotationData2Trace_DP(Vx_deg, Vy_deg, Vz_deg, Zocs_deg, Yocs_deg)

The Vx, Vy and Vz are the head rotation speeds in the EyeSeeCam coordinates The Zocs and Yocs are the eye rotations in OCS coordinates.

EyeSeeCam data

The recorded data is read from an LSL stream.

% todo: LSL streaming parameters

The lsldata is a struct and has the following fields:

escdata
escmetadata
escstr
evdata0
evdata1
evdata2
evdata3
evdata4

The lsldata.escdata contains the eye and head data. The data contains a matrix for with a row for every parameter. The total of parameters is 63. The field lsldata.escmetadata.channel lists all the names of the parameters.

%Eye data

RightEyePosX = 46;
RightEyePosY = 47;
RightEyePosZ = 48;

xeye = lsldata.escdata.Data(46,:);       % right 46, left 32
yeye = lsldata.escdata.Data(47,:);       % right 47, left 33
zeye = lsldata.escdata.Data(48,:);       % right 48, left 34

% Head data
HeadInertialVelXCal = 27;
HeadInertialVelYCal = 29;
HeadInertialVelZCal = 31;

xh = lsldata.escdata.Data(27,:);       % calibrated torion velocity data HEAD
yh = lsldata.escdata.Data(29,:);       % calibrated vertical velocity data HEAD
zh = lsldata.escdata.Data(31,:);       % calibrated horizontal velocity data HEAD