Installation#

See installing fictrac

Configuration#

There are two steps for configuring fictrac prior to running the program.

First, you must provide a text file in the fictrac/sample directory that contains the configuration parameters for your setup. At a minimum, you must define the following parameters:

  1. src_fn - defines the image source. This can be either a video file or a camera index.
  2. To set your camera index, check the order of your cameras in SpinView.
  3. Note: If you receive an error or your fictrac feed is blank, try a different camera port as you may be connected to an unavailable or incorrect port.
  4. vfov - defines the vertical field of view (angle in degrees).
  5. This can be approximated by measuring your camera angle [tan(x) = vertical field of view in the fictrac feed/distance from your camera to the ball]. 1. Example: for a setup with a distance from the camera sensor to the ball of distance_var = 17.5 (cm), and the full height of the image used by fictrac (the rectangular ROI you select for recordings in SpinView) height_var = 0.78 (cm), the vertical field of view would be:vfov = radtodeg(2*atan(height_var/(2*distance_var))).
  6. Note: to calculate the full height of the image used by fictrac (for the example above), take a picture with a ruler at the same focal point as the ball, and use the distance between lines (1 mm in this example) to calculate the height of whole the image.

  1. Note: This is an approximation and you should try several similar values to be sure you have selected a value that works best for fictrac. For example, if you calculate vfov as 3.2, try values between 2-5 as well. You can assess each value based on how well fictrac appears to map the ball and easily the map appears to break.*

Optional parameters to consider:

  1. q_factor - adjusts the resolution of the tracking window, smaller values will result in coarser, quicker tracking and vice versa. The default is set to 6, but bumping the resolution up to 8 can be helpful when trying to reduce noise (EW).
  2. thr_ratio - adjusts the white background/black foreground threshold. The default is set to 1.25, but dropping the threshold down to 1.1/0.9 can be helpful when trying to reduce noise, though the optimal value will depend on your pattern and illumination (EW).
  3. sock_port - defines the socket port for client-based connection (closed loop, phidget)
  4. This can be somewhat arbitrary, so long as it matches your python script for communicating between fictrac and the phidget. This is commonly set to 65432.
  5. Note: If FicTrac is struggling to connect a given port, just try another (e.g. 65413) so long as it is greater than 1023 (which are privileged ports).
  6. Note: Unlike in previous versions, FicTrac will run regardless of whether the socket port is “listening” (aka a python script is running).

Sample configuration file for live feed from the side:

📎 config_live.txt

Sample configuration file for live feed from behind:

📎 config_live.txt

Second, you must run the interactive configuration program (configGUI) for calibrating the region of interest for tracking your ball within the input images as well as for transforming between the camera’s and animal’s frames of reference.

  1. To configure fictrac, simply open a command terminal and type:
  2. cd <path to>\fictrac\sample
  3. ..\bin\Release\configGui.exe config_live.txt
  4. At this point, you should be prompted to set:
  5. The spherical ROI
  6. The ignore regions (optional) 1. Typically helpful to “ignore” the ball holder 1. It can be helpful to “ignore” any shadowed regions (e.g. bottom and edges of ball) depending on the evenness of your illumination. If your illumination is uneven at the edges of the ball, and they are not ignored, fictrac is more likely to get into a loop of circular motion independent of the actual motion of the ball.
  7. The XYZ axis transformation 1. In general:
    1. X should point towards the fly’s front
    2. Y should point towards the fly’s right
    3. Z should point down 1. The X or O at the end of each axis denotes whether it is pointing into or out of the view, respectively 1. When prompted to select 4 reference points, it is best to create the tightest possible square. Use the zoomed window to ensure that each reference point is perfectly aligned to the previous. If the points are not perfectly aligned, the transformation will be skewed. 1. Some example configurations for
    4. Camera behind the fly

c2a_cnrs_xy : { 235, 540, 235, 540, 235, 540, 215, 619 }

c2a_cnrs_xz : { 765, 645, 765, 645, 767, 713, 767, 713 }

c2a_cnrs_yz : { 493, 376, 509, 376, 509, 392, 493, 392 }

c2a_r : { 1.2, 1.2, 1.2 }

c2a_src : ext

c2a_t : { 0, 0, 0 }

  1. Camera on the fly's left

c2a_cnrs_xz      : { 509, 376, 526, 376, 526, 393, 509, 393 }

c2a_r            : { 0, 2.23, 2.23 }

c2a_src          : c2a_cnrs_xz

c2a_t            : { -5.558824, -7.500000, 574.678940 }

Here is an example config image from behind:

Important Note: It is good practice to configure fictrac before every experiment to ensure that the set ROIs have not changed due to slight changes in your setup.

Setting the transform based on camera position#

The c2a_r(camera to animal) parameter defines the transformation between your camera coordinates and the animal’s frame of reference. This is critical for correctly converting the rotations of the ball as seen by the camera into the rotations in the real world relative to the animal. This transform can be set in the GUI but this is error prone and really only seems to work reasonably well when the camera plane is orthogonal to one of the animal axes (i.e. from the top, behind, left, right, etc.).

To set this transform manually (rather than using the gui), c2a_src must be set to ext: c2a_src : ext

c2a_r is defined as an axis-angle representation

The default view (c2a is from the top: the X axis points forward, the Y axis points to the animal’s right, and the Z axis points down:

Here are some different configurations you might end up using

And here is matlab code that computes the axis angle representation based on azimuth and tilt, defined as illustrated above:


camera_deg_azimuth = -90;
camera_deg_tilt = 90;
R1 = rotz(-camera_deg_azimuth); 
R2 = rotx(camera_deg_tilt); 
axang = rotm2axang(R1*R2);
A = axang(1:3)*axang(4);
disp(A)

This is partcularly useful for the wind setup, where the angle needs to view the ball from an angle o avoid too much being obscured by the spout. E.g.:

To measure the azimuth angle, this can be done simply with a protractor or a ruler.

To measure the tilt angle, it might be helpful to use a digital level on the camera body, like this:

Reducing Noise#

You should consider optimizing the position and configuration of your setup in order to ensure that fictrac runs optimally and with as little noise as possible:

  1. Ball pattern - use a small, non-repeating pattern to improve tracking and avoid blotchy patterns. Many folks in the lab have found that drawing numbers and/or letters on the ball is a good option.
  2. Ensure that the pattern is solid and high-contrast, avoid speckles when possible.
  3. Camera position - ensure the entire ball is in focus
  4. Note that fictrac will have more noise along whichever axis is parallel to the camera, as less of the ball is exposed along that axis. For instance, if the camera is placed behind the fly, the forward axis will likely be more noisy. Alternatively, if the camera is placed on the side of the fly, the roll axis will likely be more noisy. etc.
  5. Illumination position - ensure the entire ball is evenly and brightly illuminated. Avoid maxing out the illumination/exposure, such that the dark portions of the pattern start to look speckled.
  6. Acquisition configuration - in SpinView:
  7. Image contrast - increase Gain and decrease Black Level under the Setting tab
  8. Image ROI - you may need to adjust your field of view so that the ball is in frame for fictrac. You can do this by adjusting the Width and Height under the Image Format tab

Running FicTrac#

  1. Configure FicTrac as described above:
  2. cd <path to>\fictrac\sample
  3. ..\bin\Release\configGui.exe config_live.txt
  4. To start FicTrac, open a command terminal and type:
  5. > cd <path to>\fictrac\sample
  6. > ..\bin\Release\fictrac.exe config_live.txt
  7. If you have successfully configured FicTrac, the command prompt should open a live window with the input image, path, and instant/accumulated maps:

  1. FicTrac will generate two outputs files:

  2. A log file (*.log) - containing debugging information about FicTrac’s execution.

  3. A data file (*.dat) - containing output data. See data_header for information about output data.

Running FicTrac From Matlab#

Helpful code for launching fictrac from Matlab:

  • Configure fictrac:
fictracCfg = 'cd/ & C: & cd/ & cd dev\fictrac\sample & ..\bin\Release\configGui.exe config_live.txt &';
[~,~] = system(fictracCfg);
  • Launch fictrac:
fictracStart = 'cd/ & C: & cd/ & cd dev\fictrac\sample & ..\bin\Release\fictrac.exe config_live.txt &';
[~,~] = system(fictracStart);
  • Launch python:
pythonStart = 'cd/ & C: & cd/ & cd dev\fictrac\scripts & python socket_client_360_cl.py &';
[~,~] = system(pythonStart);

Test Footage#

To see if FicTrac is behaving the way you expect it to, you can always try configuring and running it on some sample footage of a ball rotating clockwise three times. The file is on the server under manual, protocols, and databases > FicTrac DeBugging (too large to upload).

Data acquisition#

  • camera frame rate should be 50-80 Hz
  • NiDAQ acquisition rate will be much faster (e.g., 4-20 kHz) - this is because the NiDAQ acquisition rate is constrained to be the same for all your acquisition channels, and you are generally acquiring multiple types of data at the same time, some of which require fast sampling rates

Data processing#

  1. Convert the ball position signal from voltage to radians, then unwrap the position signal, and then fix edge cases where unwrapping fails.
  2. Downsample the position signal to a frequency that is half of your camera rate, or less (according to the Nyquist theorem). Thus, if your camera rate is 50 Hz, you should downsample to 25 Hz or less. It is recommended to use ‘resample’ instead of ‘downsample’ in Matlab, to avoid aliasing.
  3. Low-pass filter (i.e., smooth) the position signal, e.g. with a Butterworth filter with 10 Hz cutoff.
  4. Compute the time derivative of position in each axis (forward velocity, lateral velocity, rotational velocity) and multiply by downsampled frequency to obtain velocity in units of mm/s or degrees/s.
  5. Convert each axis of ball displacement to its proper fly-centric units:
  • fly forward displacement (ball pitch): mm
  • fly lateral displacement (ball roll): mm
  • fly rotational displacement (ball yaw): degrees (or radians)

The order of operations matters here:

  • Note that it is important to unwrap before downsampling; otherwise, the downsampling with interact with the wraparound phenomenon to create artifacts.
  • Note that it is important to downsample before differentiating; otherwise, the velocity signal will jump abruptly between zero and some nonzero value (because the raw signal was digitized at a sampling rate much faster than the camera rate).
  • Consecutive linear operations can be performed in any order; these include low-pass filtering (i.e., smoothing), differentiation, and unit conversion. However, the optimal low-pass filter cutoff will probably depend on whether this step is performed before or after differentiation.

Next is an example figure to show what the signal looks like when going through these different steps:

Below you can find an example Matlab code that does this processing of the signal (in this case for angular data, but can be used for the other two axes using the proper unit conversion):

Newer Package:

📎 Fictrac Analyses.zip

  • Matt: I have found this package does a better job at preserving the smaller fluctuations in the fly’s position trajectory, and is less susceptible to aliasing problems when downsampling.

Older Script:

📎 process_fictrac_data.m

More tips#

  • If you use FicTrac in open-loop mode only, then you can use the output data file saved by FicTrac (saved into an h5 file)=instead of acquiring any FicTrac data with a DAQ. For example, to calculate velocity from an h5 file of position data, Jenny interpolated the integrated ball position along a given axis (‘_intx’, Fictrac version 2.0 with modifications to output to Redis client) using the timestamps from FicTrac (approximately 50Hz, but not precisely 50Hz) with the 50Hz downsampled DAQ timestamps (which is 50Hz precisely).  She then applied the same lowpass filtering and compute the velocity using the gradient function; the output velocity signal between Fictrac output and the downsampled DAQ signal was nearly identical.
    • See 3rd & 4th tip below for comments on this method
  • You may want to save the raw video. Michael noted that the frames of the video do not match up perfectly with the individual datapoints in the output data file. If you want to do align these datasets (e.g. to align an optic flow analysis to determine when the fly is grooming). there is a debug build of FicTrac available that will save an additional text file with the information required to match up the frames. 
  • If primarily using data from the FicTrac .dat file it’s important to note that occasionally fictrac drops frames/breaks and when it does it doesn’t write a new datapoint and so there will be a brief pause of some arbitrary amount of time between the last written datapoint and the next. This can lead to nonuniform sampling rates and so if trying to compare these signals with ones acquired through a different method (e.g DAQ signals or scanimage) it is very important to take into account the exact timestamp at which each fictrac datapoint was acquired and use this information during further processing and/or to resample your fictrac data to a uniform rate. Otherwise misalignments will accumulate between the signals.
  • Another caveat to depending on the fictrac data saved in the .dat file is that the time information associated with each fictrac datapoint can occasionally be saved incorrectly, and so can introduce inaccurate lengths of times between subsequent datapoints (these can be quite long, >30seconds). If using the time information saved in the .dat, for example to resample the fictrac .dat data to a uniform sampling rate as described above, this can then produce serious errors in the alignment between your fictrac data & any other signal you are trying to compare it against (such as neural activity, or panels cue position feedback). In comparison the DAQ clock is significantly more dependable and so to avoid this issue you can route the needed fictrac signals to the DAQ (see Fictrac Part 3: closed-loop setup) which will then be acquired at a uniform sampling rate and either only use the fictrac channels acquired by the DAQ or at least use these copies to compare to the equivalent signals saved in the .dat file in order to detect instances of timepoint error in the .dat data.

Resampling example :

dat_time = seconds(cumsum(ftD.deltaTimestamp) ./ 1e9); % create timeseries of when each fictrac datapoint was acquired 
ftD.seconds = dat_time; % add dat_time to table
ftD_timetable_temp = table2timetable(ftD,'RowTimes','seconds'); % convert to timetable
ftD_timetable = retime(ftD_timetable_temp,'regular','nearest','SampleRate',fictrac_rate_round); % resample timetable to your fictrac sampling rate filling in instances of dropped frames with the nearest recorded value
% 'nearest' strategy doesn't work as well for v. long breaks but
% looks to do better overall as 'previous' can result in a small
% but sustained offset
ftD = timetable2table(ftD_timetable); % converts ftd back to table for convenience

Troubleshooting#

Richard Moore’s reddit page: https://www.reddit.com/r/fictrac/

Backward FicTrac Part 1: initial installation Computational neuroanatomy Forward