G3 Arenas Part 3: pattern and function guide • Wilson Lab Wiki

Patterns are created in MATLAB using matlab-panels and saved as .mat files locally. They are then burned to an SD card using matlab-panels, and this card is inserted into the controller box. Patterns can then be deployed to the LED panels of the G3 arena very fast and their position updated by reading variables from FicTrac, using low level C code on the controller box. The Reiser lab guide can be found here, code was written mainly by Reiser lab members. Multiple authors (inc. Mark Frye, Michael Reiser, Michael Dickinson, Stephen Holtz, Chuntao Dan, Sung Soo Kim, John Tuthill) have build a guide for panel use available here. The following is a slightly simplified version.

Your panel setup#

It is important to keep several concepts in mind. The number of unique panels required for a display may vary. Each 8x8 pixel panel is given an identity, which is set using the procedure described below. When you power up the arena, these IDs are displayed on each panel. The identities may or may not be unique. The most flexible experimental setup will require unique panel IDs. It is important to recognize that the way you create a pattern will depend upon the number of unique panels in your arena and their spatial distribution.

Building a new pattern#

The critical step in creating a pattern file in Matlab is using the make_pattern_vector command, which operates on a ‘pattern’ structure. The pattern structure has the following critical fields that arerequired before the structure may be saved in a .mat file. Here is an example of a pattaern made for Jenny’s G3 arena, the comments describe what’s happening:

%% MATLAB code for creating a G3 LED pattern
% This example code creates a single two-pixel bar.
% It assumes that you have 2 rows of 12 8x8 dot LEDs.
% It just uses the Y channel (the 96 pixel columns)
% It does not use pixels within the same column differently

%% 12 panel pattern paramters
pattern.x_num = 96; % the number of frames in the ‘x’ channel 
pattern.y_num = 1; % the number of frames in the ‘y’ channel. We are not
% really , so it is set to 2 for the two rows of panels.using the Y channel so it is set to 1.
pattern.num_panels = 24; % number of unique panel IDs required 
pattern.gs_val = 1; % grey scale value; must be either 1, 2, 3, or 4. 
% ‘1’ indicates all pixel values in pattern.Pats are binary (0 or 1)
% ‘2’ indicates all pixel values in pattern.Pats are either 0, 1, 2, or 3. 
% ‘3’ indicates all pixel values in pattern.Pats fall in the range of 0-7.
% ‘4’ indicates all pixel values in pattern.Pats fall in the range of 0-15.
pattern.row_compression = 1;
% 0 or 1. This sets the simplest available compression mode. 
% When set to 0, you produce a pattern as normal, with 8x8 pixels per panel. 
% With row compression enabled (=1), the patterns are 1/8 the size, 
% and are designed as 1x8 pixels per panel. These patterns are at least 5X faster. 

%% Define what you panel array looks like
% Pattern.Panel_map is a vector or matrix that encodes the orientation of the panel IDs.
% The number of unique panels required for a display may vary. Each has a
% unique ID. Thisis the panel pattern on Jenny's G3 setup (the 21st panel is actually missing):
% The column, 23, 11, is actually in front of the fly, which is column six.
pattern.Panel_map = [16 20 24 15 19 23 14 18 22 13 17 21 ; ...
    04 08 12 03 07 11 02 06 10 01 05 09];

%% This is the actual pattern matrix. 
% Pats is a matrix of size (L,M,N,O), where:
% --- L is the total number of pixel rows
% --- M is the number of pixel columns
% --- N is the number of frames in the ‘x’ direction, same as pattern.x_num
% --- O is the number of frames in the ‘y’ direction, same as pattern.y_num
Pats = zeros(2, pattern.x_num);
% Because we are using row compression the pattern size if just 2 'pixels' x
% 96 pixels. We only need to specify one row of pixels per panel
Pats(:,44:45) = 1; % Add a 2-pixel bar in front of the fly
% each entry in this LxMxNxO matrix is either one bit (for pattern.gs_val =1), 
% 2 bits (for pattern.gs_val =2), 3 bits (for pattern.gs_val =3), or 4 bits (for pattern.gs_val =4). 
% For example, for gs_val = 3, each pixel value can be set as 0-7, where 0 is OFF, 7 is maximal ON, 
% and the intermediate values linearly scale this intensity. 
% if a simple azimuthal shift is required (as in a yaw control experiment), 
% the function ‘ShiftMatrix.m’ can be used to create shifted maps required to depict the moving image.

%% Create full set of possible pattern iteratons
% fill in the remaining X positions by shifting each subsequent frame to
% the right by 1 pixel. 
for j = 2:96
    Pats(:,:,j,1) = ShiftMatrix(Pats(:,:,j-1,1), 1, 'r', 'y'); 
end

%% Map your pattern onto your arena
pattern.Pats = Pats;
pattern.BitMapIndex = process_panel_map(pattern);
pattern.data = Make_pattern_vector(pattern);

%% Save your pattern
directory_name = pwd();
str = [directory_name '\Pattern_1_stripe_2_width_360'];
save(str, 'pattern');

Some additional sample patterns:

📎 make_g3_pattern_verticalbar.m

📎 make_g3_pattern_verticalgrating.m

📎 make_g3_pattern_box.m

Speeding up your pattern#

The faster your arena can update with respect to the fly in closed-loop, the more faithful your virtual reality. The flicker fusion rate of the fly eye is above 200 Hz, meaning that video at a lower refresh rate than this will appear striated. In closed-loop, given the time it takes to extract variables from FicTrac, we should be aiming for about 60 Hz. In open-loop you can go much faster, e.g. 250 Hz.

If trying to push a complex pattern fast, the controller will spend more time refreshing an individual frame, and thus to keep up with the expected display rate (either set in open-loop by the function generator, or in closed-loop by the fly), the controller will drop frames.

You can benchmark your pattern (Panel_com('bench_pattern')). To get it to go faster:

Simplify the pattern. Either don’t use grayscale, reduce the number of panels used, or update the arena configuration to disable a subset of the panels in your arena (e.g. you can mask out the top and bottom row and just drive your pattern in the central 2 rows along the equator of the flies’ eye).
Use row compression. A large number of patterns we might care to try consist of identical pattern data for all rows of the pattern. When this is the case, a row-compressed pattern will just consist of one row instead of 8. This results in a speedup factor of at least 5. To enable row compression, simply make a pattern with only one row per panel, and include the setting (pattern.row_compression = 1;) in the pattern making script. See example above.
Use ‘identity’ compression. The general method outlined for making and sending patterns is simple (elegant, perhaps) but should strike no one as optimal. Many patterns contain large swaths of pixels that are simply on or off. One simple shortcut that has been implemented is to simply send the row-compressed version of the panel data for a panel that corresponds to a pattern piece that is all one value (works for grayscale too!). This feature is not used at pattern making time, but rather while the pattern is running. Identity compression can be enabled by invoking: Panel_com('ident_compress_on') from MATLAB. This methods speeds up the pattern containing a single stripe, but consider a striped drum pattern—all of the extra comparisons needed to determine if a particular panel’s piece of the pattern can be sent as a single row, will slow things down considerably and never find a compressible pattern patch.

Building a new position function#

Functions can be used to control either the x or y dimension of the panel.

Note: There is a hard cap on how long your function can be. If you notice that your function is not playing all the way through and is instead getting cut off, only to restart from the beginning, it could be that your function is too long. Try dropping the function frequency below 50Hz (can go as low as 5Hz, generally higher is better, but it is an option).

Some sample functions:

📎 make_g3_function_wiggle.m

📎 make_g3_function_dirsweep.m

Building a MATLAB script#

There are many ways to run the panels/functions via a MATLAB script. Generally, you will need to set the panel parameters (such as the pattern/function IDs, starting position, frequency, etc) and then “start” the panels.

    %set panel parameters
    Panel_com('set_pattern_id', xpatt_id); %load pattern
    pause(0.03)
    Panel_com('set_mode',[4 4]); %set xpos mode to fx, ypos mode to fx
    pause(0.03)
    Panel_com('set_position', [69 1]); %set starting x and y positions
    pause(0.03)
    Panel_com('send_gain_bias', [0 0 0 0]); %send gain/bias values
    pause(0.03)
    
    Panel_com('set_posfunc_id',[1 xfunc_id]); %set xpos fx
    pause(0.03)
    Panel_com('set_posfunc_id',[2 yfunc_id]); %set ypos fx
    pause(0.03)
    Panel_com('set_funcx_freq' , fxFreq); %set xpos fx frequency
    pause(0.03)
    Panel_com('set_funcy_freq' , fxFreq); %set ypos fx frequency
    pause(0.03)

    %run experiment
    Panel_com('start');% begin pattern motion
    Panel_com('stop');% end pattern motion
    Panel_com('all_off'); %turn panels off

Note: to ensure the pattern/functions load properly, it can be helpful to add brief (~0.03sec) pauses between Panel_com commands. MATLAB has a function for this: pause(0.03)