G3 panels along with FicTrac are used in the lab for fly virtual reality experiments. Multiple authors (inc. Mark Frye, Michael Reiser, Michael Dickinson, Stephen Holtz, Chuntao Dan, Sung Soo Kim, John Tuthill) have build a guide for panel use available here. The guide is actually written for tethered, flying flies where wing-beats are analysed rather than walking flies on a ball. The Reiser lab guide can be found online at their satellite site here.

This page provides a digest for our use scenarios. Throughout the document we tend to refer to the tiled LED array as the “arena”, composed of individual “panels”, the image they display as “patterns”, and the black box that gives it all life as the “controller”.

Hardware#

#

Arena

LED flight arena: a modular array of 8x8 dot matrix LED panels. Each panel is independently addressable – i.e. can show a different visual pattern and can display 16 intensity levels (grayscale). In the typical 12 column circular configuration, each LED (pixel) subtends (no more than) ~3.5 degrees at the retina. The arena should be powered-up when the controller power switch is toggled (note that some odd behavior might occur depending on the order of powering equipment up).

Controller#

The 3.0 controller is loaded with controller code v1.3 (black, rig mounted box), and is the interface between the PC and the arena. Both controller and code were developed in the Reiser group.

  • ADC0: analog input in mode 1 and 2 of channel x
    • This input should be in the form of L-R. That is, with a negative gain in modes 1 and 2, a negative signal will cause a decrease in channel frame and a positive signal will cause an increase in channel frame number. 
  • ADC1 : analog input in mode 1 and 2 of channel y
    • This works in the same manner as ADC0
  • ADC2: analog input in mode 3 of channel x
  • ADC3: analog input in mode 3 of channel y
  • DAC0: voltage proportional to current frame number (in the unit of volt) in mode 1, 2, 3, 4, and PC dumping mode of channel x, update analog output in mode 5 (debugging function generator) of channel x;
  • DAC1: voltage proportional to current frame number (in the unit of volt) in mode 1,2,3, 4, and PC dumping mode of channel y, update analog output in mode 5 (debugging function generator) of channel y.
  • DAC2: output from this channel is accessible from Matlab, see the set_ao command
  • DAC3: output from this channel is accessible from Matlab, see the set_ao command
  • Int0: unused;
  • Int1: timing for fetching and displaying each frame when controller works in default mode and PC dumping mode. It is set to high before controller fetches a new frame data and reset to low when the controller finishes sending the data to the arena.
  • Int2: laser trigger (optionally used)
  • Int3: unused

The DAC0, and DAC1 voltages will be values between 0V – 10V. The size of the voltage steps is set by the number of frames in X, Y for the current pattern (e.g. 96 frames in one channel would lead to voltage steps of 10/96 V; frame index 48 would be roughly 5 V). These outputs are consistent and can be used to recover the exact frame position for patterns of at least 500 frames (can only be done approximately for larger patterns). These instantaneous pattern positions are essential for off-line analysis of closed-loop behaviors, and are useful for validating open-loop experimental protocols. 

Phidget#

The phidget device is used to extract variables related to ball movement from FicTrac and send them back to the controller in order to update the LED arena very fast. Find more information on it here.

Software control#

The LED arena is controlled using Matlab. The Wilson lab has forked a copy of the relevant Matlab code, matlab-panels from the Reiser lab. With it, you can launch a GUI, ‘PControl’. You can execute patterns, but the patterns are saved to an SD card (e.g. SanDisk 32 Gb, SDHC UHS-I Card) inserted into the controller, not sent directly from Matlab. The .mat pattern files, containing the display information, is burned onto an SD card from a computer. The SD card is inserted into the Arena Controller, and the contents are sent to the panels using the Controller’s software (using low-level, C code, featuring deterministic timing).

Install#

  • You need to download matlab-panels (using for example GitHub Desktop or SourceTree) and add it to your Matlab path.
  • To do this, on Matlab’s toolstrip, you may find the option “Set Path” which allows to select one directory (or also its subfolders) and save it permanently to Matlab’s “search path”.
  • Now if you type PControlinto the Matlab command line, the GUI should pop up.
  • In order to use the serial port on the controller box you need to install the FDI driver.

Using the PControl GUI#

  • Switch on the arena. The panels will show the socket addresses.
  • Insert pre-programmed flash card containing your patterns into the controller.
  • Switch on the controller. Verify that the left-hand green PWR LED is on steady. Arena will go dark.
  • Start Matlab
  • In the Command Window, type PControl.
  • The serial connection between the PC and the controller, select: Controller->blink LED, verify the 2nd red LED for “Memory status” blinking on the controller.
  • Load a pattern in PControl with the menu Configurations->set pattern ID -> Choose a Pattern from the dropdown menu.
  • First re-zero X and Y Gain and Offset. Set Gain to a non-zero value in the appropriate channel (X or Y), hit Start, and Stop.
  • Set up control of X and Y. If using standard panels, each frame of the display will be a 96 x 32 pixel bitmap (corresponding to the number of individual LED’s around the azimuth and zenith, resp. of the arena). X and Y correspond to the two dimensions of the array of frames, they do NOT necessarily correspond to the coordinates of the display with respect to the fly.
    • Open-loop
      • Once you have loaded a pattern to display in the arena, set the menu options in the X and Y front panel drop down menus to ‘Open Loop: function X’, and ‘Open Loop: function Y’ (the default modes). Hit Start. Play with the Gain and Offset values at will. Can you figure out what X pos and Y pos do? Program a moving pattern by using the ‘functions’ menu to load periodic waveforms. Now manipulate the Gain and Offset controls and hit Start.
    • Open-loop with an external waveform
      • From the ‘patterns’ menu, load the checkerboard pattern (hopefully, it is Pattern Index 1), connect the output of a function generator to ADC2. From the X drop down menu, select ‘Position: ADC2 sets X ind’. Set the X Gain and hit Start. Vary the controls on the function generator, and verify that the pattern moves in register with function generator output. Move the checkerboard pattern horizontally by loading an internal function in the Y-channel and selecting the drop-down menu ‘Open Loop: function X’.
    • Closed-loop operation
      • The ‘closed-loop’ mode is similar to running the pattern in Open Loop with an external waveform. The only real difference is that the external function is a time-varying voltage proportional to the fly’s motion on the ball (i.e. steering torque). To give the fly control over the X pattern (e.g. load a single stripe pattern, which should occupy the X channel), plug Phidget device into ADC0. Set the X-channel drop-down menu to ‘Closed Loop: ADC0 (L-R)’. Set the X Gain to a negative value, and hit Start. Use the X Offset to balance steering asymmetry. If you wish to run the Y pattern in closed-loop, connect L-R Amplitude to ADC1 and set the Y-channel drop-down menu to ‘Closed Loop: ADC1 (L-R)’.

#

Software control detail

The PControl GUI exists as matlab-panels/controller/PControl.mwhich calls matlab-panels/controller/Panel_com.mand uses it to send commands to different serial ports. These are hard-coded as detailed above. Therefore, anything the GUI can do may be executed on the command line (or in a script) with arguments to Panel_com(‘argument’, [value]).

Here are some of the controls you can use:

  • Start – same as pushing the start button on the GUI, controller starts updating frames.
    • Arguments: none
    • Usage: Panel_com(‘start’);
    • Special case of this command for camera trigger, usage: Panel_com(‘start_w_trig’);
  • Stop – stops the controller, pattern will freeze on last displayed frame.
    • Arguments: none
    • Usage: Panel_com(‘stop');
    • Special case of this command for camera trigger, usage: Panel_com(‘stop_w_trig’);
  • All off – sets all LEDs to off.
    • Arguments: none
    • Usage: Panel_com('all_off');
  • All on – sets all LEDs to on.
    • Arguments: none
    • Usage: Panel_com('all_on');
  • Set grey level – sets all LEDs to a greyscale level from 0 to 7:
    • Arguments: none
    • Usage: 
      • Panel_com('g_level_0');   % set all panels to grey level 0;
      • Panel_com( 'g_level_7');  % set all panels to grey level 7;
  • Set pattern ID – sets the ID of the pattern
    • Arguments: a single value from corresponding to the pattern number (0 - # pats)
    • Usage: Panel_com('set_pattern_id', 3);   % set to pattern 3;
    • Note: if ID used is too large – controller will crash - Panel_com doesn’t check this
  • Set controller mode – sets the mode for the controller’s X and Y channels
    • Arguments: 2 values to set the mode for X and Y channels. 0 – open loop, 1 – closed loop, 2 – both, closed loop plus function as bias, 3 – External input sets position, 4 – Internal function generator sets velocity/position, 5 – internal function generator debug mode.
    • Usage: Panel_com(‘set_mode’, [0 1]); % X to open loop, Y to closed loop.
  • Set Pattern position – sets the position of the pattern. Controller will send this frame to panels.
    • Arguments: 2 values to set the X and Y positions, must be between 0 and num_x or num_y.
    • Usage: Panel_com('set_position', [10 1]);
    • Note:Panel_comsubtracts 1 from each of these values, because in Matlab, 1 is used as the start index, and the controller uses 0. Also, if the position value is too large – frame will not be correct - Panel_com doesn’t check this.
  • Set gain and bias – sets the gain and bias for the X and Y channels on the controller.
    • Arguments: 4 values, 1 each to set gain_x, bias_x, gain_y, bias_y. Values must be signed integers between -127 and +127. Because of this – gain values are multiplied by 10 and bias voltages are multiplied by 20.
    • Usage: Panel_com('send_gain_bias', [10 -10 0 20]); % sets gain_x = 1X, bias_x = -0.5V, gain_y = 0, bias_y = 1 V (check PControl to verify this).
    • To calculate the frames per second, and avoid the confusion above just use: FPS = gain + 2.5*bias;
  • Laser on/off – enables/disables the laser trigger. When this is on, it enables the controller outputs a trigger on DIO 1 and 2 (of opposite activation – one is high, the other is low), when the pattern is in a certain position, that is currently hard-coded in the controller.
    • Panel_com('laser_on'); OR Panel_com('laser_off');
  • Identity compression on/off – enables/disables a compression scheme on the controller.
    • Panel_com('ident_compress_on'); OR Panel_com('ident_compress_off');
  • Set trigger rate – sets the rate (frequency) of the optional camera trigger.
    • Arguments: 1 value, that must be an integer between 0 and 255.
    • Usage: Panel_com('set_trigger_rate', [100]); % sets the trigger to 100 fps.
  • Send laser pattern – Use the laser switch with a user set pattern.
    • Arguments:  Panel_com('send_laser_pattern',pattern);
    • Pattern, a numerical array with 96 elements whose value is either 0 or 1.
    • Usage: Panel_com('send_laser_pattern'[ones(1,24),zeros(1,24),ones(1,24),zeros(1,24)]);

Software control debug#

If things are not working as expected these commands may help you debug your issue.

#

Pattern, function and configuration usage

See the patterns guide for detail on how to generate patterns for your arena using Matlab. In order to use patterns you need to load then onto an SD card with help from matlab-panels. When writing patterns, position functions, and configurations to the SD card usingchoose_pats, choose_functions, and choose_configs (or PControl) a local copy is made of each file name. To be able to ‘automatically’ access the patterns on an SD card from the PControl GUI, you must be using the SD on the same machine on which it was programmed. If you need to load patterns from an SD card programmed onto another machine (or if you see that patterns and/or functions copied onto the card behave strangely) you should synchronize the SD information from the card to your PC by clicking on the controller > sync SD info menu items in the PControl GUI. In order to access these pattern names on another computer, you have to place the SD card into the other computer’s arena controller and select Sync SD info from PControl. It will ask you to reset PControl (exit and open again). Now all of the pattern/function/configuration information is also locally stored.

  • Once you have a collection of patterns that you wish to display in the arena, put them all into C:\matlabroot\patterns\. You may in fact put them any place you wish, but this is the default directory.
  • Plug a SD Flash card into a port, using a USB card reader. Verify that it appears as a drive letter (usually E).
  • In Matlab, run PControl, and select configurations > load pattern to SD.
  • In the Pattern Selection window, press Add Folder, navigate and select the folder storing your pattern files. You may also individually add the patterns, if this becomes time consuming to achieve the correct order of patterns, consider naming each pattern as “Pattern_##_id” where ## is the number you wish the pattern to be and id is an identifying name. This will ensure the patterns in the folder selected are added in the correct order.
  • Hit “OK”.
  • In the Pattern Selection Tool window, verify the patterns you want to store.
  • Press “Make Image”, then  “Burn”. If you are prompted for the drive letter at any time, type it into the prompt and press OK.
  • DO NOT take out the SD card until you see a message confirming that the SD card writing is finished.
  • Plug the SD card into the arena controller. You may need to turn the controlled off and on for it to recognize the card correctly.
  • To test the pattern, open up PControl and select configurations > set pattern ID > your pattern (from the dropdown menu).  Now you can use the PControl GUI to run your pattern by selecting values and pressing the start button.

The process for storing functions onto the SD card is similar, except you use configurations > load functions to SD. It is best to program all patterns and functions you will need for your experiments onto the SD card and then move it over to the controller. The process for storing arena configurations onto the SD card is similar, except you use configurations > load config to SD.

Debugging#

Note: as of 5/12/2021 (MC), loading patterns/functions onto the SD card only works with Matlab 2019 or older likely due to changes made in versions 2020+ to the bin2dec, hex2dec, bin2bin, dec2hex functions. We have not figured out a work around for this issue yet.

That said, playing patterns/functions appears to work fine with newer versions of Matlab.

Backward G3 Arenas Part 3: pattern and function guide Dissection guide Forward