How to Transfer Lights-On Marks in Behavior Logs

These instructions were written by Christopher Skalnik while working under Beau Alward in the Fernald Lab at Stanford University.

Overview

Let’s say that we have these types of behavior logs:

Full Scoring Logs: Records every behavior observed over the 30-minute period we analyze. These behaviors are the ones we will measure and analyze. These logs come from the morning and afternoon, and they have names like:
- Afternoon: 050118_OD1030618_TA23_Dyad_Afternoon.wmv_CS.txt
- Morning: 050118_OD1030618_TA23_Dyad_Morning.wmv_CS.txt
We sometimes call these videos “trims,” “trimmed,” or “spliced.”
First Aggression Logs: When we are creating the 30-minute video for the full scoring log, we don’t start the morning video until the fish have started behaving.¹ These logs are the ones we create when scoring for that first behavior. They have names like log052318_OD1030618_OC4C_Dyad_#.avi_CS.txt where # can be either a 1, for the first video recorded of that subject that day, or a 2 for the second video. These logs do not necessarily cover the full video because we usually stop scoring once we see and score that first behavior. The times in these logs are the ones we use when making the full scoring logs. These logs are for the un-trimmed videos.
Lights On Logs: We also score the un-trimmed videos for the lights in the recording coming on. This is because we often start the recordings while the lights in the fish facility are still off, but we want to measure the latency until various behaviors from when the lights come on. These logs record the time that the light came on as a LIGHTS ON behavior and have names like log052318_OD1030618_OC4C_Dyad_#.avi_CS_LIGHTSON.txt, where again # can be either a 1 or a 2 to signify the first or second video. The 2 is possible because sometimes the first video is so short that it is dark throughout the video.

We want to use behaviorcode or behaviorcodeAuto to statistically analyze the behavior and latency data, but these scripts don’t know how to handle lights-on information being in a separate log file that starts at a different time than the full scoring log. Therefore, we need to add a LIGHTS ON mark (in the MARKS section) to the full scoring log with a negative frame number and timestamp to accurately reflect when the lights came on relative to the behaviors in the full scoring log. We can use scorevideo_lib to do this!

Setup

Download scorevideo_lib from here. Make sure to note which release you use! If you prefer, you can also use git to clone the repository like this:
```
$ git clone https://github.com/U8NWXD/scorevideo_lib.git
$ cd scorevideo_lib
$ git checkout v0.4.1-beta
```
Note that you probably want to use the most recent release by changing v0.4.1-beta to the release you want. You should use a release and not just the master branch because master may represent work in progress that is not yet ready for use.

For the rest of this guide, I will start my paths with scorevideo_lib to refer to the root folder of the repository.
Install PyCharm. While this is not normally necessary just to run programs, PyCharm correctly tells Python how to find and import files. However, this step may not be needed later if I change the scripts to fix this themselves.
Import the copy of scorevideo_lib you downloaded into PyCharm, which should be an option from the welcome screen.
Create a virtual environment by opening the PyCharm preferences, selecting Project Interpreter under Project in the left sidebar, choosing to add a virtual environment, and finally choosing to create a new virtual environment.
Activate the virtual environment by opening a new terminal from the bottom of your PyCharm window. Then install dependencies by running
```
$ pip install -r requirements.txt
```
Move the file at scorevideo_lib/fm_behaviors.txt to scorevideo_lib/scorevideo_lib/fm_behaviors.txt. This comes from another quirk of how Python searches for and loads files, and this step may not be needed later if I fix this bug.
Create a folder at scorevideo_lib/scorevideo_lib/work/. This will be your work directory (not your working directory).
Into your work directory, copy the following:
- Full scoring logs, which should be unblinded if you blinded them.
- First aggression logs
- Lights on logs
Note that any leading log prefixing the log filenames is ignored. Also note that you should only include relevant files. Any extra files, such as first aggression logs that don’t have associated full scoring logs, will trigger errors. Similarly, all relevant files must be included. In the next step you will specify which files precisely are required, but any missing files will also trigger errors.
In scorevideo_lib/scorevideo_lib/transfer_lights_on_marks.py, find the code that looks like this:
```
# Specify regular expressions that identify logs required for every partition
PART_REQUIRED = [ExpectedFile(["_Morning."], ["_LIGHTSON.txt"]),
                 ExpectedFile(["_1."], ["_LIGHTSON.txt"])]
# Specify regular expressions that identify logs optional for every partition
PART_OPTIONAL = [ExpectedFile(["_2."], ["_LIGHTSON.txt"]),
                 ExpectedFile(["_LIGHTSON.txt"])]
# Any files in partitions not matching any of the above throw errors
```
These lines specify which files are required. To explain this, we first need some background on how scorevideo_lib matches up log files. It uses a function get_partitions in the transfer_lights_on_marks.py file to split the log file names into equivalence partitions where every partition is composed of log files whose names are “equal,” where “equal” means that the logs are for the same fish on the same day. This equality is determined by the same_fish_and_day function in the transfer_lights_on_marks.py script.

Once the partitions have been created, scorevideo_lib validates them using the PART_REQUIRED and PART_OPTIONAL constants. PART_REQUIRED is a list ExpectedFile objects, each of which describes a file. You can see the documentation here. In short, the first argument to ExpectedFile is a list of strings that are expected to present in the file name, and the second argument is a list of strings that are expected to not be present in the file name. Any file name that matches both criteria “matches” the ExpectedFile object. scorevideo_lib checks that each partition has exactly one file that matches each ExpectedFile in PART_REQUIRED and no files that match no ExpectedFile objects in the union of PART_REQUIRED and PART_OPTIONAL. In other words, the files described in PART_REQUIRED are mandatory, while the files described in PART_OPTIONAL are allowable but not required.

Note that the script also uses a name_filter function to filter out files that don’t look like logs and afternoon full scoring logs.

In my case, I used
```
# Specify regular expressions that identify logs required for every partition
PART_REQUIRED = [ExpectedFile(["_Morning."], ["_LIGHTSON.txt"]),
                 ExpectedFile(["_1."], ["_LIGHTSON.txt"]),
                 ExpectedFile(["_LIGHTSON.txt"])]
# Specify regular expressions that identify logs optional for every partition
PART_OPTIONAL = [ExpectedFile(["_2."], ["_LIGHTSON.txt"]),
                 ]
# Any files in partitions not matching any of the above throw errors
```
This requires a morning full scoring log, a first aggression log for video 1, and a lights on log. A first aggression log for video 2 is optional.

Run scorevideo_lib

You are now ready to transfer the marks. Within PyCharm, run the scorevideo_lib/scorevideo_lib/transfer_lights_on_marks.py script by clicking the green triangle in the upper right of the window. If this fails, you might need to set the working directory to scorevideo_lib/scorevideo_lib in PyCharm by choosing to Edit Configurations from the drop-down menu to the left of the green triangle.

When scorevideo_lib finishes, you should now see LIGHTS ON marks with negative frame numbers and timestamps in the full scoring logs. Now you’re ready to analyze them!

The behaviors that count as the first behavior are stored in fm_behaviors.txt in the scorevideo_lib repository. ↩