Tutorial2

From NeuroJetWiki

Contents 1 Levy Lab Tutorial #2 1.1 1.Purpose of This Tutorial 1.2 2.Necessary Knowledge 1.3 3.Background 1.4 4.Goal of This Tutorial 1.5 5.Writing the Trace Conditioning Script 1.6 6.Analyzing Results in MATLAB® 1.7 7.Iteration in NeuroJet 1.8 8.Analyzing Loop Data in MATLAB®

Levy Lab Tutorial #2

By Andrew Howe, Ben Hocking and Aprotim Sanyal

1.Purpose of This Tutorial

This is intended as an introductory tutorial for trace conditioning using the NeuroJet software.

Functions Covered: ^Calc(), @AppendData(), for loop and ^() (a.k.a. ^GetVar())

2.Necessary Knowledge

The tutorial assumes that the reader has successfully completed Tutorial #1.

3.Background

Background : The Levy Lab does a fair amount of work with the trace conditioning paradigm, where a conditioned stimulus of some duration (e.g. a tone) is followed by a trace interval (an uneventful period with no input) which is then followed by an unconditioned stimulus (e.g. a puff of air to the eye.) The subject or participant is supposed to learn to properly time a response (e.g. an eye blink) to avoid the unconditioned stimulus (UCS). Success requires that the brain somehow bridge the time gap between the presentation of the conditioned stimulus and the onset of the unconditioned stimulus in order to respond a little before the offset of the unpleasant unconditioned stimulus. This behavior is evident in neural network simulations and in vivo. The two metrics for success are prediction and recall, where prediction for our purposes means that the neural network starts firing UCS neurons within a suitable time window before the actual UCS and recall for our purposes means that the network remembers the UCS without it actually being present as an input. [see published papers]

4.Goal of This Tutorial

To produce a simple network which demonstrates recall. (Recall is easier to produce than prediction.)

5.Writing the Trace Conditioning Script

Open your favorite text editor and get ready to code! Create a new file called “Tutorial2.nj”

First, we establish the network parameters and print them to the screen as in Tutorial #1:

@SetVar(
	title           TraceTutorial1
	ni              1000
	Activity        0.1
	Con             0.1
	mu              0.05
	theta           0.5
	ResetAct        0.1
	seed            1001
	wStart          0.4
)

@PrintVar(title seed ni mu Activity theta wStart)

• Note : in NeuroJet, any zeros preceding decimal points in parameter values (Such as those in the code above) are optional.

Notice that a few of the parameters are different than they were in Tutorial #1. The settings used in this tutorial approximate the settings used in the paper entitled Activity Affects Trace Conditioning Performance in a Minimal Hippocampal Model by Sullivan & Levy for the CNS2004.

The network does not need the inhibitory K values from Tutorial #1 because it will be running competitively.

The next step is to produce the ‘tone' (CS), ‘trace interval’ and ‘air puff’ (UCS). These form subsequences which we assemble into a larger sequence, not unlike the sequence from Tutorial #1. The difference between the trace conditioning paradigm and the sequence learning paradigm of Tutorial #1 is that the middle section of the sequence contains no input. The procedure for building this sequence appears in the code below. In essence, we create four patterns: one of the tone, one of the trace interval, one of the air puff and one of the air puff blanked. This last pattern is needed for testing the network to examine the extent to which it has learned. When the air puff is omitted, the behavioral response (i.e. eye blink) should still be elicited by the tone.

Now combine the appropriate combinations of these input patterns to produce a training sequence and a testing sequence.

(Note: the symbol # is used in NeuroJet to indicate a comment.)

Code for the sequences:

#This segment produces the sequences
@CreateVar(ActiveNeurons 0)
@SetVar(ActiveNeurons ^Calc(^(Activity)*300) )
@MakeSequence(-name tone  -len 3 -non ^(ActiveNeurons) -ol 0  -st 3)
@MakeSequence(-name traceInterval -len 15 -non 0 -ol 0 -st 22)
@MakeSequence(-name airPuff -len 3 -non ^(ActiveNeurons) -ol 0 -st 3 -Nstart ^Calc(^(ActiveNeurons)+1) )
@MakeSequence(-name noPuff -len 3 -non 0 -ol 0 -st 3)

#This segment combines the sequences
@AppendData(-to training  -type seq  -from 3 tone traceInterval airPuff)
@AppendData(-to testing   -type seq  -from 3 tone traceInterval noPuff)

New Functions:

^Calc() – the calculator function in NeuroJet. Calc allows us to produce more flexible code. Above, it is used to allow for more flexible input sequences. Here, Calc() sets the number of externally stimulated neurons based on the percent of activity in the network (the value of the Activity parameter, as we saw in Tutorial #1). This ensures that more neurons will be activated for input sequences when the network has a higher overall activity level. It is sensible to assume that a more active network would also have more numerous external inputs. In many networks actually used for research here, a variable called 'me' is used instead of using the Calc() method shown above. me is not a default variable in NeuroJet, so it must be created using CreateVar() if it is used.

^() – This is shorthand for ^GetVar(). ^GetVar() allows access into the value of a variable. This appears inside our call to the ^Calc() function because we need to find out what the activity level of the network is, and that is contained within the variable ActiveNeurons.

@AppendData() – AppendData concatenates pieces of data, much like the ‘cat’ command in UNIX/LINUX. In other words, think of it as taking separate paperback books and gluing the back cover of one to the front cover of the next. Voilà! One longer book! In our case, it allows us to extend the sequence length and makes the sequence generator more flexible. The –from option specifies what should be appended, and the number preceding the list of variable names indicates the number of items to append.

(Remember, you can always check “NeuroJet -help” for more information about a function and its parameters.)

The next step in the process is to create the network and put it through its paces:

@CreateNetwork()
@Train(-name training -trials 200 -comp)
@SaveData(-from TrainingBuffer -to Tutorial2Train.fire -pad)

This code creates and trains the network and then saves the firing pattern of last training run to the file Tutorial2Train.fire.

@Test(-time 28 -Pstart 1 -Pend 21 -name testing -comp)
@SaveData(-from TestingBuffer -to Tutorial2Test.fire -pad)

This code tests the network on the experimental sequence and saves the firing pattern of the final training run to Tutorial2Train.fire and saves the test run to the file Tutorial2Test.fire.

As before, to run this network, save the file, and type “NeuroJet Tutorial2.nj” at the command prompt.

6.Analyzing Results in MATLAB®

To examine the network's performance, open MATLAB®, as in Tutorial #1, and load Tutorial2Train.fire and Tutorial2Test.fire. Spy diagram rastorgrams visualize the network firing over time, so now we will create two in order to evaluate the network’s learning.

MATLAB® command script:

load Tutorial2Train.fire;
load Tutorial2Test.fire;
spy(Tutorial2Train(:,1:120)');	% the 1:120 selects the first 120 neurons
				% the ‘ transposes the matrix
title('Training');		% adds a title to the figure
figure; 			% creates a new figure for a new graph

spy(Tutorial2Test(:,1:120)');
title('Testing');

If all has gone well, you should be able to see in the “Training” figure that the first 60 neurons are silent except when excited by the external stimuli. The figure entitled “Testing” looks very similar to the one entitled “Training”, except that the activation pattern for the unconditioned stimulus (air puff) in the testing diagram should look less crisp. Below are two actual spy diagrams:

Image:NeuroJet Tutorial2 html ma02c3e0.png Image:NeuroJet Tutorial2 html bbfb6ee.png

Due to the large number of neurons in our networks, it is very hard to examine the all the firing patterns for similarity like we did in Tutorial #1. However, there is a nice trick which allows us to examine large numbers in an easy to read colored graph. It is called the cosine comparison. MATLAB® code to generate these cosine comparisons:

for row=1:28
	for col=1:21
		sim(row,col)=sum(Tutorial2Train(col,:) ...
				 .* Tutorial2Test(row,:)) ...
				 / sum(Tutorial2Test(row,:));
	end
end
figure;
title('Cosine Comparison: Final Training to Testing');
pcolor(sim);

There should be a dark tint rising left to right diagonally in the diagram. This indicates that the firing patterns are highly similar. The diagram below shows the cosine comparison for the above graphs. The cosine comparison shown indicates that the patterns of firing are indeed quite similar.

Image:NeuroJet Tutorial2 html 772a95d1.png

7.Iteration in NeuroJet

Now we are going to add a simple looping section to your NeuroJet code. Loops work the same way that they usually work within programming, except that the syntax is a bit different for NeuoJet. The code we will add momentarily will allow us to save data about the network as it learns. Below is the code, which should replace the lines starting with @Train and @SaveData. The code below introduces '%()' and ':'.

%(i 1 30 1) {
	@Train(-name training -trials 1 -comp)
	@SaveData(-from TrainingBuffer -to Tutorial2Train:^Num2Int(^(i)):.dat -pad)
}

The '%' is the for loop indicator. It takes a variable (e.g. i), a start value (e.g. 1), an end value (e.g. 30) and a step value, which should never be less than one (e.g. 1). Then it runs exactly like a normal C++ for loop. In the example above, ^(i) accesses the loop index for individually labeling each training trial based on the ‘experience level’ of the network. The ':' which appears near the '^' is a string concatenate function. Now, we will explain the filename structure used in the @SaveData line above -- "Tutorial2Train:^Num2Int(^(i)):.dat". The “Tutorial2Train” part is going to be the first part of the name. The aforementioned prefix is appended via ':' to the label of the trial or ‘experience level’ of the network. Then the extension “.dat” is concatenated after the trial number to indicate that the file contains data.

Now try to run the script! This segment of code will replace the two lines previously training the network and saving the last training trial (they begin with @Train and @SaveData.) If you have coded carefully, 30 different training trials are written to disk.

8.Analyzing Loop Data in MATLAB®

It would be a waste of time to load each state space vector into MATLAB® by hand, so we'll use some code to help out. Some MATLAB® code which accomplishes this auto-loading appears below.

for i=1:30
	load (['Tutorial2Train',int2str(i),'.dat'])
end

The for loop is a simple MATLAB® for loop with the variable ‘i’, which runs from 1 to 30, just like in the NeuroJet code. The load command is where the magic happens. MATLAB®’s concatenate function for this case is a set of comma separated values enclosed in brackets. MATLAB® starts with ‘Tutorial2Train’ and then appends the for loop index value (after conversion to a string) and then appends the .dat file extension. Then load uses this new name to bring in the data. Now you should be able to spy diagram various training trials and see to some degree how the firing changed as the network was trained.

Now we will introduce the ReorderMovie script for MATLAB®. Neurons in our trace conditioning networks usually form firing patterns which cascade in series from the CS to the UCS. Essentially, the network auto-generates a sequence like the sequence learned in Tutorial #1, except that the patterns in the sequence do not appear visually organized in the raw data due to the random connections and large number of neurons. The ReorderMovie takes advantage of the cascading nature of the pattern and reorders all of the neurons below the externally activated neurons from first to fire firing to last to fire based on some reference file. The reference file is held constant for every frame displayed, because each frame has slightly different firing characteristics. Usually, we use the last training trial or a post-learning testing trial as the reference for reordering because the network is usually fairly stable by the end of training. From this newly reordered array, we choose random samples at regular intervals to summarize the data into something rapidly comprehensible. For a successful network, this method produces a movie across the training trials which proceeds from random noise to the beginnings of a cascade sequence to a cascade that extends the length of the trace interval. The example network used here learns very rapidly so the growth of the sequence will be rapid as well. Four frames of a network similar to the one in this tutorial appear on the next page at various stages in learning. An extension of this technique involves performing cosine comparisons on the reordered frames. As one might expect, the cosine starts out very noisy, but over training the graph gets cleaner and the diagonal line grows from the bottom left to the top right. Examples of this appear in documents in the lab.

The code used to accomplish the reordering is difficult to follow or properly modify, so the explanation above will suffice in terms of its mechanics. The current script assumes that the data are in padded arrays, as is done in this file. When the script is run, it asks for a number of pieces of information, which will be explained below, along with a few tips regarding movie generation for the adventurous code tinkerer.

First, the script asks for the prefix to your trials. All files are assumed to be in the current directory. It assumes <your input>:<TrialNumber>:.dat. Next, you input the full file name of the reference file. Then it requests the number of trials and the step value, which is the number of trials to jump for each subsequent frame. This could be useful when testing large numbers of networks because a training trial could be saved every two or more steps instead of every step. This would save I/O time and disk space for the sweep, and specific parameters could be repeated for in depth analysis later based on quick movie views. The next parameter is the index of the first neuron which is not externally activated, since externally activated neurons are not reordered. The final input parameter sets the number of samples which will be used in the reordering. Sample number begins to become important when examining 8,000 neuron networks which, even at low activities, almost fill the figure with externally activated neurons. There are currently some creative work-arounds to this problem, but only people attempting to examine large networks will need to worry about this. Someone in the lab probably has a few scripts to address this issue, so ask around.

Image:NeuroJet Tutorial2 html 1922cf19.png Image:NeuroJet Tutorial2 html m93e3980.png Image:NeuroJet Tutorial2 html 6d836c39.png Image:NeuroJet Tutorial2 html 66b482d.png

Producing movies that reside in files is somewhat difficult. MATLAB® has the capability, and we are working on writing functions to streamline the process. Check with Andrew Howe, or through his files, to find help with movies. If you render the movies in the lab, they will be in a raw format which must be converted to a smaller size. Please see the help concerning a program called transcode to accomplish this. We are working on determining the most portable format.

Now that you know some of the building blocks of NeuroJet code, and how to interpret your data, you can start creating your own code, or go on to another tutorial....