📙 How-to Guide · task-oriented

How-to Guides

Welcome to the DynVision User Guide! This section provides task-oriented guides to help you accomplish specific goals with DynVision.

Contents

Core Functionality

• Installation: Detailed instructions for installing DynVision on different platforms
• Custom Models: How to create your own neural network architectures
• Data Processing: Working with datasets and data loaders
• Temporal Data Presentation: Controlling temporal dynamics and stimulus presentation patterns
• Workflow Management: Using Snakemake for experiment orchestration
• Custom Experiments: Designing new stimulus-presentation experiments and model sweeps
• Training: Training models and generating checkpoints
• Model Testing: Evaluating trained models on test data
• Visualization: Visualizing and analyzing results
• Cluster Integration: Running DynVision on high-performance computing clusters
• Troubleshooting: Common issues and solutions

Planned Guides

The following guides are planned for future releases:

• Model Evaluation: evaluating model performance and biological plausibility
• Hyperparameter Optimization: systematic parameter tuning
• Performance Optimization: making DynVision run faster
• Transfer Learning: using pre-trained models

Common Tasks

Below are step-by-step instructions for common tasks in DynVision:

Training a Model on a Custom Dataset

1. Place your dataset in the data/raw/your_dataset/ directory.
2. Add dataset statistics in config_data.yaml.
3. Run the data preparation workflow:

   snakemake project_paths.data.raw/your_dataset/train
   

4. Convert the dataset to FFCV format:

   snakemake project_paths.data.processed/your_dataset/train_all/train.beton
   

5. Train a model on your dataset:

   snakemake train_model --config \
     model_name=DyRCNNx4 \
     data_name=your_dataset \
     model_args="{rctype:full}"
   

Comparing Different Recurrence Types

1. Set up your experiment in config_experiments.yaml (or use an existing one).
2. Run the experiment with multiple recurrence types:

   snakemake all_experiments --config \
     experiment=contrast \
     model_args="{rctype:[full,self,depthpointwise,pointdepthwise]}"
   

3. Generate comparative visualizations:

   snakemake plot_experiments_on_models
   

4. Analyze the results in the reports/figures/contrast/ directory.

Extracting Neural Responses

1. Test a model and store responses:

   snakemake test_model --config \
     model_name=DyRCNNx4 \
     model_args="{rctype:full}" \
     data_name=cifar100 \
     data_group=invertebrates \
     store_responses=100 \
     data_loader=StimulusDuration \
     data_args="{tsteps:100,stim:15}"
   

2. Load and analyze the responses in Python:

   import torch
   import matplotlib.pyplot as plt
   
   # Load responses
   responses = torch.load('models/DyRCNNx4/DyRCNNx4:rctype=full_0001_cifar100_trained_StimulusDuration:tsteps=100+stim=15_invertebrates_test_responses.pt')
   
   # Extract layer responses
   v1_response = responses['V1'].mean(dim=(0, 2, 3, 4))  # Average over all dimensions except time
   
   # Plot response time course
   plt.figure(figsize=(10, 6))
   plt.plot(v1_response.cpu())
   plt.title('V1 Response Time Course')
   plt.xlabel('Time (timesteps)')
   plt.ylabel('Average Activation')
   plt.savefig('v1_response.png')
   

Creating a Custom Recurrence Type

1. Create a new recurrence class in model_components/recurrence.py:

   class CustomRecurrence(nn.Module):
       def __init__(
           self,
           in_channels,
           kernel_size,
           bias=False,
           max_weight_init=0.05,
           **kwargs
       ):
           super().__init__()
           self.max_weight_init = max_weight_init
   
           # Implement your custom recurrence
           self.conv = nn.Conv2d(
               in_channels=in_channels,
               out_channels=in_channels,
               kernel_size=kernel_size,
               padding=kernel_size//2,
               bias=bias
           )
   
       def forward(self, x):
           return self.conv(x)
   
       def _init_parameters(self):
           nn.init.uniform_(
               self.conv.weight, a=-self.max_weight_init, b=self.max_weight_init
           )
   

2. Add your recurrence type to RecurrentConnectedConv2d in recurrence.py:

   # Inside _define_architecture method
   elif self.recurrence_type == "custom":
       self.recurrence = CustomRecurrence(
           in_channels=self.out_channels,
           **recurrence_params,
       )
   

3. Use your custom recurrence type in a model:

   snakemake experiment --config \
     model_name=DyRCNNx4 \
     model_args="{rctype:custom}" \
     experiment=contrast
   

FAQ

Q: How do I run DynVision without using Snakemake? A: While Snakemake provides the most integrated experience, you can use DynVision components directly in Python scripts. Import models from dynvision.models and use them with PyTorch Lightning's Trainer.

Q: Can I use DynVision with my existing models? A: Yes, you can wrap existing PyTorch models with DynVision's LightningBase class to leverage its training and evaluation infrastructure. See Custom Models for guidance.

Q: How do I tune hyperparameters efficiently? A: DynVision supports parameter sweeps through Snakemake's config system. Define parameter ranges in your config files and use wildcards in workflow rules to run multiple experiments. See Workflow Management for details.

Q: Can I use DynVision without a GPU?
A: Yes, DynVision works on CPU, but training will be significantly slower. Use smaller models and datasets for experimentation on CPU.

Q: How can I contribute to DynVision?
A: See the Contributing Guide for information on how to contribute to the project.

Troubleshooting

For common issues and their solutions, see the Troubleshooting Guide.

If you encounter problems not covered in the documentation, please open an issue on GitHub.