📗 Explanation · understanding-oriented

The Role of Recurrence in Visual Processing¶

Understanding-oriented

This page discusses why recurrent connections matter for modeling the ventral visual stream. For the concrete connection types DynVision implements, see the Recurrence Types reference.

Feedforward models are not enough¶

Standard convolutional networks are purely feedforward, yet the primate visual system is densely recurrent. Lateral and feedback connections are thought to support contextual modulation, figure–ground segregation, predictive processing, and robust recognition under challenging conditions (occlusion, noise, low contrast).

Empirically, recurrence has been shown to be necessary to capture the representational dynamics of the human visual system (Kietzmann et al., 2019) and to explain behavior on harder recognition tasks where feedforward models fall short (Kar et al., 2019).

What recurrence buys a model¶

DynVision lets researchers study several distinct functional contributions of recurrence:

Temporal normalization — recurrent inhibition can reproduce adaptation, sublinear temporal summation, and contrast-dependent response timing without explicit divisive-normalization operators.
Noise robustness — recurrent dynamics can stabilize representations under input perturbations, approaching human-level robustness in some regimes.
Iterative inference — recurrence allows the network to refine its estimate over biological time, rather than committing to a single feedforward pass.

A key empirical finding from DynVision is that these roles can dissociate: different architectural placements of recurrence and different metabolic-cost weightings push the network into qualitatively different dynamic regimes.

Connections in DynVision¶

Connection type	Biological analogue
Self / full lateral recurrence	Horizontal connections within a cortical area
Feedback	Top-down projections from higher to lower areas
Skip	Long-range projections between non-adjacent areas

Weight Distribution Insights¶

Analysis of learned weights reveals systematic patterns across recurrence types that explain the two functional regimes:

Weight distributions by recurrence type

Figure: Recurrent weight distributions for models trained with different connection types. Self‑recurrence produces strongly negative (inhibitory) weights; local recurrence produces a mix of negative values; full and sparser types remain near zero. Feedforward weight distributions (not shown) are consistent across all variants.

Weight distributions by activity loss weight

Figure: Feedforward and recurrent weight distributions across activity‑loss weighting. Recurrent weights remain consistently small and negative across all regularisation strengths, while feedforward weights are unaffected.

Weight distributions with alternating presentation pattern

Figure: Same weight analysis but with the 1011 alternating presentation pattern, confirming that the inhibitory‑weight regime is robust to the temporal structure of input presentation.

Feedback weight distributions

Figure: Feedback weight distributions across different feedback‑connection configurations. Multiplicative feedback shows marginally different distributions compared to additive, but no systematic noise‑robustness benefit across seeds.

Weight distributions by recurrence target

Figure: Recurrent weight distributions for models trained with different recurrence targets (input, middle, output). Middle‑target training produces qualitatively different weight structures consistent with the noise‑robustness regime.

Response Profiles¶

DynVision can capture detailed temporal response profiles that reveal how different architectural choices shape neural dynamics:

Response profiles across recurrence configs and seeds

Figure: Per‑layer response trajectories across recurrence types, recurrence targets, and random seeds. The three‑panel layout (type × target × seed) exposes consistent response‑shaping effects of recurrence independent of initialization variability.

Response profiles with skip and feedback connections

Figure: Response trajectories for models with skip and feedback connections enabled. The additional connectivity pathways produce richer temporal dynamics that propagate earlier and later than in purely feedforward networks.

Performance and Stability¶

Performance with feedback connections

Figure: Accuracy across model variants with different feedback configurations. Feedback connections provide no systematic noise‑robustness benefit across seeds; multiplicative integration is only marginally preferable to additive.

Performance by recurrence type, middle-target

Figure: Accuracy comparison across recurrence types when recurrence targets the middle layer computation. Full recurrence consistently enables the strongest noise‑robustness effect.

Training stability by recurrence target

Figure: Training stability analysis by recurrence target. The middle‑target configuration shows the most stable convergence across seeds while producing the strongest noise‑robustness gains.