Visualization Options#

In NeuroMiner, “visualization” refers to generating and inspecting the model weights, which are often represented as brain maps. This configuration menu allows you to set parameters for two distinct processes: weight vector normalization and statistical inference via permutation analysis.

1. Weight Vector Normalization#

This option allows you to normalize the weight vectors before they are visualized.

  • Menu Option: Normalize weight vectors

  • Availability: This toggle only appears if you are using an LIBSVM or LIBLIN SVM program, as specified in your training parameters.

  • Function: Toggles whether normalization is applied (Yes/No). The default is No.

2. Permutation Analysis for Statistical Inference#

The magnitude of feature weights does not directly inform you of their statistical significance (i.e., how likely they are to replicate). To assess this, you can perform a permutation analysis to create an empirical null distribution of weights for each feature. The observed weight is then compared to this distribution to derive Z-scores and P-values.

This method involves permuting outcome labels, features, or covariates. For each permutation, the models are retrained, and a null distribution of performance (e.g., BAC) and feature weights is constructed. The significance of the observed results is then calculated by comparing them to this null distribution.

The configuration menu presents two primary modes for this analysis, depending on whether dimensionality reduction (e.g., PCA) is used in your preprocessing pipeline.

Path A: Aggregated Mode (Input-Space Test)#

This is the standard, “classic” permutation test performed directly on the feature weights in the original input space.

  • Menu Option: Perform permutation test in input space

  • Function: This toggles the standard permutation test on or off.

  • Use Case: This method is appropriate when no dimensionality reduction is applied.

Important: Using this aggregated mode when dimensionality reduction (like PCA) is part of your pipeline is not recommended, as it can lead to overly conservative and inaccurate p-values. For such cases, use Path B.

Path B: Component-Wise Mode (Model-Space Test)#

This hierarchical, selective-inference mode is designed specifically for pipelines that include dimensionality reduction.

  • Menu Option: Perform model permutation test in model space and back-project only signif. components

  • Availability: This option only appears if dimensionality reduction (e.g., PCA, PLS) is detected in your preprocessing steps.

  • Function: This test operates in two stages:

    1. Model-Space Test: A permutation test is performed on the components in the reduced-dimension “model space” to see which components significantly contribute to the model.

    2. Selective Back-Projection: Only the components that pass a statistical threshold are used to “back-project” the weights into the original “input space.” This avoids cross-component cancellation and is often more sensitive.

When this mode is enabled, you will see the following sub-options:

  • Define back-projection significance threshold: Sets the p-value (e.g., 0.05) used to determine if a component is significant enough to be back-projected.

  • Correct component P values for multiple comparisons using FDR: Toggles whether to apply a False Discovery Rate correction across the components before selection.

  • Perform additional permutation test… in input-space: After the significant components are selected and back-projected, you can optionally run a second, standard input-space permutation test on this new, sparser map.

3. Common Permutation Settings#

When either Path A or Path B is enabled, you must define the parameters for the permutation analysis itself:

  • Define no. of permutations: Sets the number of permutation repetitions to build the null distribution. A higher number is more robust but computationally expensive (e.g., 5000).

  • Define permutation mode: Specifies what to permute, as described by Golland & Fischl (2003):

    • Labels: (Default) Permutes the outcome labels (e.g., patient/control).

    • Features (within-label): Permutes features within each class, preserving class structure.

    • Labels & Features: Permutes both.

    • Covariate(s): This is an EXPERT option that only appears if covariates are defined. It tests the significance of covariate correction methods (e.g., site-effect removal). For each permutation, the preprocessing is re-run with permuted covariate values before models are retrained. If selected, you will be prompted to choose which covariates to permute.

Note

Visualization Pathways: Component‑wise vs. Aggregated Back‑projection

This page explains how NeuroMiner 1.4 projects model weights back to feature space when a model is trained in a latent (component) space (e.g., PCA/ICA/NMF), and how inference differs between aggregated and component‑wise modes. It also documents permutation‑based testing and how components are aligned across CV1 folds. (Adapted from Koutsouleris, 2025.)

1) Overview

In factorization‑based pipelines (PCA, ICA, NMF, …), NeuroMiner trains the predictor on component scores \(z_c = X v_c\) with weights \(w_c\). Two display modes are available:

  • Aggregated: a single back‑projected map $\( w_{\mathrm{agg}}=\sum_{c=1}^{C} w_c\, v_c \)$

  • Component‑wise: a set of maps, one per selected component $\( w^{(\mathrm{backproj})}_c = w_c\, v_c \quad\text{for selected } c \)$

Aggregated maps give a compact “overall” picture; component‑wise maps preserve factor structure and typically offer higher statistical power.

2) Geometry & implications

With aggregation, cross‑component covariance can attenuate apparent signal (cancellation): $\( \mathrm{Var}(w_{\mathrm{agg}})=\sum_c w_c^2 \mathrm{Var}(v_c) + 2\!\!\sum_{1\le c<d\le C} \! w_c w_d\, \mathrm{Cov}(v_c, v_d). \)$

This is why feature‑level permutation tests on \(w_{\mathrm{agg}}\) often show fewer significant features than the factor‑resolved tests on \(w^{(\mathrm{backproj})}_c\).

At a glance

Aspect

Component‑wise

Aggregated

Preserves latent structure

✅

❌

Statistical power

Higher (factor‑specific)

Lower (cancellation)

Interpretability

Mechanistic, factor‑resolved

Summary‑level

Best used for

Explaining how prediction arises

Showing where overall signal lies

3) Statistical hierarchy (component‑wise mode)

  1. Component selection in model space: test whether each component contributes significantly; control FDR across components → discovery set \(\mathcal{C}_{\mathrm{sig}}\).

  2. Feature‑level inference within each selected component: run feature‑wise tests on \(w^{(\mathrm{backproj})}_c\) for \(c\in\mathcal{C}_{\mathrm{sig}}\); control FDR per component.

This is a hierarchical selective‑inference design: the feature‑level family is restricted to the already‑discovered components; no extra correction across components at the feature level.

4) Permutation inference: two modes

Let \(P\) be the set of label permutations.

4.1 Aggregated mode

Test a composite field against its permuted counterparts: $\( T_{\mathrm{agg}}=\phi(w_{\mathrm{agg}})\ \text{vs.}\ \bigl\{\phi(w^{(p)}_{\mathrm{agg}}): p\in P\bigr\}, \)\( with a feature‑wise statistic \)\phi(\cdot)\( (e.g., \)|w|$ or a Z‑score). Because factors are mixed, null variance inflates and sensitivity drops.

4.2 Component‑wise mode with selection under the null

  1. Compute a component score \(s_c\) (e.g., \(|w_c|\)). For each \(p\in P\), compute \(S^{(p)}_c\).

  2. Select components via FDR in the observed data → \(\mathcal{C}_{\mathrm{sig}}\).

  3. Mirror selection under the null per permutation using one of:

  • Rank‑matched top‑K: let \(K=|\mathcal{C}_{\mathrm{sig}}|\), select the top‑\(K\) \(S^{(p)}_c\).

  • Threshold‑matched: determine observed cutoff \(t_{\mathrm{obs}}\); select all \(c\) with \(S^{(p)}_c\ge t_{\mathrm{obs}}\) (trim/pad to \(K\) if needed).

  1. For feature \(f\) in selected slot \(c\), compute empirical p: $\( \hat p_{f,c} = \frac{E_{f,c}+1}{|P_f|+1},\quad E_{f,c}=\sum_{p\in P_f}\mathbf{1}\!\left(\bigl|w^{(p)}_{f,c}\bigr|\ge \bigl|w_{f,c}\bigr|\right). \)$ Apply FDR within each component. When using Z‑scores, compute permutation mean/variance with valid counts to handle missingness.

5) Multi‑modal shares and fusion

For early fusion, decompose the concatenated feature matrix; for intermediate fusion, decompose per modality and align components across modalities first.

  • Aggregated (modality‑level) share over components \(\mathcal{C}\): $\( L^{(m)}_p=\frac{\sum_{c\in\mathcal{C}}\bigl\|w^{(m)}_c\bigr\|_p}{\sum_{m'}\sum_{c\in\mathcal{C}}\bigl\|w^{(m')}_c\bigr\|_p} \)$

  • Component×modality share (factor‑resolved), for \(c\in\mathcal{C}_{\mathrm{sig}}\): $\( L^{(m)}_{p,c}=\frac{\bigl\|w^{(m)}_c\bigr\|_p}{\sum_{m'}\bigl\|w^{(m')}_c\bigr\|_p}. \)$

Use \(\mathcal{C}=\) all components in aggregated mode; use \(\mathcal{C}_{\mathrm{sig}}\) in component‑wise mode.

6) Alignment across CV1 folds (component matching & sign correction)

NM 1.4 aligns factorized maps so that the j‑th column refers to the same latent component across resamples:

  • Prune zero/non‑finite columns and cache a reference per outer fold.

  • Compute a similarity matrix (default: Pearson correlation).

  • Solve a maximum‑weight assignment (Hungarian / linear‑sum) with a similarity threshold; unmatched slots are left NaN.

  • Sign‑correct matched columns so orientations are consistent.

  • Propagate the alignment to p‑values, correlations, magnitudes, and multi‑modal energy summaries (component \(L_2\) norms).

See also: Component matching across folds below for an operational summary and pointers to controls.

Practical guidance

  • Prefer component‑wise mode when interpretability and sensitivity are critical (e.g., explaining predictive factors).

  • Use aggregated maps when you need a compact overview or for quick QC.

  • In multi‑modal settings, report both modality‑level and factor‑resolved shares.