effective_rank

effective_rank(matrix: torch.Tensor) -> float

Effective rank of a matrix, the entropy-based measure of Roy & Vetterli (2007).

Normalise the singular values to a distribution and take exp of its entropy. A matrix whose variance is spread evenly over k directions scores k; one dominated by a single direction scores close to 1, however large its nominal rank. Unlike a thresholded rank it has no cut-off to tune.

Parameters:

• matrix (Tensor) – A 2D tensor. Rows are samples (or output units), columns are features.

Returns:

• float – The effective rank, between 1 and min(matrix.shape); 0 for an all-zero matrix.

Source