Skip to contents

Axes Reliability

Source: vignettes/axes-reliability.Rmd
axes-reliability.Rmd

Level: Advanced. Read “Latent Group Contrasts” first.

1. Overview

This vignette estimates how reliably a circumplex instrument measures its two axes. Section 2, “What is axis reliability?”, defines the quantity and the variance components behind it. Section 3, “A worked example”, runs axes_reliability() on the simulated_items data. Section 4, “Reading the components”, reads the components and the global fit from summary(). Section 5, “Starting from a published correlation matrix”, fits the model from a correlation matrix and a sample size. The Wrap-up lists what the page covered and names the next page, “Axes Reliability Caveats”. That page states the properties of the method that shape how its output should be read. The References list the source cited.

2. What is axis reliability?

A circumplex instrument places its scales around a circle. It summarizes a person (or a profile of correlations) by a position on two orthogonal axes. Here the axes are communion (the X axis, at 0°) and agency (the Y axis, at 90°). Everything downstream uses those axis scores: a person’s projected location, and the displacement and amplitude of a Structural Summary Method profile. vignette("introduction-to-ssm-analysis") defines displacement and amplitude. So it is worth asking how reliably the instrument measures each axis.

axes_reliability() answers that question with the estimator of Strack, Jacobs, and Grosse Holtforth (2013). It fits an item-level measurement model. The model splits each item’s variance into orthogonal pieces: a general factor shared by all items, the two circumplex axes, scale specificity, and item error. For an instrument administered in blocks, it adds a block-specificity component (“Axes Reliability Caveats” explains blocks). The function reads axis reliability off the axes component alone. Reliability is then the Spearman–Brown “list-length” reliability of a composite with the axis’s effective test length, built from items that share only their axes variance.

This is a different question from the other reliability-adjacent tools in the package. ssm_sem() disattenuates (corrects for measurement error) a scale-level SSM profile. fit_structure() evaluates whether a correlation matrix has circumplex structure at all. axes_reliability() instead reports a single, interpretable number per axis: how well the instrument measures communion and agency.

3. A worked example

The package ships simulated_items, a synthetic dataset of 1–7 Likert responses from 500 respondents on 32 items. There are four items on each of the eight octant scales, in the order that octants() returns. The items were drawn from a five-component population that implies an axis reliability of about .78. The five are a general factor, the two equal axes (axes variance .18), a shared scale-specificity component of .10, block specificity, and free item error. Block specificity is zero here, because the instrument is not blockwise.

axes_reliability() needs three things: the data, a map from items to scales, and the scales’ angles. The map is a list with one character vector of item names per scale, in the same angle order as the angles you pass:

data("simulated_items")

# Four items per octant scale, in octants() order (PA, BC, DE, ..., NO).
items <- split(names(simulated_items), rep(1:8, each = 4))

res <- axes_reliability(simulated_items, items = items, angles = octants())
#> axes_reliability(): 500 complete case(s) used.
res
#> 
#> Circumplex Axes Reliability (Strack, Jacobs & Grosse Holtforth, 2013)
#> Input:        item data
#> Items:        32 (8 scales)
#> Complete N:   500
#> SEm scale:    std
#> 
#> # Per-axis reliability
#> 
#>  Axis item_n Reliability SEm   NB_Reliability
#>  X    16     0.773       0.476 0.822         
#>  Y    16     0.773       0.476 0.823         
#> 
#>   Note: the two axes share one axes-variance estimate and, with equal items
#>   per axis, carry the same reliability. This is expected, not an error.
#> 
#>   Note: the model is fit to the item correlation matrix as if it were a
#>   covariance matrix (Cudeck, 1989), and both sides of that mismatch are
#>   corrected, so these numbers differ from LISREL's, and from lavaan's own, by
#>   design.
#>   The component standard errors are adjusted to the correlation metric and
#>   are calibrated; they are typically smaller than the values printed by
#>   Strack et al. (2013), whose LISREL output carries no correction.

If your items belong to one of the package’s built-in instruments, you can pass the instrument object instead of items and angles. It supplies both the scale angles and the item membership, exactly as score() does. (The example above uses the explicit map because simulated_items is not a registered instrument.)

The header confirms how many complete cases were used. For each axis, the per-axis table reports the effective test length (item_n), the Strack axis Reliability and its standard error of measurement (SEm). It also reports the Nunnally–Bernstein reliability (NB_Reliability), a simpler formula that Section 4 compares. For a balanced instrument (the same number of items on every scale), the two axes share one axes-variance estimate and carry equal item_n. So they report the same reliability, which is expected, not an error.

The recovered reliability (about .77) lands close to the .78 built into the simulated population. The axes-variance estimate (below) recovers the population value of .18.

4. Reading the components

summary() adds the estimated variance components and the model’s global fit:

summary(res)
#> 
#> Circumplex Axes Reliability (Strack, Jacobs & Grosse Holtforth, 2013)
#> Input:        item data
#> Items:        32 (8 scales)
#> Complete N:   500
#> SEm scale:    std
#> 
#> # Per-axis reliability
#> 
#>  Axis item_n Reliability SEm   NB_Reliability
#>  X    16     0.773       0.476 0.822         
#>  Y    16     0.773       0.476 0.823         
#> 
#>   Note: the two axes share one axes-variance estimate and, with equal items
#>   per axis, carry the same reliability. This is expected, not an error.
#> 
#>   Note: the model is fit to the item correlation matrix as if it were a
#>   covariance matrix (Cudeck, 1989), and both sides of that mismatch are
#>   corrected, so these numbers differ from LISREL's, and from lavaan's own, by
#>   design.
#>   The component standard errors are adjusted to the correlation metric and
#>   are calibrated; they are typically smaller than the values printed by
#>   Strack et al. (2013), whose LISREL output carries no correction.
#> 
#> # Variance components
#> 
#>  Component         Estimate SE   
#>  general           0.051    0.005
#>  axes              0.175    0.009
#>  scale_specificity 0.093    0.008
#>  item              0.680    --   
#> 
#> # Global fit
#> 
#>   chi-square(493) = 488.27,  RMSEA = 0.000,  CFI = 1.000
#> 
#>   The global fit statistics chisq, pvalue, rmsea and cfi are scaled to that
#>   metric (Satorra & Bentler, 1994), which removes a distortion that flatters
#>   fit; df and srmr are unchanged. The scaled test can modestly over-reject at
#>   typical sample sizes: it over-flags misfit rather than flattering it; see
#>   ?axes_reliability for the measured rates.
#>   They follow lavaan's *.scaled definitions, not its *.robust ones, and
#>   differ from what fitMeasures() reports for an equivalent ML fit.

The variance components show the decomposition that the reliability rests on. The axes component is the only one that feeds reliability. The general, scale_specificity and block_specificity (when blocks were supplied) components and item error are isolated from it. This is why the Nunnally–Bernstein figure printed alongside runs higher than the Strack reliability. N–B counts scale-specificity variance as axis variance rather than isolating it. So it overestimates axis reliability whenever scale specificity is non-trivial (Strack et al., 2013, Figure 3). The gap between the two numbers shows how much scale-specific variance the simpler formula would have credited to the axes in error.

5. Starting from a published correlation matrix

You do not always have the raw data. A paper may print an item correlation matrix and nothing else, and that matrix is enough. Pass it as cormat, with the sample size it was computed from, in place of data.

R <- cor(simulated_items)
axes_reliability(
  cormat = R, items = items, angles = octants(), n = nrow(simulated_items)
)
#> 
#> Circumplex Axes Reliability (Strack, Jacobs & Grosse Holtforth, 2013)
#> Input:        correlation matrix
#> Items:        32 (8 scales)
#> Sample N:     500
#> SEm scale:    std
#> 
#> # Per-axis reliability
#> 
#>  Axis item_n Reliability SEm   NB_Reliability
#>  X    16     0.773       0.476 --            
#>  Y    16     0.773       0.476 --            
#> 
#>   Note: the two axes share one axes-variance estimate and, with equal items
#>   per axis, carry the same reliability. This is expected, not an error.
#> 
#>   Note: the Nunnally-Bernstein comparison needs the raw item scores (scale
#>   alphas and the axis-composite variance), so it is NA on the
#>   correlation-matrix path.
#> 
#>   Note: the model is fit to the item correlation matrix as if it were a
#>   covariance matrix (Cudeck, 1989), and both sides of that mismatch are
#>   corrected, so these numbers differ from LISREL's, and from lavaan's own, by
#>   design.
#>   The component standard errors are adjusted to the correlation metric and
#>   are calibrated; they are typically smaller than the values printed by
#>   Strack et al. (2013), whose LISREL output carries no correction.

The estimates are identical to the raw-data run above. The raw-data path builds exactly this matrix internally and fits it the same way. items selects and orders the matrix’s rows by name, so the matrix’s own column ordering does not matter. The matrix must be symmetric, positive definite, and have a unit diagonal (the model assumes unit-variance items).

Two things are unavailable here, and both for the same reason: they are properties of the respondents, not of their correlations. The Nunnally–Bernstein comparison is NA, which the table prints as --. It needs each scale’s alpha and the axis composite’s variance, and a correlation matrix carries neither. And sd = "raw" is refused, because there are no scale scores to take an observed SD from. If you want SEm on a raw scale, pass numeric axis SDs to sd. Both are reported rather than silently omitted, so a matrix-based result cannot be mistaken for a raw-data one.

Wrap-up

axes_reliability() gives a compact, per-axis answer to “how reliably does this instrument measure communion and agency?” It isolates the axes variance from the general and scale-specific components, which a simpler reliability formula would conflate. Use it to characterize a circumplex instrument before leaning on its axis scores. Read its output with the correlation-as-covariance, missing-data, and boundary caveats in mind. The next page to read is “Axes Reliability Caveats”, which states those caveats and the limits of the model.

References