Get started with nestedscales • nestedscales

nestedscales provides a multi-level discrete representation of a data dimension and the operations that move data between its levels. This vignette introduces the representation and the main functions on a small industry classification. The dimension packages timescales and geoscales apply the same functions to calendars and region hierarchies. The package website includes a roadmap of the package’s scope.

library(nestedscales)

Representation

A nested scale is a table with one row per atom, the finest unit of the dimension, and one column per frame, a coarser level that groups the atoms. The frames are given coarsest first. Numeric columns that are not frames are weights.

industries <- data.frame(
  section  = c("C", "C", "C", "D", "D"),
  division = c("C10", "C10", "C11", "D35", "D35"),
  class    = c("C101", "C102", "C110", "D351", "D352"),
  gva      = c(120, 80, 45, 300, 150)
)

ind <- scale_from_leaftable(
  industries,
  frames = c("section", "division", "class"),
  name   = "nace"
)
ind
#> NestedScale: nace
#> Frames (3, coarsest first):
#>   - section (2)
#>     - division (3)
#>       - class (5)
#> Atoms: 5
#> Weights: gva (default: gva)

The scale records the frames, the units at each frame, the atom key and the weights:

scale_frames(ind)
#> [1] "section"  "division" "class"
scale_units(ind, "division")
#> [1] "C10" "C11" "D35"
scale_key(ind)
#> [1] "class"
scale_weights(ind)
#> [1] "gva"

Every frame partitions the atoms. Frames need not nest in each other: a unit at one frame may belong to several units at a coarser frame, and two frames may be independent, in which case the atoms are their combinations. scale_nests(), scale_crosses() and scale_is_uniform() report the relation a given pair of frames has; no relation is assumed.

scale_nests(ind, "section", "division")
#> [1] TRUE
scale_family(ind, "section", "division")
#>   parent_frame parent child_frame child
#> 1      section      C    division   C10
#> 2      section      C    division   C11
#> 3      section      D    division   D35

Transition rules

Data keyed at one frame is converted to another frame with recast_scale(). Each value column follows a rule from SCALE_RULES:

rule	aggregation (fine to coarse)	disaggregation (coarse to fine)
`sum`	sum	split in proportion to the weight
`weighted_mean`	weighted mean	copy
`mean`	mean over atoms	copy
`copy`	the common value	copy
`sd`	standard deviation over atoms	not defined
`share`	value as a share of its parent’s total	not defined

The rule is given per value column, so extensive and intensive quantities convert in one call:

x <- data.frame(
  class = c("C101", "C102", "C110", "D351", "D352"),
  jobs  = c(12, 8, 5, 30, 20),
  wage  = c(31, 29, 27, 45, 40)
)

recast_scale(x, ind,
  from = "class", to = "section",
  rule = c(jobs = "sum", wage = "weighted_mean")
)
#>   section jobs     wage
#> 1       C   25 29.61224
#> 2       D   50 43.33333

The same function converts in the other direction. A total at the section level is split over the classes in proportion to the weight:

y <- data.frame(section = c("C", "D"), investment = c(100, 400))

recast_scale(y, ind, from = "section", to = "class", rule = "sum")
#>   class investment
#> 1  C101   48.97959
#> 2  C102   32.65306
#> 3  C110   18.36735
#> 4  D351  266.66667
#> 5  D352  133.33333

Columns that are neither the key nor a value column are identifiers and are preserved as groups, so a panel of several years or scenarios converts in one call. A rule may also be registered once for a named value column with register_scale_rule(), after which rule = can be omitted for that column.

Labels

join_scale() attaches the scale’s labels to a dataset: the unit at the keyed frame, optionally the coarser frames (attach =) and the unit’s share and weight (meta =):

join_scale(x, ind, frame = "class", attach = TRUE)
#>   class jobs wage nace.section nace.division nace
#> 1  C101   12   31            C           C10 C101
#> 2  C102    8   29            C           C10 C102
#> 3  C110    5   27            C           C11 C110
#> 4  D351   30   45            D           D35 D351
#> 5  D352   20   40            D           D35 D352

Subsets

filter_scale() keeps the atoms under given units; the result records what fraction of each weight it retains. prune_scale() collapses a scale to a coarser atom level.

manufacturing <- filter_scale(ind, "section", "C")
scale_units(manufacturing)
#> [1] "C101" "C102" "C110"
scale_coverage(manufacturing)
#>      gva 
#> 0.352518

scale_frames(prune_scale(ind, "division"))
#> [1] "section"  "division"

Several dimensions

scale_product() combines scales into one multi-dimensional index without materialising the product of their atoms. recast_product(), join_product() and filter_product() apply the single-scale operations axis by axis.

p <- scale_product(a = scale_example(), b = scale_example2())
product_size(p)
#> $axes
#> a b 
#> 7 4 
#> 
#> $total
#> [1] 28

Further operations

cluster_scale() groups units by their data and returns the scale with the clusters as a new frame.
reconcile_scale() compares aggregated data with an independent total.
as_scale_array() converts long data to a dense array over a scale, and write_scale_dataset() stores data as an Arrow dataset together with its scales.

All operations accept a data.frame, tibble, data.table, dtplyr table or arrow dataset and return the same class; lazy inputs return a query unless collect = TRUE.