Write data and the scales that index it to one folder — write_scale

The data is written as a hive-partitioned Arrow IPC dataset and the scales travel with it, so open_scale_dataset() gives back both and the result can be recast without re-declaring anything.

write_scale_dataset(
  data,
  x,
  path,
  partitioning = NULL,
  compression = "zstd",
  overwrite = FALSE
)

Arguments

data: The table to write, in any supported backend.
x: The NestedScale or ScaleProduct the data is indexed by.
path: Directory to create. It must not already contain a store unless overwrite = TRUE.
partitioning: Columns to partition the data by (hive style). NULL (default) picks nothing; a good choice is a low-cardinality identifier such as a year, which lets a filtered read skip whole files.
compression: Codec for the IPC files, e.g. "zstd" (default), "lz4" or "uncompressed".
overwrite: Replace an existing store at path.

Value

path, invisibly.

Examples

p <- scale_product(a = scale_example(), b = scale_example2())
d <- merge(
  data.frame(unit = c("U1", "U2")),
  data.frame(period = c("p1", "p2"))
)
d$v <- 1:4
dir <- file.path(tempdir(), "store")
if (requireNamespace("arrow", quietly = TRUE) &&
  requireNamespace("yaml", quietly = TRUE)) {
  write_scale_dataset(d, p, dir, overwrite = TRUE)
  ds <- open_scale_dataset(dir)
  ds
}
#> <scale_dataset> C:\Users\admin\AppData\Local\Temp\RtmpMhVAOo/store 
#>   rows:  4
#>   files: 1 (1 KB)
#>   scale: product over a x b