| Title: | Downscale and harmonize land use data |
|---|---|
| Description: | Downscale and harmonize land use data (e.g. MAgPIE or WITCH) using high resolution reference data (e.g. LUH2v2h or LUH3). |
| Authors: | Pascal Sauer [aut, cre] (affiliation: Potsdam Institute for Climate Impact Research, ORCID: <https://orcid.org/0000-0002-6856-8239>), Jan Philipp Dietrich [aut] (affiliation: Potsdam Institute for Climate Impact Research, ORCID: <https://orcid.org/0000-0002-4309-6431>) |
| Maintainer: | Pascal Sauer <[email protected]> |
| License: | LGPL (>= 3) |
| Version: | 0.51.1 |
| Built: | 2026-06-04 17:11:51 UTC |
| Source: | https://github.com/pik-piam/mrdownscale |
Downscale and harmonize land use data (e.g. MAgPIE or WITCH) using high resolution reference data (e.g. LUH2v2h or LUH3).
Maintainer: Pascal Sauer [email protected] (ORCID) (affiliation: Potsdam Institute for Climate Impact Research)
Authors:
Pascal Sauer [email protected] (ORCID) (affiliation: Potsdam Institute for Climate Impact Research)
Jan Philipp Dietrich [email protected] (ORCID) (affiliation: Potsdam Institute for Climate Impact Research)
Useful links:
Report bugs at https://github.com/pik-piam/mrdownscale/issues
Computes weights for a weighted category mapping and aggregates the weights
to the spatial resolution described by the arguments geometry and crs.
calcLandCategorizationWeight(map, geometry, crs)calcLandCategorizationWeight(map, geometry, crs)
map |
a map in form of a data.frame containing a mapping between reference categories (column name "reference"), input categories (column name "dataInput") and merged categories (column name "merge") for the data dimension of a magpie object |
geometry |
the geometry of the magpie object for which the categories should
be mapped as given in the geometry attribute of a magpie object with
geometry information |
crs |
the coordinate reference system as returned by |
This calc-function has a rather unusual shape in that sense that the arguments to be provided are not simple configuration settings but rather relative complex. For this kind of implementation it is typically advised to use tool functions. The reason that a calc function is used in this particular case instead is, that the inputs of this function usually do not change over a long period of time so that caching of results becomes key for the overall performance of the data processing, which is available for calc- but not tool-functions. If tool-functions might support caching in the future as well as a conversion to a tool function might become a sensible option.
Jan Philipp Dietrich
This function computes a version of the chosen land input data in the resolution of the land input data but harmonized to the land use information of the chosen land target data set (harmonized categories as well as harmonized transition from historic target data to simulated input data).
calcLandHarmonized(input, target, harmonizationPeriod, harmonization)calcLandHarmonized(input, target, harmonizationPeriod, harmonization)
input |
name of an input dataset, options: "magpie", "witch" |
target |
name of the land target source to be used |
harmonizationPeriod |
Two integer values, before the first given year the target dataset is used, after the second given year the input dataset is used, in between harmonize between the two datasets |
harmonization |
name of harmonization method, see |
Pascal Sauer, Jan Philipp Dietrich
This function performs the downscaling: It calculates a high resolution dataset from the low resolution input dataset and the high resolution target dataset using the given downscaling method.
calcLandHighRes( input, target, harmonizationPeriod, yearsSubset, harmonization, downscaling )calcLandHighRes( input, target, harmonizationPeriod, yearsSubset, harmonization, downscaling )
input |
name of an input dataset, options: "magpie", "witch" |
target |
name of a target dataset |
harmonizationPeriod |
Two integer values, before the first given year the target dataset is used, after the second given year the input dataset is used, in between harmonize between the two datasets |
yearsSubset |
vector of years to keep in the output dataset |
harmonization |
name of harmonization method, see |
downscaling |
name of downscaling method, currently only "magpieClassic" |
downscaled land use data
Jan Philipp Dietrich, Pascal Sauer
Prepare the land input data for the category mapping, checking data for consistency before returning. All "Land" functions deal with area data, as opposed to "Nonland" functions which deal with non-area data such as the amount of applied fertilizer. These are treated differently, because for area data other constraints apply, e.g. the total area must be constant over time.
calcLandInput(input)calcLandInput(input)
input |
name of an input dataset, options: "magpie", "witch" |
input = "magpie": includes the land use categories past (pasture, including rangeland), forestry (managed forest plantations), primforest, secdforest, urban, other (other land) and many specific crop types. Furthermore, 1st gen biofuel is added and filled with zeros. 1st gen biofuel is only modeled implicitly in magpie via demand, and because of trade it is unclear on what area 1st gen biofuel is grown, also 1st gen biofuel is quickly phased out in magpie, so we fill biofuel_1st_gen with zeros and rely on the harmonization to produce a plausible 1st gen biofuel time series. input = "witch": includes a subset of LUH land use categories: primf, primn, secdn, pastr, c4per, pltns, secdf, c3ann_irrigated, c3ann_rainfed These are given as shares. A "rest" category is added so shares sum up to 1.
land input data
Jan Philipp Dietrich, Pascal Sauer
Computes the land input data in target land categories. Splitting of land categories is performed under use of internal land weights reflecting the prevalence of a certain land category in the given area.
calcLandInputRecategorized(input, target)calcLandInputRecategorized(input, target)
input |
name of an input dataset, options: "magpie", "witch" |
target |
name of the land target source to be used |
Mapping from input to target categories is achieved via a merge of a land input mapping to reference categories and a mapping between land target categories and the same reference categories. Thereby a new source or new target can be supported by supplying a map of that new input and/or target to the reference categories.
input = "witch": The "rest" category added in calcLandInput is disaggregated into all missing land variables using the reference dataset included in mrdownscale. Thus the following variables are added, but were not at all part of the input scenario data: range, urban, c4ann_*, c3per_*, c3nfx_* This allows the harmonization and downscaling pipeline to continue with a complete dataset, but these variables should not be used or reported (at the very least they have to be checked extensively).
Jan Philipp Dietrich, Pascal Sauer
Convert the downscaled land use data to the format required by the given project.
calcLandReport( outputFormat, input, harmonizationPeriod, yearsSubset, harmonization, downscaling )calcLandReport( outputFormat, input, harmonizationPeriod, yearsSubset, harmonization, downscaling )
outputFormat |
format in which the outputs should be prepared. Options: ESM, ScenarioMIP, downscaledmagpie |
input |
name of an input dataset, options: "magpie", "witch" |
harmonizationPeriod |
Two integer values, before the first given year the target dataset is used, after the second given year the input dataset is used, in between harmonize between the two datasets |
yearsSubset |
vector of years to keep in the output dataset |
harmonization |
name of harmonization method, see |
downscaling |
name of downscaling method, currently only "magpieClassic" |
land use data
Pascal Sauer
Convert the downscaled land use data to the format required by ScenarioMIP.
calcLandReportScenarioMIP( input, harmonizationPeriod, yearsSubset, harmonization, downscaling )calcLandReportScenarioMIP( input, harmonizationPeriod, yearsSubset, harmonization, downscaling )
input |
name of an input dataset, options: "magpie", "witch" |
harmonizationPeriod |
Two integer values, before the first given year the target dataset is used, after the second given year the input dataset is used, in between harmonize between the two datasets |
yearsSubset |
vector of years to keep in the output dataset |
harmonization |
name of harmonization method, see |
downscaling |
name of downscaling method, currently only "magpieClassic" |
land use data
Pascal Sauer
Prepare the high resolution target land use dataset for harmonization and downscaling, checking data for consistency before returning.
calcLandTarget(target, endOfHistory)calcLandTarget(target, endOfHistory)
target |
name of the target dataset, one of luh2, luh2mod, luh3 luh2mod/luh3 will split secdf into pltns and secdf |
endOfHistory |
years later than this are not returned |
land target data
Pascal Sauer
Aggregated low resolution target data is extrapolated to the given years using toolExtrapolate and normalized afterwards, so that the total sum over all land types is unchanged. To account for the relationship between wood harvest area and primary land (which is no longer primary once it has been harvested) wood harvest area is calculated here even though it is a nonland variable. The share of woody land that was harvested in the historical period is calculated and then multiplied by land (already extrapolated). Primary land is then converted to secondary land, so that total reduction equals harvested area.
calcLandTargetExtrapolated(input, target, harmonizationPeriod)calcLandTargetExtrapolated(input, target, harmonizationPeriod)
input |
name of an input dataset, options: "magpie", "witch" |
target |
character, name of the target data set |
harmonizationPeriod |
Two integer values, will extrapolate to all years present in input data between harmonization start and end year |
extrapolated land target data, if calcOutput is called with supplementary = TRUE and target is luh2mod wood harvest area is also returned
Pascal Sauer
Aggregate target land data to the spatial resolution of the input data in preparation for harmonization.
calcLandTargetLowRes(input, target, endOfHistory)calcLandTargetLowRes(input, target, endOfHistory)
input |
name of an input dataset, options: "magpie", "witch" |
target |
name of a target dataset, see |
endOfHistory |
years later than this are not returned |
low resolution target land data
Pascal Sauer
Ex-Post estimation of land use transitions based on land use state information
calcLandTransitions( outputFormat, input, harmonizationPeriod, yearsSubset, harmonization, downscaling, gross )calcLandTransitions( outputFormat, input, harmonizationPeriod, yearsSubset, harmonization, downscaling, gross )
outputFormat |
format in which the outputs should be prepared. Currently, only "ESM" for earth system model compatible input data is available. |
input |
name of an input dataset, currently only "magpie" |
harmonizationPeriod |
Two integer values, before the first given year the target dataset is used, after the second given year the input dataset is used, in between harmonize between the two datasets |
yearsSubset |
vector of years to keep in the output dataset |
harmonization |
name of harmonization method, see |
downscaling |
name of downscaling method, currently only "magpieClassic" |
gross |
either boolean or a magpie object containing bidirectional transition shares relative to the area of the involved land pools (transition divided by the area of the land pool in the "from" sub dimension). If set to FALSE only net transitions will be returned. If set to TRUE an internal gross transition estimate based on average gross transitions in LUH2 in the period from 1995 to 2015 will be used. |
land use transition data
Jan Philipp Dietrich, Pascal Sauer
Prepare data to be written as a LUH-style management.nc file. Call this via calcOutput in a full function, and set calcOutput's file argument to a .nc file path.
calcManagementNC( outputFormat, input, harmonizationPeriod, yearsSubset, harmonization, downscaling )calcManagementNC( outputFormat, input, harmonizationPeriod, yearsSubset, harmonization, downscaling )
outputFormat |
options: ESM, ScenarioMIP |
input |
name of an input dataset, options: "magpie", "witch" |
harmonizationPeriod |
Two integer values, before the first given year the target dataset is used, after the second given year the input dataset is used, in between harmonize between the two datasets |
yearsSubset |
remove years from the returned data which are not in yearsSubset |
harmonization |
name of harmonization method, see |
downscaling |
name of downscaling method, currently only "magpieClassic" |
data prepared to be written as a LUH-style management.nc file
Pascal Sauer, Jan Philipp Dietrich
Harmonize nonland input data to target data using the specified method, checking data for consistency before returning.
calcNonlandHarmonized(input, target, harmonizationPeriod, harmonization)calcNonlandHarmonized(input, target, harmonizationPeriod, harmonization)
input |
name of an input dataset, currently only "magpie" |
target |
name of the target dataset |
harmonizationPeriod |
Two integer values, before the first given year the target dataset is used, after the second given year the input dataset is used, in between harmonize between the two datasets |
harmonization |
harmonization method, see |
Wood harvest biomass (bioh) is adapted to the harmonized wood harvest area by calculating kg C per mega hectare for input and target data and harmonizing it. This is then multiplied by the harmonized wood harvest area and scaled so the total harmonized bioh is reached. Harmonize absolute fertilizer in Tg yr-1, then convert to fertilizer rate in kg ha-1 yr-1.
harmonized nonland data
Pascal Sauer
Calculate a high resolution dataset from the low resolution input dataset and high resolution data.
calcNonlandHighRes( input, target, harmonizationPeriod, yearsSubset, harmonization, downscaling )calcNonlandHighRes( input, target, harmonizationPeriod, yearsSubset, harmonization, downscaling )
input |
name of an input dataset, currently only "magpie" |
target |
name of a target dataset, see |
harmonizationPeriod |
Two integer values, before the first given year the target dataset is used, after the second given year the input dataset is used, in between harmonize between the two datasets |
yearsSubset |
vector of years to keep in the output dataset |
harmonization |
name of harmonization method, see |
downscaling |
name of downscaling method, currently only "magpieClassic" |
Wood harvest area is disaggregated using the maximum possible harvest per year which is based on the downscaled land data. Bioh is disaggregated using the just disaggregated wood harvest area as weight. Fertilizer is in kg ha-1 yr-1, so we simply use the low res/region value for each cell corresponding to that region. Harvest weight type is disaggregated using the nonland target data in the first year of the harmonization period as weight.
downscaled nonland data
Pascal Sauer
## Not run: calcOutput("NonlandHighRes", input = "magpie", target = "luh2mod", harmonizationPeriod = c(2015, 2050), yearsSubset = 2015:2100, harmonization = "fade", downscaling = "magpieClassic") ## End(Not run)## Not run: calcOutput("NonlandHighRes", input = "magpie", target = "luh2mod", harmonizationPeriod = c(2015, 2050), yearsSubset = 2015:2100, harmonization = "fade", downscaling = "magpieClassic") ## End(Not run)
Prepare the nonland input data for category mapping, checking data for consistency before returning.
calcNonlandInput(input)calcNonlandInput(input)
input |
name of an input dataset, currently only "magpie" |
All "Land" functions deal with area data, as opposed to "Nonland" functions which deal with non-area data such as the amount of applied fertilizer. These are treated differently, because for area data other constraints apply, e.g. the total area must be constant over time. Fertilizer on regional level is disaggregated to cluster level using cropland as weight.
nonland input data
Pascal Sauer
Harmonize categories by mapping nonland input data categories to the categories of the nonland target dataset.
See calcLandInputRecategorized for an explanation of the mapping procedure.
calcNonlandInputRecategorized( input, target, youngShareWoodHarvestArea = 0.95, youngShareWoodHarvestWeight = 0.5 )calcNonlandInputRecategorized( input, target, youngShareWoodHarvestArea = 0.95, youngShareWoodHarvestWeight = 0.5 )
input |
name of an input dataset, currently only "magpie" |
target |
name of a target dataset, see |
youngShareWoodHarvestArea |
share of wood harvest area taken from young (instead of mature) secondary forest; default value is based on LUH value from 2014; used to disaggregate wood harvest area from secondary forest to secondary young and mature forest |
youngShareWoodHarvestWeight |
analogue to youngShareWoodHarvestArea for wood harvest weight instead of area |
Report and discard wood harvest area if there is zero wood harvest (bioh) or vice versa.
nonland data with target categories
Pascal Sauer
Convert the downscaled nonland data to the format required by the given project.
calcNonlandReport( outputFormat, harmonizationPeriod, yearsSubset, harmonization, downscaling )calcNonlandReport( outputFormat, harmonizationPeriod, yearsSubset, harmonization, downscaling )
outputFormat |
options: ESM, ScenarioMIP |
harmonizationPeriod |
Two integer values, before the first given year the target dataset is used, after the second given year the input dataset is used, in between harmonize between the two datasets |
yearsSubset |
vector of years to keep in the output dataset |
harmonization |
name of harmonization method, see |
downscaling |
name of downscaling method, currently only "magpieClassic" |
nonland data
Pascal Sauer
## Not run: calcOutput("NonlandReport", outputFormat = "ESM", harmonizationPeriod = c(2015, 2050), yearsSubset = 2015:2100, harmonization = "fade", downscaling = "magpieClassic") ## End(Not run)## Not run: calcOutput("NonlandReport", outputFormat = "ESM", harmonizationPeriod = c(2015, 2050), yearsSubset = 2015:2100, harmonization = "fade", downscaling = "magpieClassic") ## End(Not run)
Prepare the high resolution nonland target dataset for harmonization and downscaling, checking data for consistency before returning.
calcNonlandTarget(target, endOfHistory)calcNonlandTarget(target, endOfHistory)
target |
name of a target dataset, see |
endOfHistory |
years later than this are not returned |
nonland target data
Pascal Sauer
Aggregated low resolution target data is extrapolated to the given years using toolExtrapolate. To extrapolate wood harvest weight (bioh) multiply wood harvest area already extrapolated by calcLandTargetExtrapolated with the historical wood harvest rate in kg C per Mha. Fertilizer is extrapolated and returned in kg ha-1 yr-1.
calcNonlandTargetExtrapolated(input, target, harmonizationPeriod)calcNonlandTargetExtrapolated(input, target, harmonizationPeriod)
input |
name of an input dataset, currently only "magpie" |
target |
name of a target dataset, see |
harmonizationPeriod |
Two integer values, will extrapolate to all years present in input data between harmonization start and end year |
extrapolated nonland target data
Pascal Sauer
## Not run: calcOutput("NonlandTargetExtrapolated", input = "magpie", target = "luh3", harmonizationPeriod = c(2025, 2050)) ## End(Not run)## Not run: calcOutput("NonlandTargetExtrapolated", input = "magpie", target = "luh3", harmonizationPeriod = c(2025, 2050)) ## End(Not run)
Aggregate target nonland data to the spatial resolution of the input data in preparation for harmonization. Fertilizer is converted to Tg yr-1, then aggregated and converted back to kg ha-1 yr-1.
calcNonlandTargetLowRes(input, target, endOfHistory)calcNonlandTargetLowRes(input, target, endOfHistory)
input |
name of an input dataset, currently only "magpie" |
target |
name of a target dataset |
endOfHistory |
years later than this are not returned |
low resolution target nonland data
Pascal Sauer
Calculate a complete mapping from low (input dataset, clusters/countries/regions) to high resolution (target dataset, grid). As a basis the mapping from the low resolution clusters/countries/regions to grid cells is used. Cells which are present in that mapping, but not in the target dataset are discarded. Cells which are present in the target dataset, but not in the mapping are added using a nearest-neighbor approach: These cells are mapped to the same low resolution cluster/country/region as the closest cell which is already present in the mapping.
calcResolutionMapping(input, target)calcResolutionMapping(input, target)
input |
name of an input dataset, options: "magpie", "witch" |
target |
character, the target dataset |
a list including a data.frame with columns x, y, lowRes, country
Pascal Sauer
Prepare data to be written as LUH-style states.nc file. Call this via calcOutput in a full function, and set calcOutput's file argument to a .nc file path.
calcStatesNC( outputFormat, input, harmonizationPeriod, yearsSubset, harmonization, downscaling )calcStatesNC( outputFormat, input, harmonizationPeriod, yearsSubset, harmonization, downscaling )
outputFormat |
options: ESM, ScenarioMIP |
input |
name of an input dataset, options: "magpie", "witch" |
harmonizationPeriod |
Two integer values, before the first given year the target dataset is used, after the second given year the input dataset is used, in between harmonize between the two datasets |
yearsSubset |
remove years from the returned data which are not in yearsSubset |
harmonization |
name of harmonization method, see |
downscaling |
name of downscaling method, currently only "magpieClassic" |
data prepared to be written as a LUH-style states.nc file
Pascal Sauer, Jan Philipp Dietrich
Prepared data to be written as a LUH-style transitions.nc file
calcTransitionsNC( outputFormat, input, harmonizationPeriod, yearsSubset, harmonization, downscaling )calcTransitionsNC( outputFormat, input, harmonizationPeriod, yearsSubset, harmonization, downscaling )
outputFormat |
options: ESM, ScenarioMIP |
input |
name of an input dataset, options: "magpie", "witch" |
harmonizationPeriod |
Two integer values, before the first given year the target dataset is used, after the second given year the input dataset is used, in between harmonize between the two datasets |
yearsSubset |
remove years from the returned data which are not in yearsSubset |
harmonization |
name of harmonization method, see |
downscaling |
name of downscaling method, currently only "magpieClassic" |
data prepared to be written as a LUH-style transitions.nc file
Pascal Sauer, Jan Philipp Dietrich
Harmonize wood harvest area based on harmonized land data. First, wood harvest area is harmonized just like land data. Then, to ensure consistency, primary harvest is converted to seondary harvest (or vice versa) to match the primary land reduction from the harmonized land data. If secondary harvest exceeds the available secondary land, the excess is shifted to the other secondary land (secdf to secdn and vice versa). Remaining excess harvest area is reported.
calcWoodHarvestAreaHarmonized( input, target, harmonizationPeriod, harmonization )calcWoodHarvestAreaHarmonized( input, target, harmonizationPeriod, harmonization )
input |
name of an input dataset, currently only "magpie" |
target |
name of a target dataset, see |
harmonizationPeriod |
Two integer values, before the first given year the target dataset is used, after the second given year the input dataset is used, in between harmonize between the two datasets |
harmonization |
harmonization method, see |
harmonized wood harvest area data
Pascal Sauer
## Not run: calcOutput("WoodHarvestAreaHarmonized", input = "magpie", target = "luh2mod", harmonizationPeriod = c(2015, 2050), harmonization = "fade") ## End(Not run)## Not run: calcOutput("WoodHarvestAreaHarmonized", input = "magpie", target = "luh2mod", harmonizationPeriod = c(2015, 2050), harmonization = "fade") ## End(Not run)
Convert LUH2 cell area shares to absolute areas by multiplying with cell area, convert to Mha.
convertLUH2v2h(x, subtype)convertLUH2v2h(x, subtype)
x |
SpatRaster with LUH2 cell area shares |
subtype |
Only "states" is converted |
Convert LUH3 cell area shares to absolute areas by multiplying with cell area, convert to Mha.
convertLUH3(x, subtype)convertLUH3(x, subtype)
x |
SpatRaster with LUH3 cell area shares |
subtype |
Only "states" is converted |
Download the LUH2v2h dataset (states.nc, transitions.nc, management.nc, staticData_quarterdeg.nc).
downloadLUH2v2h(subtype = NULL)downloadLUH2v2h(subtype = NULL)
subtype |
one of states, management, transitions, cellArea |
metadata list with URL, DOI, title, description, author, unit, version, release date
Pascal Sauer
Download the LUH3 dataset (states, management, transitions, static).
downloadLUH3(subtype)downloadLUH3(subtype)
subtype |
one of states, management, transitions, cellArea |
metadata list with URL, title, description, author, unit, version, release date
Pascal Sauer
Downscale MAgPIE results from the given folder. The resulting tgz file will be generated in the given folder.
downscaleRun(outputfolder, ...)downscaleRun(outputfolder, ...)
outputfolder |
path to a folder containing fulldata.gdx and clustermap_*.rds, resulting tgz will be written here |
... |
arguments passed on to retrieveData, e.g. model ("DOWNSCALEDMAGPIE" or "ESM"), rev, harmonizationPeriod |
Invisibly, the path to the newly created tgz archive.
Pascal Sauer
Downscale MAgPIE results from the given folder to 0.25 degree resolution in LUH2 format for ESMs. The resulting tgz file will be generated in the given folder.
downscaleRunESM(outputdir, revision = NULL, scenario = NULL, ...)downscaleRunESM(outputdir, revision = NULL, scenario = NULL, ...)
outputdir |
path to a folder containing fulldata.gdx and clustermap_*.rds, resulting tgz will be written here |
revision |
passed on to retrieveData, default: current date |
scenario |
passed on to retrieveData, default: slightly modified folder name |
... |
additional arguments passed on to retrieveData |
Invisibly, the path to the newly created tgz archive.
Pascal Sauer
Run the pipeline to generate harmonized and downscaled MAgPIE data using landuseinit as a reference dataset. Write output in the format of avl_land_t_0.5.mz, full report on consistency checks is printed and written to report.log.
fullDOWNSCALEDMAGPIE( rev = numeric_version("0"), harmonizationPeriod = c(2015, 2050), target = "landuseinit", downscaling = "magpieClassic", harmonization = "fade" )fullDOWNSCALEDMAGPIE( rev = numeric_version("0"), harmonizationPeriod = c(2015, 2050), target = "landuseinit", downscaling = "magpieClassic", harmonization = "fade" )
rev |
revision number of the data. If not provided the current date will be used instead. When called via madrat::retrieveData rev will be converted to numeric_version. |
harmonizationPeriod |
Two integer values, before the first given year the target dataset is used, after the second given year the input dataset is used, in between harmonize between the two datasets |
target |
Name of dataset to be used as harmonization target and downscaling reference |
downscaling |
name of downscaling method, currently only "magpieClassic" |
harmonization |
name of harmonization method, see |
Pascal Sauer
Run the pipeline to generate harmonized and downscaled data to report for RESCUE and other projects where ESM compatible land use inputs are required. Write .nc files, print full report on consistency checks and write it to report.log.
fullESM( rev = numeric_version("0"), input = "magpie", fileNamePart = "", harmonizationPeriod = c(2015, 2050), yearsSubset = 2015:2100, harmonization = "fade", downscaling = "magpieClassic", compression = 2, progress = TRUE )fullESM( rev = numeric_version("0"), input = "magpie", fileNamePart = "", harmonizationPeriod = c(2015, 2050), yearsSubset = 2015:2100, harmonization = "fade", downscaling = "magpieClassic", compression = 2, progress = TRUE )
rev |
revision number of the data. If not provided the current date will be used instead. When called via madrat::retrieveData rev will be converted to numeric_version. |
input |
name of an input dataset, options: "magpie", "witch" |
fileNamePart |
scenario nc files will be called multiple-[type]_[fileNamePart]-[revision]_gn_[min(yearsSubset)]-[max(yearsSubset)].nc where type is one of states/management/transitions |
harmonizationPeriod |
Two integer values, before the first given year the target dataset is used, after the second given year the input dataset is used, in between harmonize between the two datasets |
yearsSubset |
remove years from the returned data which are not in yearsSubset |
harmonization |
name of harmonization method, see |
downscaling |
name of downscaling method, currently only "magpieClassic" |
compression |
compression level of the resulting .nc files, possible values are integers from 1-9, 1 = fastest, 9 = best compression |
progress |
boolean defining whether progress should be printed |
Pascal Sauer, Jan Philipp Dietrich
Run the pipeline to generate harmonized and downscaled data to report for ScenarioMIP. LUH3 is used as historical reference dataset for harmonization and downscaling. Write .nc files, print full report on consistency checks and write it to report.log.
fullSCENARIOMIP( rev = numeric_version("0"), input = "magpie", scenario = "", harmonizationPeriod = c(2025, 2050), yearsSubset = 1995:2100, harmonization = "fadeForest", downscaling = "magpieClassic", compression = 2, progress = TRUE )fullSCENARIOMIP( rev = numeric_version("0"), input = "magpie", scenario = "", harmonizationPeriod = c(2025, 2050), yearsSubset = 1995:2100, harmonization = "fadeForest", downscaling = "magpieClassic", compression = 2, progress = TRUE )
rev |
revision number of the data. If not provided the current date will be used instead. When called via madrat::retrieveData rev will be converted to numeric_version. |
input |
name of an input dataset, options: "magpie", "witch" |
scenario |
scenario name to be included in filenames |
harmonizationPeriod |
Two integer values, before the first given year the target dataset is used, after the second given year the input dataset is used, in between harmonize between the two datasets |
yearsSubset |
remove years from the returned data which are not in yearsSubset |
harmonization |
name of harmonization method, see |
downscaling |
name of downscaling method, currently only "magpieClassic" |
compression |
compression level of the resulting .nc files, possible values are integers from 1-9, 1 = fastest, 9 = best compression |
progress |
boolean defining whether progress should be printed |
Pascal Sauer
Read LUH2v2h data. For the states subtype, the secma and secmb categories are removed. For the management subtype, only the categories crpbf, rndwd, fulwd, fertl and irrig are read. For the transitions subtype, only the wood harvest categories bioh and harv are read. To match magpie semantics years are shifted by 1 when reading transitions.
readLUH2v2h(subtype)readLUH2v2h(subtype)
subtype |
one of states, management, transitions, cellArea |
Read LUH3 data. For the states subtype, the secma and secmb categories are removed. For the management subtype, only the categories cpbf1, cpbf2, rndwd, fulwd, fertl and irrig are read. For the transitions subtype, only the wood harvest categories bioh and harv are read. To match magpie semantics years are shifted by 1 when reading transitions. The LUH3 nc files have day-based time, which is converted to years.
readLUH3(subtype, subset)readLUH3(subtype, subset)
subtype |
one of states, management, transitions, cellArea |
subset |
which years to read |
data read from LUH3 historic nc files as SpatRaster
Pascal Sauer
Read function for data coming from the MAgPIE model.
readMagpieFulldataGdx(subtype)readMagpieFulldataGdx(subtype)
subtype |
type of data to be read in. Available options are land, crop, woodHarvestWeight, woodHarvestArea, fertilizerRegional, clustermap |
Pascal Sauer, Jan Philipp Dietrich
Add metdata nc output files
toolAddMetadataNC( ncFile, activityId, revision, harmonizationPeriod, missingValue, compression, resolution, references, targetMIP, ncTitle, referenceDataset, furtherInfoUrl )toolAddMetadataNC( ncFile, activityId, revision, harmonizationPeriod, missingValue, compression, resolution, references, targetMIP, ncTitle, referenceDataset, furtherInfoUrl )
ncFile |
file name of the respective nc file |
activityId |
a string to store in the nc attribute activityId |
revision |
a string to store in the nc attribute revision |
harmonizationPeriod |
a string to store in the nc attribute harmonizationPeriod |
missingValue |
a string to store in the nc attribute missingValue |
compression |
a string to store in the nc attribute compression |
resolution |
a string to store in the nc attribute resolution |
references |
a string to store in the nc attribute references |
targetMIP |
a string to store in the nc attribute targetMIP |
ncTitle |
a string to store in the nc attribute ncTitle |
referenceDataset |
a string to store in the nc attribute referenceDataset |
furtherInfoUrl |
a string to store in the nc attribute furtherInfoUrl |
Pascal Sauer, Jan Philipp Dietrich
Aggregate variables like c3ann_irrigated and c3ann_rainfed to just c3ann.
toolAggregateCropland( land, cropTypes = c("c3ann", "c4ann", "c3per", "c4per", "c3nfx"), ..., keepOthers = TRUE )toolAggregateCropland( land, cropTypes = c("c3ann", "c4ann", "c3per", "c4per", "c3nfx"), ..., keepOthers = TRUE )
land |
magpie object with variables starting with cropTypes in dim 3 |
cropTypes |
character vector, the variables to aggregate to |
... |
reserved for future expansion |
keepOthers |
logical, if FALSE cropTypes will be the only variables in output |
land with crop data aggregated to cropTypes
Pascal Sauer
Check wood harvest area is not exceeding land area of the corresponding type divided by timestep length. Also, check that primf and primn are reduced by at least as much as they were harvested.
toolCheckWoodHarvestArea(harvest, land, endOfHistory)toolCheckWoodHarvestArea(harvest, land, endOfHistory)
harvest |
magpie object with exactly the following categories: paste0(c("primf", "secyf", "secmf", "pltns", "primn", "secnf"), "_wood_harvest_area") |
land |
magpie object with at least the following categories: c("primf", "secdf", "pltns", "primn", "secdn") |
endOfHistory |
The last year considered part of the historical period, will check and report consistency separately for history and after |
Pascal Sauer
For each crop type (c3ann, c3nfx, c3per, c4ann, c4per) calculate irrigation share, 1st and 2nd generation biofuel shares and share of cell area (corresponding to LUH3 variables c3ann, irrig_c3ann, cpbf1_c3ann, cpbf2_c3ann and analogously for the other crop types).
toolCropData(landHighRes, cellArea)toolCropData(landHighRes, cellArea)
landHighRes |
high resolution land use data as magclass object |
cellArea |
corresponding magclass object containing cell area in Mha |
crop data as magclass object with variables irrig_*, cpbf1_*, cpbf2_*, * for all 5 crop types
Pascal Sauer
classic MAgPIE downscaling method using luscale::interpolate2 as
downscaling function.
toolDownscaleMagpieClassic(x, xTarget, xTargetLowRes, mapping)toolDownscaleMagpieClassic(x, xTarget, xTargetLowRes, mapping)
x |
magclass containing land to be downscaled |
xTarget |
magclass target land use dataset for initialization year |
xTargetLowRes |
magclass target land use dataset for initialization year in low resolution (like x) |
mapping |
mapping between |
downscaled land use dataset
Jan Philipp Dietrich, Pascal Sauer
Extrapolate a dataset into the future. A linear model is fitted for each combination of spatial entity and category, and then used to predict the value of the last requested extrapolation year. If the linear model is not significant (p > 0.05) the historical mean is used instead. A spline-based interpolation is used to create a smooth transition from historical values to the predicted value.
toolExtrapolate(x, years)toolExtrapolate(x, years)
x |
A magpie object with "year" as the temporal dimension and without any NAs |
years |
A vector of years to extrapolate to |
A magpie object like x but with the extrapolated years only
Pascal Sauer
Fill data for missing years using linear interpolation.
toolFillYearsSpatRaster(x, years = NULL)toolFillYearsSpatRaster(x, years = NULL)
x |
SpatRaster with years in layer names |
years |
data for these years will be added if they are not already present, if years is NULL all years between the first and last year in x will be filled |
SpatRaster with filled years
Pascal Sauer
Get a harmonizer function by name.
toolGetHarmonizer(harmonizerName)toolGetHarmonizer(harmonizerName)
harmonizerName |
name of a harmonizer function, currently offset, fade, fadeForest |
harmonizer function
Pascal Sauer
toolHarmonizeOffset, toolHarmonizeFade, toolHarmonizeFadeForest
The function creates a transition data set based on a state dataset which contains for every possible connection (land type A <-> land type B) the area of the smaller of these land types. This information is relevant if transitional shares should be calculated based on the smaller area of the two (assuming that the smaller area is determining how much transitions there will be).
toolGetSmallerArea(states)toolGetSmallerArea(states)
states |
magpie dataset containing states information |
A land type x land type data set containing for each possible combination the smaller area of the two land types
Jan Philipp Dietrich
Tool function for creating a harmonized data set with a smooth s-shaped transition from historic target data to simulated input data.
toolHarmonizeFade(xInput, xTarget, harmonizationPeriod, level = 3)toolHarmonizeFade(xInput, xTarget, harmonizationPeriod, level = 3)
xInput |
input data as magpie object |
xTarget |
target data as magpie object |
harmonizationPeriod |
Two integer values, before the first given year the target dataset is used, after the second given year the input dataset is used, in between harmonize between the two datasets |
level |
passed to toolReplaceExpansion |
harmonized data set as magpie object with data from input for years before the harmonization period, data from target for years after the harmonization period and a smooth transition in between.
Jan Philipp Dietrich, Pascal Sauer
Harmonize using toolHarmonizeFade, then sum up primf + secdf,
then disaggregate again in a way that minimizes primf to secdf conversion,
without exceeding linear extrapolation of primf.
toolHarmonizeFadeForest(xInput, xTarget, harmonizationPeriod)toolHarmonizeFadeForest(xInput, xTarget, harmonizationPeriod)
xInput |
input data as magpie object |
xTarget |
target data as magpie object |
harmonizationPeriod |
Two integer values, before the first given year the target dataset is used, after the second given year the input dataset is used, in between harmonize between the two datasets |
harmonized data set as magpie object with data from input for years before the harmonization period, data from target for years after the harmonization period and a smooth transition in between.
Pascal Sauer
Harmonize datasets using the offset method as implemented in mip::harmonize, originally implemented in Python as part of aneris.
toolHarmonizeOffset(xInput, xTarget, harmonizationPeriod)toolHarmonizeOffset(xInput, xTarget, harmonizationPeriod)
xInput |
magpie object to fade to, usually model projections |
xTarget |
magpie object to fade from, usually historical data, this argument is called "target" for consistency with argument names of other functions, it is somewhat misleading here, because data after the harmonizationPeriod will not be taken from target, but from input |
harmonizationPeriod |
Two integer values, before the first given year the target dataset is used, after the second given year the input dataset is used, in between harmonize between the two datasets |
magpie object with harmonized data
Pascal Sauer
Computes a land category mapping between input and target data set via retrieving mappings from these sources to an internally defined reference categorization and merging them to a direct input-to-target mapping. Mappings to reference categories are stored internally in the package and have to be added to it if new input and/or target data should be supported.
toolLandCategoriesMapping(input, target)toolLandCategoriesMapping(input, target)
input |
name of an input dataset, options: "magpie", "witch" |
target |
name of the land target source to be used |
Jan Philipp Dietrich
Return the maximum expansion from one timestep to the next.
toolMaxExpansion(x, ..., removeNA = FALSE)toolMaxExpansion(x, ..., removeNA = FALSE)
x |
A magclass object |
... |
Reserved for future use |
removeNA |
A logical indicating whether NA values should be ignored |
A numeric value indicating the maximum expansion
Pascal Sauer
Calculate the maximum possible harvest area per year based on the given land data.
toolMaxHarvestPerYear(land, split = TRUE, timestepAdjust = TRUE)toolMaxHarvestPerYear(land, split = TRUE, timestepAdjust = TRUE)
land |
magpie object with at least the following categories: c("primf", "secdf", "pltns", "primn", "secdn") |
split |
if TRUE: split secdf to secyf and secdmf, rename secdf to secnf, and add dim "wood_harvest_area" |
timestepAdjust |
if TRUE: divide values for primary land by timestep length. This makes sense, because once primary land has been harvested, it is converted to secondary land and thus cannot be harvested again. Might introduce unintended spikes when timestep length changes. |
magpie object with the maximum possible yearly wood harvest area
Pascal Sauer
Check that primary forest (primf) and primary nature/nonland (primf) are never expanding using toolExpectTrue. If they are expanding, a warning including the maximum expansion is thrown. Categories not present in x are skipped.
toolPrimExpansionCheck(x)toolPrimExpansionCheck(x)
x |
A magclass object |
Pascal Sauer
Expansion from one timestep to the next of one land type is replaced with another land type.
toolReplaceExpansion( x, from, to, ..., noteThreshold = 10^-10, warnThreshold = 10^-5, level = 1 )toolReplaceExpansion( x, from, to, ..., noteThreshold = 10^-10, warnThreshold = 10^-5, level = 1 )
x |
a magpie object |
from |
name of a land category, e.g. "primf" Expansion of this category happening from one timestep to the next will be replaced. If missing in x return x unchanged |
to |
name of another land category, e.g. "secdf" Expansion of 'from' will be replaced with expansion of 'to' |
... |
not used, will throw an error if supplied |
noteThreshold |
expansion greater than this will trigger a note that expansion was replaced |
warnThreshold |
expansion greater than this will trigger a warning that expanding considerably |
level |
passed to toolStatusMessage |
a magpie object with expansion of 'from' replaced by 'to'
Pascal Sauer
See description of calcResolutionMapping. Here we are
assuming target resolution is finer than what mapping already provides.
toolResolutionMapping(mapping, targetGrid)toolResolutionMapping(mapping, targetGrid)
mapping |
a data.frame with columns x, y, lowRes |
targetGrid |
a terra SpatRaster with the target resolution |
a data.frame with columns x, y, lowRes, country
Pascal Sauer
Scale x to make the sum of all land types is constant over time.
toolScaleConstantArea(x, ..., noteThreshold = 10^-10, warnThreshold = 10^-5)toolScaleConstantArea(x, ..., noteThreshold = 10^-10, warnThreshold = 10^-5)
x |
a magpie object |
... |
not used, will throw an error if supplied |
noteThreshold |
if the maximum difference between the scaled and the original data is greater than this, a note is triggered |
warnThreshold |
if the maximum difference between the scaled and the original data is greater than this, a warning is triggered |
x scaled
Pascal Sauer
tool function to extract transitions between categories from a land data set with at least 2 time steps. The approach is rather simplistic by assuming that expansion happens proportionally in all affected classes (equal preference of transitions across all categories).
toolTransitionsBasic(x, gross = FALSE)toolTransitionsBasic(x, gross = FALSE)
x |
magpie data set containing land data with at least two time steps to extract net transitions from |
gross |
either boolean or a magpie object containing bidirectional transition shares relative to the area of the involved land pools (transition divided by the area of the land pool in the "from" sub dimension). If set to FALSE only net transitions will be returned. If set to TRUE an internal gross transition estimate based on average gross transitions in LUH2 in the period from 1995 to 2015 will be used. |
In addition to the net effect it can also estimate gross transition. For that purpose a reference data set containing bidirectional transition shares must be provided.
If the time step length is longer than 1 year the returned object contains reference years for each period which can be repeated to retrieve the full transition between two time periods, e.g. if you provide two time steps 2000 and 2005 the return value will be the transition for year 2001. Repeating the same transition also in 2002, 2003, 2004 and 2005 will give the full transition from 2000 to 2005.
Jan Philipp Dietrich
Warn if expansion is detected one of the given variables, giving the maximum expansion in each respective variable.
toolWarnIfExpansion(x, variable)toolWarnIfExpansion(x, variable)
x |
A magclass object |
variable |
A vector of variables to be checked |
Pascal Sauer