---
title: "compareScenarios"
author: "Falk Benke"
date: "2024-04-26"
output: 
  rmarkdown::html_vignette:
    toc: true
    number_sections: true
vignette: >
  %\VignetteIndexEntry{compareScenarios}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  eval = FALSE
)
```

# Overview

`piamPlotComparison` is a lightweight framework to visually compare the results of multiple runs of IAMs, such as [REMIND](https://github.com/remindmodel/remind) or [EDGE-T](https://github.com/pik-piam/edgeTransport).

It is used in conjunction with a reporting library, such as [remind2](https://github.com/pik-piam/remind2) or [reporttransport](https://github.com/pik-piam/reporttransport). The reporting library includes Rmarkdown-files (Rmd-files) with R-code (e.g., for creating plots, creating tables) as well as descriptive text following some conventions. The framework offers functionality that reads model runs and produces either HTML- and PDF-documents using the Rmd-files via the core function `compareScenarios`.

Internally, `compareScenarios()` calls `rmarkdown::render()` on [cs_main.Rmd](https://github.com/pik-piam/piamPlotComparison/blob/master/inst/compareScenarios/cs_main.Rmd). This file loads the data from the mif-files, preprocesses the data, and includes the section-Rmd-files from the project library.

Aside from HTML- and PDF-documents as output, `compareScenarios` also allows to obtain a copy of the Rmd-files needed to create these outputs (by setting the argument `outputFormat = "Rmd"`). Rendering the resulting Rmd-files to PDF or HTML yields the same documents as calls to `compareScenarios()` with `outputFormat = "PDF"` or `"HTML"`. The advantage of being able to access these Rmd-files is the possibility to change the plotting code without changing the code of the reporting library. Moreover, Rmd-files can be used interactively in RStudio.

# Getting Started

## Add Rmd files to your reporting library

Create a folder `inst/compareScenarios` in your reporting library and place one Rmd-file for each section in your document in this folder. The section-Rmd-files follow the naming pattern `cs_NN_XXXX.Rmd`, where `NN` is replaced by a two digit number and `XXXX` is replaced by a short name of the section. 

The Rmd-files usually contain chunks with plots. For plotting, we usually use the [mip](https://github.com/pik-piam/mip) package or `ggplot`. The frameworks provides the `data` object containing your model runs and historical / reference data for plotting.

The framework takes care of preprocessing your inputs. If you need additional code to be executed as part of the setup (e.g. calculation of additional data or defining custom functions), you can do so by placing a file `preprocessing.R` in the folder `inst/compareScenarios` in your reporting library.

Once you have all the files in place, install your updated project library (or run `devtools::load_all()`).

## Run compareScenarios

A simple direct call of `piamPlotComparison::compareScenarios()` may look like this:

```{r}
library(piamPlotComparison)
compareScenarios(
  projectLibrary = "myProjectLibrary",
  mifScen = c("path/to/Base.mif", "path/to/NDC.mif"),
  mifHist = "path/to/historical.mif",
  outputFile = "CompareScenariosExample")
```

This creates a pdf called `CompareScenariosExample.pdf` using two runs `Base.mif` and `NDC.mif`, rendering all the sections found in your project library `myProjectLibrary`.

For further examples and a description of further optional arguments to the function, see `?piamPlotComparison::compareScenarios`.

## Interactive use of the Rmd-files

In this section, we want to execute individual R-code chunks in the cs-Rmd-files interactively in RStudio. First, we need access to the Rmd-files by calling `piamPlotComparison::compareScenarios()` with `outputFormat = "Rmd"` to obtain a (modified) copy of the Rmd-files.

```{r}
library(piamPlotComparison)
compareScenarios(
  projectLibrary = "myProjectLib",
  mifScen = c("path/to/scen1.mif", "path/to/scen2.mif", "path/to/scen3.mif"), # TODO.
  mifHist = "path/to/historical.mif", # TODO.
  outputDir = "path/to/where/rmds/should/be/copied/to", # TODO.
  outputFormat = "Rmd",
  # Add current time as output name to not overwrite other things:
  outputFile = format(Sys.time(), "CompScen2-%Y%m%d-%H%M%S"))
```

The code in these Rmd-files is structured in chunks and each chunk can be run separately by clicking on its play-button. We first open `cs_main.Rmd`. At the end of the file, there is a chunk
````
```{r prepare mark}
# CLICK "RUN ALL CHUNKS ABOVE" HERE TO PREPARE THE ENVIRONMENT
```
```` 
Clicking on the *triangle above bar*-button on the right executes all chunks above and by that loads and prepares the data for plotting. After that we can open any section-Rmd-file and execute a chunk to create the respective plots. The plots should appear in RStudio inside the Rmd-file below the chunk.

# Structure of cs_main.Rmd

Internally, `compareScenarios()` calls `rmarkdown::render()` on certain Rmarkdown-files (Rmd-files). To be more precise, `rmarkdown::render()` is called on [cs_main.Rmd](https://github.com/pik-piam/piamPlotComparison/blob/master/inst/compareScenarios/cs_main.Rmd). This file loads the data from the mif-files, preprocesses the data, and includes the section-Rmd-files from the project library at the very end (and optionally further Rmd-files provided by the user, see YAML-parameter `userSectionPath`).

## YAML-header

The file `cs_main.Rmd` starts with a YAML header marked by `---`. This header declares some basic information of the report, like its title and the output format. Furthermore, it contains a list `params`, which parameterizes the report. Among others, such parameters are the paths to the mif-files and certain properties that are shared for all plots in the report. Each such parameter can be changed by a respective argument in the call of `compareScenarios()`, see section *YAML Parameters* in the documentation of the function.

## Loading

The mif-files are loaded using `quitte::read.quitte()`. This function names the global region `"World"` (not `"GLO"` as `magclass::read.report()`). For each file, we obtain one data-frame with columns `model, scenario, region, variable, unit, period, value`. During preprocessing the data-frames are concatenated into one quitte object named `data`. In the sections, `data` is used to provide the data necessary for the plots.

If provided, also config files are loaded.

## Preprocessing

* Scenarios are renamed if the user specifies new names (using a named vector for the argument `mifScen` of `compareScenarios()` or by setting the parameter `mifScenNames` in the Rmd-files) or if scenario names are duplicate.
* The columns `period` (years) and `region` are filtered according to the parameters `yearsScen`, `yearsHist`, and `reg`.
* `|+|, |++|, |+++|, ...` are removed from variable names. See also `piamutils::deletePlus`. An additional column `varplus` is added to the data frame, which retains the original name including `+`. This column is used by `mip::showAreaAndBarPlotsPlus()`.


## Global Variables 

Global variables are created in `cs_main.Rmd` and are intended to be used in the plot functions of the section-Rmd-files.

The quitte-object (data frame) `data` with columns `model, scenario, region, variable, unit, period, value, varplus` provides all data that may be plotted.
It has mostly factor-columns. Thus, a vector of all available regions can be obtained by `levels(data$region)`.

Some arguments of the `mip::show...()` plot functions use the functionality provided by the base-R functions `options()` and `getOption()`. In `cs_main.Rmd` there are calls to `options()`, setting values of the global options `mip.mainReg` and `mip.yearsBarPlot`. The plotting functions may have an argument, e.g., `mainReg` with default value `getOption("mip.mainReg")`. Thus, this argument does not have to be stated for each call of the plot function (assuming the respective call of `options()` sets it to the right value).

## Custom Code

If you need to execute any additional code as part of the general setup, put it into a file `inst/compareScenarios/preprocessing.R` in your project library. Custom code will be executed after the general code. 

If you want to set `mip.histRefModel`, please do so in the `preprocessing.R` file of your project library.

# Section-Rmd-files

The section-Rmd-files follow the naming pattern `cs_NN_XXXX.Rmd`, where `NN` is replaced by a two digit number and `XXXX` is replaced by a short name of the section. If the YAML-parameter `sections` is set to `"all"`, the default, all sections of this naming pattern are included by `cs_main.Rmd`. Alternatively, `sections` can be set to a vector of individual sections in the form of `"NN_XXXX"` to only render these sections.

The section-Rmd-files consist of section and sub-section titles, marked by `#`, `##`, `###`, ..., and R-code chunks which create plots, usually by calling one of the `show...()`-functions in the `mip` package.

## userSectionPath

The user can provide one or more additional Rmd-files that are appended after the sections provided by the package. Setting `sections` to `NULL` and `userSectionPath` to a character-vector of paths to Rmd-files creates a fully user-defined report.

## Special Sections

There is a first section (section number 0) and a last section in the files `cs_00_info.Rmd` and `cs_99_further_info.Rmd`, respectively. 

### 0 Info {-}

This section contains a file reference table with scenario names and the paths to the respective mif files. It also shows if a scenario is renamed. Renaming can be forced by the user by using a named character vector as argument `mifScen` where the names indicate the new scenario name, e.g.,

```{r}
compareScenarios(
  projectLibrary = "myProjectLib",
  mifScen = c(newName1 = "path/to/scen1.mif", newName2 = "path/to/scen2.mif"),
  ...)
```

Or, if two scenarios have the same name, they are renamed automatically.

Furthermore, this section displays a description of each scenario. This requires the YAML-parameter `cfgScen` to be set (see also [99 Further Info]). The description can be set in the `scenario_config*.csv`-file via a column `description`.

### 99 Further Info {-}

If `compareScenarios()` is provided with paths to the `config.Rdata` files of the scenarios via the YAML-parameter `cfgScen`, an analysis of the configurations of the scenarios is shown.

```{r}
compareScenarios(
  projectLibrary = "myProjectLib",
  mifScen = c("path/to/scen1.mif", "path/to/scen2.mif"),
  cfgScen = c("path/to/scen1/config.RData", "path/to/scen2/config.RData"),
  ...)
```

# Content Functions

## Plot Functions

The section Rmd-files mostly consist of simple calls of a `show...()`-function from the the [mip](https://github.com/pik-piam/mip) package. These functions are designed to make the creation of re-occurring kinds of plots in `comapreScenarios` simple. Alternatively, one can build custom plots using `ggplot2`.

The `mip`-package contains following default plot functions for `comapareScenarios`: 

* `showAreaAndBarPlots()`
* `showAreaAndBarPlotsPlus()`
* `showLinePlots()`
* `showLinePlotsWithTarget()`
* `showMultiLinePlots`
* `showMultiLinePlotsByVariable()`
* `showRegiLinePlots()`

All but `showLinePlotsWithTarget()` can use the global option `mip.mainReg`.
`showAreaAndBarPlots()` and `showAreaAndBarPlotsPlus()` can use `mip.yearsBarPlot`.
`showMultiLinePlotsByVariable()` uses `mip.histRefModel`.

See the respective function documentation for more information and example plots.

## Table Functions

For creating tables in cs, see the documentation of `piamPlotComparison::calcTimeSeriesStats()` and `piamPlotComparison::showStatsTable()` and -- for an example -- the section **Tables of Significant Periods** in [`remind2/inst/compareScenarios/cs2_01_summary.Rmd`](https://github.com/pik-piam/remind2/blob/master/inst/compareScenarios/cs_01_summary.Rmd).

