---
title: "Generic Workflow Template" 
author: "Author: FirstName LastName"
date: "Last update: `r format(Sys.time(), '%d %B, %Y')`" 
output:
  BiocStyle::html_document:
    toc_float: true
    code_folding: show
  BiocStyle::pdf_document: default
package: systemPipeR
vignette: |
  %\VignetteIndexEntry{Basic Generic Template}
  %\VignetteEncoding{UTF-8}
  %\VignetteEngine{knitr::rmarkdown}
fontsize: 14pt
bibliography: bibtex.bib
---

```{css, echo=FALSE}
pre code {
white-space: pre !important;
overflow-x: scroll !important;
word-break: keep-all !important;
word-wrap: initial !important;
}
```

```{r style, echo = FALSE, results = 'asis'}
BiocStyle::markdown()
options(width=60, max.print=1000)
knitr::opts_chunk$set(
    eval=as.logical(Sys.getenv("KNITR_EVAL", "TRUE")),
    cache=as.logical(Sys.getenv("KNITR_CACHE", "TRUE")), 
    tidy.opts=list(width.cutoff=60), tidy=TRUE)
```

```{r setup, echo=FALSE, message=FALSE, warning=FALSE, eval=FALSE}
suppressPackageStartupMessages({
    library(systemPipeR)
})
```

# Workflow environment

_`systemPipeR`_ workflows can be designed and built from start to finish with a 
single command, importing from an R Markdown file or stepwise in interactive 
mode from the R console. 

This tutorial will demonstrate how to build the workflow in an interactive mode, 
appending each step. The workflow is constructed by connecting each step via 
`appendStep` method. Each `SYSargsList` instance contains instructions needed 
for processing a set of input files with a specific command-line or R software 
and the paths to the corresponding outfiles generated by a particular tool/step. 

The workflow demonstrates a simple example of a typical data analysis workflow
containing both R and command-line (CL) steps. The workflow consists of four
steps:

1. R step: export tabular data to files 
2. CL step: compress files
3. CL step: uncompress files 
4. R step: import files and plot summary statistics

Refer to our [vignette](https://www.bioconductor.org/packages/devel/bioc/vignettes/systemPipeR/inst/doc/systemPipeR.html#52_Constructing_workflows) 
for how to construct the workflow steps. If you prefer a more enriched template, 
run `systemPipeRdata::availableWF()` for pre-configured templates.

```{r create_workflow, message=FALSE, eval=FALSE}
cat(crayon::blue$bold("To use this workflow, following R packages are expected:\n"))
cat(c("'ggplot2'\n"), sep = "', '")
### pre-end
library(systemPipeR)
sal <- SPRproject()
sal
```

## Load packages 

```{r load_library, eval=FALSE, spr=TRUE}
appendStep(sal) <- LineWise(
    code = {
    library(systemPipeR)
    }, 
    step_name = "load_library"
)
```

## Export dataset to file

- Add first step as an R step using `LineWise` function

```{r export_iris, eval=FALSE, spr=TRUE}
appendStep(sal) <- LineWise(code={
    mapply(
      FUN = function(x, y) write.csv(x, y),
      x = split(iris, factor(iris$Species)),
      y = file.path("results", paste0(names(split(iris, factor(iris$Species))), ".csv"))
    )
    }, 
  step_name = "export_iris", 
  dependency = "load_library"
)
```

## Compress data

- Adding the second step, as a command-line step using `SYSargsList`

```{r gzip, eval=FALSE, spr=TRUE, spr.dep=TRUE}
targetspath <- system.file("extdata/cwl/gunzip", "targets_gunzip.txt", package = "systemPipeR")
appendStep(sal) <- SYSargsList(
    step_name = "gzip", 
    targets = targetspath, dir = TRUE,
    wf_file = "gunzip/workflow_gzip.cwl", input_file = "gunzip/gzip.yml",
    dir_path = "param/cwl",
    inputvars = c(FileName = "_FILE_PATH_", SampleName = "_SampleName_"), 
    dependency = "export_iris"
)
```

## Decompress data

- Adding the third step, as a command-line step using `SYSargsList`

```{r gunzip, eval=FALSE, spr=TRUE}
appendStep(sal) <- SYSargsList(
    step_name = "gunzip", 
    targets = "gzip", dir = TRUE,
    wf_file = "gunzip/workflow_gunzip.cwl", input_file = "gunzip/gunzip.yml",
    dir_path = "param/cwl",
    inputvars = c(gzip_file = "_FILE_PATH_", SampleName = "_SampleName_"), 
    rm_targets_col = "FileName", 
    dependency = "gzip"
)
```

## Import data back to R and perform statistical analysis and visualization 

- Adding the fourth step, as an R step using `LineWise`

```{r stats, eval=FALSE, spr=TRUE}
appendStep(sal) <- LineWise(code={
    # combine all files into one data frame
    df <- lapply(getColumn(sal, step="gunzip", 'outfiles'), function(x) read.delim(x, sep=",")[-1])
    df <- do.call(rbind, df)
    # calculate mean and sd for each species
    stats <- data.frame(cbind(mean=apply(df[,1:4], 2, mean), sd=apply(df[,1:4], 2, sd)))
    stats$species <- rownames(stats)
    # plot
    plot <- ggplot2::ggplot(stats, ggplot2::aes(x=species, y=mean, fill=species)) + 
        ggplot2::geom_bar(stat = "identity", color="black", position=ggplot2::position_dodge()) +
        ggplot2::geom_errorbar(
            ggplot2::aes(ymin=mean-sd, ymax=mean+sd), 
            width=.2,
            position=ggplot2::position_dodge(.9)
        )
    plot
    }, 
    step_name = "stats", 
    dependency = "gunzip", 
    run_step = "optional"
)
```


## Version Information

```{r sessionInfo, eval=FALSE, spr=TRUE}
appendStep(sal) <- LineWise(
    code = {
    sessionInfo()
    }, 
    step_name = "sessionInfo", 
    dependency = "stats")
```


# Manage the workflow

## Interactive job submissions in a single machine

For running the workflow, `runWF` function will execute all the steps store in 
the workflow container. The execution will be on a single machine without 
submitting to a queuing system of a computer cluster. 

```{r runWF, eval=FALSE}
sal <- runWF(sal)
```

## Visualize workflow

_`systemPipeR`_ workflows instances can be visualized with the `plotWF` function.

```{r plotWF, eval=FALSE}
plotWF(sal, rstudio = TRUE)
```

## Checking workflow status

To check the summary of the workflow, we can use:

```{r statusWF, eval=FALSE}
sal
statusWF(sal)
```

## Accessing logs report

_`systemPipeR`_ compiles all the workflow execution logs in one central location,
making it easier to check any standard output (`stdout`) or standard error
(`stderr`) for any command-line tools used on the workflow or the R code stdout.

```{r logsWF, eval=FALSE}
sal <- renderLogs(sal)
```

# About the workflow
## Tools used 
To check command-line tools used in this workflow, use `listCmdTools`, and use `listCmdModules`
to check if you have a modular system.

The following code will print out tools required in your custom SPR project in the report. 
In case you are running the workflow for the first time time and do not have a project yet, or you 
just want to browser this workflow, following code displays the tools required by default.
```{r list_tools}
if(file.exists(file.path(".SPRproject", "SYSargsList.yml"))) {
    local({
        sal <- systemPipeR::SPRproject(resume = TRUE)
        systemPipeR::listCmdTools(sal)
        systemPipeR::listCmdModules(sal)
    })
} else {
    cat(crayon::blue$bold("Tools and modules required by this workflow are:\n"))
    cat(c("gzip", "gunzip"), sep = "\n")
}
```

## Session Info
This is the session information for rendering this report. To access the session information
of workflow running, check HTML report of `renderLogs`. 
```{r report_session_info, eval=TRUE}
sessionInfo()
```

# References
