Contents

1 Installation

Most users should simply install the current version of curatedMetagenomicData from Bioconductor, unless they have good reason not to. Changes to the package are always committed to GitHub first, and it might be desirable to have the absolute latest changes by installing curatedMetagenomicData from GitHub; although, this is not recommended for most users. Finally, we have made extensive changes in curatedMetagenomicData 3.0.0 and some users might want to revert to the previous version of curatedMetagenomicData from GitHub for older analyses. The previous version of curatedMetagenomicData from GitHub, 1.20.0, will remain installable for the foreseeable future.

To install the current version of curatedMetagenomicData from Bioconductor, use BiocManager as follows.

BiocManager::install("curatedMetagenomicData")

To install the current version of curatedMetagenomicData from GitHub, use BiocManager as follows.

BiocManager::install("waldronlab/curatedMetagenomicData", dependencies = TRUE, build_vignettes = TRUE)

To install the previous version of curatedMetagenomicData from GitHub, use BiocManager as follows.

BiocManager::install("waldronlab/curatedMetagenomicData", dependencies = TRUE, build_vignettes = TRUE, ref = "v1.20.0")

2 R Packages

To demonstrate the functionality of curatedMetagenomicData, the dplyr and DT packages are needed.

library(dplyr)
library(DT)

3 Sample Metadata

The curatedMetagenomicData package contains a data.frame, sampleMetadata, of manually curated sample metadata to help users understand the nature of studies and samples available prior to returning resources. Beyond this, it serves two purposes: 1) to define study_name, which is used with the curatedMetagenomicData() method to query and return resources, and 2) to define sample_id, which is used with the returnSamples() method to return samples across studies.

To demonstrate, the first ten rows and columns (without any NA values) of sampleMetadata for the AsnicarF_2017 study are shown in the table below.

sampleMetadata %>%
    filter(study_name == "AsnicarF_2017") %>%
    select(where(~ !any(is.na(.x)))) %>%
    slice(1:10) %>%
    select(1:10) %>%
    datatable(options = list(dom = "t"), extensions = "Responsive")

4 Data Access

There are three main ways to access data from curatedMetagenomicData: 1. The curatedMetagenomicData() function to search and return individual datasets 2. The returnSamples() function to access merged datasets based on filtered metadata 3. Through the command-line interface. This is not part of the Bioconductor package or discussed here; see https://github.com/waldronlab/curatedMetagenomicDataCLI

4.1 The curatedMetagenomicData function

To access curated metagenomic data, users will use the curatedMetagenomicData() method both to query and return resources. The first argument pattern is a regular expression pattern to look for in the titles of resources available in curatedMetagenomicData; "" will return all resources. The title of each resource is a three part string with “.” as a delimiter - the fields are runDate, studyName, and dataType. The runDate is the date we created the resource and can mostly be ignored by users because if there is more than one date corresponding to a resource, the most recent one is selected automatically. It would be used if a specific runDate was needed, but only a single copy of each resource exists as of this writing.

Multiple resources can be queried or returned with a single call to curatedMetagenomicData(), only the titles of resources are returned by default. The following search term is a regex that matches all six data types from all datasets starting with *AsnicarF_20*:

curatedMetagenomicData("AsnicarF_20.+")
## 2021-03-31.AsnicarF_2017.gene_families
## 2021-03-31.AsnicarF_2017.marker_abundance
## 2021-03-31.AsnicarF_2017.marker_presence
## 2021-03-31.AsnicarF_2017.pathway_abundance
## 2021-03-31.AsnicarF_2017.pathway_coverage
## 2021-03-31.AsnicarF_2017.relative_abundance
## 2021-03-31.AsnicarF_2021.gene_families
## 2021-03-31.AsnicarF_2021.marker_abundance
## 2021-03-31.AsnicarF_2021.marker_presence
## 2021-03-31.AsnicarF_2021.pathway_abundance
## 2021-03-31.AsnicarF_2021.pathway_coverage
## 2021-03-31.AsnicarF_2021.relative_abundance

If you are more comfortable with wildcard-like searching, the following is equivalent:

glob2rx("*AsnicarF_20*") %>%
    curatedMetagenomicData()
## 2021-03-31.AsnicarF_2017.gene_families
## 2021-03-31.AsnicarF_2017.marker_abundance
## 2021-03-31.AsnicarF_2017.marker_presence
## 2021-03-31.AsnicarF_2017.pathway_abundance
## 2021-03-31.AsnicarF_2017.pathway_coverage
## 2021-03-31.AsnicarF_2017.relative_abundance
## 2021-03-31.AsnicarF_2021.gene_families
## 2021-03-31.AsnicarF_2021.marker_abundance
## 2021-03-31.AsnicarF_2021.marker_presence
## 2021-03-31.AsnicarF_2021.pathway_abundance
## 2021-03-31.AsnicarF_2021.pathway_coverage
## 2021-03-31.AsnicarF_2021.relative_abundance

When the second argument dryrun is set to FALSE, a list of SummarizedExperiment and/or TreeSummarizedExperiment objects is returned. When a single resource is requested, a single element list is returned.

curatedMetagenomicData("AsnicarF_2017.relative_abundance", dryrun = FALSE)
## $`2021-03-31.AsnicarF_2017.relative_abundance`
## class: TreeSummarizedExperiment 
## dim: 301 24 
## metadata(0):
## assays(1): relative_abundance
## rownames(301):
##   k__Bacteria|p__Proteobacteria|c__Gammaproteobacteria|o__Enterobacterales|f__Enterobacteriaceae|g__Escherichia|s__Escherichia_coli
##   k__Bacteria|p__Actinobacteria|c__Actinobacteria|o__Bifidobacteriales|f__Bifidobacteriaceae|g__Bifidobacterium|s__Bifidobacterium_bifidum
##   ...
##   k__Bacteria|p__Firmicutes|c__Bacilli|o__Lactobacillales|f__Streptococcaceae|g__Streptococcus|s__Streptococcus_gordonii
##   k__Bacteria|p__Firmicutes|c__Bacilli|o__Lactobacillales|f__Aerococcaceae|g__Abiotrophia|s__Abiotrophia_sp_HMSC24B09
## rowData names(7): Kingdom Phylum ... Genus Species
## colnames(24): MV_FEI1_t1Q14 MV_FEI2_t1Q14 ... MV_MIM5_t2M14
##   MV_MIM5_t3F15
## colData names(22): study_name subject_id ... lactating curator
## reducedDimNames(0):
## mainExpName: NULL
## altExpNames(0):
## rowLinks: a LinkDataFrame (301 rows)
## rowTree: 1 phylo tree(s) (10430 leaves)
## colLinks: NULL
## colTree: NULL

When the third argument counts is set to TRUE, relative abundance proportions are multiplied by read depth and rounded to the nearest integer prior to being returned. Also, when multiple resources are requested, the list will contain named elements corresponding to each SummarizedExperiment and/or TreeSummarizedExperiment object.

curatedMetagenomicData("AsnicarF_20.+.relative_abundance", dryrun = FALSE, counts = TRUE)
## $`2021-03-31.AsnicarF_2017.relative_abundance`
## class: TreeSummarizedExperiment 
## dim: 301 24 
## metadata(0):
## assays(1): relative_abundance
## rownames(301):
##   k__Bacteria|p__Proteobacteria|c__Gammaproteobacteria|o__Enterobacterales|f__Enterobacteriaceae|g__Escherichia|s__Escherichia_coli
##   k__Bacteria|p__Actinobacteria|c__Actinobacteria|o__Bifidobacteriales|f__Bifidobacteriaceae|g__Bifidobacterium|s__Bifidobacterium_bifidum
##   ...
##   k__Bacteria|p__Firmicutes|c__Bacilli|o__Lactobacillales|f__Streptococcaceae|g__Streptococcus|s__Streptococcus_gordonii
##   k__Bacteria|p__Firmicutes|c__Bacilli|o__Lactobacillales|f__Aerococcaceae|g__Abiotrophia|s__Abiotrophia_sp_HMSC24B09
## rowData names(7): Kingdom Phylum ... Genus Species
## colnames(24): MV_FEI1_t1Q14 MV_FEI2_t1Q14 ... MV_MIM5_t2M14
##   MV_MIM5_t3F15
## colData names(22): study_name subject_id ... lactating curator
## reducedDimNames(0):
## mainExpName: NULL
## altExpNames(0):
## rowLinks: a LinkDataFrame (301 rows)
## rowTree: 1 phylo tree(s) (10430 leaves)
## colLinks: NULL
## colTree: NULL
## 
## $`2021-03-31.AsnicarF_2021.relative_abundance`
## class: TreeSummarizedExperiment 
## dim: 639 1098 
## metadata(0):
## assays(1): relative_abundance
## rownames(639):
##   k__Bacteria|p__Bacteroidetes|c__Bacteroidia|o__Bacteroidales|f__Bacteroidaceae|g__Bacteroides|s__Bacteroides_vulgatus
##   k__Bacteria|p__Bacteroidetes|c__Bacteroidia|o__Bacteroidales|f__Bacteroidaceae|g__Bacteroides|s__Bacteroides_stercoris
##   ...
##   k__Bacteria|p__Synergistetes|c__Synergistia|o__Synergistales|f__Synergistaceae|g__Pyramidobacter|s__Pyramidobacter_sp_C12_8
##   k__Bacteria|p__Actinobacteria|c__Actinobacteria|o__Micrococcales|f__Brevibacteriaceae|g__Brevibacterium|s__Brevibacterium_aurantiacum
## rowData names(7): Kingdom Phylum ... Genus Species
## colnames(1098): SAMEA7041133 SAMEA7041134 ... SAMEA7045952 SAMEA7045953
## colData names(21): study_name subject_id ... curator BMI
## reducedDimNames(0):
## mainExpName: NULL
## altExpNames(0):
## rowLinks: a LinkDataFrame (639 rows)
## rowTree: 1 phylo tree(s) (10430 leaves)
## colLinks: NULL
## colTree: NULL

If you are interested only in taxonomic relative abundance, include *relative_abundance* in your search term, like above or with searches like glob2rx(*AsnicarF_20*abundance).

4.1.1 Merging curatedMetagenomicData() results

To merge the list elements returned from the curatedMetagenomicData() method into a single SummarizedExperiment or TreeSummarizedExperiment object, users will use the mergeData() method, provided elements are the same dataType.

curatedMetagenomicData("LiJ_20.+.marker_abundance", dryrun = FALSE) %>%
    mergeData()
## class: SummarizedExperiment 
## dim: 77712 456 
## metadata(0):
## assays(1): marker_abundance
## rownames(77712): 39491__A0A395UVM9__DXB76_04540
##   39491__A0A395UZL1__DXB76_05950 ... 1262937__R7LHU6__BN805_01557
##   712117__F3PBR4__HMPREF9056_02502
## rowData names(0):
## colnames(456): DLF005-IE DLM006-IE ... nHM612836 nHMX11726
## colData names(28): study_name subject_id ... location smoker

The mergeData() method works for every dataType and will always return the appropriate data structure (a single SummarizedExperiment or TreeSummarizedExperiment object).

curatedMetagenomicData("LiJ_20.+.pathway_abundance", dryrun = FALSE) %>%
    mergeData()
## class: SummarizedExperiment 
## dim: 27110 456 
## metadata(0):
## assays(1): pathway_abundance
## rownames(27110): UNMAPPED UNINTEGRATED ... PWY-7539:
##   6-hydroxymethyl-dihydropterin diphosphate biosynthesis III
##   (Chlamydia)|g__Prevotella.s__Prevotella_buccae LACTOSECAT-PWY:
##   lactose and galactose degradation
##   I|g__Escherichia.s__Escherichia_coli
## rowData names(0):
## colnames(456): DLF005-IE DLM006-IE ... nHM612836 nHMX11726
## colData names(28): study_name subject_id ... location smoker

This is useful for analysis across entire studies (e.g. meta-analysis); however, when doing analysis across individual samples (e.g. mega-analysis) the returnSamples() method is preferable.

curatedMetagenomicData("LiJ_20.+.relative_abundance", dryrun = FALSE) %>%
    mergeData()
## class: TreeSummarizedExperiment 
## dim: 688 456 
## metadata(0):
## assays(1): relative_abundance
## rownames(688):
##   k__Bacteria|p__Bacteroidetes|c__Bacteroidia|o__Bacteroidales|f__Bacteroidaceae|g__Bacteroides|s__Bacteroides_plebeius
##   k__Bacteria|p__Bacteroidetes|c__Bacteroidia|o__Bacteroidales|f__Bacteroidaceae|g__Bacteroides|s__Bacteroides_caccae
##   ...
##   k__Bacteria|p__Proteobacteria|c__Betaproteobacteria|o__Neisseriales|f__Neisseriaceae|g__Neisseria|s__Neisseria_elongata
##   k__Bacteria|p__Proteobacteria|c__Gammaproteobacteria|o__Enterobacterales|f__Morganellaceae|g__Providencia|s__Providencia_alcalifaciens
## rowData names(7): Kingdom Phylum ... Genus Species
## colnames(456): DLF005-IE DLM006-IE ... nHM612836 nHMX11726
## colData names(28): study_name subject_id ... location smoker
## reducedDimNames(0):
## mainExpName: NULL
## altExpNames(0):
## rowLinks: a LinkDataFrame (688 rows)
## rowTree: 1 phylo tree(s) (10430 leaves)
## colLinks: NULL
## colTree: NULL

4.2 The returnSamples() function for merged datasets based on filtered metadata

The returnSamples() takes a subset of the sampleMetadata object as input, and returns a single SummarizedExperiment or TreeSummarizedExperiment containing one of curatedMetagenomicData’s data types, with this metadata subset as its colData. To use this method, filter/select rows and columns of interest from the sampleMetadata object, maintaining at least one row, and the sampleID and study_name columns. Provide the resulting data.frame as the first argument to the returnSamples() function.

The returnSamples() method requires a second argument dataType ("gene_families", "marker_abundance", "marker_presence", "pathway_abundance", "pathway_coverage", or "relative_abundance") be specified. It is often most convenient to subset the sampleMetadata data.frame using dplyr syntax.

sampleMetadata %>%
    filter(age >= 18) %>%
    filter(!is.na(alcohol)) %>%
    filter(body_site == "stool") %>%
    select(where(~ !all(is.na(.x)))) %>%
    returnSamples("relative_abundance")
## class: TreeSummarizedExperiment 
## dim: 833 702 
## metadata(0):
## assays(1): relative_abundance
## rownames(833):
##   k__Bacteria|p__Bacteroidetes|c__Bacteroidia|o__Bacteroidales|f__Prevotellaceae|g__Prevotella|s__Prevotella_copri
##   k__Bacteria|p__Bacteroidetes|c__Bacteroidia|o__Bacteroidales|f__Prevotellaceae|g__Prevotella|s__Prevotella_sp_CAG_520
##   ...
##   k__Bacteria|p__Actinobacteria|c__Actinobacteria|o__Corynebacteriales|f__Corynebacteriaceae|g__Corynebacterium|s__Corynebacterium_aurimucosum
##   k__Bacteria|p__Actinobacteria|c__Actinobacteria|o__Corynebacteriales|f__Corynebacteriaceae|g__Corynebacterium|s__Corynebacterium_coyleae
## rowData names(7): Kingdom Phylum ... Genus Species
## colnames(702): JAS_1 JAS_10 ... YSZC12003_37879 YSZC12003_37880
## colData names(46): study_name subject_id ...
##   age_twins_started_to_live_apart zigosity
## reducedDimNames(0):
## mainExpName: NULL
## altExpNames(0):
## rowLinks: a LinkDataFrame (833 rows)
## rowTree: 1 phylo tree(s) (10430 leaves)
## colLinks: NULL
## colTree: NULL

The counts argument applies to the returnSamples() method as well, and can be passed as the third argument. Finally, users should know that any arbitrary columns added to sampleMetadata will be present in the colData of the SummarizedExperiment or TreeSummarizedExperiment object that is returned.

5 Example Analysis

To demonstrate the utility of curatedMetagenomicData, an example analysis is presented below. However, readers should know analysis is generally beyond the scope of curatedMetagenomicData and the analysis presented here is for demonstration alone. It is best to consider the output of curatedMetagenomicData as the input of analysis more than anything else.

5.1 R Packages

To demonstrate the utility of curatedMetagenomicData, the stringr, mia, scater, and vegan packages are needed.

library(stringr)
library(mia)
library(scater)
library(vegan)

5.2 Prepare Data

In our hypothetical study, let’s examine the association of alcohol consumption and stool microbial composition across all annotated samples in curatedMetagenomicData. We will examine the alpha diversity (within subject diversity), beta diversity (between subject diversity), and conclude with a few notes on differential abundance analysis.

5.2.1 Return Samples

First, as above, we use the returnSamples() method to return the relevant samples across all studies available in curatedMetagenomicData. We want adults over the age of 18, for whom alcohol consumption status is known, and we want only stool samples. The select(where... line below simply removes metadata columns which are all NA values - they exist in another study but are all NA once subsetting has been done. Lastly, the "relative_abundance" dataType is requested because it contains the relevant information about microbial composition.

alcoholStudy <-
    filter(sampleMetadata, age >= 18) %>%
    filter(!is.na(alcohol)) %>%
    filter(body_site == "stool") %>%
    select(where(~ !all(is.na(.x)))) %>%
    returnSamples("relative_abundance")

5.2.2 Mutate colData

Most of the values in the sampleMetadata data.frame (which becomes colData) are in lower snake case (e.g. snake_case) and don’t look nice in plots. Here, the values of the alcohol variable are made into title case using stringr so they will look nice in plots.

colData(alcoholStudy) <-
    colData(alcoholStudy) %>%
    as.data.frame() %>%
    mutate(alcohol = str_replace_all(alcohol, "no", "No")) %>%
    mutate(alcohol = str_replace_all(alcohol, "yes", "Yes")) %>%
    DataFrame()

5.2.3 Agglomerate Ranks

Next, the splitByRanks method from mia is used to create alternative experiments for each level of the taxonomic tree (e.g. Genus). This allows for diversity and differential abundance analysis at specific taxonomic levels; with this step complete, our data is ready to analyze.

altExps(alcoholStudy) <-
    splitByRanks(alcoholStudy)

5.3 Alpha Diversity

Alpha diversity is a measure of the within sample diversity of features (relative abundance proportions here) and seeks to quantify the evenness (i.e. are the amounts of different microbes the same) and richness (i.e. are they are large variety of microbial taxa present). The Shannon index (H’) is a commonly used measure of alpha diversity, it’s estimated here using the estimateDiversity() method from the mia package.

To quickly plot the results of alpha diversity estimation, the plotColData() method from the scater package is used along with ggplot2 syntax.

alcoholStudy %>%
    estimateDiversity(abund_values = "relative_abundance", index = "shannon") %>%
    plotColData(x = "alcohol", y = "shannon", colour_by = "alcohol", shape_by = "alcohol") +
    labs(x = "Alcohol", y = "Alpha Diversity (H')") +
    guides(color = guide_none(), shape = guide_none()) +
    theme(legend.position = "none")
Alpha Diversity - Shannon Index (H')

Figure 1: Alpha Diversity - Shannon Index (H’)

The figure suggest that those who consume alcohol have higher Shannon alpha diversity than those who do not consume alcohol; however, the difference does not appear to be significant, at least qualitatively.

5.4 Beta Diversity

Beta diversity is a measure of the between sample diversity of features (relative abundance proportions here) and seeks to quantify the magnitude of differences (or similarity) between every given pair of samples. Below it is accessed by Bray–Curtis Principal Coordinates Analysis (PCoA), which is a linear method, and Uniform Manifold Approximation and Projection (UMAP), which is a non-linear method.

5.4.1 Bray–Curtis PCoA

To calculate pairwise Bray–Curtis distance for every sample in our study we will use the runMDS() method from the scater package along with the vegdist() method from the vegan package.

To quickly plot the results of beta diversity analysis, the plotReducedDim() method from the scater package is used along with ggplot2 syntax.

alcoholStudy %>%
    runMDS(FUN = vegdist, method = "bray", exprs_values = "relative_abundance", altexp = "Genus", name = "BrayCurtis") %>%
    plotReducedDim("BrayCurtis", colour_by = "alcohol", shape_by = "alcohol") +
    labs(x = "PCo 1", y = "PCo 2") +
    guides(colour = guide_legend(title = "Alcohol"), shape = guide_legend(title = "Alcohol")) +
    theme(legend.position = c(0.90, 0.85))