Contents

Last modified: 2021-07-15 09:32:38
Compiled: Thu Jul 15 11:28:53 2021

1 Utilisation and prospects of bioimage datasets

In recent years, there has been a growing need for data analysis using machine learning in the field of bioimaging research. Machine learning is an inductive approach using data, and the construction of models, such as image segmentation and classification, involves the use of image data itself. Therefore, the publication and sharing of bioimage datasets [1] as well as knowledge creation through providing metadata to bioimages [2,3] are important issues to be discussed. At present, there is no commonly used format for sharing bioimage datasets. Also, the data is scattered among various repositories. Therefore, different image repositories manage the data in different formats (image data itself and metadata, including image format, instruments/microscopes and biosamples).

In the data analysis and quantification using those images, it is assumed that several steps of image pre-processing are performed depending on the analysis environment used. However, the implementation of supervised learning starts with finding a repository of the bioimage dataset that contains original images and their corresponding supervised labels. Once the repository is found, the image data is downloaded from the repository, the data is loaded into each environment and it is prepared in a format suitable for analytical package. These processes are time consuming before the main analysis. Also, in most of the image repositories, the data are not published in a format suitable for reading and processing in R (.Rdata, etc.), and the data are not easy to use for R users.

For performing supervised learning of bioimage data, BioImageDbs provides R list objects of the original images and their corresponding supervised labels converted into a 4D or 5D array. After retrieving the data from ExperimentHub, it can be utilised for deep learning using Keras/Tensorflow [4] and other machine learning methods, without the need for pre-processing.

On the other hand, many image analysis packages are also available on R; however, there is a lack of standardisation in image analysis. The use of common, open datasets is one of the essential steps in standardising and comparing the analytical methods. The provision of the array data of images through ExperimentHub is also intended for applications such as (1) comparing models using common-sharing data among R users and (2) applying predictions to new datasets through transfer learning and fine-tuning based on these arrays.

2 Fetch Bioimage Datasets from ExperimentHub

The BioImageDbs package provides the metadata for all BioImage databases in ExperimentHub.

The BioImageDbs package provides the metadata for bioimage datasets, which is preprocessed as array format and saved in ExperimentHub.

First we load/update the ExperimentHub resource.

library(ExperimentHub)
eh <- ExperimentHub()

Next we list all BioImageDbs entries from ExperimentHub.

query(eh, "BioImage")
## ExperimentHub with 23 records
## # snapshotDate(): 2021-05-18
## # $dataprovider: CELL TRACKING CHALLENGE (http://celltrackingchallenge.net/2...
## # $species: Homo sapiens, Mus musculus, Drosophila melanogaster
## # $rdataclass: List, magick-image
## # additional mcols(): taxonomyid, genome, description,
## #   coordinate_1_based, maintainer, rdatadateadded, preparerclass, tags,
## #   rdatapath, sourceurl, sourcetype 
## # retrieve records with, e.g., 'object[["EH6095"]]' 
## 
##            title                                                            
##   EH6095 | EM_id0001_Brain_CA1_hippocampus_region_5dTensor.rds              
##   EH6096 | EM_id0001_Brain_CA1_hippocampus_region_5dTensor_train_dataset.gif
##   EH6097 | EM_id0002_Drosophila_brain_region_5dTensor.rds                   
##   EH6098 | EM_id0002_Drosophila_brain_region_5dTensor_train_dataset.gif     
##   EH6099 | LM_id0001_DIC_C2DH_HeLa_4dTensor.rds                             
##   ...      ...                                                              
##   EH6113 | LM_id0003_Fluo_N2DH_GOWT1_5dTensor.rds                           
##   EH6114 | EM_id0003_J558L_4dTensor.rds                                     
##   EH6115 | EM_id0003_J558L_4dTensor_train_dataset.gif                       
##   EH6116 | EM_id0004_PrHudata_4dTensor.rds                                  
##   EH6117 | EM_id0004_PrHudata_4dTensor_train_dataset.gif

We can confirm the metadata in ExperimentHub in Bioconductor S3 bucket with mcols().

mcols(query(eh, "BioImage"))
## DataFrame with 23 rows and 15 columns
##                         title           dataprovider                species
##                   <character>            <character>            <character>
## EH6095 EM_id0001_Brain_CA1_.. https://www.epfl.ch/..           Mus musculus
## EH6096 EM_id0001_Brain_CA1_.. https://www.epfl.ch/..           Mus musculus
## EH6097 EM_id0002_Drosophila.. the ISBI 2012 Challe.. Drosophila melanogas..
## EH6098 EM_id0002_Drosophila.. the ISBI 2012 Challe.. Drosophila melanogas..
## EH6099 LM_id0001_DIC_C2DH_H.. CELL TRACKING CHALLE..           Homo sapiens
## ...                       ...                    ...                    ...
## EH6113 LM_id0003_Fluo_N2DH_.. CELL TRACKING CHALLE..           Mus musculus
## EH6114 EM_id0003_J558L_4dTe.. Pattern Recognition ..           Mus musculus
## EH6115 EM_id0003_J558L_4dTe.. Pattern Recognition ..           Mus musculus
## EH6116 EM_id0004_PrHudata_4.. Pattern Recognition ..           Homo sapiens
## EH6117 EM_id0004_PrHudata_4.. Pattern Recognition ..           Homo sapiens
##        taxonomyid      genome            description coordinate_1_based
##         <integer> <character>            <character>          <integer>
## EH6095      10090          NA 5D arrays with the b..                  1
## EH6096      10090          NA A animation file (.g..                  1
## EH6097       7227          NA 5D arrays with the b..                  1
## EH6098       7227          NA A animation file (.g..                  1
## EH6099       9606          NA 4D arrays with the m..                  1
## ...           ...         ...                    ...                ...
## EH6113      10090          NA 5D arrays with the m..                  1
## EH6114      10090          NA The mouse B myeloma ..                  1
## EH6115      10090          NA A animation file (.g..                  1
## EH6116       9606          NA The primary human T ..                  1
## EH6117       9606          NA A animation file (.g..                  1
##                    maintainer rdatadateadded preparerclass
##                   <character>    <character>   <character>
## EH6095 Satoshi Kume <satosh..     2021-05-18   BioImageDbs
## EH6096 Satoshi Kume <satosh..     2021-05-18   BioImageDbs
## EH6097 Satoshi Kume <satosh..     2021-05-18   BioImageDbs
## EH6098 Satoshi Kume <satosh..     2021-05-18   BioImageDbs
## EH6099 Satoshi Kume <satosh..     2021-05-18   BioImageDbs
## ...                       ...            ...           ...
## EH6113 Satoshi Kume <satosh..     2021-05-18   BioImageDbs
## EH6114 Satoshi Kume <satosh..     2021-05-18   BioImageDbs
## EH6115 Satoshi Kume <satosh..     2021-05-18   BioImageDbs
## EH6116 Satoshi Kume <satosh..     2021-05-18   BioImageDbs
## EH6117 Satoshi Kume <satosh..     2021-05-18   BioImageDbs
##                                          tags   rdataclass
##                                        <list>  <character>
## EH6095     3D images,bioimage,CellCulture,...         List
## EH6096     animation,bioimage,CellCulture,... magick-image
## EH6097      3D image,bioimage,CellCulture,...         List
## EH6098     animation,bioimage,CellCulture,... magick-image
## EH6099 bioimage,cell tracking,CellCulture,...         List
## ...                                       ...          ...
## EH6113 bioimage,cell tracking,CellCulture,...         List
## EH6114     2D images,bioimage,CellCulture,...         List
## EH6115     2D images,bioimage,CellCulture,... magick-image
## EH6116     2D images,bioimage,CellCulture,...         List
## EH6117     2D images,bioimage,CellCulture,... magick-image
##                     rdatapath              sourceurl  sourcetype
##                   <character>            <character> <character>
## EH6095 BioImageDbs/v01/EM_i.. https://github.com/k..         PNG
## EH6096 BioImageDbs/v01/EM_i.. https://github.com/k..         PNG
## EH6097 BioImageDbs/v01/EM_i.. https://github.com/k..         PNG
## EH6098 BioImageDbs/v01/EM_i.. https://github.com/k..         PNG
## EH6099 BioImageDbs/v01/LM_i.. https://github.com/k..         PNG
## ...                       ...                    ...         ...
## EH6113 BioImageDbs/v01/LM_i.. https://github.com/k..         PNG
## EH6114 BioImageDbs/v01/EM_i.. https://github.com/k..         PNG
## EH6115 BioImageDbs/v01/EM_i.. https://github.com/k..         PNG
## EH6116 BioImageDbs/v01/EM_i.. https://github.com/k..         PNG
## EH6117 BioImageDbs/v01/EM_i.. https://github.com/k..         PNG

We can retrieve only the BioImageDbs tibble files as follows.

qr <- query(eh, c("BioImageDbs", "LM_id0001"))
qr
## ExperimentHub with 5 records
## # snapshotDate(): 2021-05-18
## # $dataprovider: CELL TRACKING CHALLENGE (http://celltrackingchallenge.net/2...
## # $species: Homo sapiens
## # $rdataclass: List, magick-image
## # additional mcols(): taxonomyid, genome, description,
## #   coordinate_1_based, maintainer, rdatadateadded, preparerclass, tags,
## #   rdatapath, sourceurl, sourcetype 
## # retrieve records with, e.g., 'object[["EH6099"]]' 
## 
##            title                                                    
##   EH6099 | LM_id0001_DIC_C2DH_HeLa_4dTensor.rds                     
##   EH6100 | LM_id0001_DIC_C2DH_HeLa_4dTensor_train_dataset.gif       
##   EH6101 | LM_id0001_DIC_C2DH_HeLa_4dTensor_Binary.rds              
##   EH6102 | LM_id0001_DIC_C2DH_HeLa_4dTensor_Binary_train_dataset.gif
##   EH6103 | LM_id0001_DIC_C2DH_HeLa_5dTensor.rds
#Import data
#BioImageDbs_image_Dat <- qr[[1]]

3 5D Arrays from the ExperimentHub

The ordering of the array dimensions corresponds to the channels_last format (default) in R/Keras. The input shape of 5D array is to be batch, spatial_dim1, spatial_dim2, spatial_dim3 and channels. The number of this batch is the same as the number of the 3D image sets. The number of channels is 1 for grey images and 3 for RGB images.

4 4D Arrays from the ExperimentHub

The ordering of the array dimensions corresponds to the channels_last format (default) in R/Keras. The input shape of 4D array is to be batch, height, width and channels. The number of this batch is the same as the number of the 2D images.

5 Visualization of gif images from the ExperimentHub

As a test, we also provided gif files of some arrays for visualizations. We visualize the files using magick::image_read function.

qr <- query(eh, c("BioImageDbs", ".gif"))
qr
## ExperimentHub with 10 records
## # snapshotDate(): 2021-05-18
## # $dataprovider: CELL TRACKING CHALLENGE (http://celltrackingchallenge.net/2...
## # $species: Homo sapiens, Mus musculus, Drosophila melanogaster
## # $rdataclass: magick-image
## # additional mcols(): taxonomyid, genome, description,
## #   coordinate_1_based, maintainer, rdatadateadded, preparerclass, tags,
## #   rdatapath, sourceurl, sourcetype 
## # retrieve records with, e.g., 'object[["EH6096"]]' 
## 
##            title                                                            
##   EH6096 | EM_id0001_Brain_CA1_hippocampus_region_5dTensor_train_dataset.gif
##   EH6098 | EM_id0002_Drosophila_brain_region_5dTensor_train_dataset.gif     
##   EH6100 | LM_id0001_DIC_C2DH_HeLa_4dTensor_train_dataset.gif               
##   EH6102 | LM_id0001_DIC_C2DH_HeLa_4dTensor_Binary_train_dataset.gif        
##   EH6105 | LM_id0002_PhC_C2DH_U373_4dTensor_train_dataset.gif               
##   EH6107 | LM_id0002_PhC_C2DH_U373_4dTensor_Binary_train_dataset.gif        
##   EH6110 | LM_id0003_Fluo_N2DH_GOWT1_4dTensor_train_dataset.gif             
##   EH6112 | LM_id0003_Fluo_N2DH_GOWT1_4dTensor_Binary_train_dataset.gif      
##   EH6115 | EM_id0003_J558L_4dTensor_train_dataset.gif                       
##   EH6117 | EM_id0004_PrHudata_4dTensor_train_dataset.gif
#EM_id0001_Brain_CA1_hippocampus_region_5dTensor_train_data
qr[1]
## ExperimentHub with 1 record
## # snapshotDate(): 2021-05-18
## # names(): EH6096
## # package(): BioImageDbs
## # $dataprovider: https://www.epfl.ch/labs/cvlab/data/data-em/
## # $species: Mus musculus
## # $rdataclass: magick-image
## # $rdatadateadded: 2021-05-18
## # $title: EM_id0001_Brain_CA1_hippocampus_region_5dTensor_train_dataset.gif
## # $description: A animation file (.gif) of the train dataset of EM_id0001_Br...
## # $taxonomyid: 10090
## # $genome: NA
## # $sourcetype: PNG
## # $sourceurl: https://github.com/kumeS/BioImageDbs
## # $sourcesize: NA
## # $tags: c("animation", "bioimage", "CellCulture", "electron
## #   microscopy", "microscope", "scanning electron microscopy",
## #   "segmentation", "Tissue") 
## # retrieve record with 'object[["EH6096"]]'
##Display the gif image
#magick::image_read(qr[[1]])