Using Git with Bioconductor SVN repositories

Bioconductor packages are stored in the Bioconductor Subversion (SVN) repository. However Bioconductor also maintains Read-only mirrors for each package located on GitHub.

These instructions detail how to use these mirrors as well as git-svn to interact with the Subversion repositories from git.

The examples below use HTTPS authentication, however you are free to substitute using SSH if you prefer, in all cases USER should be replaced by your GitHub username, and REPO should be replaced by your package name.

Install Git-Svn Pre-Requisites

To install git with svn support, see the following OS specific instructions.

Windows

Download and install the Git windows client.

When using Windows, the below commands should be run using the Git Bash client which is bundled with git.

Mac OS X - Homebrew

git-svn should already be installed. Run the command

git svn --help

If it shows a help page, then git-svn is already installed. Otherwise, install Homebrew and try:

brew update
brew install git

Linux - Debian Based

sudo apt-get install git-svn

Download the update_remotes.sh script

This script will automatically create local branches to track the development and release branches from svn and the associated git mirrors. You can download this script to any directory and (optionally) put it in your PATH.

curl -O https://raw.githubusercontent.com/Bioconductor/mirror/master/update_remotes.sh

Setup

Scenario 1: Use Git Locally, No GitHub Repository

If you simply want to use git locally on your machine and do not need to have a publicly accessible git repository on GitHub (or elsewhere) you can simply clone your package from the mirror directly.

  1. git clone https://github.com/Bioconductor-mirror/REPO to clone the repository to your machine.
  2. cd REPO to switch to the REPO directory.
  3. bash /path/to/update_remotes.sh to setup the git remotes.
  4. Commit to git as you normally would.
  5. Each time you want to push git commits to svn run the following commands:
    1. git svn rebase to get the latest SVN changes.
    2. git svn dcommit --add-author-from to commit your changes to SVN. You may need to provide your SVN username git svn --username=u.name dcommit --add-author-from.

Scenario 2: Set Up Your Own GitHub Repository

If you do not already have a public git repository for package REPO the simplest thing to do is navigate to https://github.com/Bioconductor-mirror/REPO and click the Fork button in the upper right. This will create a copy of the repository on your personal account. You may want to re-enable issue tracking in your repository (it’s disabled in the read-only mirrors and forks inherit this setting). To do this, go to Settings and then click the Issues checkbox. Then perform the following steps in your terminal.

  1. git clone https://github.com/USER/REPO to clone the repository to your machine.
  2. cd REPO to switch to the REPO directory.
  3. bash /path/to/update_remotes.sh to setup the git remotes.
  4. Commit to git and push to GitHub as you normally would.
  5. Each time you want to push git commits to svn:
    1. git checkout devel to switch to the devel branch. (use release-X.X for release branches)
    2. git svn rebase to get the latest SVN changes.
    3. git merge master --log to merge your changes from the master branch or skip this step and work directly on the current branch.
    4. git svn dcommit --add-author-from to sync and commit your changes to svn. You may need to provide your SVN username git svn --username=u.name dcommit --add-author-from.

When you’re done, be sure and merge any changes from svn back into the git master branch:

git checkout master
git merge devel

Dealing with prior history / merge conflicts

If your Git repository has prior history before being added to SVN or you are running into merge conflicts when trying to run git svn rebase the best way forward is to git cherry-pick the new commits rather than trying to merge. The following workflow assumes your new commits are on the ‘master’ branch and the SVN tracking branch is called ‘devel’ (this is the case when using setup_remotes.sh).

# switch to the devel branch
git checkout devel

# lookup commit range to cherry-pick on the master branch. Copy the sha1 hashes
# for the range of commits you want to pick.
git log master

# cherry pick the commits from master to devel
git cherry-pick d9773cdb3b07c0..0bd4b4f4de831

# Fix any conflicts that arise, then run
git cherry-pick --continue

# both of these commands should now work without error
git svn info
git svn rebase

# You should now be able to commit to SVN with
git svn dcommit

If you run into any problems with this procedure please post a message to the Bioc-Devel mailing list or send an email to packages@bioconductor.org.

FAQs

How do I let users know I am using GitHub for development and contributions?

Add URL: https://github.com/USER/REPO and BugReports: https://github.com/USER/REPO/issues to your DESCRIPTION file. You can also mention your repository on the bioc-devel mailing list.

I don’t know my Subversion username and/or password. What do I do?

See Lost Subversion username or password.

How do I commit to the release version of my package?

If you are cloning the mirrors directly you can switch to the release-X.X branch of the release you would like to commit to, and then proceed as normal. (Current release version is 3.5.) If you are hosting on GitHub as well, rather than checking out bioc/master checkout bioc/release-X.X, then perform the rest of the steps as normal.

How do I get the SVN revision for a git commit, or the git commit from the SVN revision?

git svn find-rev can be used for both directions.

git svn find-rev r104237
# 3a5e1d5995322fb0569138930c3d2aaa93b1c54d

git svn find-rev 3a5e1d5995322fb0569138930c3d2aaa93b1c54d
# 104237

How can I search all Bioconductor code on GitHub?

Here’s an example search that looks through all Bioconductor software packages for the specified string.

So can I submit a pull request against any Bioconductor package?

No. The read-only mirrors are set up to automatically reject any pull request (unfortunately, GitHub does not let us disable them altogether). If you want to submit a pull request, you need to submit it to the repository that the maintainer of the package maintains (if they have chosen to create one). Check the BugReports or URL fields of the DESCRIPTION file to find the link to this repository.

I have a question or comment.

Please send your feedback to the bioc-devel mailing list.

Troubleshooting

Unable to determine upstream SVN information

The dreadful message indicating that git and svn got out of sync is Unable to determine upstream SVN information from working tree history. This can happen, for example if one forgets to git pull --rebase before trying to git svn dcommit and changes were committed to svn independently. Inspect your git log with git log --graph --decorate --pretty=oneline --abbrev-commit --all to help identify such cases.

Useful references to sort such cases out are:

Resources