Managing GitHub credentials from R, difficulty level linux

There are days when I regret switching to linux as an R user. It’s not that I’m particularly enamoured of Apple or Microsoft, and I do enjoy the freedom to tinker that linux systems provide, but without the same resourcing that underpins Windows or Mac OS, I do spent a disproportionate amount my time trying to make my long-suffering Ubuntu laptop do something that would “just work” if I’d gone with one of the more traditional options. But such is life, and besides, there’s a case to be made that the time I spend on these things is not wasted: usually, I end up learning something useful.

La la la la la. (Figure from giphy.com)

This is one of those stories.

The story is quite short…

Using GitHub credentials with R

For some years now I have been using git repositories for version control, with some ambivalence to my feelings. I absolutely love version control, and I think GitHub is a fabulous tool, but git itself gives me headaches. It feels counterintuitive and untidy, and I am resistant to learning new git tricks because of that. However, now that GitHub is moving to end password authentication for git operations, I find myself needing to do precisely that. Sigh.

Like many R users, whenever I encounter a git problem my first impulse is to see whether Happy Git and GitHub for the useR (Bryan 2018) can help me out, and true to form, it can. Having decided that I will revert to being an https girl, renouncing my flirtation with ssh, I’ve found the chapter on caching https credentials extremely useful. The usethis article on git credentials is also worth the read.

The problem can be broken into three parts:

• How do I set up an authentication token on my GitHub account?
• How do I configure my git installation to use the authentication token?
• How do I ensure that R detects these credentials?

Thanks to the fabulous work of the tidyverse team, it’s possible for R users to solve the problem in a fairly painless way. The solution has been documented repeatedly, but for the sake of completeness I’ll repeat the advice here.

Setting up the credentials

The first thing you’ll need to do is set up a GitHub token. You can do this on the GitHub website, but for an R user it’s probably easiest to use the usethis package (Wickham and Bryan 2021):

usethis::create_github_token()

This will open GitHub in a browser window, take you to the “create a new token page”, and pre-populate all the fields with sensible default values. After accepting these values, the token is created and you’ll be given a PAT, a “personal authentication token”. It’ll look something like this…

ghp_dgdfasdklfjsdklfjsadfDKFJASDLKFJ3453

…and you should immediately save this in a secure password manager, like 1password, lastpass, etc, because GitHub will only show it to you this one time.


You did save it to your password manager, right?


Right?


I mean, you might need it again.


You really might.


Yes, you.


All right then.


I’ll trust you’ve taken sensible precautions now, so let’s keep going. The next step in the process is to configure your git installation to use your token. This is, once again, quite easy to do with gitcreds (Csárdi 2020):

gitcreds::gitcreds_set()

When you call this function interactively, R will ask for your PAT. Paste it into the console, hit enter, and you are done. Your git installation is now configured to use the token. Yay! Let’s move onto the third step, which is to ensure that R will recognise and use these credentials. As it turns out, step three doesn’t require you to do anything, because it happens automatically! Functions like usethis::pr_push() recognise your credentials as soon as gitcreds sets them up, and everything works perfectly…

Quinn. (Figure from giphy.com)

… unless you’re on linux

If you’re on linux, you might find yourself in the same boat I was. The credentials you just set up work flawlessly for about 15 minutes, at which time R complains that it cannot find any credentials and you spend the next 15 minutes crying melodramatically.

When this happened to me I assumed the problem was my R environment. I tried updating gitcreds, usethis, and every other R package I could think of that might possibly be involved in communicating with git. Nothing worked. The reason nothing worked is that the problem wasn’t with R at all… it was git, and in hindsight I realise that the problem is specific to git on linux. All those beautiful people with their fancy Windows and Mac machines won’t run into the problem I encountered. They won’t spend an entire Saturday trying to teach themselves git credential management. They will never know my pain. Curse them and their superior purchasing decisions.

Daria. (Figure from giphy.com)

Just kidding. I love my quirky little Ubuntu box and I have a lot of fun learning how to fix her up every time she sets herself on fire.

Where did I leave my config?

Okay, I’m going to need to make changes to my git configuration. Although git makes it possible to store configuration locally, at the repository level, I rarely need this flexibility. The relevant information is stored in the global configuration file: on my machine, this is located at /home/danielle/.gitconfig. I can use git config to list these configuration settings, like this

git config --global --list

and at the start of this exercise the output would have looked like this:

user.name=Danielle Navarro
user.email=d.navarro@unsw.edu.au

I’m not sure why this is, but I always feel slightly more reassured when I’m able to inspect the configuration file itself. Opening my .gitconfig file shows the same information, but the formatting is slightly different in the raw file:

[user]
    name = Danielle Navarro
    email = d.navarro@unsw.edu.au

To solve the git credential problem, we’re going to need to edit this configuration information. Depending on which solution you go with, you might need to install new software too.

Don’t forget to update git

Before starting, it’s a good idea to make sure you have the latest version of git: older versions may not have the tools you need. As it happens, I had already updated git to the most recent version (2.32.0 at the time of writing), but in case anyone ends up relying on this post, here’s how you do it:

sudo add-apt-repository ppa:git-core/ppa
sudo apt update
sudo apt install git

Three solutions

1. Set a long timeout for the git cache

Recent versions of git are released with a credential cache that retains your credentials in memory temporarily. The information is never written to disk, and it expires after a time. You can tell git to use this cache as your “credential helper” by typing the following command at the terminal:

git config --global credential.helper cache

After doing this, my .gitconfig file now looks like this:

[user]
    name = Danielle Navarro
    email = d.navarro@unsw.edu.au
[credential]
    helper = cache

Unfortunately this isn’t an ideal solution, because the cache expires after 900 seconds (15 minutes). As soon as the cache expires, git loses track of your GitHub credentials and so does R. So you have to set the credentials again by calling gitcreds::gitcreds_set() and entering the PAT again. That’s annoying, but you did store the PAT in a password manager right? You were smart. You definitely aren’t going to be foolish like me, forget to store your PAT every time, and end up needing to create a new GitHub token every 15 minutes.

A simple solution to this problem is to ask git to store information in the cache for just a teeny tiny little bit longer. Instead of having the cache expire after the default 900 seconds, maybe set it to expire after 10 million seconds. That way, you’ll only have to refresh the cache using gitcreds::gitcreds_set() once every four months instead of four times an hour. Implementing this solution requires only one line of code at the terminal:

git config --global credential.helper 'cache --timeout=10000000'

After typing this, my .gitconfig file looks like this:

[user]
    name = Danielle Navarro
    email = d.navarro@unsw.edu.au
[credential]
    helper = cache --timeout=10000000

In some ways this is a bit of a hack. If cache expiry normally happens every 15 minutes, there’s something a little odd about dragging it out and making it hang around for 16 weeks. That being said, I’ve done many stranger things than this in my life. It may not be the most elegant way to solve the problem, but it works.

Trent. (Figure from giphy.com)

2. Use libsecret credential manager

It puzzled me slightly that this problem only exists for linux computers, so I did a little more reading on how git manages credentials. It turns out you don’t have to rely on the in-memory cache: you can tell git to use some other program to supply the credentials. This is what all those swanky Mac and Windows people have been doing all along. On Macs, for example, git defaults to using the OS X keychain to store credentials safely on disk. It’s possible to do the same thing on linux using libsecret (source on gitlab) and thankfully it’s not much harder to set this up than to use the “long cache” trick described in the previous section.

The first step is ensuring libsecret is installed on your machine. It probably is (or at least, it was on my Ubuntu 20.04 box), but in case it isn’t here’s the command you need

sudo apt install libsecret-1-0 libsecret-1-dev

It helps to realise that libsecret isn’t an application designed to work with git (i.e., it’s not the credential manager), nor is it the keyring where the passwords are stored. Rather, it’s a library that communicates with the keyring: I found this post useful for making sense of it. So if we want to use libsecret to access the keyring, we’re going to need a git credential manager that knows how to talk to libsecret. As it turns out, git comes with one already, you just have to build it using make:

cd /usr/share/doc/git/contrib/credential/libsecret
sudo make

This will build the git-credential-libsecret application for you and now all you have to do is tell git to use this as the “credential helper” application that supplies the GitHub credentials:

git config --global credential.helper \
  /usr/share/doc/git/contrib/credential/libsecret/git-credential-libsecret

After typing that, my .gitconfig file looks like this…

[user]
    name = Danielle Navarro
    email = d.navarro@unsw.edu.au
[credential]
    helper = /usr/share/doc/git/contrib/credential/libsecret/git-credential-libsecret

… and I’m all set and ready to go.

One thing I found handy during this step is to check that R was reading the correct configuration information. It’s possible to do this with gitcreds:

gitcreds::gitcreds_list_helpers()

[1] "/usr/share/doc/git/contrib/credential/libsecret/git-credential-libsecret"

In any case, if all the applications are talking to each other properly, the next time you call gitcreds::gitcreds_set() they’ll all send the message along: R will pass your PAT to git, git will pass it to git-credential-libsecret, git-credential-libsecret will pass it to libsecret, and the PAT will end up in your linux keychain. Whenever you need to authenticate and push some commits up to GitHub from R, it should find the credentials using the same communication channel. Everything should work swimmingly.

Quinn et al. (Figure from giphy.com)

3. Use GCM core

As far as I can tell, the libsecret credential manager is a perfectly good solution to the problem, but in the end I made a different choice: I decided to go with “git credential manager core”, or GCM Core. It’s developed by Microsoft and, perhaps unsurprisingly, it is what GitHub currently recommends. It’s slightly more painful to set up, and the installation instructions are different depending on what flavour of linux you’re running. Because I’m on Ubuntu 20.04, I downloaded the .deb file associated with the most recent release of GCM core, and then installed the application using the dpkg command:

sudo dpkg -i <path-to-deb-file>

This will build GCM core on your system, and once that’s done you can ask it to take care of the git configuration for you:

git-credential-manager-core configure

This will edit the .gitconfig file, so for me it now looks like this:

[user]
    name = Danielle Navarro
    email = d.navarro@unsw.edu.au
[credential]
    helper = 
    helper = /usr/bin/git-credential-manager-core
[credential "https://dev.azure.com"]
    useHttpPath = true

In a happier world you would be done at this point, but we don’t live in a happy world. We live in a sick sad world that has global pandemics and pineapple on pizzas. So there’s still one job left to do.

Much like the libsecret credential manager I built in the previous section, GCM core is “just” a git credential manager: it communicates with git, but it isn’t a password manager or a keyring, and it doesn’t store the PAT itself. Instead, it offers you several different options for how the PAT is to be stored. If you click through and take a look at the list, the first suggested option is to connect to a secret service API. As far as I can tell “secret service” isn’t an application, it’s a specification, and in practice it’s just a fancy way of referring to a linux keychain. Just as the libsecret credential manager needs some way of communicating with the keychain (i.e., the libsecret library itself), GCM core needs an intermediary. In fact, it turns out GCM core also uses libsecret to talk to the keychain. So that’s the option I went with. The terminal command to set this up is this:

git config --global credential.credentialStore secretservice

After running the command, my .gitconfig file looks like this:

[user]
    name = Danielle Navarro
    email = d.navarro@unsw.edu.au
[credential]
    helper = 
    helper = /usr/bin/git-credential-manager-core
    credentialStore = secretservice
[credential "https://dev.azure.com"]
    useHttpPath = true

Jane. (Figure from giphy.com)

As before, I can check that R is reading the correct configuration information…

gitcreds::gitcreds_list_helpers()

[1] "/usr/share/doc/git/contrib/credential/libsecret/git-credential-libsecret"

…and now I’m ready to go. My problems are solved. The sun is shining, the birds are singing, and git is working properly from R again. All is well in heaven and earth. Oh the sheer excitement of it all. I hope I can contain my boundless enthusiasm and joy.

Daria. (Figure from giphy.com)

References

Bryan, Jennifer. 2018. Happy Git and GitHub for the useR. GitHub.

Csárdi, Gábor. 2020. Gitcreds: Query ’Git’ Credentials from ’r’. https://CRAN.R-project.org/package=gitcreds.

Wickham, Hadley, and Jennifer Bryan. 2021. Usethis: Automate Package and Project Setup. https://CRAN.R-project.org/package=usethis.