ssh troubleshooting

In this document:



This document should help you troubleshoot ssh-related problems in installing and accessing gitolite.

This is about all the help I can give you in terms of the ssh aspect of using gitolite. If you're installing gitolite, you're a "system admin", like it or not. Ssh is therefore a necessary skill. Please take the time to learn at least enough to get passwordless access working.

I have spent more than my share of time helping people debug their misconfigured servers, simply because they tried to blame gitolite for their troubles. This stops now. I'd rather spend time on actual gitolite features, code, and documentation.

Other resources:


(common) ssh asks for a password

NOTE: This section should be useful to anyone trying to get password-less access working. It is not specific to gitolite.

You have generated a keypair on your workstation (ssh-keygen) and copied the public part of it (~/.ssh/id_rsa.pub, by default) to the server.

On the server you have appended this file to ~/.ssh/authorized_keys. Or you ran something, like the gl-setup or gl-easy-install steps during a gitolite install, which should have done that for you.

You now expect to log in without having to type in a password, but when you try, you are being asked for a password.

This is a quick checklist:


problems when using package, root, or non-root install methods

This section applies if you installed using any of the first 3 methods of install.

In these 3 modes, installation is done on the server, by logging in as some other user and doing and su - git. The admin's workstation has only one key that is known to the server's authkeys file, and this key invokes gitolite. Note that this key is not supposed to get you a shell; it is supposed to invoke gitolite.

As a result, it's a lot easier to debug. Just run ssh git@server info. If this get you the gitolite version and access info, everything is fine. If it asks you for a password, see the very first section of this document for help.

If it gets you the GNU info command output, you have shell access. This probably means you had passwordless shell access to the server before you were added as a gitolite user, and you sent that same key to your gitolite admin to include in the admin repo. This won't work -- the same key appears twice in the authkeys file now, and since the ssh server will always use the first match, the second occurrence (which invokes gitolite) is ignored.

You'll have to (create and) use a different keypair for gitolite access.

problems when using the "from-client" install method

This section applies if you installed using the from-client method.

This method of install is unique in that the admin will have 2 distinct keys to access the server. The default key (~/.ssh/id_rsa) is used to get a shell prompt and to run commands; for example, gl-easy-install uses this key to do all its server-side work.

In addition, there is a named key created just to invoke gitolite instead of starting a shell. The name is whatever you gave as the third argument to the gl-easy-install command (for example, ~/.ssh/sitaram.pub in my case).

Finally, a host gitolite para is added to ~/.ssh/config:

host gitolite
     user git
     hostname server
     identityfile ~/.ssh/sitaram

so that you can use gitolite:reponame as the URL to make ssh use the named key.

All this applies only to the admin. Normal users will only have one key and do not need any of this.

(sidebar) why two keys on client for the admin

There are two types of access the admin will make to the server: a normal login, to get a shell prompt, and gitolite access (clone/fetch/push etc). The first access needs an authkeys line without any "command=" restrictions, while the second requires a line with such a restriction.

And we can't use the same key for both because there is no way to disambiguate them; the ssh server will always (always) pick the first one in sequence when the key is offered by the ssh client.

So the next question is usually "I have other ways to get a shell on that account (like su - git from some other account), so why do I need a key for shell access at all?"

The answer to this is that the "easy install" script, being written for the most general case, needs shell access via ssh to do its stuff. If you have access otherwise, you really should use one of the other 3 install methods to install gitolite. Please see the install doc for details.

bypassing gitolite without intending to

These problems happen to the person who has utterly failed to read/heed the message that shows up at the end of running the gl-easy-install command. Both these problems are caused by using the wrong key, thus bypassing gitolite completely:

Let us recap the message that appears on a successful run of the "easy-install" program; it looks something like this (with suitable values substituted for <user>, <server>, and <port>):

IMPORTANT NOTE -- PLEASE READ!!!
    *Your* URL for cloning any repo on this server will be
        gitolite:reponame.git

    *Other* users you set up will have to use
        <user>@<server>:reponame.git
    However, if your server uses a non-standard ssh port, they should use
        ssh://<user>@<server>:<port>/reponame.git

    If this is your first time installing gitolite, please also:
        tail -31 src/gl-easy-install
    for next steps.

The first error above happens if you use git@server:reponame instead of gitolite:reponame. All your repos are actually in a subdirectory pointed to by $REPO_BASE in the rc file (default: repositories). Gitolite internally prefixes this before calling the actual git command you invoked, but since you're bypassing gitolite completely, this prefixing does not happen, and so the repo is not found.

The second error happens if you use git@server:repositories/reponame.git (assuming default $REPO_BASE setting) -- that is, you used the full unix path. Since the "prefixing" mentioned above is not required, the shell finds the repo and clones ok. But when you push, gitolite's update hook kicks in, and fails to run because some of the environment variables it is expecting are not present.

basic ssh troubleshooting for the admin

Otherwise, run these checks:

  1. ssh git@server should get you a command line without asking for a password.

    If it asks you for a password, then your id_rsa keypair changed after you ran the easy install, or someone fiddled with the ~/.ssh/authorized_keys file on the server.

    If it prints the gitolite version and access info (see doc/report-output.mkd), you managed to overwrite the id_rsa keypair with the sitaram keypair, or something equally weird. This is because a gitolite key, when used without any actual command, defaults to running gitolite's internal "info" command.

    NOTE starting with version 5.6, openssh will "Kill channel when pty allocation requests fail". This means that, instead of seeing the version and access info as described above, you may only get a message about pty allocation failure, followed by "connection closed".

  2. ssh gitolite info should print some gitolite version and access info. If you get the output of the GNU info command instead, you probably reused your id_rsa keypair as your sitaram keypair, or overwrote the sitaram keypair with the id_rsa keypair.

There are many ways to fix this, depending on where and what the damage is. The most generic way (and therefore time-taking) is to re-install gitolite from scratch:

That's a long sequence but it should work.

windows issues

On windows, I have only used msysgit, and the openssh that comes with it. Over time, I have grown to distrust putty/plink due to the number of people who seem to have trouble when those beasts are involved (I myself have never used them for any kind of git access). If you have unusual ssh problems that just don't seem to have any explanation, try removing all traces of putty/plink, including environment variables, etc., and then try again.

If you can offer an authoritative account of the complications involved, and how to resolve them and get things working, I'd be happy to credit you and include it, either directly here if it is short enough or just an external link, or in contrib/ if it's a longer piece of text.

details

Here's how it all hangs together.

files on the server

files on client

some other tips and tricks

giving shell access to gitolite users

We've managed (thanks to an idea from Jesse Keating) to make it possible for a single key to allow both gitolite access and shell access.

This is done by copying the pubkey (to which you want to give shell access) to the server and running either

cd $HOME/.gitolite  # assuming default $GL_ADMINDIR in ~/.gitolite.rc
src/gl-tool shell-add ~/foo.pub

or

gl-tool shell-add ~/foo.pub

The first method is to be used if you used the user-install mode, while the second method is for the system-install followed by user-setup mode (see doc/1-INSTALL.mkd, section on "install methods", for more on this)

IMPORTANT UPGRADE NOTE: previous implementations of this feature were crap. There was no easy/elegant way to ensure that someone who had repo admin access would not manage to get himself shell access.

Giving someone shell access requires that you should have shell access in the first place, so the simplest way is to enable it from the server side only.

losing your admin key

If you lost the admin key, and need to re-establish ownership of the gitolite-admin repository with a fresh key, take a look at the src/gl-dont-panic program. You will need shell access to the server of course. Run it without arguments to get instructions.

simulating ssh-copy-id

don't have ssh-copy-id? This is broadly what that command does, if you want to replicate it manually. The input is your pubkey, typically ~/.ssh/id_rsa.pub from your client/workstation.

[Actually, sshd requires that even directories above ~ (/, /home, typically) also must be go-w, but that needs root. And typically they're already set that way anyway. (Or if they're not, you've got bigger problems than gitolite install not working!)]