misc

[gitolite-doc.git] / dev-notes.mkd

<!-- options: toc -->

% writing your own "non-core" programs

include sidebar-toc

Gitolite has a huge bunch of existing features that gradually need to be moved
over.  Plus you may want to write your own programs to interact with it.

**This page is about *writing* hooks, commands, triggers, VREFS, and sugar
scripts.  *Installing* them, including "where and how", is described
[here][localcode]**.

Note: the [non-core][] page is the starting point for all information about
customising gitolite.

# environment variables and other inputs

In general, the following environment variables should always be available:

    GL_BINDIR
    GL_REPO_BASE
    GL_ADMIN_BASE

`GL_BINDIR` is loosely equivalent to GIT\_EXEC\_PATH in git.  `GL_REPO_BASE`
is always \$HOME/repositories, and `GL_ADMIN_BASE` is \$HOME/.gitolite.  (You
might ask why, if they're fixed values, do we need those variables.  Good
question... next!)

In addition, commands invoked by a remote client also have `GL_USER`, while
hooks have `GL_USER` as well as `GL_REPO`.

A special form of the [option][options] syntax can be used to set
[repo-specific environment variables][rsev].

Finally, note that triggers get a lot of relevant information from gitolite as
arguments; see [here][triggers] for details.

# APIs

## the shell API

The following commands exist to help you write shell scripts that interact
easily with gitolite.  Each of them responds to `-h` so please run that for
more info.

  * `gitolite access` to check access rights given repo, user, type of access
    (R, W, ...) and refname (optional).  Example use: src/commands/desc.

  * `gitolite creator` to get/check the creator of a repo.  Example use:
    src/commands/desc.

  * `gitolite git-config` to check gitolite options or git config variables
    directly from gitolite's "compiled" output, (i.e., without looking at the
    actual `repo.git/config` file or using the `git config` command).  Example
    use: src/triggers/post-compile/update-gitweb-access-list.

  * `gitolite query-rc` to check the value of an RC variable.  Example use:
    src/commands/desc.

In addition, you can also look at the comments in src/lib/Gitolite/Easy.pm
(the perl API module) for ideas.

## the perl API

...is implemented by Gitolite::Easy; the comments in src/lib/Gitolite/Easy.pm
serve as documentation.

Note that some of the perl functions called by Easy.pm will change the current
directory to something else, without saving and restoring the directory.
Patches (to Easy.pm *only*) welcome.

# writing your own...

## ...commands

Commands are standalone programs, in any language you like.  They simply
receive the arguments you append.  In addition, the env var `GL_USER` is
available if it is being run remotely.  src/commands/desc is the best example
at present.

## ...hooks

### anything but the update hook

If you want to add any hook other than the update hook, 'man githooks' is all
you need.

### update hook

If you want to add additional `update` hook functionality, do this:

  * Write and test your update hook separately from gitolite.

  * Now add the code as a VREF (see [here][localcode] for details).  Let's say
    you called it "foo".

    NOTE: if you don't care about all the extra stuff that VREFs give you over
    a normal update hook, you don't have to read that page at all; just put
    the code in the right place and go to the next step.

  * To call your new update hook to all accesses for all repos, add this to
    the end of your conf file:
.#! conf/vim-color
        repo @all
            -       VREF/foo        =   @all
.end

As you probably guessed, you can make your additional update hooks more
selective, applying them only to some repos / users / combinations.

Note: a normal update hook expects 3 arguments (ref, old SHA, new SHA).  A
VREF will get those three, followed by at least 4 more.  Your VREF should just
ignore the extra args.

## ...trigger programs

Trigger programs run at specific points in gitolite's execution, with specific
arguments being passed to them.  See the [triggers][] page for details.

You can write programs that are both manually runnable as well as callable by
trigger events, especially if they don't *need* any arguments.

## ..."sugar"

Syntactic sugar helpers are NOT complete, standalone, programs.  They must
include a perl sub called `sugar_script` that takes in a listref, and returns
a listref.  The listrefs point to a list that contains the entire conf file
(with all [include][] processing already done).  You create a new list with
contents modified as you like and return a ref to it.

There are a couple of examples in src/syntactic-sugar.

# appendix 1: repo-specific environment variables {#rsev}

A special form of the [option][options] syntax can be used to set
repo-specific environment variables that are visible to gitolite triggers and
any git hooks you may install.

For example, let's say you installed a post-update hook that initiates a CI
job.  By default, of course, this hook will be active for *all*
gitolite-managed repos.  However, you only want it to run for some specific
repos, say r1, r2, and r4.

To do that, first add this to the gitolite.conf:

.#! conf/vim-color
    repo r1 r2 r4
        option ENV.CI = 1
.end

This creates an environment variable called `GL_OPTION_CI` with the value 1,
before any trigger or hook is invoked.

Note: option names *must* start with `ENV.`, followed by a seqence of
characters composed of alphas, numbers, and the underscore character.

Now the hook running the CI job can easily decide what to do:

    # exit if $GL_OPTION_CI is not set
    [ -z $GL_OPTION_CI ] && exit

    ... rest of CI job code as before ...

Of course you can also do the opposite; i.e. decide that the listed repos
should *not* run the CI job but all other repos should:

.#! conf/vim-color
    repo @all
        option ENV.CI = 1

    repo r1 r2 r4
        option ENV.CI = ""
.end

(The hook code remains the same as before.)

<font color="gray">Before this feature was added, you could still do this, by
using the `gitolite git-config` command inside the hook code to test for
options and configs set for the repo, like:

    if gitolite git-config -q reponame gitolite-options.option-name
    then
        ...

The new method is much more convenient, as you can see.</font>

# appendix 2: log file format {#lff}

Here's a brief description of gitolite's log file format.  All fields are tab
separated.

There are two kinds of lines in the log file:

  * third field non-empty: actual log lines.  These are documented below.

  * third field empty (i.e., two tabs after the second field): extra output to
    help diagnose problems. These are NOT documented and may change without
    notice.  They can be turned off completely by setting the RC variable
    `LOG_EXTRA` to 0.

For all types of log lines, the first two fields are:

  * field 1: local time, in YYYY-MM-DD.HH:MM:SS format

  * field 2: transaction ID or "TID".  This is actually the PID of the
    outermost command that was initiated (usually "gitolite-shell").  It helps
    to keep log lines pertaining to one "run" together, even if several
    sub-commands are spawned (like for example from triggers, or even the
    update hook itself).

The third and later fields are all dependent on what type of log line it is.

The various log line formats are:

  * **start**

    This line denotes the beginning of a gitolite operation.

      * field 3: 'ssh' or 'http'

      * field 4: `ARGV=<comma-separated list of command line arguments>`

        Usually this is just the gitolite username (the argument to
        gitolite-shell, as supplied by the forced command in the authorized
        keys file).  If you're [giving shell access][giving-shell] to some
        users, this would be "-s,username".  That's two command line arguments
        ("-s" and the username), comma separated.

      * field 5: `SOC=<original command from client>`

        This is the command exactly as the git client sent it, or the user
        typed it.  Typically this is one of these:

            git-upload-pack 'reponame'
            git-receive-pack 'reponame'

        (including the single quotes, because that's what the client sent).

        Gitolite commands are also recorded as is, so this could be something
        like `info` or perhaps `perms -l reponame` etc.

      * field 6: `FROM=<ip address>`

  * **pre-git**

    This log line appears when the first access check succeeds (i.e., before
    git-upload-pack or git-receive-pack is called).

      * field 3: 'pre\_git'
      * field 4: reponame
      * field 5: username
      * field 6: 'R' or 'W'
      * field 7: 'any'
      * field 8: the refex that allowed access

  * **update**

    This log line appears when the second access check succeeds (i.e., when
    the update hook decides to allow the push).

      * field 3: 'update'
      * field 4: reponame
      * field 5: username
      * field 6: 'W', '+', 'C', 'D', 'WM', '+M', 'CM', or 'DM'
      * field 7: ref name (like 'refs/heads/master')
      * field 8: old SHA
      * field 9: new SHA
      * field 10: the refex that allowed access

    The values in field 6 reflect the possible [write types][write-types], but
    note that there is a difference between what the log file contains and
    what the gitolite.conf file contains (e.g., `+` versus `RW+`).  There's
    another subtle difference for those who are not thinking clearly: the
    `RW+` in the conf file is a permission, but it would show up as a `+` in
    the log file only if an *actual* force push had happened, otherwise it
    would be just `W`.

    By the way, notice that fields 7 to 9 are exactly what git itself supplies
    the update hook (see 'man githooks').

    <font color="gray">There is a special version of this line that appears
    when someone [bypasses][bypass] gitolite's access control to push directly
    on the server.  The 'reponame' (field 4) is replaced by the full path of
    the repo being pushed to, the username (field 5) is replaced by the Unix
    userid in parentheses, and the operation code (field 6) is
    'bypass'.</font>

  * **create**

    This log line appears when a wild repo was auto-created by a user.

      * field 3: 'create'
      * field 4: reponame
      * field 5: username
      * field 6: 'R' or 'W'

    Field 6 is 'perms-c' if the wild repo is created using the perms command's
    '-c' option.

  * **end**

    This indicates the end of the transaction.  Normally, you should not see
    any more lines with the same TID after this line.

      * field 3: 'END'

  * **warnings** and **errors**

    Typically seen when access is denied.

      * field 3: 'warn' or 'die'
      * field 4: message.  Parts of the message (like reponame, username, etc)
        are not split out into fields, though.

  * **cli**

    This logs gitolite sub-commands run directly on the server, like `gitolite
    setup` etc.

      * field 3: 'cli'
      * field 4: 'gitolite'
      * field 5 to end: arguments supplied to gitolite command, one per field

# appendix 3: @@gray((v3.6.1+))@@ syslog

Gitolite allows you to send log entries to syslog.  To do that, uncomment one
of the commented out values for LOG\_DEST in the rc file.  If your rc file
does not have any such lines, add one of the following lines just after the
LOG\_EXTRA line:

    # use this to log only to syslog
    LOG_DEST                        => 'syslog',

    # use this to log to syslog and the normal gitolite log
    LOG_DEST                        => 'syslog,normal',

Please note:

*   The first two fields described in the previous section (time, TID) are
    different.  Syslog takes care of putting in the time, and the TID is
    appended to the ident, so instead of just 'gitolite', you get
    'gitolite[$GL_TID]'.

    This means actual log lines will look something like this, since syslog
    appends the actual PID to the ident as well:

        Jun  3 09:26:11 sita-lt gitolite[14950][14950]: ssh ARGV=admin  SOC=git-receive-pack 'gitolite-admin'   FROM=::1
        Jun  3 09:26:11 sita-lt gitolite[14950][14950]: pre_git gitolite-admin  admin   W   any refs/.*
        Jun  3 09:26:11 sita-lt gitolite[14950][14991]: update  gitolite-admin  admin   W   refs/heads/master   [snip]
        Jun  3 09:26:13 sita-lt gitolite[14950][14950]: END

*   Normal log messages use the 'info' priority, while LOG\_EXTRA messages
    (see previous section) use the 'debug' priority.

    It may be useful to send the debug output to a different output file.
    Unlike in the normal gitolite logs, where there is an extra tab character
    (or, an empty field, depending on how you look at it), the syslog lines do
    not let you easily distinguish between the main log lines and the
    LOG\_EXTRA lines.