contrib/triggers/file_mirror

   1 #!/usr/bin/perl
   2 use strict;
   3 use warnings;
   4
   5 # Use an external (non-gitolite) mirror to backup gitolite repos.  They will
   6 # be automatically kept uptodate as people push to your gitolite server.  If
   7 # your server should die and you create a new one, you can quickly and easily
   8 # get everything back from the external mirror with a few simple commands.
   9
  10 #       -------------------------------------------------------------
  11 #       SEE WARNINGS/CAVEATS AND INSTRUCTIONS AT THE END OF THIS FILE
  12 #       -------------------------------------------------------------
  13
  14 # ----------------------------------------------------------------------
  15
  16 use lib $ENV{GL_LIBDIR};
  17 use Gitolite::Easy;
  18
  19 my ( $trigger, $repo, $dummy, $op ) = @ARGV;
  20 exit 0 unless $trigger eq 'POST_GIT' or $trigger eq 'POST_CREATE';
  21 exit 0 if     $trigger eq 'POST_GIT' and $op ne 'W';
  22
  23 chdir("$rc{GL_REPO_BASE}/$repo.git") or _die "chdir failed: $!\n";
  24
  25 my %config = config( $repo, "gitolite-options\\.mirror\\.extslave" );
  26 for my $slave ( values %config ) {
  27     _do($slave);
  28
  29     # processing one slave is sufficient for restoring!
  30     last if $trigger eq 'POST_CREATE';
  31 }
  32
  33 # in shell, that would be something like:
  34 #   gitolite git-config -r $repo gitolite-options\\.mirror\\.extslave | cut -f3 | while read slave
  35 #   do
  36 #       ...
  37
  38 # ----------------------------------------------------------------------
  39
  40 sub _do {
  41     my $url = shift;
  42
  43     if ( $trigger eq 'POST_CREATE' ) {
  44         # brand new repo just created; needs to be populated from mirror
  45
  46         # For your urls you will need a way to somehow query the server and
  47         # ask if the repo is present; it's upto you how you do it.
  48         my $path = $url;
  49         $path =~ s(^file://)();
  50         return unless -d $path;
  51
  52         # now fetch.  Maybe we can put a "-q" in there?
  53         system( "git", "fetch", $url, "+refs/*:refs/*" );
  54
  55     } elsif ( $trigger eq 'POST_GIT' ) {
  56         # someone just pushed; we need to update our mirrors
  57
  58         # need to create the repo on the mirror.  Again, it's upto you how you
  59         # make sure there's a repo on the mirror that can receive the push.
  60         make_repo($url);    # in case it doesn't already exist
  61
  62         # now push
  63         system( "git", "push", "--mirror", $url );
  64     }
  65 }
  66
  67 sub make_repo {
  68     my $url = shift;
  69     # in this example, the URL is 'file:///...'; for other urls, presumably
  70     # the url tells you enough about how to *create* a repo.
  71
  72     my $path = $url;
  73     $path =~ s(^file://)();
  74     return if -d $path;
  75     system( "git", "init", "--bare", $path );
  76 }
  77
  78 __END__
  79
  80 WARNINGS
  81 --------
  82
  83 1.  THIS IS SAMPLE CODE.  You will AT LEAST have to customise the _do() and
  84     make_repo() functions above based on what your remote URLs are.  For
  85     example, I don't even know how to create a repo from the command line if
  86     your external store is, say, github!
  87
  88 2.  THIS DOES NOT WORK FOR WILD REPOs.  It can be made to work, with a few
  89     extra steps to backup and restore the "gl-perms" and "gl-creator" files.
  90
  91     "Left as an exercise for the reader!"
  92
  93 DESIGN NOTES
  94 ------------
  95
  96 This is really just a combination of "upstream" (see src/triggers/upstream)
  97 and mirroring (gitolite mirroring does allow a slave to be non-gitolite, as
  98 long as the ssh stuff is done the same way).
  99
 100 The main difference is that gitolite mirroring expects peers to all talk ssh,
 101 whereas this method lets you use other protocols.  Specifically, since this
 102 whole thing was started off by someone wanting to put his repos on s3
 103 (apparently jgit can talk to s3 directly), you can modify the two functions to
 104 deal with whatever remote server you have.
 105
 106 LANGUAGE
 107 --------
 108
 109 This doesn't have to be in perl.  Shell equivalent for the only gitolite
 110 specific code is supplied; the rest of the code is fairly straightforward.
 111
 112 SETUP
 113 -----
 114
 115 1.  Put this code into your LOCAL_CODE directory under "triggers"; see
 116     non-core.html for details.
 117
 118 2.  Add these lines to your rc file, just before the ENABLE line.  (I'm
 119     assuming a v3.4 or later installation here).
 120
 121         POST_CREATE => [ 'file_mirror' ],
 122         POST_GIT => [ 'file_mirror' ],
 123
 124 3.  Backup your rc file, since you may have other changes in it that you'll
 125     want to preserve.
 126
 127 4.  Do something like this in your gitolite.conf file:
 128
 129         repo @all
 130             option mirror.extslave-1    =   file:///tmp/he1/%GL_REPO.git
 131             option mirror.extslave-2    =   file:///tmp/he2/%GL_REPO.git
 132
 133     As you can see, since this is just for demo/test, we're using a couple of
 134     temp directories to serve as our "remotes" using the file:// protocol.
 135
 136 5.  Do a one-time manual sync of all the repos (subsequent syncs happen on
 137     each push):
 138
 139         gitolite list-phy-repos | xargs -I xx gitolite trigger POST_GIT xx admin W
 140
 141     (This is a little trick we're playing on the trigger stuff, but it should
 142     work fine.  Just make sure that, if you have other things in your POST_GIT
 143     trigger list, they're not affected in some way.  'gitolite query-rc
 144     POST_GIT' will tell you what else you have.)
 145
 146 That takes care of the "setup" and "regular backup".
 147
 148 RESTORE
 149 -------
 150
 151 1.  Install gitolite normally.  You'll get the usual two repos.
 152
 153 2.  Restore the previously backed up rc file to replace the default one that
 154     gitolite created.  At the very least, the rc file should have the
 155     POST_CREATE and POST_GIT entries.
 156
 157         ---------------------------------------------------------
 158         IF YOU FORGET THIS STEP, NASTY THINGS WILL HAPPEN TO YOU!
 159         ---------------------------------------------------------
 160
 161 3.  Clone the admin repo from one of your backup servers to some temp dir.  In
 162     our example,
 163
 164         git clone /tmp/he1/gitolite-admin.git old-ga
 165
 166 4.  'cd' to that clone and force push to your *new* admin repo:
 167
 168         cd old-ga
 169         git push -f admin:gitolite-admin
 170
 171 That's it.  As each repo gets created by the admin push, they'll get populated
 172 by the backed up stuff due to the POST_CREATE trigger.