technical/hash-function-transition.html

   1 <!DOCTYPE html>
   2 <html xmlns="http://www.w3.org/1999/xhtml" lang="en">
   3 <head>
   4 <meta charset="UTF-8"/>
   5 <meta http-equiv="X-UA-Compatible" content="IE=edge"/>
   6 <meta name="viewport" content="width=device-width, initial-scale=1.0"/>
   7 <meta name="generator" content="Asciidoctor 2.0.20"/>
   8 <title>Git hash function transition</title>
   9 <link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700"/>
  10 <style>
  11 /*! Asciidoctor default stylesheet | MIT License | https://asciidoctor.org */
  12 /* Uncomment the following line when using as a custom stylesheet */
  13 /* @import "https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700"; */
  14 html{font-family:sans-serif;-webkit-text-size-adjust:100%}
  15 a{background:none}
  16 a:focus{outline:thin dotted}
  17 a:active,a:hover{outline:0}
  18 h1{font-size:2em;margin:.67em 0}
  19 b,strong{font-weight:bold}
  20 abbr{font-size:.9em}
  21 abbr[title]{cursor:help;border-bottom:1px dotted #dddddf;text-decoration:none}
  22 dfn{font-style:italic}
  23 hr{height:0}
  24 mark{background:#ff0;color:#000}
  25 code,kbd,pre,samp{font-family:monospace;font-size:1em}
  26 pre{white-space:pre-wrap}
  27 q{quotes:"\201C" "\201D" "\2018" "\2019"}
  28 small{font-size:80%}
  29 sub,sup{font-size:75%;line-height:0;position:relative;vertical-align:baseline}
  30 sup{top:-.5em}
  31 sub{bottom:-.25em}
  32 img{border:0}
  33 svg:not(:root){overflow:hidden}
  34 figure{margin:0}
  35 audio,video{display:inline-block}
  36 audio:not([controls]){display:none;height:0}
  37 fieldset{border:1px solid silver;margin:0 2px;padding:.35em .625em .75em}
  38 legend{border:0;padding:0}
  39 button,input,select,textarea{font-family:inherit;font-size:100%;margin:0}
  40 button,input{line-height:normal}
  41 button,select{text-transform:none}
  42 button,html input[type=button],input[type=reset],input[type=submit]{-webkit-appearance:button;cursor:pointer}
  43 button[disabled],html input[disabled]{cursor:default}
  44 input[type=checkbox],input[type=radio]{padding:0}
  45 button::-moz-focus-inner,input::-moz-focus-inner{border:0;padding:0}
  46 textarea{overflow:auto;vertical-align:top}
  47 table{border-collapse:collapse;border-spacing:0}
  48 *,::before,::after{box-sizing:border-box}
  49 html,body{font-size:100%}
  50 body{background:#fff;color:rgba(0,0,0,.8);padding:0;margin:0;font-family:"Noto Serif","DejaVu Serif",serif;line-height:1;position:relative;cursor:auto;-moz-tab-size:4;-o-tab-size:4;tab-size:4;word-wrap:anywhere;-moz-osx-font-smoothing:grayscale;-webkit-font-smoothing:antialiased}
  51 a:hover{cursor:pointer}
  52 img,object,embed{max-width:100%;height:auto}
  53 object,embed{height:100%}
  54 img{-ms-interpolation-mode:bicubic}
  55 .left{float:left!important}
  56 .right{float:right!important}
  57 .text-left{text-align:left!important}
  58 .text-right{text-align:right!important}
  59 .text-center{text-align:center!important}
  60 .text-justify{text-align:justify!important}
  61 .hide{display:none}
  62 img,object,svg{display:inline-block;vertical-align:middle}
  63 textarea{height:auto;min-height:50px}
  64 select{width:100%}
  65 .subheader,.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{line-height:1.45;color:#7a2518;font-weight:400;margin-top:0;margin-bottom:.25em}
  66 div,dl,dt,dd,ul,ol,li,h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6,pre,form,p,blockquote,th,td{margin:0;padding:0}
  67 a{color:#2156a5;text-decoration:underline;line-height:inherit}
  68 a:hover,a:focus{color:#1d4b8f}
  69 a img{border:0}
  70 p{line-height:1.6;margin-bottom:1.25em;text-rendering:optimizeLegibility}
  71 p aside{font-size:.875em;line-height:1.35;font-style:italic}
  72 h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{font-family:"Open Sans","DejaVu Sans",sans-serif;font-weight:300;font-style:normal;color:#ba3925;text-rendering:optimizeLegibility;margin-top:1em;margin-bottom:.5em;line-height:1.0125em}
  73 h1 small,h2 small,h3 small,#toctitle small,.sidebarblock>.content>.title small,h4 small,h5 small,h6 small{font-size:60%;color:#e99b8f;line-height:0}
  74 h1{font-size:2.125em}
  75 h2{font-size:1.6875em}
  76 h3,#toctitle,.sidebarblock>.content>.title{font-size:1.375em}
  77 h4,h5{font-size:1.125em}
  78 h6{font-size:1em}
  79 hr{border:solid #dddddf;border-width:1px 0 0;clear:both;margin:1.25em 0 1.1875em}
  80 em,i{font-style:italic;line-height:inherit}
  81 strong,b{font-weight:bold;line-height:inherit}
  82 small{font-size:60%;line-height:inherit}
  83 code{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;font-weight:400;color:rgba(0,0,0,.9)}
  84 ul,ol,dl{line-height:1.6;margin-bottom:1.25em;list-style-position:outside;font-family:inherit}
  85 ul,ol{margin-left:1.5em}
  86 ul li ul,ul li ol{margin-left:1.25em;margin-bottom:0}
  87 ul.circle{list-style-type:circle}
  88 ul.disc{list-style-type:disc}
  89 ul.square{list-style-type:square}
  90 ul.circle ul:not([class]),ul.disc ul:not([class]),ul.square ul:not([class]){list-style:inherit}
  91 ol li ul,ol li ol{margin-left:1.25em;margin-bottom:0}
  92 dl dt{margin-bottom:.3125em;font-weight:bold}
  93 dl dd{margin-bottom:1.25em}
  94 blockquote{margin:0 0 1.25em;padding:.5625em 1.25em 0 1.1875em;border-left:1px solid #ddd}
  95 blockquote,blockquote p{line-height:1.6;color:rgba(0,0,0,.85)}
  96 @media screen and (min-width:768px){h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2}
  97 h1{font-size:2.75em}
  98 h2{font-size:2.3125em}
  99 h3,#toctitle,.sidebarblock>.content>.title{font-size:1.6875em}
 100 h4{font-size:1.4375em}}
 101 table{background:#fff;margin-bottom:1.25em;border:1px solid #dedede;word-wrap:normal}
 102 table thead,table tfoot{background:#f7f8f7}
 103 table thead tr th,table thead tr td,table tfoot tr th,table tfoot tr td{padding:.5em .625em .625em;font-size:inherit;color:rgba(0,0,0,.8);text-align:left}
 104 table tr th,table tr td{padding:.5625em .625em;font-size:inherit;color:rgba(0,0,0,.8)}
 105 table tr.even,table tr.alt{background:#f8f8f7}
 106 table thead tr th,table tfoot tr th,table tbody tr td,table tr td,table tfoot tr td{line-height:1.6}
 107 h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2;word-spacing:-.05em}
 108 h1 strong,h2 strong,h3 strong,#toctitle strong,.sidebarblock>.content>.title strong,h4 strong,h5 strong,h6 strong{font-weight:400}
 109 .center{margin-left:auto;margin-right:auto}
 110 .stretch{width:100%}
 111 .clearfix::before,.clearfix::after,.float-group::before,.float-group::after{content:" ";display:table}
 112 .clearfix::after,.float-group::after{clear:both}
 113 :not(pre).nobreak{word-wrap:normal}
 114 :not(pre).nowrap{white-space:nowrap}
 115 :not(pre).pre-wrap{white-space:pre-wrap}
 116 :not(pre):not([class^=L])>code{font-size:.9375em;font-style:normal!important;letter-spacing:0;padding:.1em .5ex;word-spacing:-.15em;background:#f7f7f8;border-radius:4px;line-height:1.45;text-rendering:optimizeSpeed}
 117 pre{color:rgba(0,0,0,.9);font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;line-height:1.45;text-rendering:optimizeSpeed}
 118 pre code,pre pre{color:inherit;font-size:inherit;line-height:inherit}
 119 pre>code{display:block}
 120 pre.nowrap,pre.nowrap pre{white-space:pre;word-wrap:normal}
 121 em em{font-style:normal}
 122 strong strong{font-weight:400}
 123 .keyseq{color:rgba(51,51,51,.8)}
 124 kbd{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;display:inline-block;color:rgba(0,0,0,.8);font-size:.65em;line-height:1.45;background:#f7f7f7;border:1px solid #ccc;border-radius:3px;box-shadow:0 1px 0 rgba(0,0,0,.2),inset 0 0 0 .1em #fff;margin:0 .15em;padding:.2em .5em;vertical-align:middle;position:relative;top:-.1em;white-space:nowrap}
 125 .keyseq kbd:first-child{margin-left:0}
 126 .keyseq kbd:last-child{margin-right:0}
 127 .menuseq,.menuref{color:#000}
 128 .menuseq b:not(.caret),.menuref{font-weight:inherit}
 129 .menuseq{word-spacing:-.02em}
 130 .menuseq b.caret{font-size:1.25em;line-height:.8}
 131 .menuseq i.caret{font-weight:bold;text-align:center;width:.45em}
 132 b.button::before,b.button::after{position:relative;top:-1px;font-weight:400}
 133 b.button::before{content:"[";padding:0 3px 0 2px}
 134 b.button::after{content:"]";padding:0 2px 0 3px}
 135 p a>code:hover{color:rgba(0,0,0,.9)}
 136 #header,#content,#footnotes,#footer{width:100%;margin:0 auto;max-width:62.5em;*zoom:1;position:relative;padding-left:.9375em;padding-right:.9375em}
 137 #header::before,#header::after,#content::before,#content::after,#footnotes::before,#footnotes::after,#footer::before,#footer::after{content:" ";display:table}
 138 #header::after,#content::after,#footnotes::after,#footer::after{clear:both}
 139 #content{margin-top:1.25em}
 140 #content::before{content:none}
 141 #header>h1:first-child{color:rgba(0,0,0,.85);margin-top:2.25rem;margin-bottom:0}
 142 #header>h1:first-child+#toc{margin-top:8px;border-top:1px solid #dddddf}
 143 #header>h1:only-child,body.toc2 #header>h1:nth-last-child(2){border-bottom:1px solid #dddddf;padding-bottom:8px}
 144 #header .details{border-bottom:1px solid #dddddf;line-height:1.45;padding-top:.25em;padding-bottom:.25em;padding-left:.25em;color:rgba(0,0,0,.6);display:flex;flex-flow:row wrap}
 145 #header .details span:first-child{margin-left:-.125em}
 146 #header .details span.email a{color:rgba(0,0,0,.85)}
 147 #header .details br{display:none}
 148 #header .details br+span::before{content:"\00a0\2013\00a0"}
 149 #header .details br+span.author::before{content:"\00a0\22c5\00a0";color:rgba(0,0,0,.85)}
 150 #header .details br+span#revremark::before{content:"\00a0|\00a0"}
 151 #header #revnumber{text-transform:capitalize}
 152 #header #revnumber::after{content:"\00a0"}
 153 #content>h1:first-child:not([class]){color:rgba(0,0,0,.85);border-bottom:1px solid #dddddf;padding-bottom:8px;margin-top:0;padding-top:1rem;margin-bottom:1.25rem}
 154 #toc{border-bottom:1px solid #e7e7e9;padding-bottom:.5em}
 155 #toc>ul{margin-left:.125em}
 156 #toc ul.sectlevel0>li>a{font-style:italic}
 157 #toc ul.sectlevel0 ul.sectlevel1{margin:.5em 0}
 158 #toc ul{font-family:"Open Sans","DejaVu Sans",sans-serif;list-style-type:none}
 159 #toc li{line-height:1.3334;margin-top:.3334em}
 160 #toc a{text-decoration:none}
 161 #toc a:active{text-decoration:underline}
 162 #toctitle{color:#7a2518;font-size:1.2em}
 163 @media screen and (min-width:768px){#toctitle{font-size:1.375em}
 164 body.toc2{padding-left:15em;padding-right:0}
 165 #toc.toc2{margin-top:0!important;background:#f8f8f7;position:fixed;width:15em;left:0;top:0;border-right:1px solid #e7e7e9;border-top-width:0!important;border-bottom-width:0!important;z-index:1000;padding:1.25em 1em;height:100%;overflow:auto}
 166 #toc.toc2 #toctitle{margin-top:0;margin-bottom:.8rem;font-size:1.2em}
 167 #toc.toc2>ul{font-size:.9em;margin-bottom:0}
 168 #toc.toc2 ul ul{margin-left:0;padding-left:1em}
 169 #toc.toc2 ul.sectlevel0 ul.sectlevel1{padding-left:0;margin-top:.5em;margin-bottom:.5em}
 170 body.toc2.toc-right{padding-left:0;padding-right:15em}
 171 body.toc2.toc-right #toc.toc2{border-right-width:0;border-left:1px solid #e7e7e9;left:auto;right:0}}
 172 @media screen and (min-width:1280px){body.toc2{padding-left:20em;padding-right:0}
 173 #toc.toc2{width:20em}
 174 #toc.toc2 #toctitle{font-size:1.375em}
 175 #toc.toc2>ul{font-size:.95em}
 176 #toc.toc2 ul ul{padding-left:1.25em}
 177 body.toc2.toc-right{padding-left:0;padding-right:20em}}
 178 #content #toc{border:1px solid #e0e0dc;margin-bottom:1.25em;padding:1.25em;background:#f8f8f7;border-radius:4px}
 179 #content #toc>:first-child{margin-top:0}
 180 #content #toc>:last-child{margin-bottom:0}
 181 #footer{max-width:none;background:rgba(0,0,0,.8);padding:1.25em}
 182 #footer-text{color:hsla(0,0%,100%,.8);line-height:1.44}
 183 #content{margin-bottom:.625em}
 184 .sect1{padding-bottom:.625em}
 185 @media screen and (min-width:768px){#content{margin-bottom:1.25em}
 186 .sect1{padding-bottom:1.25em}}
 187 .sect1:last-child{padding-bottom:0}
 188 .sect1+.sect1{border-top:1px solid #e7e7e9}
 189 #content h1>a.anchor,h2>a.anchor,h3>a.anchor,#toctitle>a.anchor,.sidebarblock>.content>.title>a.anchor,h4>a.anchor,h5>a.anchor,h6>a.anchor{position:absolute;z-index:1001;width:1.5ex;margin-left:-1.5ex;display:block;text-decoration:none!important;visibility:hidden;text-align:center;font-weight:400}
 190 #content h1>a.anchor::before,h2>a.anchor::before,h3>a.anchor::before,#toctitle>a.anchor::before,.sidebarblock>.content>.title>a.anchor::before,h4>a.anchor::before,h5>a.anchor::before,h6>a.anchor::before{content:"\00A7";font-size:.85em;display:block;padding-top:.1em}
 191 #content h1:hover>a.anchor,#content h1>a.anchor:hover,h2:hover>a.anchor,h2>a.anchor:hover,h3:hover>a.anchor,#toctitle:hover>a.anchor,.sidebarblock>.content>.title:hover>a.anchor,h3>a.anchor:hover,#toctitle>a.anchor:hover,.sidebarblock>.content>.title>a.anchor:hover,h4:hover>a.anchor,h4>a.anchor:hover,h5:hover>a.anchor,h5>a.anchor:hover,h6:hover>a.anchor,h6>a.anchor:hover{visibility:visible}
 192 #content h1>a.link,h2>a.link,h3>a.link,#toctitle>a.link,.sidebarblock>.content>.title>a.link,h4>a.link,h5>a.link,h6>a.link{color:#ba3925;text-decoration:none}
 193 #content h1>a.link:hover,h2>a.link:hover,h3>a.link:hover,#toctitle>a.link:hover,.sidebarblock>.content>.title>a.link:hover,h4>a.link:hover,h5>a.link:hover,h6>a.link:hover{color:#a53221}
 194 details,.audioblock,.imageblock,.literalblock,.listingblock,.stemblock,.videoblock{margin-bottom:1.25em}
 195 details{margin-left:1.25rem}
 196 details>summary{cursor:pointer;display:block;position:relative;line-height:1.6;margin-bottom:.625rem;outline:none;-webkit-tap-highlight-color:transparent}
 197 details>summary::-webkit-details-marker{display:none}
 198 details>summary::before{content:"";border:solid transparent;border-left:solid;border-width:.3em 0 .3em .5em;position:absolute;top:.5em;left:-1.25rem;transform:translateX(15%)}
 199 details[open]>summary::before{border:solid transparent;border-top:solid;border-width:.5em .3em 0;transform:translateY(15%)}
 200 details>summary::after{content:"";width:1.25rem;height:1em;position:absolute;top:.3em;left:-1.25rem}
 201 .admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{text-rendering:optimizeLegibility;text-align:left;font-family:"Noto Serif","DejaVu Serif",serif;font-size:1rem;font-style:italic}
 202 table.tableblock.fit-content>caption.title{white-space:nowrap;width:0}
 203 .paragraph.lead>p,#preamble>.sectionbody>[class=paragraph]:first-of-type p{font-size:1.21875em;line-height:1.6;color:rgba(0,0,0,.85)}
 204 .admonitionblock>table{border-collapse:separate;border:0;background:none;width:100%}
 205 .admonitionblock>table td.icon{text-align:center;width:80px}
 206 .admonitionblock>table td.icon img{max-width:none}
 207 .admonitionblock>table td.icon .title{font-weight:bold;font-family:"Open Sans","DejaVu Sans",sans-serif;text-transform:uppercase}
 208 .admonitionblock>table td.content{padding-left:1.125em;padding-right:1.25em;border-left:1px solid #dddddf;color:rgba(0,0,0,.6);word-wrap:anywhere}
 209 .admonitionblock>table td.content>:last-child>:last-child{margin-bottom:0}
 210 .exampleblock>.content{border:1px solid #e6e6e6;margin-bottom:1.25em;padding:1.25em;background:#fff;border-radius:4px}
 211 .sidebarblock{border:1px solid #dbdbd6;margin-bottom:1.25em;padding:1.25em;background:#f3f3f2;border-radius:4px}
 212 .sidebarblock>.content>.title{color:#7a2518;margin-top:0;text-align:center}
 213 .exampleblock>.content>:first-child,.sidebarblock>.content>:first-child{margin-top:0}
 214 .exampleblock>.content>:last-child,.exampleblock>.content>:last-child>:last-child,.exampleblock>.content .olist>ol>li:last-child>:last-child,.exampleblock>.content .ulist>ul>li:last-child>:last-child,.exampleblock>.content .qlist>ol>li:last-child>:last-child,.sidebarblock>.content>:last-child,.sidebarblock>.content>:last-child>:last-child,.sidebarblock>.content .olist>ol>li:last-child>:last-child,.sidebarblock>.content .ulist>ul>li:last-child>:last-child,.sidebarblock>.content .qlist>ol>li:last-child>:last-child{margin-bottom:0}
 215 .literalblock pre,.listingblock>.content>pre{border-radius:4px;overflow-x:auto;padding:1em;font-size:.8125em}
 216 @media screen and (min-width:768px){.literalblock pre,.listingblock>.content>pre{font-size:.90625em}}
 217 @media screen and (min-width:1280px){.literalblock pre,.listingblock>.content>pre{font-size:1em}}
 218 .literalblock pre,.listingblock>.content>pre:not(.highlight),.listingblock>.content>pre[class=highlight],.listingblock>.content>pre[class^="highlight "]{background:#f7f7f8}
 219 .literalblock.output pre{color:#f7f7f8;background:rgba(0,0,0,.9)}
 220 .listingblock>.content{position:relative}
 221 .listingblock code[data-lang]::before{display:none;content:attr(data-lang);position:absolute;font-size:.75em;top:.425rem;right:.5rem;line-height:1;text-transform:uppercase;color:inherit;opacity:.5}
 222 .listingblock:hover code[data-lang]::before{display:block}
 223 .listingblock.terminal pre .command::before{content:attr(data-prompt);padding-right:.5em;color:inherit;opacity:.5}
 224 .listingblock.terminal pre .command:not([data-prompt])::before{content:"$"}
 225 .listingblock pre.highlightjs{padding:0}
 226 .listingblock pre.highlightjs>code{padding:1em;border-radius:4px}
 227 .listingblock pre.prettyprint{border-width:0}
 228 .prettyprint{background:#f7f7f8}
 229 pre.prettyprint .linenums{line-height:1.45;margin-left:2em}
 230 pre.prettyprint li{background:none;list-style-type:inherit;padding-left:0}
 231 pre.prettyprint li code[data-lang]::before{opacity:1}
 232 pre.prettyprint li:not(:first-child) code[data-lang]::before{display:none}
 233 table.linenotable{border-collapse:separate;border:0;margin-bottom:0;background:none}
 234 table.linenotable td[class]{color:inherit;vertical-align:top;padding:0;line-height:inherit;white-space:normal}
 235 table.linenotable td.code{padding-left:.75em}
 236 table.linenotable td.linenos,pre.pygments .linenos{border-right:1px solid;opacity:.35;padding-right:.5em;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none}
 237 pre.pygments span.linenos{display:inline-block;margin-right:.75em}
 238 .quoteblock{margin:0 1em 1.25em 1.5em;display:table}
 239 .quoteblock:not(.excerpt)>.title{margin-left:-1.5em;margin-bottom:.75em}
 240 .quoteblock blockquote,.quoteblock p{color:rgba(0,0,0,.85);font-size:1.15rem;line-height:1.75;word-spacing:.1em;letter-spacing:0;font-style:italic;text-align:justify}
 241 .quoteblock blockquote{margin:0;padding:0;border:0}
 242 .quoteblock blockquote::before{content:"\201c";float:left;font-size:2.75em;font-weight:bold;line-height:.6em;margin-left:-.6em;color:#7a2518;text-shadow:0 1px 2px rgba(0,0,0,.1)}
 243 .quoteblock blockquote>.paragraph:last-child p{margin-bottom:0}
 244 .quoteblock .attribution{margin-top:.75em;margin-right:.5ex;text-align:right}
 245 .verseblock{margin:0 1em 1.25em}
 246 .verseblock pre{font-family:"Open Sans","DejaVu Sans",sans-serif;font-size:1.15rem;color:rgba(0,0,0,.85);font-weight:300;text-rendering:optimizeLegibility}
 247 .verseblock pre strong{font-weight:400}
 248 .verseblock .attribution{margin-top:1.25rem;margin-left:.5ex}
 249 .quoteblock .attribution,.verseblock .attribution{font-size:.9375em;line-height:1.45;font-style:italic}
 250 .quoteblock .attribution br,.verseblock .attribution br{display:none}
 251 .quoteblock .attribution cite,.verseblock .attribution cite{display:block;letter-spacing:-.025em;color:rgba(0,0,0,.6)}
 252 .quoteblock.abstract blockquote::before,.quoteblock.excerpt blockquote::before,.quoteblock .quoteblock blockquote::before{display:none}
 253 .quoteblock.abstract blockquote,.quoteblock.abstract p,.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{line-height:1.6;word-spacing:0}
 254 .quoteblock.abstract{margin:0 1em 1.25em;display:block}
 255 .quoteblock.abstract>.title{margin:0 0 .375em;font-size:1.15em;text-align:center}
 256 .quoteblock.excerpt>blockquote,.quoteblock .quoteblock{padding:0 0 .25em 1em;border-left:.25em solid #dddddf}
 257 .quoteblock.excerpt,.quoteblock .quoteblock{margin-left:0}
 258 .quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{color:inherit;font-size:1.0625rem}
 259 .quoteblock.excerpt .attribution,.quoteblock .quoteblock .attribution{color:inherit;font-size:.85rem;text-align:left;margin-right:0}
 260 p.tableblock:last-child{margin-bottom:0}
 261 td.tableblock>.content{margin-bottom:1.25em;word-wrap:anywhere}
 262 td.tableblock>.content>:last-child{margin-bottom:-1.25em}
 263 table.tableblock,th.tableblock,td.tableblock{border:0 solid #dedede}
 264 table.grid-all>*>tr>*{border-width:1px}
 265 table.grid-cols>*>tr>*{border-width:0 1px}
 266 table.grid-rows>*>tr>*{border-width:1px 0}
 267 table.frame-all{border-width:1px}
 268 table.frame-ends{border-width:1px 0}
 269 table.frame-sides{border-width:0 1px}
 270 table.frame-none>colgroup+*>:first-child>*,table.frame-sides>colgroup+*>:first-child>*{border-top-width:0}
 271 table.frame-none>:last-child>:last-child>*,table.frame-sides>:last-child>:last-child>*{border-bottom-width:0}
 272 table.frame-none>*>tr>:first-child,table.frame-ends>*>tr>:first-child{border-left-width:0}
 273 table.frame-none>*>tr>:last-child,table.frame-ends>*>tr>:last-child{border-right-width:0}
 274 table.stripes-all>*>tr,table.stripes-odd>*>tr:nth-of-type(odd),table.stripes-even>*>tr:nth-of-type(even),table.stripes-hover>*>tr:hover{background:#f8f8f7}
 275 th.halign-left,td.halign-left{text-align:left}
 276 th.halign-right,td.halign-right{text-align:right}
 277 th.halign-center,td.halign-center{text-align:center}
 278 th.valign-top,td.valign-top{vertical-align:top}
 279 th.valign-bottom,td.valign-bottom{vertical-align:bottom}
 280 th.valign-middle,td.valign-middle{vertical-align:middle}
 281 table thead th,table tfoot th{font-weight:bold}
 282 tbody tr th{background:#f7f8f7}
 283 tbody tr th,tbody tr th p,tfoot tr th,tfoot tr th p{color:rgba(0,0,0,.8);font-weight:bold}
 284 p.tableblock>code:only-child{background:none;padding:0}
 285 p.tableblock{font-size:1em}
 286 ol{margin-left:1.75em}
 287 ul li ol{margin-left:1.5em}
 288 dl dd{margin-left:1.125em}
 289 dl dd:last-child,dl dd:last-child>:last-child{margin-bottom:0}
 290 li p,ul dd,ol dd,.olist .olist,.ulist .ulist,.ulist .olist,.olist .ulist{margin-bottom:.625em}
 291 ul.checklist,ul.none,ol.none,ul.no-bullet,ol.no-bullet,ol.unnumbered,ul.unstyled,ol.unstyled{list-style-type:none}
 292 ul.no-bullet,ol.no-bullet,ol.unnumbered{margin-left:.625em}
 293 ul.unstyled,ol.unstyled{margin-left:0}
 294 li>p:empty:only-child::before{content:"";display:inline-block}
 295 ul.checklist>li>p:first-child{margin-left:-1em}
 296 ul.checklist>li>p:first-child>.fa-square-o:first-child,ul.checklist>li>p:first-child>.fa-check-square-o:first-child{width:1.25em;font-size:.8em;position:relative;bottom:.125em}
 297 ul.checklist>li>p:first-child>input[type=checkbox]:first-child{margin-right:.25em}
 298 ul.inline{display:flex;flex-flow:row wrap;list-style:none;margin:0 0 .625em -1.25em}
 299 ul.inline>li{margin-left:1.25em}
 300 .unstyled dl dt{font-weight:400;font-style:normal}
 301 ol.arabic{list-style-type:decimal}
 302 ol.decimal{list-style-type:decimal-leading-zero}
 303 ol.loweralpha{list-style-type:lower-alpha}
 304 ol.upperalpha{list-style-type:upper-alpha}
 305 ol.lowerroman{list-style-type:lower-roman}
 306 ol.upperroman{list-style-type:upper-roman}
 307 ol.lowergreek{list-style-type:lower-greek}
 308 .hdlist>table,.colist>table{border:0;background:none}
 309 .hdlist>table>tbody>tr,.colist>table>tbody>tr{background:none}
 310 td.hdlist1,td.hdlist2{vertical-align:top;padding:0 .625em}
 311 td.hdlist1{font-weight:bold;padding-bottom:1.25em}
 312 td.hdlist2{word-wrap:anywhere}
 313 .literalblock+.colist,.listingblock+.colist{margin-top:-.5em}
 314 .colist td:not([class]):first-child{padding:.4em .75em 0;line-height:1;vertical-align:top}
 315 .colist td:not([class]):first-child img{max-width:none}
 316 .colist td:not([class]):last-child{padding:.25em 0}
 317 .thumb,.th{line-height:0;display:inline-block;border:4px solid #fff;box-shadow:0 0 0 1px #ddd}
 318 .imageblock.left{margin:.25em .625em 1.25em 0}
 319 .imageblock.right{margin:.25em 0 1.25em .625em}
 320 .imageblock>.title{margin-bottom:0}
 321 .imageblock.thumb,.imageblock.th{border-width:6px}
 322 .imageblock.thumb>.title,.imageblock.th>.title{padding:0 .125em}
 323 .image.left,.image.right{margin-top:.25em;margin-bottom:.25em;display:inline-block;line-height:0}
 324 .image.left{margin-right:.625em}
 325 .image.right{margin-left:.625em}
 326 a.image{text-decoration:none;display:inline-block}
 327 a.image object{pointer-events:none}
 328 sup.footnote,sup.footnoteref{font-size:.875em;position:static;vertical-align:super}
 329 sup.footnote a,sup.footnoteref a{text-decoration:none}
 330 sup.footnote a:active,sup.footnoteref a:active{text-decoration:underline}
 331 #footnotes{padding-top:.75em;padding-bottom:.75em;margin-bottom:.625em}
 332 #footnotes hr{width:20%;min-width:6.25em;margin:-.25em 0 .75em;border-width:1px 0 0}
 333 #footnotes .footnote{padding:0 .375em 0 .225em;line-height:1.3334;font-size:.875em;margin-left:1.2em;margin-bottom:.2em}
 334 #footnotes .footnote a:first-of-type{font-weight:bold;text-decoration:none;margin-left:-1.05em}
 335 #footnotes .footnote:last-of-type{margin-bottom:0}
 336 #content #footnotes{margin-top:-.625em;margin-bottom:0;padding:.75em 0}
 337 div.unbreakable{page-break-inside:avoid}
 338 .big{font-size:larger}
 339 .small{font-size:smaller}
 340 .underline{text-decoration:underline}
 341 .overline{text-decoration:overline}
 342 .line-through{text-decoration:line-through}
 343 .aqua{color:#00bfbf}
 344 .aqua-background{background:#00fafa}
 345 .black{color:#000}
 346 .black-background{background:#000}
 347 .blue{color:#0000bf}
 348 .blue-background{background:#0000fa}
 349 .fuchsia{color:#bf00bf}
 350 .fuchsia-background{background:#fa00fa}
 351 .gray{color:#606060}
 352 .gray-background{background:#7d7d7d}
 353 .green{color:#006000}
 354 .green-background{background:#007d00}
 355 .lime{color:#00bf00}
 356 .lime-background{background:#00fa00}
 357 .maroon{color:#600000}
 358 .maroon-background{background:#7d0000}
 359 .navy{color:#000060}
 360 .navy-background{background:#00007d}
 361 .olive{color:#606000}
 362 .olive-background{background:#7d7d00}
 363 .purple{color:#600060}
 364 .purple-background{background:#7d007d}
 365 .red{color:#bf0000}
 366 .red-background{background:#fa0000}
 367 .silver{color:#909090}
 368 .silver-background{background:#bcbcbc}
 369 .teal{color:#006060}
 370 .teal-background{background:#007d7d}
 371 .white{color:#bfbfbf}
 372 .white-background{background:#fafafa}
 373 .yellow{color:#bfbf00}
 374 .yellow-background{background:#fafa00}
 375 span.icon>.fa{cursor:default}
 376 a span.icon>.fa{cursor:inherit}
 377 .admonitionblock td.icon [class^="fa icon-"]{font-size:2.5em;text-shadow:1px 1px 2px rgba(0,0,0,.5);cursor:default}
 378 .admonitionblock td.icon .icon-note::before{content:"\f05a";color:#19407c}
 379 .admonitionblock td.icon .icon-tip::before{content:"\f0eb";text-shadow:1px 1px 2px rgba(155,155,0,.8);color:#111}
 380 .admonitionblock td.icon .icon-warning::before{content:"\f071";color:#bf6900}
 381 .admonitionblock td.icon .icon-caution::before{content:"\f06d";color:#bf3400}
 382 .admonitionblock td.icon .icon-important::before{content:"\f06a";color:#bf0000}
 383 .conum[data-value]{display:inline-block;color:#fff!important;background:rgba(0,0,0,.8);border-radius:50%;text-align:center;font-size:.75em;width:1.67em;height:1.67em;line-height:1.67em;font-family:"Open Sans","DejaVu Sans",sans-serif;font-style:normal;font-weight:bold}
 384 .conum[data-value] *{color:#fff!important}
 385 .conum[data-value]+b{display:none}
 386 .conum[data-value]::after{content:attr(data-value)}
 387 pre .conum[data-value]{position:relative;top:-.125em}
 388 b.conum *{color:inherit!important}
 389 .conum:not([data-value]):empty{display:none}
 390 dt,th.tableblock,td.content,div.footnote{text-rendering:optimizeLegibility}
 391 h1,h2,p,td.content,span.alt,summary{letter-spacing:-.01em}
 392 p strong,td.content strong,div.footnote strong{letter-spacing:-.005em}
 393 p,blockquote,dt,td.content,td.hdlist1,span.alt,summary{font-size:1.0625rem}
 394 p{margin-bottom:1.25rem}
 395 .sidebarblock p,.sidebarblock dt,.sidebarblock td.content,p.tableblock{font-size:1em}
 396 .exampleblock>.content{background:#fffef7;border-color:#e0e0dc;box-shadow:0 1px 4px #e0e0dc}
 397 .print-only{display:none!important}
 398 @page{margin:1.25cm .75cm}
 399 @media print{*{box-shadow:none!important;text-shadow:none!important}
 400 html{font-size:80%}
 401 a{color:inherit!important;text-decoration:underline!important}
 402 a.bare,a[href^="#"],a[href^="mailto:"]{text-decoration:none!important}
 403 a[href^="http:"]:not(.bare)::after,a[href^="https:"]:not(.bare)::after{content:"(" attr(href) ")";display:inline-block;font-size:.875em;padding-left:.25em}
 404 abbr[title]{border-bottom:1px dotted}
 405 abbr[title]::after{content:" (" attr(title) ")"}
 406 pre,blockquote,tr,img,object,svg{page-break-inside:avoid}
 407 thead{display:table-header-group}
 408 svg{max-width:100%}
 409 p,blockquote,dt,td.content{font-size:1em;orphans:3;widows:3}
 410 h2,h3,#toctitle,.sidebarblock>.content>.title{page-break-after:avoid}
 411 #header,#content,#footnotes,#footer{max-width:none}
 412 #toc,.sidebarblock,.exampleblock>.content{background:none!important}
 413 #toc{border-bottom:1px solid #dddddf!important;padding-bottom:0!important}
 414 body.book #header{text-align:center}
 415 body.book #header>h1:first-child{border:0!important;margin:2.5em 0 1em}
 416 body.book #header .details{border:0!important;display:block;padding:0!important}
 417 body.book #header .details span:first-child{margin-left:0!important}
 418 body.book #header .details br{display:block}
 419 body.book #header .details br+span::before{content:none!important}
 420 body.book #toc{border:0!important;text-align:left!important;padding:0!important;margin:0!important}
 421 body.book #toc,body.book #preamble,body.book h1.sect0,body.book .sect1>h2{page-break-before:always}
 422 .listingblock code[data-lang]::before{display:block}
 423 #footer{padding:0 .9375em}
 424 .hide-on-print{display:none!important}
 425 .print-only{display:block!important}
 426 .hide-for-print{display:none!important}
 427 .show-for-print{display:inherit!important}}
 428 @media amzn-kf8,print{#header>h1:first-child{margin-top:1.25rem}
 429 .sect1{padding:0!important}
 430 .sect1+.sect1{border:0}
 431 #footer{background:none}
 432 #footer-text{color:rgba(0,0,0,.6);font-size:.9em}}
 433 @media amzn-kf8{#header,#content,#footnotes,#footer{padding:0}}
 434 </style>
 435 </head>
 436 <body class="article">
 437 <div id="header">
 438 <h1>Git hash function transition</h1>
 439 </div>
 440 <div id="content">
 441 <div class="sect1">
 442 <h2 id="_objective">Objective</h2>
 443 <div class="sectionbody">
 444 <div class="paragraph">
 445 <p>Migrate Git from SHA-1 to a stronger hash function.</p>
 446 </div>
 447 </div>
 448 </div>
 449 <div class="sect1">
 450 <h2 id="_background">Background</h2>
 451 <div class="sectionbody">
 452 <div class="paragraph">
 453 <p>At its core, the Git version control system is a content addressable
 454 filesystem. It uses the SHA-1 hash function to name content. For
 455 example, files, directories, and revisions are referred to by hash
 456 values unlike in other traditional version control systems where files
 457 or versions are referred to via sequential numbers. The use of a hash
 458 function to address its content delivers a few advantages:</p>
 459 </div>
 460 <div class="ulist">
 461 <ul>
 462 <li>
 463 <p>Integrity checking is easy. Bit flips, for example, are easily
 464 detected, as the hash of corrupted content does not match its name.</p>
 465 </li>
 466 <li>
 467 <p>Lookup of objects is fast.</p>
 468 </li>
 469 </ul>
 470 </div>
 471 <div class="paragraph">
 472 <p>Using a cryptographically secure hash function brings additional
 473 advantages:</p>
 474 </div>
 475 <div class="ulist">
 476 <ul>
 477 <li>
 478 <p>Object names can be signed and third parties can trust the hash to
 479 address the signed object and all objects it references.</p>
 480 </li>
 481 <li>
 482 <p>Communication using Git protocol and out of band communication
 483 methods have a short reliable string that can be used to reliably
 484 address stored content.</p>
 485 </li>
 486 </ul>
 487 </div>
 488 <div class="paragraph">
 489 <p>Over time some flaws in SHA-1 have been discovered by security
 490 researchers. On 23 February 2017 the SHAttered attack
 491 (<a href="https://shattered.io" class="bare">https://shattered.io</a>) demonstrated a practical SHA-1 hash collision.</p>
 492 </div>
 493 <div class="paragraph">
 494 <p>Git v2.13.0 and later subsequently moved to a hardened SHA-1
 495 implementation by default, which isn&#8217;t vulnerable to the SHAttered
 496 attack, but SHA-1 is still weak.</p>
 497 </div>
 498 <div class="paragraph">
 499 <p>Thus it&#8217;s considered prudent to move past any variant of SHA-1
 500 to a new hash. There&#8217;s no guarantee that future attacks on SHA-1 won&#8217;t
 501 be published in the future, and those attacks may not have viable
 502 mitigations.</p>
 503 </div>
 504 <div class="paragraph">
 505 <p>If SHA-1 and its variants were to be truly broken, Git&#8217;s hash function
 506 could not be considered cryptographically secure any more. This would
 507 impact the communication of hash values because we could not trust
 508 that a given hash value represented the known good version of content
 509 that the speaker intended.</p>
 510 </div>
 511 <div class="paragraph">
 512 <p>SHA-1 still possesses the other properties such as fast object lookup
 513 and safe error checking, but other hash functions are equally suitable
 514 that are believed to be cryptographically secure.</p>
 515 </div>
 516 </div>
 517 </div>
 518 <div class="sect1">
 519 <h2 id="_choice_of_hash">Choice of Hash</h2>
 520 <div class="sectionbody">
 521 <div class="paragraph">
 522 <p>The hash to replace the hardened SHA-1 should be stronger than SHA-1
 523 was: we would like it to be trustworthy and useful in practice for at
 524 least 10 years.</p>
 525 </div>
 526 <div class="paragraph">
 527 <p>Some other relevant properties:</p>
 528 </div>
 529 <div class="olist arabic">
 530 <ol class="arabic">
 531 <li>
 532 <p>A 256-bit hash (long enough to match common security practice; not
 533 excessively long to hurt performance and disk usage).</p>
 534 </li>
 535 <li>
 536 <p>High quality implementations should be widely available (e.g., in
 537 OpenSSL and Apple CommonCrypto).</p>
 538 </li>
 539 <li>
 540 <p>The hash function&#8217;s properties should match Git&#8217;s needs (e.g. Git
 541 requires collision and 2nd preimage resistance and does not require
 542 length extension resistance).</p>
 543 </li>
 544 <li>
 545 <p>As a tiebreaker, the hash should be fast to compute (fortunately
 546 many contenders are faster than SHA-1).</p>
 547 </li>
 548 </ol>
 549 </div>
 550 <div class="paragraph">
 551 <p>There were several contenders for a successor hash to SHA-1, including
 552 SHA-256, SHA-512/256, SHA-256x16, K12, and BLAKE2bp-256.</p>
 553 </div>
 554 <div class="paragraph">
 555 <p>In late 2018 the project picked SHA-256 as its successor hash.</p>
 556 </div>
 557 <div class="paragraph">
 558 <p>See 0ed8d8da374 (doc hash-function-transition: pick SHA-256 as
 559 NewHash, 2018-08-04) and numerous mailing list threads at the time,
 560 particularly the one starting at
 561 <a href="https://lore.kernel.org/git/20180609224913.GC38834@genre.crustytoothpaste.net/" class="bare">https://lore.kernel.org/git/20180609224913.GC38834@genre.crustytoothpaste.net/</a>
 562 for more information.</p>
 563 </div>
 564 </div>
 565 </div>
 566 <div class="sect1">
 567 <h2 id="_goals">Goals</h2>
 568 <div class="sectionbody">
 569 <div class="olist arabic">
 570 <ol class="arabic">
 571 <li>
 572 <p>The transition to SHA-256 can be done one local repository at a time.</p>
 573 <div class="olist loweralpha">
 574 <ol class="loweralpha">
 575 <li>
 576 <p>Requiring no action by any other party.</p>
 577 </li>
 578 <li>
 579 <p>A SHA-256 repository can communicate with SHA-1 Git servers
 580 (push/fetch).</p>
 581 </li>
 582 <li>
 583 <p>Users can use SHA-1 and SHA-256 identifiers for objects
 584 interchangeably (see "Object names on the command line", below).</p>
 585 </li>
 586 <li>
 587 <p>New signed objects make use of a stronger hash function than
 588 SHA-1 for their security guarantees.</p>
 589 </li>
 590 </ol>
 591 </div>
 592 </li>
 593 <li>
 594 <p>Allow a complete transition away from SHA-1.</p>
 595 <div class="olist loweralpha">
 596 <ol class="loweralpha">
 597 <li>
 598 <p>Local metadata for SHA-1 compatibility can be removed from a
 599 repository if compatibility with SHA-1 is no longer needed.</p>
 600 </li>
 601 </ol>
 602 </div>
 603 </li>
 604 <li>
 605 <p>Maintainability throughout the process.</p>
 606 <div class="olist loweralpha">
 607 <ol class="loweralpha">
 608 <li>
 609 <p>The object format is kept simple and consistent.</p>
 610 </li>
 611 <li>
 612 <p>Creation of a generalized repository conversion tool.</p>
 613 </li>
 614 </ol>
 615 </div>
 616 </li>
 617 </ol>
 618 </div>
 619 </div>
 620 </div>
 621 <div class="sect1">
 622 <h2 id="_non_goals">Non-Goals</h2>
 623 <div class="sectionbody">
 624 <div class="olist arabic">
 625 <ol class="arabic">
 626 <li>
 627 <p>Add SHA-256 support to Git protocol. This is valuable and the
 628 logical next step but it is out of scope for this initial design.</p>
 629 </li>
 630 <li>
 631 <p>Transparently improving the security of existing SHA-1 signed
 632 objects.</p>
 633 </li>
 634 <li>
 635 <p>Intermixing objects using multiple hash functions in a single
 636 repository.</p>
 637 </li>
 638 <li>
 639 <p>Taking the opportunity to fix other bugs in Git&#8217;s formats and
 640 protocols.</p>
 641 </li>
 642 <li>
 643 <p>Shallow clones and fetches into a SHA-256 repository. (This will
 644 change when we add SHA-256 support to Git protocol.)</p>
 645 </li>
 646 <li>
 647 <p>Skip fetching some submodules of a project into a SHA-256
 648 repository. (This also depends on SHA-256 support in Git
 649 protocol.)</p>
 650 </li>
 651 </ol>
 652 </div>
 653 </div>
 654 </div>
 655 <div class="sect1">
 656 <h2 id="_overview">Overview</h2>
 657 <div class="sectionbody">
 658 <div class="paragraph">
 659 <p>We introduce a new repository format extension. Repositories with this
 660 extension enabled use SHA-256 instead of SHA-1 to name their objects.
 661 This affects both object names and object content&#8201;&#8212;&#8201;both the names
 662 of objects and all references to other objects within an object are
 663 switched to the new hash function.</p>
 664 </div>
 665 <div class="paragraph">
 666 <p>SHA-256 repositories cannot be read by older versions of Git.</p>
 667 </div>
 668 <div class="paragraph">
 669 <p>Alongside the packfile, a SHA-256 repository stores a bidirectional
 670 mapping between SHA-256 and SHA-1 object names. The mapping is generated
 671 locally and can be verified using "git fsck". Object lookups use this
 672 mapping to allow naming objects using either their SHA-1 and SHA-256 names
 673 interchangeably.</p>
 674 </div>
 675 <div class="paragraph">
 676 <p>"git cat-file" and "git hash-object" gain options to display an object
 677 in its SHA-1 form and write an object given its SHA-1 form. This
 678 requires all objects referenced by that object to be present in the
 679 object database so that they can be named using the appropriate name
 680 (using the bidirectional hash mapping).</p>
 681 </div>
 682 <div class="paragraph">
 683 <p>Fetches from a SHA-1 based server convert the fetched objects into
 684 SHA-256 form and record the mapping in the bidirectional mapping table
 685 (see below for details). Pushes to a SHA-1 based server convert the
 686 objects being pushed into SHA-1 form so the server does not have to be
 687 aware of the hash function the client is using.</p>
 688 </div>
 689 </div>
 690 </div>
 691 <div class="sect1">
 692 <h2 id="_detailed_design">Detailed Design</h2>
 693 <div class="sectionbody">
 694 <div class="sect2">
 695 <h3 id="_repository_format_extension">Repository format extension</h3>
 696 <div class="paragraph">
 697 <p>A SHA-256 repository uses repository format version <code>1</code> (see
 698 <a href="../gitrepository-layout.html">gitrepository-layout(5)</a>) with <code>extensions.objectFormat</code> and
 699 <code>extensions.compatObjectFormat</code> (see <a href="../git-config.html">git-config(1)</a>) set to:</p>
 700 </div>
 701 <div class="literalblock">
 702 <div class="content">
 703 <pre>[core]
 704         repositoryFormatVersion = 1
 705 [extensions]
 706         objectFormat = sha256
 707         compatObjectFormat = sha1</pre>
 708 </div>
 709 </div>
 710 <div class="paragraph">
 711 <p>The combination of setting <code>core.repositoryFormatVersion=1</code> and
 712 populating <code>extensions.</code>* ensures that all versions of Git later than
 713 <code>v0.99.9l</code> will die instead of trying to operate on the SHA-256
 714 repository, instead producing an error message.</p>
 715 </div>
 716 <div class="literalblock">
 717 <div class="content">
 718 <pre># Between v0.99.9l and v2.7.0
 719 $ git status
 720 fatal: Expected git repo version &lt;= 0, found 1
 721 # After v2.7.0
 722 $ git status
 723 fatal: unknown repository extensions found:
 724         objectformat
 725         compatobjectformat</pre>
 726 </div>
 727 </div>
 728 <div class="paragraph">
 729 <p>See the "Transition plan" section below for more details on these
 730 repository extensions.</p>
 731 </div>
 732 </div>
 733 <div class="sect2">
 734 <h3 id="_object_names">Object names</h3>
 735 <div class="paragraph">
 736 <p>Objects can be named by their 40 hexadecimal digit SHA-1 name or 64
 737 hexadecimal digit SHA-256 name, plus names derived from those (see
 738 gitrevisions(7)).</p>
 739 </div>
 740 <div class="paragraph">
 741 <p>The SHA-1 name of an object is the SHA-1 of the concatenation of its
 742 type, length, a nul byte, and the object&#8217;s SHA-1 content. This is the
 743 traditional &lt;sha1&gt; used in Git to name objects.</p>
 744 </div>
 745 <div class="paragraph">
 746 <p>The SHA-256 name of an object is the SHA-256 of the concatenation of its
 747 type, length, a nul byte, and the object&#8217;s SHA-256 content.</p>
 748 </div>
 749 </div>
 750 <div class="sect2">
 751 <h3 id="_object_format">Object format</h3>
 752 <div class="paragraph">
 753 <p>The content as a byte sequence of a tag, commit, or tree object named
 754 by SHA-1 and SHA-256 differ because an object named by SHA-256 name refers to
 755 other objects by their SHA-256 names and an object named by SHA-1 name
 756 refers to other objects by their SHA-1 names.</p>
 757 </div>
 758 <div class="paragraph">
 759 <p>The SHA-256 content of an object is the same as its SHA-1 content, except
 760 that objects referenced by the object are named using their SHA-256 names
 761 instead of SHA-1 names. Because a blob object does not refer to any
 762 other object, its SHA-1 content and SHA-256 content are the same.</p>
 763 </div>
 764 <div class="paragraph">
 765 <p>The format allows round-trip conversion between SHA-256 content and
 766 SHA-1 content.</p>
 767 </div>
 768 </div>
 769 <div class="sect2">
 770 <h3 id="_object_storage">Object storage</h3>
 771 <div class="paragraph">
 772 <p>Loose objects use zlib compression and packed objects use the packed
 773 format described in <a href="../gitformat-pack.html">gitformat-pack(5)</a>, just like
 774 today. The content that is compressed and stored uses SHA-256 content
 775 instead of SHA-1 content.</p>
 776 </div>
 777 </div>
 778 <div class="sect2">
 779 <h3 id="_pack_index">Pack index</h3>
 780 <div class="paragraph">
 781 <p>Pack index (.idx) files use a new v3 format that supports multiple
 782 hash functions. They have the following format (all integers are in
 783 network byte order):</p>
 784 </div>
 785 <div class="ulist">
 786 <ul>
 787 <li>
 788 <p>A header appears at the beginning and consists of the following:</p>
 789 <div class="ulist">
 790 <ul>
 791 <li>
 792 <p>The 4-byte pack index signature: <em>\377t0c</em></p>
 793 </li>
 794 <li>
 795 <p>4-byte version number: 3</p>
 796 </li>
 797 <li>
 798 <p>4-byte length of the header section, including the signature and
 799 version number</p>
 800 </li>
 801 <li>
 802 <p>4-byte number of objects contained in the pack</p>
 803 </li>
 804 <li>
 805 <p>4-byte number of object formats in this pack index: 2</p>
 806 </li>
 807 <li>
 808 <p>For each object format:</p>
 809 <div class="ulist">
 810 <ul>
 811 <li>
 812 <p>4-byte format identifier (e.g., <em>sha1</em> for SHA-1)</p>
 813 </li>
 814 <li>
 815 <p>4-byte length in bytes of shortened object names. This is the
 816 shortest possible length needed to make names in the shortened
 817 object name table unambiguous.</p>
 818 </li>
 819 <li>
 820 <p>4-byte integer, recording where tables relating to this format
 821 are stored in this index file, as an offset from the beginning.</p>
 822 </li>
 823 </ul>
 824 </div>
 825 </li>
 826 <li>
 827 <p>4-byte offset to the trailer from the beginning of this file.</p>
 828 </li>
 829 <li>
 830 <p>Zero or more additional key/value pairs (4-byte key, 4-byte
 831 value). Only one key is supported: <em>PSRC</em>. See the "Loose objects
 832 and unreachable objects" section for supported values and how this
 833 is used.  All other keys are reserved. Readers must ignore
 834 unrecognized keys.</p>
 835 </li>
 836 </ul>
 837 </div>
 838 </li>
 839 <li>
 840 <p>Zero or more NUL bytes. This can optionally be used to improve the
 841 alignment of the full object name table below.</p>
 842 </li>
 843 <li>
 844 <p>Tables for the first object format:</p>
 845 <div class="ulist">
 846 <ul>
 847 <li>
 848 <p>A sorted table of shortened object names.  These are prefixes of
 849 the names of all objects in this pack file, packed together
 850 without offset values to reduce the cache footprint of the binary
 851 search for a specific object name.</p>
 852 </li>
 853 <li>
 854 <p>A table of full object names in pack order. This allows resolving
 855 a reference to "the nth object in the pack file" (from a
 856 reachability bitmap or from the next table of another object
 857 format) to its object name.</p>
 858 </li>
 859 <li>
 860 <p>A table of 4-byte values mapping object name order to pack order.
 861 For an object in the table of sorted shortened object names, the
 862 value at the corresponding index in this table is the index in the
 863 previous table for that same object.
 864 This can be used to look up the object in reachability bitmaps or
 865 to look up its name in another object format.</p>
 866 </li>
 867 <li>
 868 <p>A table of 4-byte CRC32 values of the packed object data, in the
 869 order that the objects appear in the pack file. This is to allow
 870 compressed data to be copied directly from pack to pack during
 871 repacking without undetected data corruption.</p>
 872 </li>
 873 <li>
 874 <p>A table of 4-byte offset values. For an object in the table of
 875 sorted shortened object names, the value at the corresponding
 876 index in this table indicates where that object can be found in
 877 the pack file. These are usually 31-bit pack file offsets, but
 878 large offsets are encoded as an index into the next table with the
 879 most significant bit set.</p>
 880 </li>
 881 <li>
 882 <p>A table of 8-byte offset entries (empty for pack files less than
 883 2 GiB). Pack files are organized with heavily used objects toward
 884 the front, so most object references should not need to refer to
 885 this table.</p>
 886 </li>
 887 </ul>
 888 </div>
 889 </li>
 890 <li>
 891 <p>Zero or more NUL bytes.</p>
 892 </li>
 893 <li>
 894 <p>Tables for the second object format, with the same layout as above,
 895 up to and not including the table of CRC32 values.</p>
 896 </li>
 897 <li>
 898 <p>Zero or more NUL bytes.</p>
 899 </li>
 900 <li>
 901 <p>The trailer consists of the following:</p>
 902 <div class="ulist">
 903 <ul>
 904 <li>
 905 <p>A copy of the 20-byte SHA-256 checksum at the end of the
 906 corresponding packfile.</p>
 907 </li>
 908 <li>
 909 <p>20-byte SHA-256 checksum of all of the above.</p>
 910 </li>
 911 </ul>
 912 </div>
 913 </li>
 914 </ul>
 915 </div>
 916 </div>
 917 <div class="sect2">
 918 <h3 id="_loose_object_index">Loose object index</h3>
 919 <div class="paragraph">
 920 <p>A new file $GIT_OBJECT_DIR/loose-object-idx contains information about
 921 all loose objects. Its format is</p>
 922 </div>
 923 <div class="literalblock">
 924 <div class="content">
 925 <pre># loose-object-idx
 926 (sha256-name SP sha1-name LF)*</pre>
 927 </div>
 928 </div>
 929 <div class="paragraph">
 930 <p>where the object names are in hexadecimal format. The file is not
 931 sorted.</p>
 932 </div>
 933 <div class="paragraph">
 934 <p>The loose object index is protected against concurrent writes by a
 935 lock file $GIT_OBJECT_DIR/loose-object-idx.lock. To add a new loose
 936 object:</p>
 937 </div>
 938 <div class="olist arabic">
 939 <ol class="arabic">
 940 <li>
 941 <p>Write the loose object to a temporary file, like today.</p>
 942 </li>
 943 <li>
 944 <p>Open loose-object-idx.lock with O_CREAT | O_EXCL to acquire the lock.</p>
 945 </li>
 946 <li>
 947 <p>Rename the loose object into place.</p>
 948 </li>
 949 <li>
 950 <p>Open loose-object-idx with O_APPEND and write the new object</p>
 951 </li>
 952 <li>
 953 <p>Unlink loose-object-idx.lock to release the lock.</p>
 954 </li>
 955 </ol>
 956 </div>
 957 <div class="paragraph">
 958 <p>To remove entries (e.g. in "git pack-refs" or "git-prune"):</p>
 959 </div>
 960 <div class="olist arabic">
 961 <ol class="arabic">
 962 <li>
 963 <p>Open loose-object-idx.lock with O_CREAT | O_EXCL to acquire the
 964 lock.</p>
 965 </li>
 966 <li>
 967 <p>Write the new content to loose-object-idx.lock.</p>
 968 </li>
 969 <li>
 970 <p>Unlink any loose objects being removed.</p>
 971 </li>
 972 <li>
 973 <p>Rename to replace loose-object-idx, releasing the lock.</p>
 974 </li>
 975 </ol>
 976 </div>
 977 </div>
 978 <div class="sect2">
 979 <h3 id="_translation_table">Translation table</h3>
 980 <div class="paragraph">
 981 <p>The index files support a bidirectional mapping between SHA-1 names
 982 and SHA-256 names. The lookup proceeds similarly to ordinary object
 983 lookups. For example, to convert a SHA-1 name to a SHA-256 name:</p>
 984 </div>
 985 <div class="olist arabic">
 986 <ol class="arabic">
 987 <li>
 988 <p>Look for the object in idx files. If a match is present in the
 989 idx&#8217;s sorted list of truncated SHA-1 names, then:</p>
 990 <div class="olist loweralpha">
 991 <ol class="loweralpha">
 992 <li>
 993 <p>Read the corresponding entry in the SHA-1 name order to pack
 994 name order mapping.</p>
 995 </li>
 996 <li>
 997 <p>Read the corresponding entry in the full SHA-1 name table to
 998 verify we found the right object. If it is, then</p>
 999 </li>
1000 <li>
1001 <p>Read the corresponding entry in the full SHA-256 name table.
1002 That is the object&#8217;s SHA-256 name.</p>
1003 </li>
1004 </ol>
1005 </div>
1006 </li>
1007 <li>
1008 <p>Check for a loose object. Read lines from loose-object-idx until
1009 we find a match.</p>
1010 </li>
1011 </ol>
1012 </div>
1013 <div class="paragraph">
1014 <p>Step (1) takes the same amount of time as an ordinary object lookup:
1015 O(number of packs * log(objects per pack)). Step (2) takes O(number of
1016 loose objects) time. To maintain good performance it will be necessary
1017 to keep the number of loose objects low. See the "Loose objects and
1018 unreachable objects" section below for more details.</p>
1019 </div>
1020 <div class="paragraph">
1021 <p>Since all operations that make new objects (e.g., "git commit") add
1022 the new objects to the corresponding index, this mapping is possible
1023 for all objects in the object store.</p>
1024 </div>
1025 </div>
1026 <div class="sect2">
1027 <h3 id="_reading_an_objects_sha_1_content">Reading an object&#8217;s SHA-1 content</h3>
1028 <div class="paragraph">
1029 <p>The SHA-1 content of an object can be read by converting all SHA-256 names
1030 of its SHA-256 content references to SHA-1 names using the translation table.</p>
1031 </div>
1032 </div>
1033 <div class="sect2">
1034 <h3 id="_fetch">Fetch</h3>
1035 <div class="paragraph">
1036 <p>Fetching from a SHA-1 based server requires translating between SHA-1
1037 and SHA-256 based representations on the fly.</p>
1038 </div>
1039 <div class="paragraph">
1040 <p>SHA-1s named in the ref advertisement that are present on the client
1041 can be translated to SHA-256 and looked up as local objects using the
1042 translation table.</p>
1043 </div>
1044 <div class="paragraph">
1045 <p>Negotiation proceeds as today. Any "have"s generated locally are
1046 converted to SHA-1 before being sent to the server, and SHA-1s
1047 mentioned by the server are converted to SHA-256 when looking them up
1048 locally.</p>
1049 </div>
1050 <div class="paragraph">
1051 <p>After negotiation, the server sends a packfile containing the
1052 requested objects. We convert the packfile to SHA-256 format using
1053 the following steps:</p>
1054 </div>
1055 <div class="olist arabic">
1056 <ol class="arabic">
1057 <li>
1058 <p>index-pack: inflate each object in the packfile and compute its
1059 SHA-1. Objects can contain deltas in OBJ_REF_DELTA format against
1060 objects the client has locally. These objects can be looked up
1061 using the translation table and their SHA-1 content read as
1062 described above to resolve the deltas.</p>
1063 </li>
1064 <li>
1065 <p>topological sort: starting at the "want"s from the negotiation
1066 phase, walk through objects in the pack and emit a list of them,
1067 excluding blobs, in reverse topologically sorted order, with each
1068 object coming later in the list than all objects it references.
1069 (This list only contains objects reachable from the "wants". If the
1070 pack from the server contained additional extraneous objects, then
1071 they will be discarded.)</p>
1072 </li>
1073 <li>
1074 <p>convert to SHA-256: open a new SHA-256 packfile. Read the topologically
1075 sorted list just generated. For each object, inflate its
1076 SHA-1 content, convert to SHA-256 content, and write it to the SHA-256
1077 pack. Record the new SHA-1&#8592;&#8594;SHA-256 mapping entry for use in the idx.</p>
1078 </li>
1079 <li>
1080 <p>sort: reorder entries in the new pack to match the order of objects
1081 in the pack the server generated and include blobs. Write a SHA-256 idx
1082 file</p>
1083 </li>
1084 <li>
1085 <p>clean up: remove the SHA-1 based pack file, index, and
1086 topologically sorted list obtained from the server in steps 1
1087 and 2.</p>
1088 </li>
1089 </ol>
1090 </div>
1091 <div class="paragraph">
1092 <p>Step 3 requires every object referenced by the new object to be in the
1093 translation table. This is why the topological sort step is necessary.</p>
1094 </div>
1095 <div class="paragraph">
1096 <p>As an optimization, step 1 could write a file describing what non-blob
1097 objects each object it has inflated from the packfile references. This
1098 makes the topological sort in step 2 possible without inflating the
1099 objects in the packfile for a second time. The objects need to be
1100 inflated again in step 3, for a total of two inflations.</p>
1101 </div>
1102 <div class="paragraph">
1103 <p>Step 4 is probably necessary for good read-time performance. "git
1104 pack-objects" on the server optimizes the pack file for good data
1105 locality (see Documentation/technical/pack-heuristics.txt).</p>
1106 </div>
1107 <div class="paragraph">
1108 <p>Details of this process are likely to change. It will take some
1109 experimenting to get this to perform well.</p>
1110 </div>
1111 </div>
1112 <div class="sect2">
1113 <h3 id="_push">Push</h3>
1114 <div class="paragraph">
1115 <p>Push is simpler than fetch because the objects referenced by the
1116 pushed objects are already in the translation table. The SHA-1 content
1117 of each object being pushed can be read as described in the "Reading
1118 an object&#8217;s SHA-1 content" section to generate the pack written by git
1119 send-pack.</p>
1120 </div>
1121 </div>
1122 <div class="sect2">
1123 <h3 id="_signed_commits">Signed Commits</h3>
1124 <div class="paragraph">
1125 <p>We add a new field "gpgsig-sha256" to the commit object format to allow
1126 signing commits without relying on SHA-1. It is similar to the
1127 existing "gpgsig" field. Its signed payload is the SHA-256 content of the
1128 commit object with any "gpgsig" and "gpgsig-sha256" fields removed.</p>
1129 </div>
1130 <div class="paragraph">
1131 <p>This means commits can be signed</p>
1132 </div>
1133 <div class="olist arabic">
1134 <ol class="arabic">
1135 <li>
1136 <p>using SHA-1 only, as in existing signed commit objects</p>
1137 </li>
1138 <li>
1139 <p>using both SHA-1 and SHA-256, by using both gpgsig-sha256 and gpgsig
1140 fields.</p>
1141 </li>
1142 <li>
1143 <p>using only SHA-256, by only using the gpgsig-sha256 field.</p>
1144 </li>
1145 </ol>
1146 </div>
1147 <div class="paragraph">
1148 <p>Old versions of "git verify-commit" can verify the gpgsig signature in
1149 cases (1) and (2) without modifications and view case (3) as an
1150 ordinary unsigned commit.</p>
1151 </div>
1152 </div>
1153 <div class="sect2">
1154 <h3 id="_signed_tags">Signed Tags</h3>
1155 <div class="paragraph">
1156 <p>We add a new field "gpgsig-sha256" to the tag object format to allow
1157 signing tags without relying on SHA-1. Its signed payload is the
1158 SHA-256 content of the tag with its gpgsig-sha256 field and "-----BEGIN PGP
1159 SIGNATURE-----" delimited in-body signature removed.</p>
1160 </div>
1161 <div class="paragraph">
1162 <p>This means tags can be signed</p>
1163 </div>
1164 <div class="olist arabic">
1165 <ol class="arabic">
1166 <li>
1167 <p>using SHA-1 only, as in existing signed tag objects</p>
1168 </li>
1169 <li>
1170 <p>using both SHA-1 and SHA-256, by using gpgsig-sha256 and an in-body
1171 signature.</p>
1172 </li>
1173 <li>
1174 <p>using only SHA-256, by only using the gpgsig-sha256 field.</p>
1175 </li>
1176 </ol>
1177 </div>
1178 </div>
1179 <div class="sect2">
1180 <h3 id="_mergetag_embedding">Mergetag embedding</h3>
1181 <div class="paragraph">
1182 <p>The mergetag field in the SHA-1 content of a commit contains the
1183 SHA-1 content of a tag that was merged by that commit.</p>
1184 </div>
1185 <div class="paragraph">
1186 <p>The mergetag field in the SHA-256 content of the same commit contains the
1187 SHA-256 content of the same tag.</p>
1188 </div>
1189 </div>
1190 <div class="sect2">
1191 <h3 id="_submodules">Submodules</h3>
1192 <div class="paragraph">
1193 <p>To convert recorded submodule pointers, you need to have the converted
1194 submodule repository in place. The translation table of the submodule
1195 can be used to look up the new hash.</p>
1196 </div>
1197 </div>
1198 <div class="sect2">
1199 <h3 id="_loose_objects_and_unreachable_objects">Loose objects and unreachable objects</h3>
1200 <div class="paragraph">
1201 <p>Fast lookups in the loose-object-idx require that the number of loose
1202 objects not grow too high.</p>
1203 </div>
1204 <div class="paragraph">
1205 <p>"git gc --auto" currently waits for there to be 6700 loose objects
1206 present before consolidating them into a packfile. We will need to
1207 measure to find a more appropriate threshold for it to use.</p>
1208 </div>
1209 <div class="paragraph">
1210 <p>"git gc --auto" currently waits for there to be 50 packs present
1211 before combining packfiles. Packing loose objects more aggressively
1212 may cause the number of pack files to grow too quickly. This can be
1213 mitigated by using a strategy similar to Martin Fick&#8217;s exponential
1214 rolling garbage collection script:
1215 <a href="https://gerrit-review.googlesource.com/c/gerrit/+/35215" class="bare">https://gerrit-review.googlesource.com/c/gerrit/+/35215</a></p>
1216 </div>
1217 <div class="paragraph">
1218 <p>"git gc" currently expels any unreachable objects it encounters in
1219 pack files to loose objects in an attempt to prevent a race when
1220 pruning them (in case another process is simultaneously writing a new
1221 object that refers to the about-to-be-deleted object). This leads to
1222 an explosion in the number of loose objects present and disk space
1223 usage due to the objects in delta form being replaced with independent
1224 loose objects.  Worse, the race is still present for loose objects.</p>
1225 </div>
1226 <div class="paragraph">
1227 <p>Instead, "git gc" will need to move unreachable objects to a new
1228 packfile marked as UNREACHABLE_GARBAGE (using the PSRC field; see
1229 below). To avoid the race when writing new objects referring to an
1230 about-to-be-deleted object, code paths that write new objects will
1231 need to copy any objects from UNREACHABLE_GARBAGE packs that they
1232 refer to new, non-UNREACHABLE_GARBAGE packs (or loose objects).
1233 UNREACHABLE_GARBAGE are then safe to delete if their creation time (as
1234 indicated by the file&#8217;s mtime) is long enough ago.</p>
1235 </div>
1236 <div class="paragraph">
1237 <p>To avoid a proliferation of UNREACHABLE_GARBAGE packs, they can be
1238 combined under certain circumstances. If "gc.garbageTtl" is set to
1239 greater than one day, then packs created within a single calendar day,
1240 UTC, can be coalesced together. The resulting packfile would have an
1241 mtime before midnight on that day, so this makes the effective maximum
1242 ttl the garbageTtl + 1 day. If "gc.garbageTtl" is less than one day,
1243 then we divide the calendar day into intervals one-third of that ttl
1244 in duration. Packs created within the same interval can be coalesced
1245 together. The resulting packfile would have an mtime before the end of
1246 the interval, so this makes the effective maximum ttl equal to the
1247 garbageTtl * 4/3.</p>
1248 </div>
1249 <div class="paragraph">
1250 <p>This rule comes from Thirumala Reddy Mutchukota&#8217;s JGit change
1251 <a href="https://git.eclipse.org/r/90465" class="bare">https://git.eclipse.org/r/90465</a>.</p>
1252 </div>
1253 <div class="paragraph">
1254 <p>The UNREACHABLE_GARBAGE setting goes in the PSRC field of the pack
1255 index. More generally, that field indicates where a pack came from:</p>
1256 </div>
1257 <div class="ulist">
1258 <ul>
1259 <li>
1260 <p>1 (PACK_SOURCE_RECEIVE) for a pack received over the network</p>
1261 </li>
1262 <li>
1263 <p>2 (PACK_SOURCE_AUTO) for a pack created by a lightweight
1264 "gc --auto" operation</p>
1265 </li>
1266 <li>
1267 <p>3 (PACK_SOURCE_GC) for a pack created by a full gc</p>
1268 </li>
1269 <li>
1270 <p>4 (PACK_SOURCE_UNREACHABLE_GARBAGE) for potential garbage
1271 discovered by gc</p>
1272 </li>
1273 <li>
1274 <p>5 (PACK_SOURCE_INSERT) for locally created objects that were
1275 written directly to a pack file, e.g. from "git add ."</p>
1276 </li>
1277 </ul>
1278 </div>
1279 <div class="paragraph">
1280 <p>This information can be useful for debugging and for "gc --auto" to
1281 make appropriate choices about which packs to coalesce.</p>
1282 </div>
1283 </div>
1284 </div>
1285 </div>
1286 <div class="sect1">
1287 <h2 id="_caveats">Caveats</h2>
1288 <div class="sectionbody">
1289 <div class="sect2">
1290 <h3 id="_invalid_objects">Invalid objects</h3>
1291 <div class="paragraph">
1292 <p>The conversion from SHA-1 content to SHA-256 content retains any
1293 brokenness in the original object (e.g., tree entry modes encoded with
1294 leading 0, tree objects whose paths are not sorted correctly, and
1295 commit objects without an author or committer). This is a deliberate
1296 feature of the design to allow the conversion to round-trip.</p>
1297 </div>
1298 <div class="paragraph">
1299 <p>More profoundly broken objects (e.g., a commit with a truncated "tree"
1300 header line) cannot be converted but were not usable by current Git
1301 anyway.</p>
1302 </div>
1303 </div>
1304 <div class="sect2">
1305 <h3 id="_shallow_clone_and_submodules">Shallow clone and submodules</h3>
1306 <div class="paragraph">
1307 <p>Because it requires all referenced objects to be available in the
1308 locally generated translation table, this design does not support
1309 shallow clone or unfetched submodules. Protocol improvements might
1310 allow lifting this restriction.</p>
1311 </div>
1312 </div>
1313 <div class="sect2">
1314 <h3 id="_alternates">Alternates</h3>
1315 <div class="paragraph">
1316 <p>For the same reason, a SHA-256 repository cannot borrow objects from a
1317 SHA-1 repository using objects/info/alternates or
1318 $GIT_ALTERNATE_OBJECT_REPOSITORIES.</p>
1319 </div>
1320 </div>
1321 <div class="sect2">
1322 <h3 id="_git_notes">git notes</h3>
1323 <div class="paragraph">
1324 <p>The "git notes" tool annotates objects using their SHA-1 name as key.
1325 This design does not describe a way to migrate notes trees to use
1326 SHA-256 names. That migration is expected to happen separately (for
1327 example using a file at the root of the notes tree to describe which
1328 hash it uses).</p>
1329 </div>
1330 </div>
1331 <div class="sect2">
1332 <h3 id="_server_side_cost">Server-side cost</h3>
1333 <div class="paragraph">
1334 <p>Until Git protocol gains SHA-256 support, using SHA-256 based storage
1335 on public-facing Git servers is strongly discouraged. Once Git
1336 protocol gains SHA-256 support, SHA-256 based servers are likely not
1337 to support SHA-1 compatibility, to avoid what may be a very expensive
1338 hash re-encode during clone and to encourage peers to modernize.</p>
1339 </div>
1340 <div class="paragraph">
1341 <p>The design described here allows fetches by SHA-1 clients of a
1342 personal SHA-256 repository because it&#8217;s not much more difficult than
1343 allowing pushes from that repository. This support needs to be guarded
1344 by a configuration option&#8201;&#8212;&#8201;servers like git.kernel.org that serve a
1345 large number of clients would not be expected to bear that cost.</p>
1346 </div>
1347 </div>
1348 <div class="sect2">
1349 <h3 id="_meaning_of_signatures">Meaning of signatures</h3>
1350 <div class="paragraph">
1351 <p>The signed payload for signed commits and tags does not explicitly
1352 name the hash used to identify objects. If some day Git adopts a new
1353 hash function with the same length as the current SHA-1 (40
1354 hexadecimal digit) or SHA-256 (64 hexadecimal digit) objects then the
1355 intent behind the PGP signed payload in an object signature is
1356 unclear:</p>
1357 </div>
1358 <div class="literalblock">
1359 <div class="content">
1360 <pre>object e7e07d5a4fcc2a203d9873968ad3e6bd4d7419d7
1361 type commit
1362 tag v2.12.0
1363 tagger Junio C Hamano &lt;gitster@pobox.com&gt; 1487962205 -0800</pre>
1364 </div>
1365 </div>
1366 <div class="literalblock">
1367 <div class="content">
1368 <pre>Git 2.12</pre>
1369 </div>
1370 </div>
1371 <div class="paragraph">
1372 <p>Does this mean Git v2.12.0 is the commit with SHA-1 name
1373 e7e07d5a4fcc2a203d9873968ad3e6bd4d7419d7 or the commit with
1374 new-40-digit-hash-name e7e07d5a4fcc2a203d9873968ad3e6bd4d7419d7?</p>
1375 </div>
1376 <div class="paragraph">
1377 <p>Fortunately SHA-256 and SHA-1 have different lengths. If Git starts
1378 using another hash with the same length to name objects, then it will
1379 need to change the format of signed payloads using that hash to
1380 address this issue.</p>
1381 </div>
1382 </div>
1383 <div class="sect2">
1384 <h3 id="_object_names_on_the_command_line">Object names on the command line</h3>
1385 <div class="paragraph">
1386 <p>To support the transition (see Transition plan below), this design
1387 supports four different modes of operation:</p>
1388 </div>
1389 <div class="olist arabic">
1390 <ol class="arabic">
1391 <li>
1392 <p>("dark launch") Treat object names input by the user as SHA-1 and
1393 convert any object names written to output to SHA-1, but store
1394 objects using SHA-256.  This allows users to test the code with no
1395 visible behavior change except for performance.  This allows
1396 running even tests that assume the SHA-1 hash function, to
1397 sanity-check the behavior of the new mode.</p>
1398 </li>
1399 <li>
1400 <p>("early transition") Allow both SHA-1 and SHA-256 object names in
1401 input. Any object names written to output use SHA-1. This allows
1402 users to continue to make use of SHA-1 to communicate with peers
1403 (e.g. by email) that have not migrated yet and prepares for mode 3.</p>
1404 </li>
1405 <li>
1406 <p>("late transition") Allow both SHA-1 and SHA-256 object names in
1407 input. Any object names written to output use SHA-256. In this
1408 mode, users are using a more secure object naming method by
1409 default.  The disruption is minimal as long as most of their peers
1410 are in mode 2 or mode 3.</p>
1411 </li>
1412 <li>
1413 <p>("post-transition") Treat object names input by the user as
1414 SHA-256 and write output using SHA-256. This is safer than mode 3
1415 because there is less risk that input is incorrectly interpreted
1416 using the wrong hash function.</p>
1417 </li>
1418 </ol>
1419 </div>
1420 <div class="paragraph">
1421 <p>The mode is specified in configuration.</p>
1422 </div>
1423 <div class="paragraph">
1424 <p>The user can also explicitly specify which format to use for a
1425 particular revision specifier and for output, overriding the mode. For
1426 example:</p>
1427 </div>
1428 <div class="literalblock">
1429 <div class="content">
1430 <pre>git --output-format=sha1 log abac87a^{sha1}..f787cac^{sha256}</pre>
1431 </div>
1432 </div>
1433 </div>
1434 </div>
1435 </div>
1436 <div class="sect1">
1437 <h2 id="_transition_plan">Transition plan</h2>
1438 <div class="sectionbody">
1439 <div class="paragraph">
1440 <p>Some initial steps can be implemented independently of one another:</p>
1441 </div>
1442 <div class="ulist">
1443 <ul>
1444 <li>
1445 <p>adding a hash function API (vtable)</p>
1446 </li>
1447 <li>
1448 <p>teaching fsck to tolerate the gpgsig-sha256 field</p>
1449 </li>
1450 <li>
1451 <p>excluding gpgsig-* from the fields copied by "git commit --amend"</p>
1452 </li>
1453 <li>
1454 <p>annotating tests that depend on SHA-1 values with a SHA1 test
1455 prerequisite</p>
1456 </li>
1457 <li>
1458 <p>using "struct object_id", GIT_MAX_RAWSZ, and GIT_MAX_HEXSZ
1459 consistently instead of "unsigned char *" and the hardcoded
1460 constants 20 and 40.</p>
1461 </li>
1462 <li>
1463 <p>introducing index v3</p>
1464 </li>
1465 <li>
1466 <p>adding support for the PSRC field and safer object pruning</p>
1467 </li>
1468 </ul>
1469 </div>
1470 <div class="paragraph">
1471 <p>The first user-visible change is the introduction of the objectFormat
1472 extension (without compatObjectFormat). This requires:</p>
1473 </div>
1474 <div class="ulist">
1475 <ul>
1476 <li>
1477 <p>teaching fsck about this mode of operation</p>
1478 </li>
1479 <li>
1480 <p>using the hash function API (vtable) when computing object names</p>
1481 </li>
1482 <li>
1483 <p>signing objects and verifying signatures</p>
1484 </li>
1485 <li>
1486 <p>rejecting attempts to fetch from or push to an incompatible
1487 repository</p>
1488 </li>
1489 </ul>
1490 </div>
1491 <div class="paragraph">
1492 <p>Next comes introduction of compatObjectFormat:</p>
1493 </div>
1494 <div class="ulist">
1495 <ul>
1496 <li>
1497 <p>implementing the loose-object-idx</p>
1498 </li>
1499 <li>
1500 <p>translating object names between object formats</p>
1501 </li>
1502 <li>
1503 <p>translating object content between object formats</p>
1504 </li>
1505 <li>
1506 <p>generating and verifying signatures in the compat format</p>
1507 </li>
1508 <li>
1509 <p>adding appropriate index entries when adding a new object to the
1510 object store</p>
1511 </li>
1512 <li>
1513 <p>--output-format option</p>
1514 </li>
1515 <li>
1516 <p>^{sha1} and ^{sha256} revision notation</p>
1517 </li>
1518 <li>
1519 <p>configuration to specify default input and output format (see
1520 "Object names on the command line" above)</p>
1521 </li>
1522 </ul>
1523 </div>
1524 <div class="paragraph">
1525 <p>The next step is supporting fetches and pushes to SHA-1 repositories:</p>
1526 </div>
1527 <div class="ulist">
1528 <ul>
1529 <li>
1530 <p>allow pushes to a repository using the compat format</p>
1531 </li>
1532 <li>
1533 <p>generate a topologically sorted list of the SHA-1 names of fetched
1534 objects</p>
1535 </li>
1536 <li>
1537 <p>convert the fetched packfile to SHA-256 format and generate an idx
1538 file</p>
1539 </li>
1540 <li>
1541 <p>re-sort to match the order of objects in the fetched packfile</p>
1542 </li>
1543 </ul>
1544 </div>
1545 <div class="paragraph">
1546 <p>The infrastructure supporting fetch also allows converting an existing
1547 repository. In converted repositories and new clones, end users can
1548 gain support for the new hash function without any visible change in
1549 behavior (see "dark launch" in the "Object names on the command line"
1550 section). In particular this allows users to verify SHA-256 signatures
1551 on objects in the repository, and it should ensure the transition code
1552 is stable in production in preparation for using it more widely.</p>
1553 </div>
1554 <div class="paragraph">
1555 <p>Over time projects would encourage their users to adopt the "early
1556 transition" and then "late transition" modes to take advantage of the
1557 new, more futureproof SHA-256 object names.</p>
1558 </div>
1559 <div class="paragraph">
1560 <p>When objectFormat and compatObjectFormat are both set, commands
1561 generating signatures would generate both SHA-1 and SHA-256 signatures
1562 by default to support both new and old users.</p>
1563 </div>
1564 <div class="paragraph">
1565 <p>In projects using SHA-256 heavily, users could be encouraged to adopt
1566 the "post-transition" mode to avoid accidentally making implicit use
1567 of SHA-1 object names.</p>
1568 </div>
1569 <div class="paragraph">
1570 <p>Once a critical mass of users have upgraded to a version of Git that
1571 can verify SHA-256 signatures and have converted their existing
1572 repositories to support verifying them, we can add support for a
1573 setting to generate only SHA-256 signatures. This is expected to be at
1574 least a year later.</p>
1575 </div>
1576 <div class="paragraph">
1577 <p>That is also a good moment to advertise the ability to convert
1578 repositories to use SHA-256 only, stripping out all SHA-1 related
1579 metadata. This improves performance by eliminating translation
1580 overhead and security by avoiding the possibility of accidentally
1581 relying on the safety of SHA-1.</p>
1582 </div>
1583 <div class="paragraph">
1584 <p>Updating Git&#8217;s protocols to allow a server to specify which hash
1585 functions it supports is also an important part of this transition. It
1586 is not discussed in detail in this document but this transition plan
1587 assumes it happens. :)</p>
1588 </div>
1589 </div>
1590 </div>
1591 <div class="sect1">
1592 <h2 id="_alternatives_considered">Alternatives considered</h2>
1593 <div class="sectionbody">
1594 <div class="sect2">
1595 <h3 id="_upgrading_everyone_working_on_a_particular_project_on_a_flag_day">Upgrading everyone working on a particular project on a flag day</h3>
1596 <div class="paragraph">
1597 <p>Projects like the Linux kernel are large and complex enough that
1598 flipping the switch for all projects based on the repository at once
1599 is infeasible.</p>
1600 </div>
1601 <div class="paragraph">
1602 <p>Not only would all developers and server operators supporting
1603 developers have to switch on the same flag day, but supporting tooling
1604 (continuous integration, code review, bug trackers, etc) would have to
1605 be adapted as well. This also makes it difficult to get early feedback
1606 from some project participants testing before it is time for mass
1607 adoption.</p>
1608 </div>
1609 </div>
1610 <div class="sect2">
1611 <h3 id="_using_hash_functions_in_parallel">Using hash functions in parallel</h3>
1612 <div class="paragraph">
1613 <p>(e.g. <a href="https://lore.kernel.org/git/22708.8913.864049.452252@chiark.greenend.org.uk/" class="bare">https://lore.kernel.org/git/22708.8913.864049.452252@chiark.greenend.org.uk/</a> )
1614 Objects newly created would be addressed by the new hash, but inside
1615 such an object (e.g. commit) it is still possible to address objects
1616 using the old hash function.</p>
1617 </div>
1618 <div class="ulist">
1619 <ul>
1620 <li>
1621 <p>You cannot trust its history (needed for bisectability) in the
1622 future without further work</p>
1623 </li>
1624 <li>
1625 <p>Maintenance burden as the number of supported hash functions grows
1626 (they will never go away, so they accumulate). In this proposal, by
1627 comparison, converted objects lose all references to SHA-1.</p>
1628 </li>
1629 </ul>
1630 </div>
1631 </div>
1632 <div class="sect2">
1633 <h3 id="_signed_objects_with_multiple_hashes">Signed objects with multiple hashes</h3>
1634 <div class="paragraph">
1635 <p>Instead of introducing the gpgsig-sha256 field in commit and tag objects
1636 for SHA-256 content based signatures, an earlier version of this design
1637 added "hash sha256 &lt;SHA-256 name&gt;" fields to strengthen the existing
1638 SHA-1 content based signatures.</p>
1639 </div>
1640 <div class="paragraph">
1641 <p>In other words, a single signature was used to attest to the object
1642 content using both hash functions. This had some advantages:</p>
1643 </div>
1644 <div class="ulist">
1645 <ul>
1646 <li>
1647 <p>Using one signature instead of two speeds up the signing process.</p>
1648 </li>
1649 <li>
1650 <p>Having one signed payload with both hashes allows the signer to
1651 attest to the SHA-1 name and SHA-256 name referring to the same object.</p>
1652 </li>
1653 <li>
1654 <p>All users consume the same signature. Broken signatures are likely
1655 to be detected quickly using current versions of git.</p>
1656 </li>
1657 </ul>
1658 </div>
1659 <div class="paragraph">
1660 <p>However, it also came with disadvantages:</p>
1661 </div>
1662 <div class="ulist">
1663 <ul>
1664 <li>
1665 <p>Verifying a signed object requires access to the SHA-1 names of all
1666 objects it references, even after the transition is complete and
1667 translation table is no longer needed for anything else. To support
1668 this, the design added fields such as "hash sha1 tree &lt;SHA-1 name&gt;"
1669 and "hash sha1 parent &lt;SHA-1 name&gt;" to the SHA-256 content of a signed
1670 commit, complicating the conversion process.</p>
1671 </li>
1672 <li>
1673 <p>Allowing signed objects without a SHA-1 (for after the transition is
1674 complete) complicated the design further, requiring a "nohash sha1"
1675 field to suppress including "hash sha1" fields in the SHA-256 content
1676 and signed payload.</p>
1677 </li>
1678 </ul>
1679 </div>
1680 </div>
1681 <div class="sect2">
1682 <h3 id="_lazily_populated_translation_table">Lazily populated translation table</h3>
1683 <div class="paragraph">
1684 <p>Some of the work of building the translation table could be deferred to
1685 push time, but that would significantly complicate and slow down pushes.
1686 Calculating the SHA-1 name at object creation time at the same time it is
1687 being streamed to disk and having its SHA-256 name calculated should be
1688 an acceptable cost.</p>
1689 </div>
1690 </div>
1691 </div>
1692 </div>
1693 <div class="sect1">
1694 <h2 id="_document_history">Document History</h2>
1695 <div class="sectionbody">
1696 <div class="paragraph">
1697 <p>2017-03-03
1698 <a href="mailto:bmwill@google.com">bmwill@google.com</a>, <a href="mailto:jonathantanmy@google.com">jonathantanmy@google.com</a>, <a href="mailto:jrnieder@gmail.com">jrnieder@gmail.com</a>,
1699 <a href="mailto:sbeller@google.com">sbeller@google.com</a></p>
1700 </div>
1701 <div class="ulist">
1702 <ul>
1703 <li>
1704 <p>Initial version sent to <a href="https://lore.kernel.org/git/20170304011251.GA26789@aiede.mtv.corp.google.com" class="bare">https://lore.kernel.org/git/20170304011251.GA26789@aiede.mtv.corp.google.com</a></p>
1705 </li>
1706 </ul>
1707 </div>
1708 <div class="paragraph">
1709 <p>2017-03-03 <a href="mailto:jrnieder@gmail.com">jrnieder@gmail.com</a>
1710 Incorporated suggestions from jonathantanmy and sbeller:</p>
1711 </div>
1712 <div class="ulist">
1713 <ul>
1714 <li>
1715 <p>Describe purpose of signed objects with each hash type</p>
1716 </li>
1717 <li>
1718 <p>Redefine signed object verification using object content under the
1719 first hash function</p>
1720 </li>
1721 </ul>
1722 </div>
1723 <div class="paragraph">
1724 <p>2017-03-06 <a href="mailto:jrnieder@gmail.com">jrnieder@gmail.com</a></p>
1725 </div>
1726 <div class="ulist">
1727 <ul>
1728 <li>
1729 <p>Use SHA3-256 instead of SHA2 (thanks, Linus and brian m. carlson).[1][2]</p>
1730 </li>
1731 <li>
1732 <p>Make SHA3-based signatures a separate field, avoiding the need for
1733 "hash" and "nohash" fields (thanks to peff[3]).</p>
1734 </li>
1735 <li>
1736 <p>Add a sorting phase to fetch (thanks to Junio for noticing the need
1737 for this).</p>
1738 </li>
1739 <li>
1740 <p>Omit blobs from the topological sort during fetch (thanks to peff).</p>
1741 </li>
1742 <li>
1743 <p>Discuss alternates, git notes, and git servers in the caveats
1744 section (thanks to Junio Hamano, brian m. carlson[4], and Shawn
1745 Pearce).</p>
1746 </li>
1747 <li>
1748 <p>Clarify language throughout (thanks to various commenters,
1749 especially Junio).</p>
1750 </li>
1751 </ul>
1752 </div>
1753 <div class="paragraph">
1754 <p>2017-09-27 <a href="mailto:jrnieder@gmail.com">jrnieder@gmail.com</a>, <a href="mailto:sbeller@google.com">sbeller@google.com</a></p>
1755 </div>
1756 <div class="ulist">
1757 <ul>
1758 <li>
1759 <p>Use placeholder NewHash instead of SHA3-256</p>
1760 </li>
1761 <li>
1762 <p>Describe criteria for picking a hash function.</p>
1763 </li>
1764 <li>
1765 <p>Include a transition plan (thanks especially to Brandon Williams
1766 for fleshing these ideas out)</p>
1767 </li>
1768 <li>
1769 <p>Define the translation table (thanks, Shawn Pearce[5], Jonathan
1770 Tan, and Masaya Suzuki)</p>
1771 </li>
1772 <li>
1773 <p>Avoid loose object overhead by packing more aggressively in
1774 "git gc --auto"</p>
1775 </li>
1776 </ul>
1777 </div>
1778 <div class="paragraph">
1779 <p>Later history:</p>
1780 </div>
1781 <div class="ulist">
1782 <ul>
1783 <li>
1784 <p>See the history of this file in git.git for the history of subsequent
1785 edits. This document history is no longer being maintained as it
1786 would now be superfluous to the commit log</p>
1787 </li>
1788 </ul>
1789 </div>
1790 <div class="paragraph">
1791 <p>References:</p>
1792 </div>
1793 <div class="literalblock">
1794 <div class="content">
1795 <pre>[1] https://lore.kernel.org/git/CA+55aFzJtejiCjV0e43+9oR3QuJK2PiFiLQemytoLpyJWe6P9w@mail.gmail.com/
1796 [2] https://lore.kernel.org/git/CA+55aFz+gkAsDZ24zmePQuEs1XPS9BP_s8O7Q4wQ7LV7X5-oDA@mail.gmail.com/
1797 [3] https://lore.kernel.org/git/20170306084353.nrns455dvkdsfgo5@sigill.intra.peff.net/
1798 [4] https://lore.kernel.org/git/20170304224936.rqqtkdvfjgyezsht@genre.crustytoothpaste.net
1799 [5] https://lore.kernel.org/git/CAJo=hJtoX9=AyLHHpUJS7fueV9ciZ_MNpnEPHUz8Whui6g9F0A@mail.gmail.com/</pre>
1800 </div>
1801 </div>
1802 </div>
1803 </div>
1804 </div>
1805 <div id="footer">
1806 <div id="footer-text">
1807 Last updated 2024-11-07 20:36:53 -0800
1808 </div>
1809 </div>
1810 </body>
1811 </html>