technical/hash-function-transition.html

   1 <!DOCTYPE html>
   2 <html xmlns="http://www.w3.org/1999/xhtml" lang="en">
   3 <head>
   4 <meta charset="UTF-8"/>
   5 <meta http-equiv="X-UA-Compatible" content="IE=edge"/>
   6 <meta name="viewport" content="width=device-width, initial-scale=1.0"/>
   7 <meta name="generator" content="Asciidoctor 2.0.20"/>
   8 <title>Git hash function transition</title>
   9 <link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700"/>
  10 <style>
  11 /*! Asciidoctor default stylesheet | MIT License | https://asciidoctor.org */
  12 /* Uncomment the following line when using as a custom stylesheet */
  13 /* @import "https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700"; */
  14 html{font-family:sans-serif;-webkit-text-size-adjust:100%}
  15 a{background:none}
  16 a:focus{outline:thin dotted}
  17 a:active,a:hover{outline:0}
  18 h1{font-size:2em;margin:.67em 0}
  19 b,strong{font-weight:bold}
  20 abbr{font-size:.9em}
  21 abbr[title]{cursor:help;border-bottom:1px dotted #dddddf;text-decoration:none}
  22 dfn{font-style:italic}
  23 hr{height:0}
  24 mark{background:#ff0;color:#000}
  25 code,kbd,pre,samp{font-family:monospace;font-size:1em}
  26 pre{white-space:pre-wrap}
  27 q{quotes:"\201C" "\201D" "\2018" "\2019"}
  28 small{font-size:80%}
  29 sub,sup{font-size:75%;line-height:0;position:relative;vertical-align:baseline}
  30 sup{top:-.5em}
  31 sub{bottom:-.25em}
  32 img{border:0}
  33 svg:not(:root){overflow:hidden}
  34 figure{margin:0}
  35 audio,video{display:inline-block}
  36 audio:not([controls]){display:none;height:0}
  37 fieldset{border:1px solid silver;margin:0 2px;padding:.35em .625em .75em}
  38 legend{border:0;padding:0}
  39 button,input,select,textarea{font-family:inherit;font-size:100%;margin:0}
  40 button,input{line-height:normal}
  41 button,select{text-transform:none}
  42 button,html input[type=button],input[type=reset],input[type=submit]{-webkit-appearance:button;cursor:pointer}
  43 button[disabled],html input[disabled]{cursor:default}
  44 input[type=checkbox],input[type=radio]{padding:0}
  45 button::-moz-focus-inner,input::-moz-focus-inner{border:0;padding:0}
  46 textarea{overflow:auto;vertical-align:top}
  47 table{border-collapse:collapse;border-spacing:0}
  48 *,::before,::after{box-sizing:border-box}
  49 html,body{font-size:100%}
  50 body{background:#fff;color:rgba(0,0,0,.8);padding:0;margin:0;font-family:"Noto Serif","DejaVu Serif",serif;line-height:1;position:relative;cursor:auto;-moz-tab-size:4;-o-tab-size:4;tab-size:4;word-wrap:anywhere;-moz-osx-font-smoothing:grayscale;-webkit-font-smoothing:antialiased}
  51 a:hover{cursor:pointer}
  52 img,object,embed{max-width:100%;height:auto}
  53 object,embed{height:100%}
  54 img{-ms-interpolation-mode:bicubic}
  55 .left{float:left!important}
  56 .right{float:right!important}
  57 .text-left{text-align:left!important}
  58 .text-right{text-align:right!important}
  59 .text-center{text-align:center!important}
  60 .text-justify{text-align:justify!important}
  61 .hide{display:none}
  62 img,object,svg{display:inline-block;vertical-align:middle}
  63 textarea{height:auto;min-height:50px}
  64 select{width:100%}
  65 .subheader,.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{line-height:1.45;color:#7a2518;font-weight:400;margin-top:0;margin-bottom:.25em}
  66 div,dl,dt,dd,ul,ol,li,h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6,pre,form,p,blockquote,th,td{margin:0;padding:0}
  67 a{color:#2156a5;text-decoration:underline;line-height:inherit}
  68 a:hover,a:focus{color:#1d4b8f}
  69 a img{border:0}
  70 p{line-height:1.6;margin-bottom:1.25em;text-rendering:optimizeLegibility}
  71 p aside{font-size:.875em;line-height:1.35;font-style:italic}
  72 h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{font-family:"Open Sans","DejaVu Sans",sans-serif;font-weight:300;font-style:normal;color:#ba3925;text-rendering:optimizeLegibility;margin-top:1em;margin-bottom:.5em;line-height:1.0125em}
  73 h1 small,h2 small,h3 small,#toctitle small,.sidebarblock>.content>.title small,h4 small,h5 small,h6 small{font-size:60%;color:#e99b8f;line-height:0}
  74 h1{font-size:2.125em}
  75 h2{font-size:1.6875em}
  76 h3,#toctitle,.sidebarblock>.content>.title{font-size:1.375em}
  77 h4,h5{font-size:1.125em}
  78 h6{font-size:1em}
  79 hr{border:solid #dddddf;border-width:1px 0 0;clear:both;margin:1.25em 0 1.1875em}
  80 em,i{font-style:italic;line-height:inherit}
  81 strong,b{font-weight:bold;line-height:inherit}
  82 small{font-size:60%;line-height:inherit}
  83 code{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;font-weight:400;color:rgba(0,0,0,.9)}
  84 ul,ol,dl{line-height:1.6;margin-bottom:1.25em;list-style-position:outside;font-family:inherit}
  85 ul,ol{margin-left:1.5em}
  86 ul li ul,ul li ol{margin-left:1.25em;margin-bottom:0}
  87 ul.circle{list-style-type:circle}
  88 ul.disc{list-style-type:disc}
  89 ul.square{list-style-type:square}
  90 ul.circle ul:not([class]),ul.disc ul:not([class]),ul.square ul:not([class]){list-style:inherit}
  91 ol li ul,ol li ol{margin-left:1.25em;margin-bottom:0}
  92 dl dt{margin-bottom:.3125em;font-weight:bold}
  93 dl dd{margin-bottom:1.25em}
  94 blockquote{margin:0 0 1.25em;padding:.5625em 1.25em 0 1.1875em;border-left:1px solid #ddd}
  95 blockquote,blockquote p{line-height:1.6;color:rgba(0,0,0,.85)}
  96 @media screen and (min-width:768px){h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2}
  97 h1{font-size:2.75em}
  98 h2{font-size:2.3125em}
  99 h3,#toctitle,.sidebarblock>.content>.title{font-size:1.6875em}
 100 h4{font-size:1.4375em}}
 101 table{background:#fff;margin-bottom:1.25em;border:1px solid #dedede;word-wrap:normal}
 102 table thead,table tfoot{background:#f7f8f7}
 103 table thead tr th,table thead tr td,table tfoot tr th,table tfoot tr td{padding:.5em .625em .625em;font-size:inherit;color:rgba(0,0,0,.8);text-align:left}
 104 table tr th,table tr td{padding:.5625em .625em;font-size:inherit;color:rgba(0,0,0,.8)}
 105 table tr.even,table tr.alt{background:#f8f8f7}
 106 table thead tr th,table tfoot tr th,table tbody tr td,table tr td,table tfoot tr td{line-height:1.6}
 107 h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2;word-spacing:-.05em}
 108 h1 strong,h2 strong,h3 strong,#toctitle strong,.sidebarblock>.content>.title strong,h4 strong,h5 strong,h6 strong{font-weight:400}
 109 .center{margin-left:auto;margin-right:auto}
 110 .stretch{width:100%}
 111 .clearfix::before,.clearfix::after,.float-group::before,.float-group::after{content:" ";display:table}
 112 .clearfix::after,.float-group::after{clear:both}
 113 :not(pre).nobreak{word-wrap:normal}
 114 :not(pre).nowrap{white-space:nowrap}
 115 :not(pre).pre-wrap{white-space:pre-wrap}
 116 :not(pre):not([class^=L])>code{font-size:.9375em;font-style:normal!important;letter-spacing:0;padding:.1em .5ex;word-spacing:-.15em;background:#f7f7f8;border-radius:4px;line-height:1.45;text-rendering:optimizeSpeed}
 117 pre{color:rgba(0,0,0,.9);font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;line-height:1.45;text-rendering:optimizeSpeed}
 118 pre code,pre pre{color:inherit;font-size:inherit;line-height:inherit}
 119 pre>code{display:block}
 120 pre.nowrap,pre.nowrap pre{white-space:pre;word-wrap:normal}
 121 em em{font-style:normal}
 122 strong strong{font-weight:400}
 123 .keyseq{color:rgba(51,51,51,.8)}
 124 kbd{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;display:inline-block;color:rgba(0,0,0,.8);font-size:.65em;line-height:1.45;background:#f7f7f7;border:1px solid #ccc;border-radius:3px;box-shadow:0 1px 0 rgba(0,0,0,.2),inset 0 0 0 .1em #fff;margin:0 .15em;padding:.2em .5em;vertical-align:middle;position:relative;top:-.1em;white-space:nowrap}
 125 .keyseq kbd:first-child{margin-left:0}
 126 .keyseq kbd:last-child{margin-right:0}
 127 .menuseq,.menuref{color:#000}
 128 .menuseq b:not(.caret),.menuref{font-weight:inherit}
 129 .menuseq{word-spacing:-.02em}
 130 .menuseq b.caret{font-size:1.25em;line-height:.8}
 131 .menuseq i.caret{font-weight:bold;text-align:center;width:.45em}
 132 b.button::before,b.button::after{position:relative;top:-1px;font-weight:400}
 133 b.button::before{content:"[";padding:0 3px 0 2px}
 134 b.button::after{content:"]";padding:0 2px 0 3px}
 135 p a>code:hover{color:rgba(0,0,0,.9)}
 136 #header,#content,#footnotes,#footer{width:100%;margin:0 auto;max-width:62.5em;*zoom:1;position:relative;padding-left:.9375em;padding-right:.9375em}
 137 #header::before,#header::after,#content::before,#content::after,#footnotes::before,#footnotes::after,#footer::before,#footer::after{content:" ";display:table}
 138 #header::after,#content::after,#footnotes::after,#footer::after{clear:both}
 139 #content{margin-top:1.25em}
 140 #content::before{content:none}
 141 #header>h1:first-child{color:rgba(0,0,0,.85);margin-top:2.25rem;margin-bottom:0}
 142 #header>h1:first-child+#toc{margin-top:8px;border-top:1px solid #dddddf}
 143 #header>h1:only-child,body.toc2 #header>h1:nth-last-child(2){border-bottom:1px solid #dddddf;padding-bottom:8px}
 144 #header .details{border-bottom:1px solid #dddddf;line-height:1.45;padding-top:.25em;padding-bottom:.25em;padding-left:.25em;color:rgba(0,0,0,.6);display:flex;flex-flow:row wrap}
 145 #header .details span:first-child{margin-left:-.125em}
 146 #header .details span.email a{color:rgba(0,0,0,.85)}
 147 #header .details br{display:none}
 148 #header .details br+span::before{content:"\00a0\2013\00a0"}
 149 #header .details br+span.author::before{content:"\00a0\22c5\00a0";color:rgba(0,0,0,.85)}
 150 #header .details br+span#revremark::before{content:"\00a0|\00a0"}
 151 #header #revnumber{text-transform:capitalize}
 152 #header #revnumber::after{content:"\00a0"}
 153 #content>h1:first-child:not([class]){color:rgba(0,0,0,.85);border-bottom:1px solid #dddddf;padding-bottom:8px;margin-top:0;padding-top:1rem;margin-bottom:1.25rem}
 154 #toc{border-bottom:1px solid #e7e7e9;padding-bottom:.5em}
 155 #toc>ul{margin-left:.125em}
 156 #toc ul.sectlevel0>li>a{font-style:italic}
 157 #toc ul.sectlevel0 ul.sectlevel1{margin:.5em 0}
 158 #toc ul{font-family:"Open Sans","DejaVu Sans",sans-serif;list-style-type:none}
 159 #toc li{line-height:1.3334;margin-top:.3334em}
 160 #toc a{text-decoration:none}
 161 #toc a:active{text-decoration:underline}
 162 #toctitle{color:#7a2518;font-size:1.2em}
 163 @media screen and (min-width:768px){#toctitle{font-size:1.375em}
 164 body.toc2{padding-left:15em;padding-right:0}
 165 #toc.toc2{margin-top:0!important;background:#f8f8f7;position:fixed;width:15em;left:0;top:0;border-right:1px solid #e7e7e9;border-top-width:0!important;border-bottom-width:0!important;z-index:1000;padding:1.25em 1em;height:100%;overflow:auto}
 166 #toc.toc2 #toctitle{margin-top:0;margin-bottom:.8rem;font-size:1.2em}
 167 #toc.toc2>ul{font-size:.9em;margin-bottom:0}
 168 #toc.toc2 ul ul{margin-left:0;padding-left:1em}
 169 #toc.toc2 ul.sectlevel0 ul.sectlevel1{padding-left:0;margin-top:.5em;margin-bottom:.5em}
 170 body.toc2.toc-right{padding-left:0;padding-right:15em}
 171 body.toc2.toc-right #toc.toc2{border-right-width:0;border-left:1px solid #e7e7e9;left:auto;right:0}}
 172 @media screen and (min-width:1280px){body.toc2{padding-left:20em;padding-right:0}
 173 #toc.toc2{width:20em}
 174 #toc.toc2 #toctitle{font-size:1.375em}
 175 #toc.toc2>ul{font-size:.95em}
 176 #toc.toc2 ul ul{padding-left:1.25em}
 177 body.toc2.toc-right{padding-left:0;padding-right:20em}}
 178 #content #toc{border:1px solid #e0e0dc;margin-bottom:1.25em;padding:1.25em;background:#f8f8f7;border-radius:4px}
 179 #content #toc>:first-child{margin-top:0}
 180 #content #toc>:last-child{margin-bottom:0}
 181 #footer{max-width:none;background:rgba(0,0,0,.8);padding:1.25em}
 182 #footer-text{color:hsla(0,0%,100%,.8);line-height:1.44}
 183 #content{margin-bottom:.625em}
 184 .sect1{padding-bottom:.625em}
 185 @media screen and (min-width:768px){#content{margin-bottom:1.25em}
 186 .sect1{padding-bottom:1.25em}}
 187 .sect1:last-child{padding-bottom:0}
 188 .sect1+.sect1{border-top:1px solid #e7e7e9}
 189 #content h1>a.anchor,h2>a.anchor,h3>a.anchor,#toctitle>a.anchor,.sidebarblock>.content>.title>a.anchor,h4>a.anchor,h5>a.anchor,h6>a.anchor{position:absolute;z-index:1001;width:1.5ex;margin-left:-1.5ex;display:block;text-decoration:none!important;visibility:hidden;text-align:center;font-weight:400}
 190 #content h1>a.anchor::before,h2>a.anchor::before,h3>a.anchor::before,#toctitle>a.anchor::before,.sidebarblock>.content>.title>a.anchor::before,h4>a.anchor::before,h5>a.anchor::before,h6>a.anchor::before{content:"\00A7";font-size:.85em;display:block;padding-top:.1em}
 191 #content h1:hover>a.anchor,#content h1>a.anchor:hover,h2:hover>a.anchor,h2>a.anchor:hover,h3:hover>a.anchor,#toctitle:hover>a.anchor,.sidebarblock>.content>.title:hover>a.anchor,h3>a.anchor:hover,#toctitle>a.anchor:hover,.sidebarblock>.content>.title>a.anchor:hover,h4:hover>a.anchor,h4>a.anchor:hover,h5:hover>a.anchor,h5>a.anchor:hover,h6:hover>a.anchor,h6>a.anchor:hover{visibility:visible}
 192 #content h1>a.link,h2>a.link,h3>a.link,#toctitle>a.link,.sidebarblock>.content>.title>a.link,h4>a.link,h5>a.link,h6>a.link{color:#ba3925;text-decoration:none}
 193 #content h1>a.link:hover,h2>a.link:hover,h3>a.link:hover,#toctitle>a.link:hover,.sidebarblock>.content>.title>a.link:hover,h4>a.link:hover,h5>a.link:hover,h6>a.link:hover{color:#a53221}
 194 details,.audioblock,.imageblock,.literalblock,.listingblock,.stemblock,.videoblock{margin-bottom:1.25em}
 195 details{margin-left:1.25rem}
 196 details>summary{cursor:pointer;display:block;position:relative;line-height:1.6;margin-bottom:.625rem;outline:none;-webkit-tap-highlight-color:transparent}
 197 details>summary::-webkit-details-marker{display:none}
 198 details>summary::before{content:"";border:solid transparent;border-left:solid;border-width:.3em 0 .3em .5em;position:absolute;top:.5em;left:-1.25rem;transform:translateX(15%)}
 199 details[open]>summary::before{border:solid transparent;border-top:solid;border-width:.5em .3em 0;transform:translateY(15%)}
 200 details>summary::after{content:"";width:1.25rem;height:1em;position:absolute;top:.3em;left:-1.25rem}
 201 .admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{text-rendering:optimizeLegibility;text-align:left;font-family:"Noto Serif","DejaVu Serif",serif;font-size:1rem;font-style:italic}
 202 table.tableblock.fit-content>caption.title{white-space:nowrap;width:0}
 203 .paragraph.lead>p,#preamble>.sectionbody>[class=paragraph]:first-of-type p{font-size:1.21875em;line-height:1.6;color:rgba(0,0,0,.85)}
 204 .admonitionblock>table{border-collapse:separate;border:0;background:none;width:100%}
 205 .admonitionblock>table td.icon{text-align:center;width:80px}
 206 .admonitionblock>table td.icon img{max-width:none}
 207 .admonitionblock>table td.icon .title{font-weight:bold;font-family:"Open Sans","DejaVu Sans",sans-serif;text-transform:uppercase}
 208 .admonitionblock>table td.content{padding-left:1.125em;padding-right:1.25em;border-left:1px solid #dddddf;color:rgba(0,0,0,.6);word-wrap:anywhere}
 209 .admonitionblock>table td.content>:last-child>:last-child{margin-bottom:0}
 210 .exampleblock>.content{border:1px solid #e6e6e6;margin-bottom:1.25em;padding:1.25em;background:#fff;border-radius:4px}
 211 .sidebarblock{border:1px solid #dbdbd6;margin-bottom:1.25em;padding:1.25em;background:#f3f3f2;border-radius:4px}
 212 .sidebarblock>.content>.title{color:#7a2518;margin-top:0;text-align:center}
 213 .exampleblock>.content>:first-child,.sidebarblock>.content>:first-child{margin-top:0}
 214 .exampleblock>.content>:last-child,.exampleblock>.content>:last-child>:last-child,.exampleblock>.content .olist>ol>li:last-child>:last-child,.exampleblock>.content .ulist>ul>li:last-child>:last-child,.exampleblock>.content .qlist>ol>li:last-child>:last-child,.sidebarblock>.content>:last-child,.sidebarblock>.content>:last-child>:last-child,.sidebarblock>.content .olist>ol>li:last-child>:last-child,.sidebarblock>.content .ulist>ul>li:last-child>:last-child,.sidebarblock>.content .qlist>ol>li:last-child>:last-child{margin-bottom:0}
 215 .literalblock pre,.listingblock>.content>pre{border-radius:4px;overflow-x:auto;padding:1em;font-size:.8125em}
 216 @media screen and (min-width:768px){.literalblock pre,.listingblock>.content>pre{font-size:.90625em}}
 217 @media screen and (min-width:1280px){.literalblock pre,.listingblock>.content>pre{font-size:1em}}
 218 .literalblock pre,.listingblock>.content>pre:not(.highlight),.listingblock>.content>pre[class=highlight],.listingblock>.content>pre[class^="highlight "]{background:#f7f7f8}
 219 .literalblock.output pre{color:#f7f7f8;background:rgba(0,0,0,.9)}
 220 .listingblock>.content{position:relative}
 221 .listingblock code[data-lang]::before{display:none;content:attr(data-lang);position:absolute;font-size:.75em;top:.425rem;right:.5rem;line-height:1;text-transform:uppercase;color:inherit;opacity:.5}
 222 .listingblock:hover code[data-lang]::before{display:block}
 223 .listingblock.terminal pre .command::before{content:attr(data-prompt);padding-right:.5em;color:inherit;opacity:.5}
 224 .listingblock.terminal pre .command:not([data-prompt])::before{content:"$"}
 225 .listingblock pre.highlightjs{padding:0}
 226 .listingblock pre.highlightjs>code{padding:1em;border-radius:4px}
 227 .listingblock pre.prettyprint{border-width:0}
 228 .prettyprint{background:#f7f7f8}
 229 pre.prettyprint .linenums{line-height:1.45;margin-left:2em}
 230 pre.prettyprint li{background:none;list-style-type:inherit;padding-left:0}
 231 pre.prettyprint li code[data-lang]::before{opacity:1}
 232 pre.prettyprint li:not(:first-child) code[data-lang]::before{display:none}
 233 table.linenotable{border-collapse:separate;border:0;margin-bottom:0;background:none}
 234 table.linenotable td[class]{color:inherit;vertical-align:top;padding:0;line-height:inherit;white-space:normal}
 235 table.linenotable td.code{padding-left:.75em}
 236 table.linenotable td.linenos,pre.pygments .linenos{border-right:1px solid;opacity:.35;padding-right:.5em;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none}
 237 pre.pygments span.linenos{display:inline-block;margin-right:.75em}
 238 .quoteblock{margin:0 1em 1.25em 1.5em;display:table}
 239 .quoteblock:not(.excerpt)>.title{margin-left:-1.5em;margin-bottom:.75em}
 240 .quoteblock blockquote,.quoteblock p{color:rgba(0,0,0,.85);font-size:1.15rem;line-height:1.75;word-spacing:.1em;letter-spacing:0;font-style:italic;text-align:justify}
 241 .quoteblock blockquote{margin:0;padding:0;border:0}
 242 .quoteblock blockquote::before{content:"\201c";float:left;font-size:2.75em;font-weight:bold;line-height:.6em;margin-left:-.6em;color:#7a2518;text-shadow:0 1px 2px rgba(0,0,0,.1)}
 243 .quoteblock blockquote>.paragraph:last-child p{margin-bottom:0}
 244 .quoteblock .attribution{margin-top:.75em;margin-right:.5ex;text-align:right}
 245 .verseblock{margin:0 1em 1.25em}
 246 .verseblock pre{font-family:"Open Sans","DejaVu Sans",sans-serif;font-size:1.15rem;color:rgba(0,0,0,.85);font-weight:300;text-rendering:optimizeLegibility}
 247 .verseblock pre strong{font-weight:400}
 248 .verseblock .attribution{margin-top:1.25rem;margin-left:.5ex}
 249 .quoteblock .attribution,.verseblock .attribution{font-size:.9375em;line-height:1.45;font-style:italic}
 250 .quoteblock .attribution br,.verseblock .attribution br{display:none}
 251 .quoteblock .attribution cite,.verseblock .attribution cite{display:block;letter-spacing:-.025em;color:rgba(0,0,0,.6)}
 252 .quoteblock.abstract blockquote::before,.quoteblock.excerpt blockquote::before,.quoteblock .quoteblock blockquote::before{display:none}
 253 .quoteblock.abstract blockquote,.quoteblock.abstract p,.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{line-height:1.6;word-spacing:0}
 254 .quoteblock.abstract{margin:0 1em 1.25em;display:block}
 255 .quoteblock.abstract>.title{margin:0 0 .375em;font-size:1.15em;text-align:center}
 256 .quoteblock.excerpt>blockquote,.quoteblock .quoteblock{padding:0 0 .25em 1em;border-left:.25em solid #dddddf}
 257 .quoteblock.excerpt,.quoteblock .quoteblock{margin-left:0}
 258 .quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{color:inherit;font-size:1.0625rem}
 259 .quoteblock.excerpt .attribution,.quoteblock .quoteblock .attribution{color:inherit;font-size:.85rem;text-align:left;margin-right:0}
 260 p.tableblock:last-child{margin-bottom:0}
 261 td.tableblock>.content{margin-bottom:1.25em;word-wrap:anywhere}
 262 td.tableblock>.content>:last-child{margin-bottom:-1.25em}
 263 table.tableblock,th.tableblock,td.tableblock{border:0 solid #dedede}
 264 table.grid-all>*>tr>*{border-width:1px}
 265 table.grid-cols>*>tr>*{border-width:0 1px}
 266 table.grid-rows>*>tr>*{border-width:1px 0}
 267 table.frame-all{border-width:1px}
 268 table.frame-ends{border-width:1px 0}
 269 table.frame-sides{border-width:0 1px}
 270 table.frame-none>colgroup+*>:first-child>*,table.frame-sides>colgroup+*>:first-child>*{border-top-width:0}
 271 table.frame-none>:last-child>:last-child>*,table.frame-sides>:last-child>:last-child>*{border-bottom-width:0}
 272 table.frame-none>*>tr>:first-child,table.frame-ends>*>tr>:first-child{border-left-width:0}
 273 table.frame-none>*>tr>:last-child,table.frame-ends>*>tr>:last-child{border-right-width:0}
 274 table.stripes-all>*>tr,table.stripes-odd>*>tr:nth-of-type(odd),table.stripes-even>*>tr:nth-of-type(even),table.stripes-hover>*>tr:hover{background:#f8f8f7}
 275 th.halign-left,td.halign-left{text-align:left}
 276 th.halign-right,td.halign-right{text-align:right}
 277 th.halign-center,td.halign-center{text-align:center}
 278 th.valign-top,td.valign-top{vertical-align:top}
 279 th.valign-bottom,td.valign-bottom{vertical-align:bottom}
 280 th.valign-middle,td.valign-middle{vertical-align:middle}
 281 table thead th,table tfoot th{font-weight:bold}
 282 tbody tr th{background:#f7f8f7}
 283 tbody tr th,tbody tr th p,tfoot tr th,tfoot tr th p{color:rgba(0,0,0,.8);font-weight:bold}
 284 p.tableblock>code:only-child{background:none;padding:0}
 285 p.tableblock{font-size:1em}
 286 ol{margin-left:1.75em}
 287 ul li ol{margin-left:1.5em}
 288 dl dd{margin-left:1.125em}
 289 dl dd:last-child,dl dd:last-child>:last-child{margin-bottom:0}
 290 li p,ul dd,ol dd,.olist .olist,.ulist .ulist,.ulist .olist,.olist .ulist{margin-bottom:.625em}
 291 ul.checklist,ul.none,ol.none,ul.no-bullet,ol.no-bullet,ol.unnumbered,ul.unstyled,ol.unstyled{list-style-type:none}
 292 ul.no-bullet,ol.no-bullet,ol.unnumbered{margin-left:.625em}
 293 ul.unstyled,ol.unstyled{margin-left:0}
 294 li>p:empty:only-child::before{content:"";display:inline-block}
 295 ul.checklist>li>p:first-child{margin-left:-1em}
 296 ul.checklist>li>p:first-child>.fa-square-o:first-child,ul.checklist>li>p:first-child>.fa-check-square-o:first-child{width:1.25em;font-size:.8em;position:relative;bottom:.125em}
 297 ul.checklist>li>p:first-child>input[type=checkbox]:first-child{margin-right:.25em}
 298 ul.inline{display:flex;flex-flow:row wrap;list-style:none;margin:0 0 .625em -1.25em}
 299 ul.inline>li{margin-left:1.25em}
 300 .unstyled dl dt{font-weight:400;font-style:normal}
 301 ol.arabic{list-style-type:decimal}
 302 ol.decimal{list-style-type:decimal-leading-zero}
 303 ol.loweralpha{list-style-type:lower-alpha}
 304 ol.upperalpha{list-style-type:upper-alpha}
 305 ol.lowerroman{list-style-type:lower-roman}
 306 ol.upperroman{list-style-type:upper-roman}
 307 ol.lowergreek{list-style-type:lower-greek}
 308 .hdlist>table,.colist>table{border:0;background:none}
 309 .hdlist>table>tbody>tr,.colist>table>tbody>tr{background:none}
 310 td.hdlist1,td.hdlist2{vertical-align:top;padding:0 .625em}
 311 td.hdlist1{font-weight:bold;padding-bottom:1.25em}
 312 td.hdlist2{word-wrap:anywhere}
 313 .literalblock+.colist,.listingblock+.colist{margin-top:-.5em}
 314 .colist td:not([class]):first-child{padding:.4em .75em 0;line-height:1;vertical-align:top}
 315 .colist td:not([class]):first-child img{max-width:none}
 316 .colist td:not([class]):last-child{padding:.25em 0}
 317 .thumb,.th{line-height:0;display:inline-block;border:4px solid #fff;box-shadow:0 0 0 1px #ddd}
 318 .imageblock.left{margin:.25em .625em 1.25em 0}
 319 .imageblock.right{margin:.25em 0 1.25em .625em}
 320 .imageblock>.title{margin-bottom:0}
 321 .imageblock.thumb,.imageblock.th{border-width:6px}
 322 .imageblock.thumb>.title,.imageblock.th>.title{padding:0 .125em}
 323 .image.left,.image.right{margin-top:.25em;margin-bottom:.25em;display:inline-block;line-height:0}
 324 .image.left{margin-right:.625em}
 325 .image.right{margin-left:.625em}
 326 a.image{text-decoration:none;display:inline-block}
 327 a.image object{pointer-events:none}
 328 sup.footnote,sup.footnoteref{font-size:.875em;position:static;vertical-align:super}
 329 sup.footnote a,sup.footnoteref a{text-decoration:none}
 330 sup.footnote a:active,sup.footnoteref a:active{text-decoration:underline}
 331 #footnotes{padding-top:.75em;padding-bottom:.75em;margin-bottom:.625em}
 332 #footnotes hr{width:20%;min-width:6.25em;margin:-.25em 0 .75em;border-width:1px 0 0}
 333 #footnotes .footnote{padding:0 .375em 0 .225em;line-height:1.3334;font-size:.875em;margin-left:1.2em;margin-bottom:.2em}
 334 #footnotes .footnote a:first-of-type{font-weight:bold;text-decoration:none;margin-left:-1.05em}
 335 #footnotes .footnote:last-of-type{margin-bottom:0}
 336 #content #footnotes{margin-top:-.625em;margin-bottom:0;padding:.75em 0}
 337 div.unbreakable{page-break-inside:avoid}
 338 .big{font-size:larger}
 339 .small{font-size:smaller}
 340 .underline{text-decoration:underline}
 341 .overline{text-decoration:overline}
 342 .line-through{text-decoration:line-through}
 343 .aqua{color:#00bfbf}
 344 .aqua-background{background:#00fafa}
 345 .black{color:#000}
 346 .black-background{background:#000}
 347 .blue{color:#0000bf}
 348 .blue-background{background:#0000fa}
 349 .fuchsia{color:#bf00bf}
 350 .fuchsia-background{background:#fa00fa}
 351 .gray{color:#606060}
 352 .gray-background{background:#7d7d7d}
 353 .green{color:#006000}
 354 .green-background{background:#007d00}
 355 .lime{color:#00bf00}
 356 .lime-background{background:#00fa00}
 357 .maroon{color:#600000}
 358 .maroon-background{background:#7d0000}
 359 .navy{color:#000060}
 360 .navy-background{background:#00007d}
 361 .olive{color:#606000}
 362 .olive-background{background:#7d7d00}
 363 .purple{color:#600060}
 364 .purple-background{background:#7d007d}
 365 .red{color:#bf0000}
 366 .red-background{background:#fa0000}
 367 .silver{color:#909090}
 368 .silver-background{background:#bcbcbc}
 369 .teal{color:#006060}
 370 .teal-background{background:#007d7d}
 371 .white{color:#bfbfbf}
 372 .white-background{background:#fafafa}
 373 .yellow{color:#bfbf00}
 374 .yellow-background{background:#fafa00}
 375 span.icon>.fa{cursor:default}
 376 a span.icon>.fa{cursor:inherit}
 377 .admonitionblock td.icon [class^="fa icon-"]{font-size:2.5em;text-shadow:1px 1px 2px rgba(0,0,0,.5);cursor:default}
 378 .admonitionblock td.icon .icon-note::before{content:"\f05a";color:#19407c}
 379 .admonitionblock td.icon .icon-tip::before{content:"\f0eb";text-shadow:1px 1px 2px rgba(155,155,0,.8);color:#111}
 380 .admonitionblock td.icon .icon-warning::before{content:"\f071";color:#bf6900}
 381 .admonitionblock td.icon .icon-caution::before{content:"\f06d";color:#bf3400}
 382 .admonitionblock td.icon .icon-important::before{content:"\f06a";color:#bf0000}
 383 .conum[data-value]{display:inline-block;color:#fff!important;background:rgba(0,0,0,.8);border-radius:50%;text-align:center;font-size:.75em;width:1.67em;height:1.67em;line-height:1.67em;font-family:"Open Sans","DejaVu Sans",sans-serif;font-style:normal;font-weight:bold}
 384 .conum[data-value] *{color:#fff!important}
 385 .conum[data-value]+b{display:none}
 386 .conum[data-value]::after{content:attr(data-value)}
 387 pre .conum[data-value]{position:relative;top:-.125em}
 388 b.conum *{color:inherit!important}
 389 .conum:not([data-value]):empty{display:none}
 390 dt,th.tableblock,td.content,div.footnote{text-rendering:optimizeLegibility}
 391 h1,h2,p,td.content,span.alt,summary{letter-spacing:-.01em}
 392 p strong,td.content strong,div.footnote strong{letter-spacing:-.005em}
 393 p,blockquote,dt,td.content,td.hdlist1,span.alt,summary{font-size:1.0625rem}
 394 p{margin-bottom:1.25rem}
 395 .sidebarblock p,.sidebarblock dt,.sidebarblock td.content,p.tableblock{font-size:1em}
 396 .exampleblock>.content{background:#fffef7;border-color:#e0e0dc;box-shadow:0 1px 4px #e0e0dc}
 397 .print-only{display:none!important}
 398 @page{margin:1.25cm .75cm}
 399 @media print{*{box-shadow:none!important;text-shadow:none!important}
 400 html{font-size:80%}
 401 a{color:inherit!important;text-decoration:underline!important}
 402 a.bare,a[href^="#"],a[href^="mailto:"]{text-decoration:none!important}
 403 a[href^="http:"]:not(.bare)::after,a[href^="https:"]:not(.bare)::after{content:"(" attr(href) ")";display:inline-block;font-size:.875em;padding-left:.25em}
 404 abbr[title]{border-bottom:1px dotted}
 405 abbr[title]::after{content:" (" attr(title) ")"}
 406 pre,blockquote,tr,img,object,svg{page-break-inside:avoid}
 407 thead{display:table-header-group}
 408 svg{max-width:100%}
 409 p,blockquote,dt,td.content{font-size:1em;orphans:3;widows:3}
 410 h2,h3,#toctitle,.sidebarblock>.content>.title{page-break-after:avoid}
 411 #header,#content,#footnotes,#footer{max-width:none}
 412 #toc,.sidebarblock,.exampleblock>.content{background:none!important}
 413 #toc{border-bottom:1px solid #dddddf!important;padding-bottom:0!important}
 414 body.book #header{text-align:center}
 415 body.book #header>h1:first-child{border:0!important;margin:2.5em 0 1em}
 416 body.book #header .details{border:0!important;display:block;padding:0!important}
 417 body.book #header .details span:first-child{margin-left:0!important}
 418 body.book #header .details br{display:block}
 419 body.book #header .details br+span::before{content:none!important}
 420 body.book #toc{border:0!important;text-align:left!important;padding:0!important;margin:0!important}
 421 body.book #toc,body.book #preamble,body.book h1.sect0,body.book .sect1>h2{page-break-before:always}
 422 .listingblock code[data-lang]::before{display:block}
 423 #footer{padding:0 .9375em}
 424 .hide-on-print{display:none!important}
 425 .print-only{display:block!important}
 426 .hide-for-print{display:none!important}
 427 .show-for-print{display:inherit!important}}
 428 @media amzn-kf8,print{#header>h1:first-child{margin-top:1.25rem}
 429 .sect1{padding:0!important}
 430 .sect1+.sect1{border:0}
 431 #footer{background:none}
 432 #footer-text{color:rgba(0,0,0,.6);font-size:.9em}}
 433 @media amzn-kf8{#header,#content,#footnotes,#footer{padding:0}}
 434 </style>
 435 </head>
 436 <body class="article">
 437 <div id="header">
 438 <h1>Git hash function transition</h1>
 439 <div class="details">
 440 <span id="revdate">2024-10-02</span>
 441 </div>
 442 </div>
 443 <div id="content">
 444 <div class="sect1">
 445 <h2 id="_objective">Objective</h2>
 446 <div class="sectionbody">
 447 <div class="paragraph">
 448 <p>Migrate Git from SHA-1 to a stronger hash function.</p>
 449 </div>
 450 </div>
 451 </div>
 452 <div class="sect1">
 453 <h2 id="_background">Background</h2>
 454 <div class="sectionbody">
 455 <div class="paragraph">
 456 <p>At its core, the Git version control system is a content addressable
 457 filesystem. It uses the SHA-1 hash function to name content. For
 458 example, files, directories, and revisions are referred to by hash
 459 values unlike in other traditional version control systems where files
 460 or versions are referred to via sequential numbers. The use of a hash
 461 function to address its content delivers a few advantages:</p>
 462 </div>
 463 <div class="ulist">
 464 <ul>
 465 <li>
 466 <p>Integrity checking is easy. Bit flips, for example, are easily
 467 detected, as the hash of corrupted content does not match its name.</p>
 468 </li>
 469 <li>
 470 <p>Lookup of objects is fast.</p>
 471 </li>
 472 </ul>
 473 </div>
 474 <div class="paragraph">
 475 <p>Using a cryptographically secure hash function brings additional
 476 advantages:</p>
 477 </div>
 478 <div class="ulist">
 479 <ul>
 480 <li>
 481 <p>Object names can be signed and third parties can trust the hash to
 482 address the signed object and all objects it references.</p>
 483 </li>
 484 <li>
 485 <p>Communication using Git protocol and out of band communication
 486 methods have a short reliable string that can be used to reliably
 487 address stored content.</p>
 488 </li>
 489 </ul>
 490 </div>
 491 <div class="paragraph">
 492 <p>Over time some flaws in SHA-1 have been discovered by security
 493 researchers. On 23 February 2017 the SHAttered attack
 494 (<a href="https://shattered.io" class="bare">https://shattered.io</a>) demonstrated a practical SHA-1 hash collision.</p>
 495 </div>
 496 <div class="paragraph">
 497 <p>Git v2.13.0 and later subsequently moved to a hardened SHA-1
 498 implementation by default, which isn&#8217;t vulnerable to the SHAttered
 499 attack, but SHA-1 is still weak.</p>
 500 </div>
 501 <div class="paragraph">
 502 <p>Thus it&#8217;s considered prudent to move past any variant of SHA-1
 503 to a new hash. There&#8217;s no guarantee that future attacks on SHA-1 won&#8217;t
 504 be published in the future, and those attacks may not have viable
 505 mitigations.</p>
 506 </div>
 507 <div class="paragraph">
 508 <p>If SHA-1 and its variants were to be truly broken, Git&#8217;s hash function
 509 could not be considered cryptographically secure any more. This would
 510 impact the communication of hash values because we could not trust
 511 that a given hash value represented the known good version of content
 512 that the speaker intended.</p>
 513 </div>
 514 <div class="paragraph">
 515 <p>SHA-1 still possesses the other properties such as fast object lookup
 516 and safe error checking, but other hash functions are equally suitable
 517 that are believed to be cryptographically secure.</p>
 518 </div>
 519 </div>
 520 </div>
 521 <div class="sect1">
 522 <h2 id="_choice_of_hash">Choice of Hash</h2>
 523 <div class="sectionbody">
 524 <div class="paragraph">
 525 <p>The hash to replace the hardened SHA-1 should be stronger than SHA-1
 526 was: we would like it to be trustworthy and useful in practice for at
 527 least 10 years.</p>
 528 </div>
 529 <div class="paragraph">
 530 <p>Some other relevant properties:</p>
 531 </div>
 532 <div class="olist arabic">
 533 <ol class="arabic">
 534 <li>
 535 <p>A 256-bit hash (long enough to match common security practice; not
 536 excessively long to hurt performance and disk usage).</p>
 537 </li>
 538 <li>
 539 <p>High quality implementations should be widely available (e.g., in
 540 OpenSSL and Apple CommonCrypto).</p>
 541 </li>
 542 <li>
 543 <p>The hash function&#8217;s properties should match Git&#8217;s needs (e.g. Git
 544 requires collision and 2nd preimage resistance and does not require
 545 length extension resistance).</p>
 546 </li>
 547 <li>
 548 <p>As a tiebreaker, the hash should be fast to compute (fortunately
 549 many contenders are faster than SHA-1).</p>
 550 </li>
 551 </ol>
 552 </div>
 553 <div class="paragraph">
 554 <p>There were several contenders for a successor hash to SHA-1, including
 555 SHA-256, SHA-512/256, SHA-256x16, K12, and BLAKE2bp-256.</p>
 556 </div>
 557 <div class="paragraph">
 558 <p>In late 2018 the project picked SHA-256 as its successor hash.</p>
 559 </div>
 560 <div class="paragraph">
 561 <p>See 0ed8d8da374 (doc hash-function-transition: pick SHA-256 as
 562 NewHash, 2018-08-04) and numerous mailing list threads at the time,
 563 particularly the one starting at
 564 <a href="https://lore.kernel.org/git/20180609224913.GC38834@genre.crustytoothpaste.net/" class="bare">https://lore.kernel.org/git/20180609224913.GC38834@genre.crustytoothpaste.net/</a>
 565 for more information.</p>
 566 </div>
 567 </div>
 568 </div>
 569 <div class="sect1">
 570 <h2 id="_goals">Goals</h2>
 571 <div class="sectionbody">
 572 <div class="olist arabic">
 573 <ol class="arabic">
 574 <li>
 575 <p>The transition to SHA-256 can be done one local repository at a time.</p>
 576 <div class="olist loweralpha">
 577 <ol class="loweralpha">
 578 <li>
 579 <p>Requiring no action by any other party.</p>
 580 </li>
 581 <li>
 582 <p>A SHA-256 repository can communicate with SHA-1 Git servers
 583 (push/fetch).</p>
 584 </li>
 585 <li>
 586 <p>Users can use SHA-1 and SHA-256 identifiers for objects
 587 interchangeably (see "Object names on the command line", below).</p>
 588 </li>
 589 <li>
 590 <p>New signed objects make use of a stronger hash function than
 591 SHA-1 for their security guarantees.</p>
 592 </li>
 593 </ol>
 594 </div>
 595 </li>
 596 <li>
 597 <p>Allow a complete transition away from SHA-1.</p>
 598 <div class="olist loweralpha">
 599 <ol class="loweralpha">
 600 <li>
 601 <p>Local metadata for SHA-1 compatibility can be removed from a
 602 repository if compatibility with SHA-1 is no longer needed.</p>
 603 </li>
 604 </ol>
 605 </div>
 606 </li>
 607 <li>
 608 <p>Maintainability throughout the process.</p>
 609 <div class="olist loweralpha">
 610 <ol class="loweralpha">
 611 <li>
 612 <p>The object format is kept simple and consistent.</p>
 613 </li>
 614 <li>
 615 <p>Creation of a generalized repository conversion tool.</p>
 616 </li>
 617 </ol>
 618 </div>
 619 </li>
 620 </ol>
 621 </div>
 622 </div>
 623 </div>
 624 <div class="sect1">
 625 <h2 id="_non_goals">Non-Goals</h2>
 626 <div class="sectionbody">
 627 <div class="olist arabic">
 628 <ol class="arabic">
 629 <li>
 630 <p>Add SHA-256 support to Git protocol. This is valuable and the
 631 logical next step but it is out of scope for this initial design.</p>
 632 </li>
 633 <li>
 634 <p>Transparently improving the security of existing SHA-1 signed
 635 objects.</p>
 636 </li>
 637 <li>
 638 <p>Intermixing objects using multiple hash functions in a single
 639 repository.</p>
 640 </li>
 641 <li>
 642 <p>Taking the opportunity to fix other bugs in Git&#8217;s formats and
 643 protocols.</p>
 644 </li>
 645 <li>
 646 <p>Shallow clones and fetches into a SHA-256 repository. (This will
 647 change when we add SHA-256 support to Git protocol.)</p>
 648 </li>
 649 <li>
 650 <p>Skip fetching some submodules of a project into a SHA-256
 651 repository. (This also depends on SHA-256 support in Git
 652 protocol.)</p>
 653 </li>
 654 </ol>
 655 </div>
 656 </div>
 657 </div>
 658 <div class="sect1">
 659 <h2 id="_overview">Overview</h2>
 660 <div class="sectionbody">
 661 <div class="paragraph">
 662 <p>We introduce a new repository format extension. Repositories with this
 663 extension enabled use SHA-256 instead of SHA-1 to name their objects.
 664 This affects both object names and object content&#8201;&#8212;&#8201;both the names
 665 of objects and all references to other objects within an object are
 666 switched to the new hash function.</p>
 667 </div>
 668 <div class="paragraph">
 669 <p>SHA-256 repositories cannot be read by older versions of Git.</p>
 670 </div>
 671 <div class="paragraph">
 672 <p>Alongside the packfile, a SHA-256 repository stores a bidirectional
 673 mapping between SHA-256 and SHA-1 object names. The mapping is generated
 674 locally and can be verified using "git fsck". Object lookups use this
 675 mapping to allow naming objects using either their SHA-1 and SHA-256 names
 676 interchangeably.</p>
 677 </div>
 678 <div class="paragraph">
 679 <p>"git cat-file" and "git hash-object" gain options to display an object
 680 in its SHA-1 form and write an object given its SHA-1 form. This
 681 requires all objects referenced by that object to be present in the
 682 object database so that they can be named using the appropriate name
 683 (using the bidirectional hash mapping).</p>
 684 </div>
 685 <div class="paragraph">
 686 <p>Fetches from a SHA-1 based server convert the fetched objects into
 687 SHA-256 form and record the mapping in the bidirectional mapping table
 688 (see below for details). Pushes to a SHA-1 based server convert the
 689 objects being pushed into SHA-1 form so the server does not have to be
 690 aware of the hash function the client is using.</p>
 691 </div>
 692 </div>
 693 </div>
 694 <div class="sect1">
 695 <h2 id="_detailed_design">Detailed Design</h2>
 696 <div class="sectionbody">
 697 <div class="sect2">
 698 <h3 id="_repository_format_extension">Repository format extension</h3>
 699 <div class="paragraph">
 700 <p>A SHA-256 repository uses repository format version <code>1</code> (see
 701 Documentation/technical/repository-version.txt) with extensions
 702 <code>objectFormat</code> and <code>compatObjectFormat</code>:</p>
 703 </div>
 704 <div class="literalblock">
 705 <div class="content">
 706 <pre>[core]
 707         repositoryFormatVersion = 1
 708 [extensions]
 709         objectFormat = sha256
 710         compatObjectFormat = sha1</pre>
 711 </div>
 712 </div>
 713 <div class="paragraph">
 714 <p>The combination of setting <code>core.repositoryFormatVersion=1</code> and
 715 populating <code>extensions.*</code> ensures that all versions of Git later than
 716 <code>v0.99.9l</code> will die instead of trying to operate on the SHA-256
 717 repository, instead producing an error message.</p>
 718 </div>
 719 <div class="literalblock">
 720 <div class="content">
 721 <pre># Between v0.99.9l and v2.7.0
 722 $ git status
 723 fatal: Expected git repo version &lt;= 0, found 1
 724 # After v2.7.0
 725 $ git status
 726 fatal: unknown repository extensions found:
 727         objectformat
 728         compatobjectformat</pre>
 729 </div>
 730 </div>
 731 <div class="paragraph">
 732 <p>See the "Transition plan" section below for more details on these
 733 repository extensions.</p>
 734 </div>
 735 </div>
 736 <div class="sect2">
 737 <h3 id="_object_names">Object names</h3>
 738 <div class="paragraph">
 739 <p>Objects can be named by their 40 hexadecimal digit SHA-1 name or 64
 740 hexadecimal digit SHA-256 name, plus names derived from those (see
 741 gitrevisions(7)).</p>
 742 </div>
 743 <div class="paragraph">
 744 <p>The SHA-1 name of an object is the SHA-1 of the concatenation of its
 745 type, length, a nul byte, and the object&#8217;s SHA-1 content. This is the
 746 traditional &lt;sha1&gt; used in Git to name objects.</p>
 747 </div>
 748 <div class="paragraph">
 749 <p>The SHA-256 name of an object is the SHA-256 of the concatenation of its
 750 type, length, a nul byte, and the object&#8217;s SHA-256 content.</p>
 751 </div>
 752 </div>
 753 <div class="sect2">
 754 <h3 id="_object_format">Object format</h3>
 755 <div class="paragraph">
 756 <p>The content as a byte sequence of a tag, commit, or tree object named
 757 by SHA-1 and SHA-256 differ because an object named by SHA-256 name refers to
 758 other objects by their SHA-256 names and an object named by SHA-1 name
 759 refers to other objects by their SHA-1 names.</p>
 760 </div>
 761 <div class="paragraph">
 762 <p>The SHA-256 content of an object is the same as its SHA-1 content, except
 763 that objects referenced by the object are named using their SHA-256 names
 764 instead of SHA-1 names. Because a blob object does not refer to any
 765 other object, its SHA-1 content and SHA-256 content are the same.</p>
 766 </div>
 767 <div class="paragraph">
 768 <p>The format allows round-trip conversion between SHA-256 content and
 769 SHA-1 content.</p>
 770 </div>
 771 </div>
 772 <div class="sect2">
 773 <h3 id="_object_storage">Object storage</h3>
 774 <div class="paragraph">
 775 <p>Loose objects use zlib compression and packed objects use the packed
 776 format described in <a href="../gitformat-pack.html">gitformat-pack(5)</a>, just like
 777 today. The content that is compressed and stored uses SHA-256 content
 778 instead of SHA-1 content.</p>
 779 </div>
 780 </div>
 781 <div class="sect2">
 782 <h3 id="_pack_index">Pack index</h3>
 783 <div class="paragraph">
 784 <p>Pack index (.idx) files use a new v3 format that supports multiple
 785 hash functions. They have the following format (all integers are in
 786 network byte order):</p>
 787 </div>
 788 <div class="ulist">
 789 <ul>
 790 <li>
 791 <p>A header appears at the beginning and consists of the following:</p>
 792 <div class="ulist">
 793 <ul>
 794 <li>
 795 <p>The 4-byte pack index signature: <em>\377t0c</em></p>
 796 </li>
 797 <li>
 798 <p>4-byte version number: 3</p>
 799 </li>
 800 <li>
 801 <p>4-byte length of the header section, including the signature and
 802 version number</p>
 803 </li>
 804 <li>
 805 <p>4-byte number of objects contained in the pack</p>
 806 </li>
 807 <li>
 808 <p>4-byte number of object formats in this pack index: 2</p>
 809 </li>
 810 <li>
 811 <p>For each object format:</p>
 812 <div class="ulist">
 813 <ul>
 814 <li>
 815 <p>4-byte format identifier (e.g., <em>sha1</em> for SHA-1)</p>
 816 </li>
 817 <li>
 818 <p>4-byte length in bytes of shortened object names. This is the
 819 shortest possible length needed to make names in the shortened
 820 object name table unambiguous.</p>
 821 </li>
 822 <li>
 823 <p>4-byte integer, recording where tables relating to this format
 824 are stored in this index file, as an offset from the beginning.</p>
 825 </li>
 826 </ul>
 827 </div>
 828 </li>
 829 <li>
 830 <p>4-byte offset to the trailer from the beginning of this file.</p>
 831 </li>
 832 <li>
 833 <p>Zero or more additional key/value pairs (4-byte key, 4-byte
 834 value). Only one key is supported: <em>PSRC</em>. See the "Loose objects
 835 and unreachable objects" section for supported values and how this
 836 is used.  All other keys are reserved. Readers must ignore
 837 unrecognized keys.</p>
 838 </li>
 839 </ul>
 840 </div>
 841 </li>
 842 <li>
 843 <p>Zero or more NUL bytes. This can optionally be used to improve the
 844 alignment of the full object name table below.</p>
 845 </li>
 846 <li>
 847 <p>Tables for the first object format:</p>
 848 <div class="ulist">
 849 <ul>
 850 <li>
 851 <p>A sorted table of shortened object names.  These are prefixes of
 852 the names of all objects in this pack file, packed together
 853 without offset values to reduce the cache footprint of the binary
 854 search for a specific object name.</p>
 855 </li>
 856 <li>
 857 <p>A table of full object names in pack order. This allows resolving
 858 a reference to "the nth object in the pack file" (from a
 859 reachability bitmap or from the next table of another object
 860 format) to its object name.</p>
 861 </li>
 862 <li>
 863 <p>A table of 4-byte values mapping object name order to pack order.
 864 For an object in the table of sorted shortened object names, the
 865 value at the corresponding index in this table is the index in the
 866 previous table for that same object.
 867 This can be used to look up the object in reachability bitmaps or
 868 to look up its name in another object format.</p>
 869 </li>
 870 <li>
 871 <p>A table of 4-byte CRC32 values of the packed object data, in the
 872 order that the objects appear in the pack file. This is to allow
 873 compressed data to be copied directly from pack to pack during
 874 repacking without undetected data corruption.</p>
 875 </li>
 876 <li>
 877 <p>A table of 4-byte offset values. For an object in the table of
 878 sorted shortened object names, the value at the corresponding
 879 index in this table indicates where that object can be found in
 880 the pack file. These are usually 31-bit pack file offsets, but
 881 large offsets are encoded as an index into the next table with the
 882 most significant bit set.</p>
 883 </li>
 884 <li>
 885 <p>A table of 8-byte offset entries (empty for pack files less than
 886 2 GiB). Pack files are organized with heavily used objects toward
 887 the front, so most object references should not need to refer to
 888 this table.</p>
 889 </li>
 890 </ul>
 891 </div>
 892 </li>
 893 <li>
 894 <p>Zero or more NUL bytes.</p>
 895 </li>
 896 <li>
 897 <p>Tables for the second object format, with the same layout as above,
 898 up to and not including the table of CRC32 values.</p>
 899 </li>
 900 <li>
 901 <p>Zero or more NUL bytes.</p>
 902 </li>
 903 <li>
 904 <p>The trailer consists of the following:</p>
 905 <div class="ulist">
 906 <ul>
 907 <li>
 908 <p>A copy of the 20-byte SHA-256 checksum at the end of the
 909 corresponding packfile.</p>
 910 </li>
 911 <li>
 912 <p>20-byte SHA-256 checksum of all of the above.</p>
 913 </li>
 914 </ul>
 915 </div>
 916 </li>
 917 </ul>
 918 </div>
 919 </div>
 920 <div class="sect2">
 921 <h3 id="_loose_object_index">Loose object index</h3>
 922 <div class="paragraph">
 923 <p>A new file $GIT_OBJECT_DIR/loose-object-idx contains information about
 924 all loose objects. Its format is</p>
 925 </div>
 926 <div class="literalblock">
 927 <div class="content">
 928 <pre># loose-object-idx
 929 (sha256-name SP sha1-name LF)*</pre>
 930 </div>
 931 </div>
 932 <div class="paragraph">
 933 <p>where the object names are in hexadecimal format. The file is not
 934 sorted.</p>
 935 </div>
 936 <div class="paragraph">
 937 <p>The loose object index is protected against concurrent writes by a
 938 lock file $GIT_OBJECT_DIR/loose-object-idx.lock. To add a new loose
 939 object:</p>
 940 </div>
 941 <div class="olist arabic">
 942 <ol class="arabic">
 943 <li>
 944 <p>Write the loose object to a temporary file, like today.</p>
 945 </li>
 946 <li>
 947 <p>Open loose-object-idx.lock with O_CREAT | O_EXCL to acquire the lock.</p>
 948 </li>
 949 <li>
 950 <p>Rename the loose object into place.</p>
 951 </li>
 952 <li>
 953 <p>Open loose-object-idx with O_APPEND and write the new object</p>
 954 </li>
 955 <li>
 956 <p>Unlink loose-object-idx.lock to release the lock.</p>
 957 </li>
 958 </ol>
 959 </div>
 960 <div class="paragraph">
 961 <p>To remove entries (e.g. in "git pack-refs" or "git-prune"):</p>
 962 </div>
 963 <div class="olist arabic">
 964 <ol class="arabic">
 965 <li>
 966 <p>Open loose-object-idx.lock with O_CREAT | O_EXCL to acquire the
 967 lock.</p>
 968 </li>
 969 <li>
 970 <p>Write the new content to loose-object-idx.lock.</p>
 971 </li>
 972 <li>
 973 <p>Unlink any loose objects being removed.</p>
 974 </li>
 975 <li>
 976 <p>Rename to replace loose-object-idx, releasing the lock.</p>
 977 </li>
 978 </ol>
 979 </div>
 980 </div>
 981 <div class="sect2">
 982 <h3 id="_translation_table">Translation table</h3>
 983 <div class="paragraph">
 984 <p>The index files support a bidirectional mapping between SHA-1 names
 985 and SHA-256 names. The lookup proceeds similarly to ordinary object
 986 lookups. For example, to convert a SHA-1 name to a SHA-256 name:</p>
 987 </div>
 988 <div class="olist arabic">
 989 <ol class="arabic">
 990 <li>
 991 <p>Look for the object in idx files. If a match is present in the
 992 idx&#8217;s sorted list of truncated SHA-1 names, then:</p>
 993 <div class="olist loweralpha">
 994 <ol class="loweralpha">
 995 <li>
 996 <p>Read the corresponding entry in the SHA-1 name order to pack
 997 name order mapping.</p>
 998 </li>
 999 <li>
1000 <p>Read the corresponding entry in the full SHA-1 name table to
1001 verify we found the right object. If it is, then</p>
1002 </li>
1003 <li>
1004 <p>Read the corresponding entry in the full SHA-256 name table.
1005 That is the object&#8217;s SHA-256 name.</p>
1006 </li>
1007 </ol>
1008 </div>
1009 </li>
1010 <li>
1011 <p>Check for a loose object. Read lines from loose-object-idx until
1012 we find a match.</p>
1013 </li>
1014 </ol>
1015 </div>
1016 <div class="paragraph">
1017 <p>Step (1) takes the same amount of time as an ordinary object lookup:
1018 O(number of packs * log(objects per pack)). Step (2) takes O(number of
1019 loose objects) time. To maintain good performance it will be necessary
1020 to keep the number of loose objects low. See the "Loose objects and
1021 unreachable objects" section below for more details.</p>
1022 </div>
1023 <div class="paragraph">
1024 <p>Since all operations that make new objects (e.g., "git commit") add
1025 the new objects to the corresponding index, this mapping is possible
1026 for all objects in the object store.</p>
1027 </div>
1028 </div>
1029 <div class="sect2">
1030 <h3 id="_reading_an_objects_sha_1_content">Reading an object&#8217;s SHA-1 content</h3>
1031 <div class="paragraph">
1032 <p>The SHA-1 content of an object can be read by converting all SHA-256 names
1033 of its SHA-256 content references to SHA-1 names using the translation table.</p>
1034 </div>
1035 </div>
1036 <div class="sect2">
1037 <h3 id="_fetch">Fetch</h3>
1038 <div class="paragraph">
1039 <p>Fetching from a SHA-1 based server requires translating between SHA-1
1040 and SHA-256 based representations on the fly.</p>
1041 </div>
1042 <div class="paragraph">
1043 <p>SHA-1s named in the ref advertisement that are present on the client
1044 can be translated to SHA-256 and looked up as local objects using the
1045 translation table.</p>
1046 </div>
1047 <div class="paragraph">
1048 <p>Negotiation proceeds as today. Any "have"s generated locally are
1049 converted to SHA-1 before being sent to the server, and SHA-1s
1050 mentioned by the server are converted to SHA-256 when looking them up
1051 locally.</p>
1052 </div>
1053 <div class="paragraph">
1054 <p>After negotiation, the server sends a packfile containing the
1055 requested objects. We convert the packfile to SHA-256 format using
1056 the following steps:</p>
1057 </div>
1058 <div class="olist arabic">
1059 <ol class="arabic">
1060 <li>
1061 <p>index-pack: inflate each object in the packfile and compute its
1062 SHA-1. Objects can contain deltas in OBJ_REF_DELTA format against
1063 objects the client has locally. These objects can be looked up
1064 using the translation table and their SHA-1 content read as
1065 described above to resolve the deltas.</p>
1066 </li>
1067 <li>
1068 <p>topological sort: starting at the "want"s from the negotiation
1069 phase, walk through objects in the pack and emit a list of them,
1070 excluding blobs, in reverse topologically sorted order, with each
1071 object coming later in the list than all objects it references.
1072 (This list only contains objects reachable from the "wants". If the
1073 pack from the server contained additional extraneous objects, then
1074 they will be discarded.)</p>
1075 </li>
1076 <li>
1077 <p>convert to SHA-256: open a new SHA-256 packfile. Read the topologically
1078 sorted list just generated. For each object, inflate its
1079 SHA-1 content, convert to SHA-256 content, and write it to the SHA-256
1080 pack. Record the new SHA-1&#8592;&#8594;SHA-256 mapping entry for use in the idx.</p>
1081 </li>
1082 <li>
1083 <p>sort: reorder entries in the new pack to match the order of objects
1084 in the pack the server generated and include blobs. Write a SHA-256 idx
1085 file</p>
1086 </li>
1087 <li>
1088 <p>clean up: remove the SHA-1 based pack file, index, and
1089 topologically sorted list obtained from the server in steps 1
1090 and 2.</p>
1091 </li>
1092 </ol>
1093 </div>
1094 <div class="paragraph">
1095 <p>Step 3 requires every object referenced by the new object to be in the
1096 translation table. This is why the topological sort step is necessary.</p>
1097 </div>
1098 <div class="paragraph">
1099 <p>As an optimization, step 1 could write a file describing what non-blob
1100 objects each object it has inflated from the packfile references. This
1101 makes the topological sort in step 2 possible without inflating the
1102 objects in the packfile for a second time. The objects need to be
1103 inflated again in step 3, for a total of two inflations.</p>
1104 </div>
1105 <div class="paragraph">
1106 <p>Step 4 is probably necessary for good read-time performance. "git
1107 pack-objects" on the server optimizes the pack file for good data
1108 locality (see Documentation/technical/pack-heuristics.txt).</p>
1109 </div>
1110 <div class="paragraph">
1111 <p>Details of this process are likely to change. It will take some
1112 experimenting to get this to perform well.</p>
1113 </div>
1114 </div>
1115 <div class="sect2">
1116 <h3 id="_push">Push</h3>
1117 <div class="paragraph">
1118 <p>Push is simpler than fetch because the objects referenced by the
1119 pushed objects are already in the translation table. The SHA-1 content
1120 of each object being pushed can be read as described in the "Reading
1121 an object&#8217;s SHA-1 content" section to generate the pack written by git
1122 send-pack.</p>
1123 </div>
1124 </div>
1125 <div class="sect2">
1126 <h3 id="_signed_commits">Signed Commits</h3>
1127 <div class="paragraph">
1128 <p>We add a new field "gpgsig-sha256" to the commit object format to allow
1129 signing commits without relying on SHA-1. It is similar to the
1130 existing "gpgsig" field. Its signed payload is the SHA-256 content of the
1131 commit object with any "gpgsig" and "gpgsig-sha256" fields removed.</p>
1132 </div>
1133 <div class="paragraph">
1134 <p>This means commits can be signed</p>
1135 </div>
1136 <div class="olist arabic">
1137 <ol class="arabic">
1138 <li>
1139 <p>using SHA-1 only, as in existing signed commit objects</p>
1140 </li>
1141 <li>
1142 <p>using both SHA-1 and SHA-256, by using both gpgsig-sha256 and gpgsig
1143 fields.</p>
1144 </li>
1145 <li>
1146 <p>using only SHA-256, by only using the gpgsig-sha256 field.</p>
1147 </li>
1148 </ol>
1149 </div>
1150 <div class="paragraph">
1151 <p>Old versions of "git verify-commit" can verify the gpgsig signature in
1152 cases (1) and (2) without modifications and view case (3) as an
1153 ordinary unsigned commit.</p>
1154 </div>
1155 </div>
1156 <div class="sect2">
1157 <h3 id="_signed_tags">Signed Tags</h3>
1158 <div class="paragraph">
1159 <p>We add a new field "gpgsig-sha256" to the tag object format to allow
1160 signing tags without relying on SHA-1. Its signed payload is the
1161 SHA-256 content of the tag with its gpgsig-sha256 field and "-----BEGIN PGP
1162 SIGNATURE-----" delimited in-body signature removed.</p>
1163 </div>
1164 <div class="paragraph">
1165 <p>This means tags can be signed</p>
1166 </div>
1167 <div class="olist arabic">
1168 <ol class="arabic">
1169 <li>
1170 <p>using SHA-1 only, as in existing signed tag objects</p>
1171 </li>
1172 <li>
1173 <p>using both SHA-1 and SHA-256, by using gpgsig-sha256 and an in-body
1174 signature.</p>
1175 </li>
1176 <li>
1177 <p>using only SHA-256, by only using the gpgsig-sha256 field.</p>
1178 </li>
1179 </ol>
1180 </div>
1181 </div>
1182 <div class="sect2">
1183 <h3 id="_mergetag_embedding">Mergetag embedding</h3>
1184 <div class="paragraph">
1185 <p>The mergetag field in the SHA-1 content of a commit contains the
1186 SHA-1 content of a tag that was merged by that commit.</p>
1187 </div>
1188 <div class="paragraph">
1189 <p>The mergetag field in the SHA-256 content of the same commit contains the
1190 SHA-256 content of the same tag.</p>
1191 </div>
1192 </div>
1193 <div class="sect2">
1194 <h3 id="_submodules">Submodules</h3>
1195 <div class="paragraph">
1196 <p>To convert recorded submodule pointers, you need to have the converted
1197 submodule repository in place. The translation table of the submodule
1198 can be used to look up the new hash.</p>
1199 </div>
1200 </div>
1201 <div class="sect2">
1202 <h3 id="_loose_objects_and_unreachable_objects">Loose objects and unreachable objects</h3>
1203 <div class="paragraph">
1204 <p>Fast lookups in the loose-object-idx require that the number of loose
1205 objects not grow too high.</p>
1206 </div>
1207 <div class="paragraph">
1208 <p>"git gc --auto" currently waits for there to be 6700 loose objects
1209 present before consolidating them into a packfile. We will need to
1210 measure to find a more appropriate threshold for it to use.</p>
1211 </div>
1212 <div class="paragraph">
1213 <p>"git gc --auto" currently waits for there to be 50 packs present
1214 before combining packfiles. Packing loose objects more aggressively
1215 may cause the number of pack files to grow too quickly. This can be
1216 mitigated by using a strategy similar to Martin Fick&#8217;s exponential
1217 rolling garbage collection script:
1218 <a href="https://gerrit-review.googlesource.com/c/gerrit/+/35215" class="bare">https://gerrit-review.googlesource.com/c/gerrit/+/35215</a></p>
1219 </div>
1220 <div class="paragraph">
1221 <p>"git gc" currently expels any unreachable objects it encounters in
1222 pack files to loose objects in an attempt to prevent a race when
1223 pruning them (in case another process is simultaneously writing a new
1224 object that refers to the about-to-be-deleted object). This leads to
1225 an explosion in the number of loose objects present and disk space
1226 usage due to the objects in delta form being replaced with independent
1227 loose objects.  Worse, the race is still present for loose objects.</p>
1228 </div>
1229 <div class="paragraph">
1230 <p>Instead, "git gc" will need to move unreachable objects to a new
1231 packfile marked as UNREACHABLE_GARBAGE (using the PSRC field; see
1232 below). To avoid the race when writing new objects referring to an
1233 about-to-be-deleted object, code paths that write new objects will
1234 need to copy any objects from UNREACHABLE_GARBAGE packs that they
1235 refer to new, non-UNREACHABLE_GARBAGE packs (or loose objects).
1236 UNREACHABLE_GARBAGE are then safe to delete if their creation time (as
1237 indicated by the file&#8217;s mtime) is long enough ago.</p>
1238 </div>
1239 <div class="paragraph">
1240 <p>To avoid a proliferation of UNREACHABLE_GARBAGE packs, they can be
1241 combined under certain circumstances. If "gc.garbageTtl" is set to
1242 greater than one day, then packs created within a single calendar day,
1243 UTC, can be coalesced together. The resulting packfile would have an
1244 mtime before midnight on that day, so this makes the effective maximum
1245 ttl the garbageTtl + 1 day. If "gc.garbageTtl" is less than one day,
1246 then we divide the calendar day into intervals one-third of that ttl
1247 in duration. Packs created within the same interval can be coalesced
1248 together. The resulting packfile would have an mtime before the end of
1249 the interval, so this makes the effective maximum ttl equal to the
1250 garbageTtl * 4/3.</p>
1251 </div>
1252 <div class="paragraph">
1253 <p>This rule comes from Thirumala Reddy Mutchukota&#8217;s JGit change
1254 <a href="https://git.eclipse.org/r/90465" class="bare">https://git.eclipse.org/r/90465</a>.</p>
1255 </div>
1256 <div class="paragraph">
1257 <p>The UNREACHABLE_GARBAGE setting goes in the PSRC field of the pack
1258 index. More generally, that field indicates where a pack came from:</p>
1259 </div>
1260 <div class="ulist">
1261 <ul>
1262 <li>
1263 <p>1 (PACK_SOURCE_RECEIVE) for a pack received over the network</p>
1264 </li>
1265 <li>
1266 <p>2 (PACK_SOURCE_AUTO) for a pack created by a lightweight
1267 "gc --auto" operation</p>
1268 </li>
1269 <li>
1270 <p>3 (PACK_SOURCE_GC) for a pack created by a full gc</p>
1271 </li>
1272 <li>
1273 <p>4 (PACK_SOURCE_UNREACHABLE_GARBAGE) for potential garbage
1274 discovered by gc</p>
1275 </li>
1276 <li>
1277 <p>5 (PACK_SOURCE_INSERT) for locally created objects that were
1278 written directly to a pack file, e.g. from "git add ."</p>
1279 </li>
1280 </ul>
1281 </div>
1282 <div class="paragraph">
1283 <p>This information can be useful for debugging and for "gc --auto" to
1284 make appropriate choices about which packs to coalesce.</p>
1285 </div>
1286 </div>
1287 </div>
1288 </div>
1289 <div class="sect1">
1290 <h2 id="_caveats">Caveats</h2>
1291 <div class="sectionbody">
1292 <div class="sect2">
1293 <h3 id="_invalid_objects">Invalid objects</h3>
1294 <div class="paragraph">
1295 <p>The conversion from SHA-1 content to SHA-256 content retains any
1296 brokenness in the original object (e.g., tree entry modes encoded with
1297 leading 0, tree objects whose paths are not sorted correctly, and
1298 commit objects without an author or committer). This is a deliberate
1299 feature of the design to allow the conversion to round-trip.</p>
1300 </div>
1301 <div class="paragraph">
1302 <p>More profoundly broken objects (e.g., a commit with a truncated "tree"
1303 header line) cannot be converted but were not usable by current Git
1304 anyway.</p>
1305 </div>
1306 </div>
1307 <div class="sect2">
1308 <h3 id="_shallow_clone_and_submodules">Shallow clone and submodules</h3>
1309 <div class="paragraph">
1310 <p>Because it requires all referenced objects to be available in the
1311 locally generated translation table, this design does not support
1312 shallow clone or unfetched submodules. Protocol improvements might
1313 allow lifting this restriction.</p>
1314 </div>
1315 </div>
1316 <div class="sect2">
1317 <h3 id="_alternates">Alternates</h3>
1318 <div class="paragraph">
1319 <p>For the same reason, a SHA-256 repository cannot borrow objects from a
1320 SHA-1 repository using objects/info/alternates or
1321 $GIT_ALTERNATE_OBJECT_REPOSITORIES.</p>
1322 </div>
1323 </div>
1324 <div class="sect2">
1325 <h3 id="_git_notes">git notes</h3>
1326 <div class="paragraph">
1327 <p>The "git notes" tool annotates objects using their SHA-1 name as key.
1328 This design does not describe a way to migrate notes trees to use
1329 SHA-256 names. That migration is expected to happen separately (for
1330 example using a file at the root of the notes tree to describe which
1331 hash it uses).</p>
1332 </div>
1333 </div>
1334 <div class="sect2">
1335 <h3 id="_server_side_cost">Server-side cost</h3>
1336 <div class="paragraph">
1337 <p>Until Git protocol gains SHA-256 support, using SHA-256 based storage
1338 on public-facing Git servers is strongly discouraged. Once Git
1339 protocol gains SHA-256 support, SHA-256 based servers are likely not
1340 to support SHA-1 compatibility, to avoid what may be a very expensive
1341 hash re-encode during clone and to encourage peers to modernize.</p>
1342 </div>
1343 <div class="paragraph">
1344 <p>The design described here allows fetches by SHA-1 clients of a
1345 personal SHA-256 repository because it&#8217;s not much more difficult than
1346 allowing pushes from that repository. This support needs to be guarded
1347 by a configuration option&#8201;&#8212;&#8201;servers like git.kernel.org that serve a
1348 large number of clients would not be expected to bear that cost.</p>
1349 </div>
1350 </div>
1351 <div class="sect2">
1352 <h3 id="_meaning_of_signatures">Meaning of signatures</h3>
1353 <div class="paragraph">
1354 <p>The signed payload for signed commits and tags does not explicitly
1355 name the hash used to identify objects. If some day Git adopts a new
1356 hash function with the same length as the current SHA-1 (40
1357 hexadecimal digit) or SHA-256 (64 hexadecimal digit) objects then the
1358 intent behind the PGP signed payload in an object signature is
1359 unclear:</p>
1360 </div>
1361 <div class="literalblock">
1362 <div class="content">
1363 <pre>object e7e07d5a4fcc2a203d9873968ad3e6bd4d7419d7
1364 type commit
1365 tag v2.12.0
1366 tagger Junio C Hamano &lt;gitster@pobox.com&gt; 1487962205 -0800</pre>
1367 </div>
1368 </div>
1369 <div class="literalblock">
1370 <div class="content">
1371 <pre>Git 2.12</pre>
1372 </div>
1373 </div>
1374 <div class="paragraph">
1375 <p>Does this mean Git v2.12.0 is the commit with SHA-1 name
1376 e7e07d5a4fcc2a203d9873968ad3e6bd4d7419d7 or the commit with
1377 new-40-digit-hash-name e7e07d5a4fcc2a203d9873968ad3e6bd4d7419d7?</p>
1378 </div>
1379 <div class="paragraph">
1380 <p>Fortunately SHA-256 and SHA-1 have different lengths. If Git starts
1381 using another hash with the same length to name objects, then it will
1382 need to change the format of signed payloads using that hash to
1383 address this issue.</p>
1384 </div>
1385 </div>
1386 <div class="sect2">
1387 <h3 id="_object_names_on_the_command_line">Object names on the command line</h3>
1388 <div class="paragraph">
1389 <p>To support the transition (see Transition plan below), this design
1390 supports four different modes of operation:</p>
1391 </div>
1392 <div class="olist arabic">
1393 <ol class="arabic">
1394 <li>
1395 <p>("dark launch") Treat object names input by the user as SHA-1 and
1396 convert any object names written to output to SHA-1, but store
1397 objects using SHA-256.  This allows users to test the code with no
1398 visible behavior change except for performance.  This allows
1399 running even tests that assume the SHA-1 hash function, to
1400 sanity-check the behavior of the new mode.</p>
1401 </li>
1402 <li>
1403 <p>("early transition") Allow both SHA-1 and SHA-256 object names in
1404 input. Any object names written to output use SHA-1. This allows
1405 users to continue to make use of SHA-1 to communicate with peers
1406 (e.g. by email) that have not migrated yet and prepares for mode 3.</p>
1407 </li>
1408 <li>
1409 <p>("late transition") Allow both SHA-1 and SHA-256 object names in
1410 input. Any object names written to output use SHA-256. In this
1411 mode, users are using a more secure object naming method by
1412 default.  The disruption is minimal as long as most of their peers
1413 are in mode 2 or mode 3.</p>
1414 </li>
1415 <li>
1416 <p>("post-transition") Treat object names input by the user as
1417 SHA-256 and write output using SHA-256. This is safer than mode 3
1418 because there is less risk that input is incorrectly interpreted
1419 using the wrong hash function.</p>
1420 </li>
1421 </ol>
1422 </div>
1423 <div class="paragraph">
1424 <p>The mode is specified in configuration.</p>
1425 </div>
1426 <div class="paragraph">
1427 <p>The user can also explicitly specify which format to use for a
1428 particular revision specifier and for output, overriding the mode. For
1429 example:</p>
1430 </div>
1431 <div class="literalblock">
1432 <div class="content">
1433 <pre>git --output-format=sha1 log abac87a^{sha1}..f787cac^{sha256}</pre>
1434 </div>
1435 </div>
1436 </div>
1437 </div>
1438 </div>
1439 <div class="sect1">
1440 <h2 id="_transition_plan">Transition plan</h2>
1441 <div class="sectionbody">
1442 <div class="paragraph">
1443 <p>Some initial steps can be implemented independently of one another:</p>
1444 </div>
1445 <div class="ulist">
1446 <ul>
1447 <li>
1448 <p>adding a hash function API (vtable)</p>
1449 </li>
1450 <li>
1451 <p>teaching fsck to tolerate the gpgsig-sha256 field</p>
1452 </li>
1453 <li>
1454 <p>excluding gpgsig-* from the fields copied by "git commit --amend"</p>
1455 </li>
1456 <li>
1457 <p>annotating tests that depend on SHA-1 values with a SHA1 test
1458 prerequisite</p>
1459 </li>
1460 <li>
1461 <p>using "struct object_id", GIT_MAX_RAWSZ, and GIT_MAX_HEXSZ
1462 consistently instead of "unsigned char *" and the hardcoded
1463 constants 20 and 40.</p>
1464 </li>
1465 <li>
1466 <p>introducing index v3</p>
1467 </li>
1468 <li>
1469 <p>adding support for the PSRC field and safer object pruning</p>
1470 </li>
1471 </ul>
1472 </div>
1473 <div class="paragraph">
1474 <p>The first user-visible change is the introduction of the objectFormat
1475 extension (without compatObjectFormat). This requires:</p>
1476 </div>
1477 <div class="ulist">
1478 <ul>
1479 <li>
1480 <p>teaching fsck about this mode of operation</p>
1481 </li>
1482 <li>
1483 <p>using the hash function API (vtable) when computing object names</p>
1484 </li>
1485 <li>
1486 <p>signing objects and verifying signatures</p>
1487 </li>
1488 <li>
1489 <p>rejecting attempts to fetch from or push to an incompatible
1490 repository</p>
1491 </li>
1492 </ul>
1493 </div>
1494 <div class="paragraph">
1495 <p>Next comes introduction of compatObjectFormat:</p>
1496 </div>
1497 <div class="ulist">
1498 <ul>
1499 <li>
1500 <p>implementing the loose-object-idx</p>
1501 </li>
1502 <li>
1503 <p>translating object names between object formats</p>
1504 </li>
1505 <li>
1506 <p>translating object content between object formats</p>
1507 </li>
1508 <li>
1509 <p>generating and verifying signatures in the compat format</p>
1510 </li>
1511 <li>
1512 <p>adding appropriate index entries when adding a new object to the
1513 object store</p>
1514 </li>
1515 <li>
1516 <p>--output-format option</p>
1517 </li>
1518 <li>
1519 <p>^{sha1} and ^{sha256} revision notation</p>
1520 </li>
1521 <li>
1522 <p>configuration to specify default input and output format (see
1523 "Object names on the command line" above)</p>
1524 </li>
1525 </ul>
1526 </div>
1527 <div class="paragraph">
1528 <p>The next step is supporting fetches and pushes to SHA-1 repositories:</p>
1529 </div>
1530 <div class="ulist">
1531 <ul>
1532 <li>
1533 <p>allow pushes to a repository using the compat format</p>
1534 </li>
1535 <li>
1536 <p>generate a topologically sorted list of the SHA-1 names of fetched
1537 objects</p>
1538 </li>
1539 <li>
1540 <p>convert the fetched packfile to SHA-256 format and generate an idx
1541 file</p>
1542 </li>
1543 <li>
1544 <p>re-sort to match the order of objects in the fetched packfile</p>
1545 </li>
1546 </ul>
1547 </div>
1548 <div class="paragraph">
1549 <p>The infrastructure supporting fetch also allows converting an existing
1550 repository. In converted repositories and new clones, end users can
1551 gain support for the new hash function without any visible change in
1552 behavior (see "dark launch" in the "Object names on the command line"
1553 section). In particular this allows users to verify SHA-256 signatures
1554 on objects in the repository, and it should ensure the transition code
1555 is stable in production in preparation for using it more widely.</p>
1556 </div>
1557 <div class="paragraph">
1558 <p>Over time projects would encourage their users to adopt the "early
1559 transition" and then "late transition" modes to take advantage of the
1560 new, more futureproof SHA-256 object names.</p>
1561 </div>
1562 <div class="paragraph">
1563 <p>When objectFormat and compatObjectFormat are both set, commands
1564 generating signatures would generate both SHA-1 and SHA-256 signatures
1565 by default to support both new and old users.</p>
1566 </div>
1567 <div class="paragraph">
1568 <p>In projects using SHA-256 heavily, users could be encouraged to adopt
1569 the "post-transition" mode to avoid accidentally making implicit use
1570 of SHA-1 object names.</p>
1571 </div>
1572 <div class="paragraph">
1573 <p>Once a critical mass of users have upgraded to a version of Git that
1574 can verify SHA-256 signatures and have converted their existing
1575 repositories to support verifying them, we can add support for a
1576 setting to generate only SHA-256 signatures. This is expected to be at
1577 least a year later.</p>
1578 </div>
1579 <div class="paragraph">
1580 <p>That is also a good moment to advertise the ability to convert
1581 repositories to use SHA-256 only, stripping out all SHA-1 related
1582 metadata. This improves performance by eliminating translation
1583 overhead and security by avoiding the possibility of accidentally
1584 relying on the safety of SHA-1.</p>
1585 </div>
1586 <div class="paragraph">
1587 <p>Updating Git&#8217;s protocols to allow a server to specify which hash
1588 functions it supports is also an important part of this transition. It
1589 is not discussed in detail in this document but this transition plan
1590 assumes it happens. :)</p>
1591 </div>
1592 </div>
1593 </div>
1594 <div class="sect1">
1595 <h2 id="_alternatives_considered">Alternatives considered</h2>
1596 <div class="sectionbody">
1597 <div class="sect2">
1598 <h3 id="_upgrading_everyone_working_on_a_particular_project_on_a_flag_day">Upgrading everyone working on a particular project on a flag day</h3>
1599 <div class="paragraph">
1600 <p>Projects like the Linux kernel are large and complex enough that
1601 flipping the switch for all projects based on the repository at once
1602 is infeasible.</p>
1603 </div>
1604 <div class="paragraph">
1605 <p>Not only would all developers and server operators supporting
1606 developers have to switch on the same flag day, but supporting tooling
1607 (continuous integration, code review, bug trackers, etc) would have to
1608 be adapted as well. This also makes it difficult to get early feedback
1609 from some project participants testing before it is time for mass
1610 adoption.</p>
1611 </div>
1612 </div>
1613 <div class="sect2">
1614 <h3 id="_using_hash_functions_in_parallel">Using hash functions in parallel</h3>
1615 <div class="paragraph">
1616 <p>(e.g. <a href="https://lore.kernel.org/git/22708.8913.864049.452252@chiark.greenend.org.uk/" class="bare">https://lore.kernel.org/git/22708.8913.864049.452252@chiark.greenend.org.uk/</a> )
1617 Objects newly created would be addressed by the new hash, but inside
1618 such an object (e.g. commit) it is still possible to address objects
1619 using the old hash function.</p>
1620 </div>
1621 <div class="ulist">
1622 <ul>
1623 <li>
1624 <p>You cannot trust its history (needed for bisectability) in the
1625 future without further work</p>
1626 </li>
1627 <li>
1628 <p>Maintenance burden as the number of supported hash functions grows
1629 (they will never go away, so they accumulate). In this proposal, by
1630 comparison, converted objects lose all references to SHA-1.</p>
1631 </li>
1632 </ul>
1633 </div>
1634 </div>
1635 <div class="sect2">
1636 <h3 id="_signed_objects_with_multiple_hashes">Signed objects with multiple hashes</h3>
1637 <div class="paragraph">
1638 <p>Instead of introducing the gpgsig-sha256 field in commit and tag objects
1639 for SHA-256 content based signatures, an earlier version of this design
1640 added "hash sha256 &lt;SHA-256 name&gt;" fields to strengthen the existing
1641 SHA-1 content based signatures.</p>
1642 </div>
1643 <div class="paragraph">
1644 <p>In other words, a single signature was used to attest to the object
1645 content using both hash functions. This had some advantages:</p>
1646 </div>
1647 <div class="ulist">
1648 <ul>
1649 <li>
1650 <p>Using one signature instead of two speeds up the signing process.</p>
1651 </li>
1652 <li>
1653 <p>Having one signed payload with both hashes allows the signer to
1654 attest to the SHA-1 name and SHA-256 name referring to the same object.</p>
1655 </li>
1656 <li>
1657 <p>All users consume the same signature. Broken signatures are likely
1658 to be detected quickly using current versions of git.</p>
1659 </li>
1660 </ul>
1661 </div>
1662 <div class="paragraph">
1663 <p>However, it also came with disadvantages:</p>
1664 </div>
1665 <div class="ulist">
1666 <ul>
1667 <li>
1668 <p>Verifying a signed object requires access to the SHA-1 names of all
1669 objects it references, even after the transition is complete and
1670 translation table is no longer needed for anything else. To support
1671 this, the design added fields such as "hash sha1 tree &lt;SHA-1 name&gt;"
1672 and "hash sha1 parent &lt;SHA-1 name&gt;" to the SHA-256 content of a signed
1673 commit, complicating the conversion process.</p>
1674 </li>
1675 <li>
1676 <p>Allowing signed objects without a SHA-1 (for after the transition is
1677 complete) complicated the design further, requiring a "nohash sha1"
1678 field to suppress including "hash sha1" fields in the SHA-256 content
1679 and signed payload.</p>
1680 </li>
1681 </ul>
1682 </div>
1683 </div>
1684 <div class="sect2">
1685 <h3 id="_lazily_populated_translation_table">Lazily populated translation table</h3>
1686 <div class="paragraph">
1687 <p>Some of the work of building the translation table could be deferred to
1688 push time, but that would significantly complicate and slow down pushes.
1689 Calculating the SHA-1 name at object creation time at the same time it is
1690 being streamed to disk and having its SHA-256 name calculated should be
1691 an acceptable cost.</p>
1692 </div>
1693 </div>
1694 </div>
1695 </div>
1696 <div class="sect1">
1697 <h2 id="_document_history">Document History</h2>
1698 <div class="sectionbody">
1699 <div class="paragraph">
1700 <p>2017-03-03
1701 <a href="mailto:bmwill@google.com">bmwill@google.com</a>, <a href="mailto:jonathantanmy@google.com">jonathantanmy@google.com</a>, <a href="mailto:jrnieder@gmail.com">jrnieder@gmail.com</a>,
1702 <a href="mailto:sbeller@google.com">sbeller@google.com</a></p>
1703 </div>
1704 <div class="ulist">
1705 <ul>
1706 <li>
1707 <p>Initial version sent to <a href="https://lore.kernel.org/git/20170304011251.GA26789@aiede.mtv.corp.google.com" class="bare">https://lore.kernel.org/git/20170304011251.GA26789@aiede.mtv.corp.google.com</a></p>
1708 </li>
1709 </ul>
1710 </div>
1711 <div class="paragraph">
1712 <p>2017-03-03 <a href="mailto:jrnieder@gmail.com">jrnieder@gmail.com</a>
1713 Incorporated suggestions from jonathantanmy and sbeller:</p>
1714 </div>
1715 <div class="ulist">
1716 <ul>
1717 <li>
1718 <p>Describe purpose of signed objects with each hash type</p>
1719 </li>
1720 <li>
1721 <p>Redefine signed object verification using object content under the
1722 first hash function</p>
1723 </li>
1724 </ul>
1725 </div>
1726 <div class="paragraph">
1727 <p>2017-03-06 <a href="mailto:jrnieder@gmail.com">jrnieder@gmail.com</a></p>
1728 </div>
1729 <div class="ulist">
1730 <ul>
1731 <li>
1732 <p>Use SHA3-256 instead of SHA2 (thanks, Linus and brian m. carlson).[1][2]</p>
1733 </li>
1734 <li>
1735 <p>Make SHA3-based signatures a separate field, avoiding the need for
1736 "hash" and "nohash" fields (thanks to peff[3]).</p>
1737 </li>
1738 <li>
1739 <p>Add a sorting phase to fetch (thanks to Junio for noticing the need
1740 for this).</p>
1741 </li>
1742 <li>
1743 <p>Omit blobs from the topological sort during fetch (thanks to peff).</p>
1744 </li>
1745 <li>
1746 <p>Discuss alternates, git notes, and git servers in the caveats
1747 section (thanks to Junio Hamano, brian m. carlson[4], and Shawn
1748 Pearce).</p>
1749 </li>
1750 <li>
1751 <p>Clarify language throughout (thanks to various commenters,
1752 especially Junio).</p>
1753 </li>
1754 </ul>
1755 </div>
1756 <div class="paragraph">
1757 <p>2017-09-27 <a href="mailto:jrnieder@gmail.com">jrnieder@gmail.com</a>, <a href="mailto:sbeller@google.com">sbeller@google.com</a></p>
1758 </div>
1759 <div class="ulist">
1760 <ul>
1761 <li>
1762 <p>Use placeholder NewHash instead of SHA3-256</p>
1763 </li>
1764 <li>
1765 <p>Describe criteria for picking a hash function.</p>
1766 </li>
1767 <li>
1768 <p>Include a transition plan (thanks especially to Brandon Williams
1769 for fleshing these ideas out)</p>
1770 </li>
1771 <li>
1772 <p>Define the translation table (thanks, Shawn Pearce[5], Jonathan
1773 Tan, and Masaya Suzuki)</p>
1774 </li>
1775 <li>
1776 <p>Avoid loose object overhead by packing more aggressively in
1777 "git gc --auto"</p>
1778 </li>
1779 </ul>
1780 </div>
1781 <div class="paragraph">
1782 <p>Later history:</p>
1783 </div>
1784 <div class="ulist">
1785 <ul>
1786 <li>
1787 <p>See the history of this file in git.git for the history of subsequent
1788 edits. This document history is no longer being maintained as it
1789 would now be superfluous to the commit log</p>
1790 </li>
1791 </ul>
1792 </div>
1793 <div class="paragraph">
1794 <p>References:</p>
1795 </div>
1796 <div class="literalblock">
1797 <div class="content">
1798 <pre>[1] https://lore.kernel.org/git/CA+55aFzJtejiCjV0e43+9oR3QuJK2PiFiLQemytoLpyJWe6P9w@mail.gmail.com/
1799 [2] https://lore.kernel.org/git/CA+55aFz+gkAsDZ24zmePQuEs1XPS9BP_s8O7Q4wQ7LV7X5-oDA@mail.gmail.com/
1800 [3] https://lore.kernel.org/git/20170306084353.nrns455dvkdsfgo5@sigill.intra.peff.net/
1801 [4] https://lore.kernel.org/git/20170304224936.rqqtkdvfjgyezsht@genre.crustytoothpaste.net
1802 [5] https://lore.kernel.org/git/CAJo=hJtoX9=AyLHHpUJS7fueV9ciZ_MNpnEPHUz8Whui6g9F0A@mail.gmail.com/</pre>
1803 </div>
1804 </div>
1805 </div>
1806 </div>
1807 </div>
1808 <div id="footer">
1809 <div id="footer-text">
1810 Last updated 2023-01-30 14:44:53 -0800
1811 </div>
1812 </div>
1813 </body>
1814 </html>