technical/racy-git.html

   1 <?xml version="1.0" encoding="UTF-8"?>
   2 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
   3     "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
   4 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
   5 <head>
   6 <meta http-equiv="Content-Type" content="application/xhtml+xml; charset=UTF-8" />
   7 <meta name="generator" content="AsciiDoc 10.2.0" />
   8 <title>Use of index and Racy Git problem</title>
   9 <style type="text/css">
  10 /* Shared CSS for AsciiDoc xhtml11 and html5 backends */
  11
  12 /* Default font. */
  13 body {
  14   font-family: Georgia,serif;
  15 }
  16
  17 /* Title font. */
  18 h1, h2, h3, h4, h5, h6,
  19 div.title, caption.title,
  20 thead, p.table.header,
  21 #toctitle,
  22 #author, #revnumber, #revdate, #revremark,
  23 #footer {
  24   font-family: Arial,Helvetica,sans-serif;
  25 }
  26
  27 body {
  28   margin: 1em 5% 1em 5%;
  29 }
  30
  31 a {
  32   color: blue;
  33   text-decoration: underline;
  34 }
  35 a:visited {
  36   color: fuchsia;
  37 }
  38
  39 em {
  40   font-style: italic;
  41   color: navy;
  42 }
  43
  44 strong {
  45   font-weight: bold;
  46   color: #083194;
  47 }
  48
  49 h1, h2, h3, h4, h5, h6 {
  50   color: #527bbd;
  51   margin-top: 1.2em;
  52   margin-bottom: 0.5em;
  53   line-height: 1.3;
  54 }
  55
  56 h1, h2, h3 {
  57   border-bottom: 2px solid silver;
  58 }
  59 h2 {
  60   padding-top: 0.5em;
  61 }
  62 h3 {
  63   float: left;
  64 }
  65 h3 + * {
  66   clear: left;
  67 }
  68 h5 {
  69   font-size: 1.0em;
  70 }
  71
  72 div.sectionbody {
  73   margin-left: 0;
  74 }
  75
  76 hr {
  77   border: 1px solid silver;
  78 }
  79
  80 p {
  81   margin-top: 0.5em;
  82   margin-bottom: 0.5em;
  83 }
  84
  85 ul, ol, li > p {
  86   margin-top: 0;
  87 }
  88 ul > li     { color: #aaa; }
  89 ul > li > * { color: black; }
  90
  91 .monospaced, code, pre {
  92   font-family: "Courier New", Courier, monospace;
  93   font-size: inherit;
  94   color: navy;
  95   padding: 0;
  96   margin: 0;
  97 }
  98 pre {
  99   white-space: pre-wrap;
 100 }
 101
 102 #author {
 103   color: #527bbd;
 104   font-weight: bold;
 105   font-size: 1.1em;
 106 }
 107 #email {
 108 }
 109 #revnumber, #revdate, #revremark {
 110 }
 111
 112 #footer {
 113   font-size: small;
 114   border-top: 2px solid silver;
 115   padding-top: 0.5em;
 116   margin-top: 4.0em;
 117 }
 118 #footer-text {
 119   float: left;
 120   padding-bottom: 0.5em;
 121 }
 122 #footer-badges {
 123   float: right;
 124   padding-bottom: 0.5em;
 125 }
 126
 127 #preamble {
 128   margin-top: 1.5em;
 129   margin-bottom: 1.5em;
 130 }
 131 div.imageblock, div.exampleblock, div.verseblock,
 132 div.quoteblock, div.literalblock, div.listingblock, div.sidebarblock,
 133 div.admonitionblock {
 134   margin-top: 1.0em;
 135   margin-bottom: 1.5em;
 136 }
 137 div.admonitionblock {
 138   margin-top: 2.0em;
 139   margin-bottom: 2.0em;
 140   margin-right: 10%;
 141   color: #606060;
 142 }
 143
 144 div.content { /* Block element content. */
 145   padding: 0;
 146 }
 147
 148 /* Block element titles. */
 149 div.title, caption.title {
 150   color: #527bbd;
 151   font-weight: bold;
 152   text-align: left;
 153   margin-top: 1.0em;
 154   margin-bottom: 0.5em;
 155 }
 156 div.title + * {
 157   margin-top: 0;
 158 }
 159
 160 td div.title:first-child {
 161   margin-top: 0.0em;
 162 }
 163 div.content div.title:first-child {
 164   margin-top: 0.0em;
 165 }
 166 div.content + div.title {
 167   margin-top: 0.0em;
 168 }
 169
 170 div.sidebarblock > div.content {
 171   background: #ffffee;
 172   border: 1px solid #dddddd;
 173   border-left: 4px solid #f0f0f0;
 174   padding: 0.5em;
 175 }
 176
 177 div.listingblock > div.content {
 178   border: 1px solid #dddddd;
 179   border-left: 5px solid #f0f0f0;
 180   background: #f8f8f8;
 181   padding: 0.5em;
 182 }
 183
 184 div.quoteblock, div.verseblock {
 185   padding-left: 1.0em;
 186   margin-left: 1.0em;
 187   margin-right: 10%;
 188   border-left: 5px solid #f0f0f0;
 189   color: #888;
 190 }
 191
 192 div.quoteblock > div.attribution {
 193   padding-top: 0.5em;
 194   text-align: right;
 195 }
 196
 197 div.verseblock > pre.content {
 198   font-family: inherit;
 199   font-size: inherit;
 200 }
 201 div.verseblock > div.attribution {
 202   padding-top: 0.75em;
 203   text-align: left;
 204 }
 205 /* DEPRECATED: Pre version 8.2.7 verse style literal block. */
 206 div.verseblock + div.attribution {
 207   text-align: left;
 208 }
 209
 210 div.admonitionblock .icon {
 211   vertical-align: top;
 212   font-size: 1.1em;
 213   font-weight: bold;
 214   text-decoration: underline;
 215   color: #527bbd;
 216   padding-right: 0.5em;
 217 }
 218 div.admonitionblock td.content {
 219   padding-left: 0.5em;
 220   border-left: 3px solid #dddddd;
 221 }
 222
 223 div.exampleblock > div.content {
 224   border-left: 3px solid #dddddd;
 225   padding-left: 0.5em;
 226 }
 227
 228 div.imageblock div.content { padding-left: 0; }
 229 span.image img { border-style: none; vertical-align: text-bottom; }
 230 a.image:visited { color: white; }
 231
 232 dl {
 233   margin-top: 0.8em;
 234   margin-bottom: 0.8em;
 235 }
 236 dt {
 237   margin-top: 0.5em;
 238   margin-bottom: 0;
 239   font-style: normal;
 240   color: navy;
 241 }
 242 dd > *:first-child {
 243   margin-top: 0.1em;
 244 }
 245
 246 ul, ol {
 247     list-style-position: outside;
 248 }
 249 ol.arabic {
 250   list-style-type: decimal;
 251 }
 252 ol.loweralpha {
 253   list-style-type: lower-alpha;
 254 }
 255 ol.upperalpha {
 256   list-style-type: upper-alpha;
 257 }
 258 ol.lowerroman {
 259   list-style-type: lower-roman;
 260 }
 261 ol.upperroman {
 262   list-style-type: upper-roman;
 263 }
 264
 265 div.compact ul, div.compact ol,
 266 div.compact p, div.compact p,
 267 div.compact div, div.compact div {
 268   margin-top: 0.1em;
 269   margin-bottom: 0.1em;
 270 }
 271
 272 tfoot {
 273   font-weight: bold;
 274 }
 275 td > div.verse {
 276   white-space: pre;
 277 }
 278
 279 div.hdlist {
 280   margin-top: 0.8em;
 281   margin-bottom: 0.8em;
 282 }
 283 div.hdlist tr {
 284   padding-bottom: 15px;
 285 }
 286 dt.hdlist1.strong, td.hdlist1.strong {
 287   font-weight: bold;
 288 }
 289 td.hdlist1 {
 290   vertical-align: top;
 291   font-style: normal;
 292   padding-right: 0.8em;
 293   color: navy;
 294 }
 295 td.hdlist2 {
 296   vertical-align: top;
 297 }
 298 div.hdlist.compact tr {
 299   margin: 0;
 300   padding-bottom: 0;
 301 }
 302
 303 .comment {
 304   background: yellow;
 305 }
 306
 307 .footnote, .footnoteref {
 308   font-size: 0.8em;
 309 }
 310
 311 span.footnote, span.footnoteref {
 312   vertical-align: super;
 313 }
 314
 315 #footnotes {
 316   margin: 20px 0 20px 0;
 317   padding: 7px 0 0 0;
 318 }
 319
 320 #footnotes div.footnote {
 321   margin: 0 0 5px 0;
 322 }
 323
 324 #footnotes hr {
 325   border: none;
 326   border-top: 1px solid silver;
 327   height: 1px;
 328   text-align: left;
 329   margin-left: 0;
 330   width: 20%;
 331   min-width: 100px;
 332 }
 333
 334 div.colist td {
 335   padding-right: 0.5em;
 336   padding-bottom: 0.3em;
 337   vertical-align: top;
 338 }
 339 div.colist td img {
 340   margin-top: 0.3em;
 341 }
 342
 343 @media print {
 344   #footer-badges { display: none; }
 345 }
 346
 347 #toc {
 348   margin-bottom: 2.5em;
 349 }
 350
 351 #toctitle {
 352   color: #527bbd;
 353   font-size: 1.1em;
 354   font-weight: bold;
 355   margin-top: 1.0em;
 356   margin-bottom: 0.1em;
 357 }
 358
 359 div.toclevel0, div.toclevel1, div.toclevel2, div.toclevel3, div.toclevel4 {
 360   margin-top: 0;
 361   margin-bottom: 0;
 362 }
 363 div.toclevel2 {
 364   margin-left: 2em;
 365   font-size: 0.9em;
 366 }
 367 div.toclevel3 {
 368   margin-left: 4em;
 369   font-size: 0.9em;
 370 }
 371 div.toclevel4 {
 372   margin-left: 6em;
 373   font-size: 0.9em;
 374 }
 375
 376 span.aqua { color: aqua; }
 377 span.black { color: black; }
 378 span.blue { color: blue; }
 379 span.fuchsia { color: fuchsia; }
 380 span.gray { color: gray; }
 381 span.green { color: green; }
 382 span.lime { color: lime; }
 383 span.maroon { color: maroon; }
 384 span.navy { color: navy; }
 385 span.olive { color: olive; }
 386 span.purple { color: purple; }
 387 span.red { color: red; }
 388 span.silver { color: silver; }
 389 span.teal { color: teal; }
 390 span.white { color: white; }
 391 span.yellow { color: yellow; }
 392
 393 span.aqua-background { background: aqua; }
 394 span.black-background { background: black; }
 395 span.blue-background { background: blue; }
 396 span.fuchsia-background { background: fuchsia; }
 397 span.gray-background { background: gray; }
 398 span.green-background { background: green; }
 399 span.lime-background { background: lime; }
 400 span.maroon-background { background: maroon; }
 401 span.navy-background { background: navy; }
 402 span.olive-background { background: olive; }
 403 span.purple-background { background: purple; }
 404 span.red-background { background: red; }
 405 span.silver-background { background: silver; }
 406 span.teal-background { background: teal; }
 407 span.white-background { background: white; }
 408 span.yellow-background { background: yellow; }
 409
 410 span.big { font-size: 2em; }
 411 span.small { font-size: 0.6em; }
 412
 413 span.underline { text-decoration: underline; }
 414 span.overline { text-decoration: overline; }
 415 span.line-through { text-decoration: line-through; }
 416
 417 div.unbreakable { page-break-inside: avoid; }
 418
 419
 420 /*
 421  * xhtml11 specific
 422  *
 423  * */
 424
 425 div.tableblock {
 426   margin-top: 1.0em;
 427   margin-bottom: 1.5em;
 428 }
 429 div.tableblock > table {
 430   border: 3px solid #527bbd;
 431 }
 432 thead, p.table.header {
 433   font-weight: bold;
 434   color: #527bbd;
 435 }
 436 p.table {
 437   margin-top: 0;
 438 }
 439 /* Because the table frame attribute is overridden by CSS in most browsers. */
 440 div.tableblock > table[frame="void"] {
 441   border-style: none;
 442 }
 443 div.tableblock > table[frame="hsides"] {
 444   border-left-style: none;
 445   border-right-style: none;
 446 }
 447 div.tableblock > table[frame="vsides"] {
 448   border-top-style: none;
 449   border-bottom-style: none;
 450 }
 451
 452
 453 /*
 454  * html5 specific
 455  *
 456  * */
 457
 458 table.tableblock {
 459   margin-top: 1.0em;
 460   margin-bottom: 1.5em;
 461 }
 462 thead, p.tableblock.header {
 463   font-weight: bold;
 464   color: #527bbd;
 465 }
 466 p.tableblock {
 467   margin-top: 0;
 468 }
 469 table.tableblock {
 470   border-width: 3px;
 471   border-spacing: 0px;
 472   border-style: solid;
 473   border-color: #527bbd;
 474   border-collapse: collapse;
 475 }
 476 th.tableblock, td.tableblock {
 477   border-width: 1px;
 478   padding: 4px;
 479   border-style: solid;
 480   border-color: #527bbd;
 481 }
 482
 483 table.tableblock.frame-topbot {
 484   border-left-style: hidden;
 485   border-right-style: hidden;
 486 }
 487 table.tableblock.frame-sides {
 488   border-top-style: hidden;
 489   border-bottom-style: hidden;
 490 }
 491 table.tableblock.frame-none {
 492   border-style: hidden;
 493 }
 494
 495 th.tableblock.halign-left, td.tableblock.halign-left {
 496   text-align: left;
 497 }
 498 th.tableblock.halign-center, td.tableblock.halign-center {
 499   text-align: center;
 500 }
 501 th.tableblock.halign-right, td.tableblock.halign-right {
 502   text-align: right;
 503 }
 504
 505 th.tableblock.valign-top, td.tableblock.valign-top {
 506   vertical-align: top;
 507 }
 508 th.tableblock.valign-middle, td.tableblock.valign-middle {
 509   vertical-align: middle;
 510 }
 511 th.tableblock.valign-bottom, td.tableblock.valign-bottom {
 512   vertical-align: bottom;
 513 }
 514
 515
 516 /*
 517  * manpage specific
 518  *
 519  * */
 520
 521 body.manpage h1 {
 522   padding-top: 0.5em;
 523   padding-bottom: 0.5em;
 524   border-top: 2px solid silver;
 525   border-bottom: 2px solid silver;
 526 }
 527 body.manpage h2 {
 528   border-style: none;
 529 }
 530 body.manpage div.sectionbody {
 531   margin-left: 3em;
 532 }
 533
 534 @media print {
 535   body.manpage div#toc { display: none; }
 536 }
 537
 538
 539 </style>
 540 <script type="text/javascript">
 541 /*<![CDATA[*/
 542 var asciidoc = {  // Namespace.
 543
 544 /////////////////////////////////////////////////////////////////////
 545 // Table Of Contents generator
 546 /////////////////////////////////////////////////////////////////////
 547
 548 /* Author: Mihai Bazon, September 2002
 549  * http://students.infoiasi.ro/~mishoo
 550  *
 551  * Table Of Content generator
 552  * Version: 0.4
 553  *
 554  * Feel free to use this script under the terms of the GNU General Public
 555  * License, as long as you do not remove or alter this notice.
 556  */
 557
 558  /* modified by Troy D. Hanson, September 2006. License: GPL */
 559  /* modified by Stuart Rackham, 2006, 2009. License: GPL */
 560
 561 // toclevels = 1..4.
 562 toc: function (toclevels) {
 563
 564   function getText(el) {
 565     var text = "";
 566     for (var i = el.firstChild; i != null; i = i.nextSibling) {
 567       if (i.nodeType == 3 /* Node.TEXT_NODE */) // IE doesn't speak constants.
 568         text += i.data;
 569       else if (i.firstChild != null)
 570         text += getText(i);
 571     }
 572     return text;
 573   }
 574
 575   function TocEntry(el, text, toclevel) {
 576     this.element = el;
 577     this.text = text;
 578     this.toclevel = toclevel;
 579   }
 580
 581   function tocEntries(el, toclevels) {
 582     var result = new Array;
 583     var re = new RegExp('[hH]([1-'+(toclevels+1)+'])');
 584     // Function that scans the DOM tree for header elements (the DOM2
 585     // nodeIterator API would be a better technique but not supported by all
 586     // browsers).
 587     var iterate = function (el) {
 588       for (var i = el.firstChild; i != null; i = i.nextSibling) {
 589         if (i.nodeType == 1 /* Node.ELEMENT_NODE */) {
 590           var mo = re.exec(i.tagName);
 591           if (mo && (i.getAttribute("class") || i.getAttribute("className")) != "float") {
 592             result[result.length] = new TocEntry(i, getText(i), mo[1]-1);
 593           }
 594           iterate(i);
 595         }
 596       }
 597     }
 598     iterate(el);
 599     return result;
 600   }
 601
 602   var toc = document.getElementById("toc");
 603   if (!toc) {
 604     return;
 605   }
 606
 607   // Delete existing TOC entries in case we're reloading the TOC.
 608   var tocEntriesToRemove = [];
 609   var i;
 610   for (i = 0; i < toc.childNodes.length; i++) {
 611     var entry = toc.childNodes[i];
 612     if (entry.nodeName.toLowerCase() == 'div'
 613      && entry.getAttribute("class")
 614      && entry.getAttribute("class").match(/^toclevel/))
 615       tocEntriesToRemove.push(entry);
 616   }
 617   for (i = 0; i < tocEntriesToRemove.length; i++) {
 618     toc.removeChild(tocEntriesToRemove[i]);
 619   }
 620
 621   // Rebuild TOC entries.
 622   var entries = tocEntries(document.getElementById("content"), toclevels);
 623   for (var i = 0; i < entries.length; ++i) {
 624     var entry = entries[i];
 625     if (entry.element.id == "")
 626       entry.element.id = "_toc_" + i;
 627     var a = document.createElement("a");
 628     a.href = "#" + entry.element.id;
 629     a.appendChild(document.createTextNode(entry.text));
 630     var div = document.createElement("div");
 631     div.appendChild(a);
 632     div.className = "toclevel" + entry.toclevel;
 633     toc.appendChild(div);
 634   }
 635   if (entries.length == 0)
 636     toc.parentNode.removeChild(toc);
 637 },
 638
 639
 640 /////////////////////////////////////////////////////////////////////
 641 // Footnotes generator
 642 /////////////////////////////////////////////////////////////////////
 643
 644 /* Based on footnote generation code from:
 645  * http://www.brandspankingnew.net/archive/2005/07/format_footnote.html
 646  */
 647
 648 footnotes: function () {
 649   // Delete existing footnote entries in case we're reloading the footnodes.
 650   var i;
 651   var noteholder = document.getElementById("footnotes");
 652   if (!noteholder) {
 653     return;
 654   }
 655   var entriesToRemove = [];
 656   for (i = 0; i < noteholder.childNodes.length; i++) {
 657     var entry = noteholder.childNodes[i];
 658     if (entry.nodeName.toLowerCase() == 'div' && entry.getAttribute("class") == "footnote")
 659       entriesToRemove.push(entry);
 660   }
 661   for (i = 0; i < entriesToRemove.length; i++) {
 662     noteholder.removeChild(entriesToRemove[i]);
 663   }
 664
 665   // Rebuild footnote entries.
 666   var cont = document.getElementById("content");
 667   var spans = cont.getElementsByTagName("span");
 668   var refs = {};
 669   var n = 0;
 670   for (i=0; i<spans.length; i++) {
 671     if (spans[i].className == "footnote") {
 672       n++;
 673       var note = spans[i].getAttribute("data-note");
 674       if (!note) {
 675         // Use [\s\S] in place of . so multi-line matches work.
 676         // Because JavaScript has no s (dotall) regex flag.
 677         note = spans[i].innerHTML.match(/\s*\[([\s\S]*)]\s*/)[1];
 678         spans[i].innerHTML =
 679           "[<a id='_footnoteref_" + n + "' href='#_footnote_" + n +
 680           "' title='View footnote' class='footnote'>" + n + "</a>]";
 681         spans[i].setAttribute("data-note", note);
 682       }
 683       noteholder.innerHTML +=
 684         "<div class='footnote' id='_footnote_" + n + "'>" +
 685         "<a href='#_footnoteref_" + n + "' title='Return to text'>" +
 686         n + "</a>. " + note + "</div>";
 687       var id =spans[i].getAttribute("id");
 688       if (id != null) refs["#"+id] = n;
 689     }
 690   }
 691   if (n == 0)
 692     noteholder.parentNode.removeChild(noteholder);
 693   else {
 694     // Process footnoterefs.
 695     for (i=0; i<spans.length; i++) {
 696       if (spans[i].className == "footnoteref") {
 697         var href = spans[i].getElementsByTagName("a")[0].getAttribute("href");
 698         href = href.match(/#.*/)[0];  // Because IE return full URL.
 699         n = refs[href];
 700         spans[i].innerHTML =
 701           "[<a href='#_footnote_" + n +
 702           "' title='View footnote' class='footnote'>" + n + "</a>]";
 703       }
 704     }
 705   }
 706 },
 707
 708 install: function(toclevels) {
 709   var timerId;
 710
 711   function reinstall() {
 712     asciidoc.footnotes();
 713     if (toclevels) {
 714       asciidoc.toc(toclevels);
 715     }
 716   }
 717
 718   function reinstallAndRemoveTimer() {
 719     clearInterval(timerId);
 720     reinstall();
 721   }
 722
 723   timerId = setInterval(reinstall, 500);
 724   if (document.addEventListener)
 725     document.addEventListener("DOMContentLoaded", reinstallAndRemoveTimer, false);
 726   else
 727     window.onload = reinstallAndRemoveTimer;
 728 }
 729
 730 }
 731 asciidoc.install();
 732 /*]]>*/
 733 </script>
 734 </head>
 735 <body class="article">
 736 <div id="header">
 737 <h1>Use of index and Racy Git problem</h1>
 738 <span id="revdate">2024-07-12</span>
 739 </div>
 740 <div id="content">
 741 <div class="sect1">
 742 <h2 id="_background">Background</h2>
 743 <div class="sectionbody">
 744 <div class="paragraph"><p>The index is one of the most important data structures in Git.
 745 It represents a virtual working tree state by recording list of
 746 paths and their object names and serves as a staging area to
 747 write out the next tree object to be committed.  The state is
 748 "virtual" in the sense that it does not necessarily have to, and
 749 often does not, match the files in the working tree.</p></div>
 750 <div class="paragraph"><p>There are cases where Git needs to examine the differences between the
 751 virtual working tree state in the index and the files in the
 752 working tree.  The most obvious case is when the user asks <code>git
 753 diff</code> (or its low level implementation, <code>git diff-files</code>) or
 754 <code>git-ls-files --modified</code>.  In addition, Git internally checks
 755 if the files in the working tree are different from what are
 756 recorded in the index to avoid stomping on local changes in them
 757 during patch application, switching branches, and merging.</p></div>
 758 <div class="paragraph"><p>In order to speed up this comparison between the files in the
 759 working tree and the index entries, the index entries record the
 760 information obtained from the filesystem via <code>lstat(2)</code> system
 761 call when they were last updated.  When checking if they differ,
 762 Git first runs <code>lstat(2)</code> on the files and compares the result
 763 with this information (this is what was originally done by the
 764 <code>ce_match_stat()</code> function, but the current code does it in
 765 <code>ce_match_stat_basic()</code> function).  If some of these "cached
 766 stat information" fields do not match, Git can tell that the
 767 files are modified without even looking at their contents.</p></div>
 768 <div class="paragraph"><p>Note: not all members in <code>struct stat</code> obtained via <code>lstat(2)</code>
 769 are used for this comparison.  For example, <code>st_atime</code> obviously
 770 is not useful.  Currently, Git compares the file type (regular
 771 files vs symbolic links) and executable bits (only for regular
 772 files) from <code>st_mode</code> member, <code>st_mtime</code> and <code>st_ctime</code>
 773 timestamps, <code>st_uid</code>, <code>st_gid</code>, <code>st_ino</code>, and <code>st_size</code> members.
 774 With a <code>USE_STDEV</code> compile-time option, <code>st_dev</code> is also
 775 compared, but this is not enabled by default because this member
 776 is not stable on network filesystems.  With <code>USE_NSEC</code>
 777 compile-time option, <code>st_mtim.tv_nsec</code> and <code>st_ctim.tv_nsec</code>
 778 members are also compared. On Linux, this is not enabled by default
 779 because in-core timestamps can have finer granularity than
 780 on-disk timestamps, resulting in meaningless changes when an
 781 inode is evicted from the inode cache.  See commit 8ce13b0
 782 of git://git.kernel.org/pub/scm/linux/kernel/git/tglx/history.git
 783 ([PATCH] Sync in core time granularity with filesystems,
 784 2005-01-04). This patch is included in kernel 2.6.11 and newer, but
 785 only fixes the issue for file systems with exactly 1 ns or 1 s
 786 resolution. Other file systems are still broken in current Linux
 787 kernels (e.g. CEPH, CIFS, NTFS, UDF), see
 788 <a href="https://lore.kernel.org/lkml/5577240D.7020309@gmail.com/">https://lore.kernel.org/lkml/5577240D.7020309@gmail.com/</a></p></div>
 789 </div>
 790 </div>
 791 <div class="sect1">
 792 <h2 id="_racy_git">Racy Git</h2>
 793 <div class="sectionbody">
 794 <div class="paragraph"><p>There is one slight problem with the optimization based on the
 795 cached stat information.  Consider this sequence:</p></div>
 796 <div class="literalblock">
 797 <div class="content">
 798 <pre><code>: modify 'foo'
 799 $ git update-index 'foo'
 800 : modify 'foo' again, in-place, without changing its size</code></pre>
 801 </div></div>
 802 <div class="paragraph"><p>The first <code>update-index</code> computes the object name of the
 803 contents of file <code>foo</code> and updates the index entry for <code>foo</code>
 804 along with the <code>struct stat</code> information.  If the modification
 805 that follows it happens very fast so that the file&#8217;s <code>st_mtime</code>
 806 timestamp does not change, after this sequence, the cached stat
 807 information the index entry records still exactly match what you
 808 would see in the filesystem, even though the file <code>foo</code> is now
 809 different.
 810 This way, Git can incorrectly think files in the working tree
 811 are unmodified even though they actually are.  This is called
 812 the "racy Git" problem (discovered by Pasky), and the entries
 813 that appear clean when they may not be because of this problem
 814 are called "racily clean".</p></div>
 815 <div class="paragraph"><p>To avoid this problem, Git does two things:</p></div>
 816 <div class="olist arabic"><ol class="arabic">
 817 <li>
 818 <p>
 819 When the cached stat information says the file has not been
 820   modified, and the <code>st_mtime</code> is the same as (or newer than)
 821   the timestamp of the index file itself (which is the time <code>git
 822   update-index foo</code> finished running in the above example), it
 823   also compares the contents with the object registered in the
 824   index entry to make sure they match.
 825 </p>
 826 </li>
 827 <li>
 828 <p>
 829 When the index file is updated that contains racily clean
 830   entries, cached <code>st_size</code> information is truncated to zero
 831   before writing a new version of the index file.
 832 </p>
 833 </li>
 834 </ol></div>
 835 <div class="paragraph"><p>Because the index file itself is written after collecting all
 836 the stat information from updated paths, <code>st_mtime</code> timestamp of
 837 it is usually the same as or newer than any of the paths the
 838 index contains.  And no matter how quick the modification that
 839 follows <code>git update-index foo</code> finishes, the resulting
 840 <code>st_mtime</code> timestamp on <code>foo</code> cannot get a value earlier
 841 than the index file.  Therefore, index entries that can be
 842 racily clean are limited to the ones that have the same
 843 timestamp as the index file itself.</p></div>
 844 <div class="paragraph"><p>The callers that want to check if an index entry matches the
 845 corresponding file in the working tree continue to call
 846 <code>ce_match_stat()</code>, but with this change, <code>ce_match_stat()</code> uses
 847 <code>ce_modified_check_fs()</code> to see if racily clean ones are
 848 actually clean after comparing the cached stat information using
 849 <code>ce_match_stat_basic()</code>.</p></div>
 850 <div class="paragraph"><p>The problem the latter solves is this sequence:</p></div>
 851 <div class="literalblock">
 852 <div class="content">
 853 <pre><code>$ git update-index 'foo'
 854 : modify 'foo' in-place without changing its size
 855 : wait for enough time
 856 $ git update-index 'bar'</code></pre>
 857 </div></div>
 858 <div class="paragraph"><p>Without the latter, the timestamp of the index file gets a newer
 859 value, and falsely clean entry <code>foo</code> would not be caught by the
 860 timestamp comparison check done with the former logic anymore.
 861 The latter makes sure that the cached stat information for <code>foo</code>
 862 would never match with the file in the working tree, so later
 863 checks by <code>ce_match_stat_basic()</code> would report that the index entry
 864 does not match the file and Git does not have to fall back on more
 865 expensive <code>ce_modified_check_fs()</code>.</p></div>
 866 </div>
 867 </div>
 868 <div class="sect1">
 869 <h2 id="_runtime_penalty">Runtime penalty</h2>
 870 <div class="sectionbody">
 871 <div class="paragraph"><p>The runtime penalty of falling back to <code>ce_modified_check_fs()</code>
 872 from <code>ce_match_stat()</code> can be very expensive when there are many
 873 racily clean entries.  An obvious way to artificially create
 874 this situation is to give the same timestamp to all the files in
 875 the working tree in a large project, run <code>git update-index</code> on
 876 them, and give the same timestamp to the index file:</p></div>
 877 <div class="literalblock">
 878 <div class="content">
 879 <pre><code>$ date &gt;.datestamp
 880 $ git ls-files | xargs touch -r .datestamp
 881 $ git ls-files | git update-index --stdin
 882 $ touch -r .datestamp .git/index</code></pre>
 883 </div></div>
 884 <div class="paragraph"><p>This will make all index entries racily clean.  The linux project, for
 885 example, there are over 20,000 files in the working tree.  On my
 886 Athlon 64 X2 3800+, after the above:</p></div>
 887 <div class="literalblock">
 888 <div class="content">
 889 <pre><code>$ /usr/bin/time git diff-files
 890 1.68user 0.54system 0:02.22elapsed 100%CPU (0avgtext+0avgdata 0maxresident)k
 891 0inputs+0outputs (0major+67111minor)pagefaults 0swaps
 892 $ git update-index MAINTAINERS
 893 $ /usr/bin/time git diff-files
 894 0.02user 0.12system 0:00.14elapsed 100%CPU (0avgtext+0avgdata 0maxresident)k
 895 0inputs+0outputs (0major+935minor)pagefaults 0swaps</code></pre>
 896 </div></div>
 897 <div class="paragraph"><p>Running <code>git update-index</code> in the middle checked the racily
 898 clean entries, and left the cached <code>st_mtime</code> for all the paths
 899 intact because they were actually clean (so this step took about
 900 the same amount of time as the first <code>git diff-files</code>).  After
 901 that, they are not racily clean anymore but are truly clean, so
 902 the second invocation of <code>git diff-files</code> fully took advantage
 903 of the cached stat information.</p></div>
 904 </div>
 905 </div>
 906 <div class="sect1">
 907 <h2 id="_avoiding_runtime_penalty">Avoiding runtime penalty</h2>
 908 <div class="sectionbody">
 909 <div class="paragraph"><p>In order to avoid the above runtime penalty, post 1.4.2 Git used
 910 to have a code that made sure the index file
 911 got a timestamp newer than the youngest files in the index when
 912 there were many young files with the same timestamp as the
 913 resulting index file otherwise would have by waiting
 914 before finishing writing the index file out.</p></div>
 915 <div class="paragraph"><p>I suspected that in practice the situation where many paths in the
 916 index are all racily clean was quite rare.  The only code paths
 917 that can record recent timestamp for large number of paths are:</p></div>
 918 <div class="olist arabic"><ol class="arabic">
 919 <li>
 920 <p>
 921 Initial <code>git add .</code> of a large project.
 922 </p>
 923 </li>
 924 <li>
 925 <p>
 926 <code>git checkout</code> of a large project from an empty index into an
 927   unpopulated working tree.
 928 </p>
 929 </li>
 930 </ol></div>
 931 <div class="paragraph"><p>Note: switching branches with <code>git checkout</code> keeps the cached
 932 stat information of existing working tree files that are the
 933 same between the current branch and the new branch, which are
 934 all older than the resulting index file, and they will not
 935 become racily clean.  Only the files that are actually checked
 936 out can become racily clean.</p></div>
 937 <div class="paragraph"><p>In a large project where raciness avoidance cost really matters,
 938 however, the initial computation of all object names in the
 939 index takes more than one second, and the index file is written
 940 out after all that happens.  Therefore the timestamp of the
 941 index file will be more than one second later than the
 942 youngest file in the working tree.  This means that in these
 943 cases there actually will not be any racily clean entry in
 944 the resulting index.</p></div>
 945 <div class="paragraph"><p>Based on this discussion, the current code does not use the
 946 "workaround" to avoid the runtime penalty that does not exist in
 947 practice anymore.  This was done with commit 0fc82cff on Aug 15,
 948 2006.</p></div>
 949 </div>
 950 </div>
 951 </div>
 952 <div id="footnotes"><hr /></div>
 953 <div id="footer">
 954 <div id="footer-text">
 955 Last updated
 956  2023-10-23 14:43:46 PDT
 957 </div>
 958 </div>
 959 </body>
 960 </html>