MultiPackIndex.find() hands out shared mutable state — concurrent lookups return wrong object offsets #291

Activity

1. Digital-shaman87 commented on Oct 7, 2026

   

   Digital-shaman87

   on Oct 7, 2026

   More actions

   Independent confirmation, with a minimal reproduction, a note on how widely this now triggers, and a request to prioritise it.

   Still reproducible on the newest release (7.8.0)

   Repository: an ordinary ~56 MB repo (4 packs, ~48k objects) whose objects/pack/multi-pack-index was written by stock Git. Same object store copied for every run; 30 rounds × 32 threads, each thread opening the same 1,600 existing object ids:

   JGit	core.multiPackIndex	Result
   7.7.0	true (default)	17–22 spurious MissingObjectException: Missing unknown <id>
   7.7.1	true	27
   7.8.0	true	15 / 27 / 22 (three runs)
   7.8.0	false	0
   7.7.0	false	0

   Every reported id exists (git cat-file -t succeeds). The shared PackOffset in MultiPackIndexV1.find() is still present in 7.8.0 and on master.

   // args: <gitDir> <file with one object id per line> <rounds> <threads>
   import org.eclipse.jgit.lib.*;
   import org.eclipse.jgit.storage.file.*;
   import java.io.File; import java.nio.file.*; import java.util.*;
   import java.util.concurrent.*; import java.util.concurrent.atomic.*;
   public class Stress {
     public static void main(String[] a) throws Exception {
       List<String> ids = Files.readAllLines(Paths.get(a[1]));
       int rounds = Integer.parseInt(a[2]), threads = Integer.parseInt(a[3]);
       AtomicInteger fails = new AtomicInteger();
       for (int r = 0; r < rounds; r++) {
         new WindowCacheConfig().install(); // cold start each round
         try (Repository repo = new FileRepositoryBuilder().setGitDir(new File(a[0])).setMustExist(true).build()) {
           ExecutorService ex = Executors.newFixedThreadPool(threads); CountDownLatch go = new CountDownLatch(1);
           List<Future<?>> fs = new ArrayList<>();
           for (int t = 0; t < threads; t++) { final int off = t; fs.add(ex.submit(() -> { go.await();
             try (ObjectReader rd = repo.newObjectReader()) {
               for (int i = 0; i < ids.size(); i++) {
                 try { rd.open(ObjectId.fromString(ids.get((i + off * 97) % ids.size()))).getSize(); }
                 catch (Exception e) { fails.incrementAndGet(); } } }
             return null; })); }
           go.countDown(); for (Future<?> f : fs) f.get(); ex.shutdown();
         }
       }
       System.out.println((fails.get() == 0 ? "GREEN" : "RED") + " fails=" + fails.get());
     }
   }

   Why this is now urgent: stock Git creates the midx automatically

   With Git 2.55 (Git for Windows 2.55.0.windows.5 here, default configuration, no maintenance.*/repack.* settings), ordinary commits run git maintenance run --auto, whose default geometric strategy repacks with a multi-pack-index once about 100 loose objects accumulate (maintenance.geometric-repack.auto, default 100). In our case the midx appeared during a normal working session. So users do not opt in to a midx; they get one, and from then on every multithreaded JGit consumer of that repository can fail at random.

   Concrete downstream impact: SonarQube's SCM blame (multithreaded JGit blame in the scanner engine) fails the whole analysis non-deterministically with MissingObjectException ("Error during SonarScanner Engine execution"). The scanner engines we checked all bundle an affected JGit: sonar-scanner-engine 13.7.0 (SonarQube Community Build 26.9) → JGit 7.7.0; 13.12.0 → 7.7.1; 13.15.0 → 7.8.0. A newer SonarQube does not help until JGit is fixed.

   Workaround for anyone hitting this: git config core.multiPackIndex false in the repository that owns the objects (for a clone using objects/info/alternates, the alternate's repository config is the one JGit reads); JGit then ignores the existing midx.

   Suggested fix

   Do not share the result between calls: have MultiPackIndexV1.find() return a new PackOffset per call (it is a small value object), or have PackMidx.MidxPackIndex.findOffset() copy the fields under the reader's own instance. Either removes the shared mutable state; the stress test above makes a regression test (it fails within seconds on a repo with a midx).

   Given that stock Git now writes a midx by default and that the failure is silent data "loss" (existing objects reported missing), could this be prioritised for the next 7.x service release? Happy to run a candidate build against the reproduction.