BICEPS

BASIC INFORMATION

Short name: BICEPS
Long name: BICEPS Benchmarking

Company: Easter-eggs
Country: France

Call: F4Fp-SME-COD210921  (see call details)
Proposal number: F4Fp-SME-COD210921-01

SUMMARY REMARKS & TESTBEDS

BICEPS proposal to the June 2021 call for NGI Zero Discovery was accepted. It is a theoretical storage solution designed in 2021 to address the technical challenges that appear when storing billions of small (~4KB) objects. It was validated with preliminary benchmarks on grid5000 during the summer of 2021. The goal is to write an implementation integrated into the Software Heritage codebase.

The primary purpose of BICEPS is to deliver performances to significantly reduce the infrastructure cost required to store billions of objects on a distributed storage at a petabyte scale and speed up mirroring and processing of the entire corpus. It is beneficial to the Software Heritage project because it allows to store many more objects and enable researchers to verify results in a matter of weeks instead of months. Experiments validating the newer implementation must be run on a large clusters to continuously verify it matches the expectations. The capabilities provided by the Fed4FIRE+ Federation are necessary to the success of BICEPS.

MATERIALS