Crash in the FreeDesktop GitLab infrastructure affecting the repositories of many projects

The development infrastructure supported by the FreeDesktop community based on the GitLab platform (gitlab.freedesktop.org) was unavailable due to the failure of two SSD drives at once in a distributed storage based on the Ceph FS. No predictions are made yet as to whether it will be possible to restore all the up-to-date data of internal GitLab services (mirrors worked for git repositories, but data on bug tracking and code reviews may be partially lost).

It was not possible to restore the storage for the Kubernetes cluster from the first run, after which the administrators went to bed to continue the restoration with a fresh mind. The work is still limited to the intention to raise the storage using the capabilities of the Ceph FS to ensure fault tolerance and store redundant data with their replication to different nodes. The existence and relevance of individual backups has not yet been discussed in the discussion.

The FreeDesktop project switched to GitLab as their primary collaborative development platform in 2018, and used it not only to access repositories, but also to track bugs, code reviews, write documentation, and test in continuous integration systems. The repository mirrors remain available on GitHub.

The Freedesktop.org infrastructure maintains over 1200 open source project repositories. Projects such as Mesa, Wayland, X.Org Server, D-Bus, Pipewire, PulseAudio, GStreamer, NetworkManager, libinput, PolKit, and FreeType are used as the primary GitLab platform on Freedesktop servers. The systemd project is formally FreeDesktop, but uses GitHub as its primary development platform. The LibreOffice project, which also partially uses the FreeDesktop infrastructure, uses its own server based on Gerrit to receive changes.

Source: opennet.ru

Add a comment