Library Genesis is a true gem of the Internet. This online library, offering free access to over 2.7 million books, has taken a long-awaited step this week. One of the library's web mirrors now allows users to download files through IPFS — a distributed file system.
So, the collection of Library Genesis books has been uploaded to IPFS, pinned, and linked to search functionality. This means it has become a bit harder to deprive people of access to our shared cultural and scientific heritage.
About LibGen
In the early 2000s, the still unregulated Internet hosted dozens of collections of scientific books. The largest collections that I can recall — KoLXo3, mehmat, and mirknig — contained tens of thousands of textbooks, publications, and other important djvu and pdf files for students by 2007.
Like any other file dumps, these collections suffered from common navigation issues. For instance, the Kolkhoz library existed on over 20 DVDs. The most in-demand parts of the library were moved by older students to the dormitory file sharing system, and if something rare was needed, woe to you! At the very least, you ended up buying beer for the owner of the disks.
Nevertheless, the collections were still of tangible size. Although searching by file names often hit a wall due to the creativity of the file creator, a manual full scan could help retrieve the needed book after tirelessly scrolling through dozens of pages.
In 2008, on rutracker.ru (then torrents.ru), an enthusiast published torrents that compiled existing book collections into one large heap. In the same thread, a person began the meticulous work of organizing the uploaded files and creating a web interface. This is how Library Genesis came to be.
Since 2008, LibGen has evolved and expanded its own bookshelves through the efforts of the community. Book metadata has been edited, preserved, and distributed as MySQL dumps for anyone to access. The altruistic approach to metadata has led to the emergence of numerous mirrors and increased the survival of the entire project, despite growing fragmentation.
An important milestone in the life of the library was the mirroring of the Sci-Hub database, which started in 2013. Thanks to the collaboration of the two systems, an unprecedented quality set of data was concentrated in one place — scientific and literary books along with scientific publications. I have a suspicion that a single dump of the combined LibGen and Sci-Hub database would be enough to restore the scientific and technological progress of civilization in the event of its loss during a catastrophe.
Today, the library remains quite stable, offering a web interface that allows users to search the collection and download found files.
LibGen on IPFS
And although the social significance of LibGen is obvious, so too are the reasons that constantly put the library at risk of closure. This has led maintainers of the mirrors to seek new ways to ensure stability. One such way has been the publication of the collection on IPFS.
IPFS has been around for quite some time. When the technology emerged, great hopes were placed on it, not all of which have been fulfilled. Nevertheless, the development of the network continues, and the emergence of LibGen within it may boost new contributions and benefit the network itself.
Simply put, IPFS can be described as a file system stretched across an indefinite number of nodes in the network. Participants in the peer-to-peer network can cache files on their end and distribute them to others. Files are addressed not by paths, but by hashes of their content.
Some time ago, participants from LibGen announced IPFS hashes and began distributing files. This week, links to files on IPFS have started appearing in the search results of some LibGen mirrors. Moreover, thanks to the efforts of activists from the Internet Archive team and coverage of events on reddit, there is now an influx of additional seeders both on IPFS and for distributing the original torrents.
It is still unknown whether the IPFS hashes will appear in the dumps of the LibGen database, but it seems that this is to be expected. The ability to download the collection's metadata along with the IPFS hashes will lower the entry threshold for creating one's own mirror, increase the stability of the entire library, and bring the creators' dream of the library closer to realization.
P.S. A resource has been created for those wishing to help the project , which provides instructions on how to set up IPFS.
Source: habr.com
