Virginia Polytechnic Institute for discussion among Linux kernel developers of a set of patches implementing a distributed thread execution system (Distributed Thread Execution), which allows applications to run on multiple computers with distribution and transparent migration of threads between hosts. With Popcorn, applications can be launched on one host and then seamlessly moved to another host without stopping their execution. In multithreaded programs, individual threads are allowed to migrate to other hosts.
Unlike the project , which allows saving the state of a process and resuming execution on another system, Popcorn provides seamless and dynamic migration between hosts during application execution, requiring no action from the user and ensuring consistency of virtual memory on all hosts where threads are executed simultaneously.
The Popcorn software stack consists of the Linux kernel and with tests demonstrating how system calls in Popcorn can be used for migrating threads in distributed-executing applications. At the kernel level, extensions to the virtual memory subsystem are proposed, implementing distributed shared memory, allowing processes on different hosts to access a shared and consistent virtual address space. Consistency of virtual memory pages is ensured through a protocol that replicates memory pages on the host upon read access and invalidates memory pages upon write operations.
Interaction between hosts is facilitated by a message handler operating at the kernel level, transmitting messages over a TCP socket. It is noted that TCP/IP is used to simplify debugging and testing during development. Developers understand that from a security and performance standpoint, TCP/IP is not the best way to transfer the contents of kernel structures and memory pages between hosts. All hosts running distributed applications must have an identical level of trust. Once the core algorithms are stabilized, a more efficient transport method will be implemented.
Popcorn se razvija od 2014. godine kao istraživački projekt za proučavanje mogućnosti kreiranja distribuiranih aplikacija, čiji se tokovi mogu izvršavati na različitim čvorovima u računarskim sistemima, u kojima se mogu kombinovati jezgre zasnovane na različitim arhitekturama skupa instrukcija (Xeon/Xeon-Phi, ARM/x86, CPU/GPU/FPGA). Predloženi set zakrpa za programere jezgra Linux podržava samo izvršavanje na hostovima sa CPU x86, ali postoji i funkcionalnija verzija Popcorn Linux, koja omogućava izvršavanje aplikacija na hostovima sa različitim arhitekturama CPU (x86 i ARM). Za korišćenje Popcorn-a u heterogenim okruženjima potrebno je koristiti posebni na bazi LLVM. Pri distribuiranom izvršavanju na hostovima sa jednom arhitekturom, ponovna kompilacija posebnim kompajlerom nije potrebna.
Poleg tega je mogoče omeniti sličnog projekta sa implementacijom početnog prototipa API za pokretanje podprocesa na drugim računarima u klasteru (kao fork(), ali premešta razgranati proces na drugi računar).
Kod je napisan na jeziku Rust i za sada omogućava kloniranje samo najjednostavnijih procesa, u kojima se ne koriste sistemski resursi, kao što su datoteke. Pri pozivu telefork, memorija i povezane strukture procesa kloniraju se na drugi host, na kojem je pokrenut serverski obradnik (telepad). Pomoću ptrace, odraz memorije procesa se serijalizuje i zajedno sa stanjem procesa i registrima prenosi na drugi host. API takođe omogućava čuvanje stanja procesa u datoteku i obnavljanje preko nje.
Izvor: opennet.ru
