Tom qab peb thiab ib nrab xyoo ntawm txoj kev loj hlob, thawj qhov kev tso tawm ruaj khov ntawm GNU Wget2 qhov project tau nthuav tawm, tsim ib qho kev hloov kho tshiab ntawm qhov kev pab cuam rau automating recursive rub tawm ntawm GNU Wget cov ntsiab lus. GNU Wget2 tau tsim thiab sau dua tshiab los ntawm kos thiab yog qhov tseem ceeb rau kev txav cov haujlwm yooj yim ntawm tus neeg siv lub vev xaib mus rau hauv lub tsev qiv ntawv libwget, uas tuaj yeem siv sib cais hauv cov ntawv thov. Cov khoom siv hluav taws xob tau tso cai raws li GPLv3+, thiab lub tsev qiv ntawv tau tso cai raws li LGPLv3+.
Hloov chaw maj mam rov ua dua cov cai hauv paus uas twb muaj lawm, nws tau txiav txim siab rov ua txhua yam los ntawm kos thiab tsim kom muaj ib ceg Wget2 cais los siv cov tswv yim rau kev txhim kho, nce kev ua haujlwm thiab hloov pauv uas cuam tshuam kev sib raug zoo. Nrog rau kev zam ntawm kev tsis lees paub ntawm FTP raws tu qauv thiab WARC hom, wget2 tuaj yeem ua qhov hloov pauv pob tshab rau cov khoom siv wget classic hauv ntau lub sijhawm.
Uas tau hais tias, wget2 muaj qee cov ntaub ntawv sib txawv ntawm tus cwj pwm, muab txog 30 qhov kev xaiv ntxiv, thiab tsis txhawb nqa ntau lub kaum os. Nrog rau kev ua cov kev xaiv xws li "-nug-password", "-header", "-exclude-directories", "-ftp*", "-warc*", "-limit-rate", "-relative" tau ua. nres "thiab"--unlink".
Qhov tseem ceeb innovation muaj xws li:
- Txav functionality mus rau lub tsev qiv ntawv libwget.
- Hloov mus rau multi-threaded architecture.
- Muaj peev xwm tsim kom muaj ntau yam kev sib txuas nyob rau hauv parallel thiab rub mus rau ntau cov xov. Nws tseem tuaj yeem ua kom sib npaug ntawm qhov rub tawm ntawm ib cov ntaub ntawv muab faib ua cov blocks siv qhov "-chunk-size" kev xaiv.
- HTTP/2 raws tu qauv txhawb nqa.
- Siv If-Modified-Since HTTP header los rub tawm cov ntaub ntawv hloov kho xwb.
- Hloov mus rau kev siv sab nraud bandwidth limiters xws li trickle.
- Kev them nyiaj yug rau Txais-Encoding header, compressed cov ntaub ntawv hloov lwm lub tsev, thiab brotli, zstd, lzip, gzip, deflate, lzma, thiab bzip2 compression algorithms.
- Kev them nyiaj yug rau TLS 1.3, OCSP (Online Certificate Status Protocol) rau kev tshuaj xyuas cov ntawv pov thawj tshem tawm, HSTS (HTTP Kev Nyab Xeeb Kev Nyab Xeeb Kev Nyab Xeeb) rau kev yuam kom hloov mus rau HTTPS thiab HPKP (HTTP Public Key Pinning) rau daim ntawv pov thawj khi.
- Muaj peev xwm siv GnuTLS, WolfSSL thiab OpenSSL ua backends rau TLS.
- Txhawb kom qhib ceev ceev ntawm TCP kev sib txuas (TCP FastOpen).
- Ua-hauv Metalink hom kev txhawb nqa.
- Kev them nyiaj yug rau cov npe sau npe thoob ntiaj teb (IDNA2008).
- Lub peev xwm ua haujlwm ib txhij los ntawm ntau lub npe servers (ib qho kwj yuav raug thauj los ntawm ib lub npe, thiab qhov thib ob los ntawm lwm qhov).
- Built-in kev txhawb nqa rau cov xov xwm pub hauv Atom thiab RSS hom (piv txwv li, rau kev luam theej duab thiab rub tawm cov txuas). RSS/Atom cov ntaub ntawv tuaj yeem rub tawm los ntawm cov ntaub ntawv hauv zos lossis hauv lub network.
- Kev them nyiaj yug rau rho tawm URLs los ntawm Sitemap. Muaj cov parser rau rho tawm cov ntawv txuas los ntawm CSS thiab XML cov ntaub ntawv.
- Kev them nyiaj yug rau 'suav nrog' cov lus qhia hauv cov ntaub ntawv teeb tsa thiab faib cov chaw hla ntau cov ntaub ntawv (/etc/wget/conf.d/*.conf).
- Built-in DNS nug caching mechanism.
- Muaj peev xwm recoding cov ntsiab lus los ntawm kev hloov cov ntaub ntawv encoding.
- Kev suav nyiaj rau cov ntaub ntawv "robots.txt" thaum rov rub tawm.
- Txhim khu kev sau hom nrog fsync() hu tom qab txuag cov ntaub ntawv.
- Kev muaj peev xwm rov pib cuam tshuam TLS ntu, nrog rau cache thiab txuag TLS kev sib kho tsis ua haujlwm rau cov ntaub ntawv.
- "--input-file-" hom rau kev thauj khoom URLs los ntawm tus qauv nkag kwj.
- Kev tshuaj xyuas qhov Scope ntawm Cov Ncuav Qab Zib tiv thaiv cov npe ntawm cov npe ntawm pej xeem cov lus xaus (Public Suffix List) kom cais tawm ntawm lwm qhov chaw sib txawv tuav hauv tib qib thib ob (piv txwv li, "a.github.io" thiab "b.github. ib").
- Txhawb rub tawm ICEcast / SHOUTcast streaming.
Tau qhov twg los: opennet.ru