Txog ib txoj kev coj txawv txawv rau txuag chaw hard disk

Lwm tus neeg siv xav sau ib daim ntawv tshiab rau lub hard drive, tab sis nws tsis muaj chaw txaus los ua qhov no. Kuv kuj tsis xav rho tawm dab tsi, vim "txhua yam tseem ceeb heev thiab tsim nyog." Thiab peb yuav ua li cas nrog nws?

Tsis muaj leej twg muaj teeb meem no. Muaj cov ntaub ntawv terabytes ntawm peb lub hard drive, thiab qhov nyiaj no tsis zoo li yuav txo qis. Tab sis nws txawv li cas? Thaum kawg, tag nrho cov ntaub ntawv tsuas yog cov khoom ntawm ib qho ntev thiab, feem ntau yuav, tus tshiab tsis txawv ntau ntawm qhov uas twb tau khaws cia lawm.

Nws yog qhov tseeb tias kev tshawb nrhiav cov ntaub ntawv uas twb tau khaws cia rau hauv lub hard drive yog, yog tias tsis ua haujlwm, ces tsawg kawg tsis yog ib txoj haujlwm zoo. Ntawm qhov tod tes, yog qhov txawv me me, ces koj tuaj yeem kho nws me ntsis ...

Txog ib txoj kev coj txawv txawv rau txuag chaw hard disk

TL; DR - qhov thib ob sim los tham txog txoj kev coj txawv txawv ntawm kev ua kom cov ntaub ntawv siv JPEG cov ntaub ntawv, tam sim no hauv daim ntawv nkag siab ntau dua.

Hais txog cov khoom thiab qhov sib txawv

Yog tias koj nqa ob daim ntawv tsis sib xws, ces qhov nruab nrab ntawm ib nrab ntawm cov khoom lawv muaj coincide. Tseeb tiag, ntawm cov layouts ua tau rau txhua khub ('00, 01, 10, 11β€²), raws nraim ib nrab muaj qhov tseem ceeb, txhua yam yog yooj yim ntawm no.

Tab sis tau kawg, yog tias peb tsuas yog nqa ob cov ntaub ntawv thiab haum ib qho mus rau qhov thib ob, ces peb yuav poob ib qho ntawm lawv. Yog tias peb khaws cov kev hloov pauv, peb tsuas yog rov ua dua tshiab delta encoding, uas tshwm sim zoo kawg nkaus tsis muaj peb, txawm hais tias nws tsis yog feem ntau siv rau tib lub hom phiaj. Peb tuaj yeem sim embed ib ntu me me rau hauv ib qho loj dua, tab sis txawm li ntawd los peb pheej pheej hmoo poob qhov tseem ceeb ntawm cov ntaub ntawv yog tias peb siv nws nrog txhua yam.

Ntawm dab tsi thiab dab tsi ces qhov txawv yuav raug tshem tawm? Zoo, uas yog, cov ntaub ntawv tshiab sau los ntawm cov neeg siv tsuas yog ib ntu ntawm cov khoom, uas peb tsis tuaj yeem ua dab tsi los ntawm nws tus kheej. Tom qab ntawd koj tsuas yog yuav tsum nrhiav cov khoom zoo li no ntawm lub hard drive uas lawv tuaj yeem hloov pauv yam tsis tas yuav khaws qhov sib txawv, kom koj thiaj li muaj sia nyob lawv qhov poob yam tsis muaj qhov tshwm sim loj. Thiab nws ua rau kev txiav txim siab hloov tsis yog cov ntaub ntawv ntawm FS nws tus kheej, tab sis qee cov ntaub ntawv tsawg dua hauv nws. Tab sis qhov twg thiab yuav ua li cas?

Txoj kev haum

Lossy compressed ntaub ntawv tuaj cawm. Tag nrho cov jpegs, mp3s thiab lwm yam, txawm hais tias lossy compression, muaj ib pawg ntawm cov khoom uas tuaj yeem hloov pauv yam xyuam xim. Nws muaj peev xwm siv cov txuj ci siab heev uas imperceptibly hloov lawv cov khoom nyob rau ntau theem ntawm encoding. Tos. Cov tswv yim Advanced... imperceptible modification... ib me ntsis rau lwm... nws yuav luag zoo li steganography!

Tseeb, embed ib cov ntaub ntawv rau hauv lwm tus yog reminiscent ntawm nws txoj kev zoo li tsis muaj lwm yam. Kuv kuj zoo siab los ntawm qhov tsis pom kev ntawm kev hloov pauv rau tib neeg lub siab. Qhov twg txoj kev diverge yog nyob rau hauv secrecy: peb txoj hauj lwm los mus rau cov neeg siv nkag mus rau cov ntaub ntawv ntxiv rau nws hard drive; nws tsuas yog ua phem rau nws. Nws yuav tsis nco qab lawm.

Yog li ntawd, txawm tias peb tuaj yeem siv lawv, peb yuav tsum tau ua qee qhov kev hloov kho. Thiab ces kuv yuav qhia thiab qhia lawv siv cov piv txwv ntawm ib tug ntawm cov uas twb muaj lawm txoj kev thiab ib tug ntau yam ntaub ntawv hom.

Hais txog jackals

Yog tias koj nyem nws tiag tiag, nws yog qhov compressible tshaj plaws hauv ntiaj teb. Peb, ntawm chav kawm, tham txog JPEG cov ntaub ntawv. Tsis tsuas yog muaj tons ntawm cov cuab yeej thiab cov txheej txheem uas twb muaj lawm rau embedding cov ntaub ntawv rau hauv nws, tab sis nws yog hom duab nrov tshaj plaws hauv ntiaj teb no.

Txog ib txoj kev coj txawv txawv rau txuag chaw hard disk

Txawm li cas los xij, txhawm rau kom tsis txhob koom nrog kev yug menyuam dev, koj yuav tsum txwv koj qhov kev ua si hauv cov ntaub ntawv ntawm hom ntawv no. Tsis muaj leej twg nyiam monochrome squares uas tshwm sim vim yog compression ntau dhau, yog li koj yuav tsum txwv koj tus kheej ua haujlwm nrog cov ntaub ntawv compressed, tsis txhob recoding. Tshwj xeeb tshaj yog, nrog cov lej sib npaug, uas nyob twj ywm tom qab kev khiav hauj lwm lub luag haujlwm rau cov ntaub ntawv poob - DCT thiab quantization, uas zoo kawg nkaus tso tawm hauv cov txheej txheem encoding (ua tsaug rau wiki ntawm Bauman National Library):
Txog ib txoj kev coj txawv txawv rau txuag chaw hard disk

Muaj ntau txoj hauv kev rau optimizing cov ntaub ntawv jpeg. Yog tsis muaj optimization (jpegtran), muaj optimization "tsis poob", uas ua tau pab lwm yam, tab sis peb tsis quav ntsej txog lawv. Tom qab tag nrho, yog hais tias tus neeg siv yog npaj txhij rau embed ib cov ntaub ntawv mus rau lwm yam thiaj li yuav ua rau kom dawb disk qhov chaw, ces nws yog optimized nws dluab ntev dhau los, los yog tsis xav ua qhov no tag nrho vim ntshai poob zoo.

F5

Tag nrho tsev neeg ntawm algorithms haum rau cov xwm txheej no, uas koj tuaj yeem paub koj tus kheej nrog hauv qhov kev nthuav qhia zoo no. Qhov siab tshaj plaws ntawm lawv yog algorithm F5 los ntawm Andreas Westfeld, ua hauj lwm nrog cov coefficients ntawm brightness tivthaiv, txij li thaum tib neeg lub qhov muag yog qhov tsawg rhiab heev rau nws cov kev hloov. Ntxiv mus, nws siv cov txheej txheem embedding raws li matrix encoding, uas ua rau nws muaj peev xwm hloov tau tsawg dua thaum embedding tib tus nqi ntawm cov ntaub ntawv, qhov loj ntawm lub thawv siv.

Cov kev hloov pauv lawv tus kheej npau taws kom txo qis tus nqi ntawm cov coefficients los ntawm ib qho hauv qee qhov xwm txheej (uas yog, tsis yog ib txwm), uas tso cai rau koj siv F5 txhawm rau txhim kho cov ntaub ntawv khaws cia ntawm koj lub hard drive. Lub ntsiab lus yog tias cov coefficient tom qab qhov kev hloov pauv no feem ntau yuav muaj tsawg dua cov khoom tom qab Huffman encoding vim qhov kev faib tawm ntawm cov txiaj ntsig hauv JPEG, thiab xoom tshiab yuav muab qhov nce thaum lawv siv RLE.

Cov kev hloov kho tsim nyog tau npau taws kom tshem tawm qhov kev lav phib xaub rau kev zais (password rearrangement), uas txuag cov peev txheej thiab ua tiav lub sijhawm, thiab ntxiv cov txheej txheem rau kev ua haujlwm nrog ntau cov ntaub ntawv es tsis txhob siv ib zaug. Tus nyeem ntawv tsis zoo li yuav txaus siab rau cov txheej txheem kev hloov pauv hauv kev nthuav dav ntxiv, yog li cia peb mus rau qhov kev piav qhia ntawm kev siv.

Tshiab thev naus laus zis

Txhawm rau ua kom pom tias txoj hauv kev no ua haujlwm li cas, kuv tau siv txoj hauv kev hauv C ntshiab thiab ua tiav ntau qhov kev ua kom zoo dua ob qho tib si ntawm kev ua kom nrawm thiab nco (koj tsis tuaj yeem xav txog tias cov duab no hnyav npaum li cas yam tsis muaj compression, txawm tias ua ntej DCT). Hla-platform tiav nrog kev sib xyaw ntawm cov tsev qiv ntawv libjpeg, pcre ua ΠΈ me me, uas peb ua tsaug rau lawv. Tag nrho cov no tau muab tso ua ke los ntawm 'ua', yog li cov neeg siv Windows xav nruab qee Cygwin rau lawv tus kheej rau kev ntsuam xyuas, lossis cuam tshuam nrog Visual Studio thiab cov tsev qiv ntawv ntawm lawv tus kheej.

Kev siv yog muaj nyob rau hauv daim ntawv ntawm console utility thiab tsev qiv ntawv. Cov neeg txaus siab tuaj yeem paub ntau ntxiv txog kev siv cov ntawv tom kawg hauv readme hauv qhov chaw cia khoom ntawm Github, qhov txuas uas kuv yuav xa mus rau qhov kawg ntawm cov ntawv tshaj tawm.

Yuav siv li cas?

Ua tib zoo. Cov duab siv rau ntim yog xaiv los ntawm kev tshawb nrhiav siv cov lus qhia tsis tu ncua hauv cov npe hauv paus. Thaum ua tiav, cov ntaub ntawv tuaj yeem txav tau, hloov npe thiab theej ntawm qhov yuav nyob hauv nws thaj tsam, hloov cov ntaub ntawv thiab kev khiav haujlwm, thiab lwm yam. Txawm li cas los xij, koj yuav tsum ceev faj heev thiab tsis txhob hloov cov ntsiab lus tam sim ntawd. Poob tus nqi ntawm ib qho me ntsis tuaj yeem ua rau nws tsis tuaj yeem rov qab tau cov ntaub ntawv.

Tom qab ua tiav, qhov hluav taws xob tso tawm cov ntaub ntawv tshwj xeeb uas muaj tag nrho cov ntaub ntawv tsim nyog rau kev tshem tawm, suav nrog cov ntaub ntawv hais txog cov duab siv. Los ntawm nws tus kheej, nws hnyav txog ob peb kilobytes thiab tsis muaj kev cuam tshuam loj rau qhov chaw nyob disk.

Koj tuaj yeem txheeb xyuas qhov muaj peev xwm siv tau tus chij '-a': './f5ar -a [search folder] [Perl-compatible li niaj zaus qhia]'. Kev ntim khoom yog ua tiav nrog cov lus txib './f5ar -p [search folder] [Perl-tshaj li niaj zaus qhia] [packed file] [archive name]', thiab unpacking nrog './f5ar -u [cov ntaub ntawv khaws tseg] [rov qab cov ntaub ntawv npe ]' ib.

Ua qauv qhia ua haujlwm

Txhawm rau ua kom pom qhov ua tau zoo ntawm txoj kev, kuv tau tso ib phau ntawm 225 cov duab dawb kiag li ntawm cov dev los ntawm kev pabcuam Unsplash thiab pom nyob rau hauv cov ntaub ntawv loj pdf ntawm 45 meters ntawm qhov thib ob ntim Art of Programming Knuta.

Cov txheej txheem yog yooj yim heev:

$ du -sh knuth.pdf dogs/
44M knuth.pdf
633M dogs/

$ ./f5ar -p dogs/ .*jpg knuth.pdf dogs.f5ar
Reading compressing file... ok
Initializing the archive... ok
Analysing library capacity... done in 17.0s
Detected somewhat guaranteed capacity of 48439359 bytes
Detected possible capacity of upto 102618787 bytes
Compressing... done in 39.4s
Saving the archive... ok

$ ./f5ar -u dogs/dogs.f5ar knuth_unpacked.pdf
Initializing the archive... ok
Reading the archive file... ok
Filling the archive with files... done in 1.4s
Decompressing... done in 21.0s
Writing extracted data... ok

$ sha1sum knuth.pdf knuth_unpacked.pdf
5bd1f496d2e45e382f33959eae5ab15da12cd666 knuth.pdf
5bd1f496d2e45e382f33959eae5ab15da12cd666 knuth_unpacked.pdf

$ du -sh dogs/
551M dogs/

Screenshots rau cov kiv cua

Txog ib txoj kev coj txawv txawv rau txuag chaw hard disk

Cov ntaub ntawv unpacked tuaj yeem thiab tseem yuav tsum tau nyeem:

Txog ib txoj kev coj txawv txawv rau txuag chaw hard disk

Raws li koj tuaj yeem pom, los ntawm tus thawj 633 + 36 == 669 megabytes ntawm cov ntaub ntawv ntawm lub hard drive, peb tuaj rau ntau qab ntxiag 551. Qhov sib txawv radical no tau piav qhia los ntawm qhov txo qis ntawm cov txiaj ntsig ntawm cov coefficients, uas cuam tshuam rau lawv. tom qab lossless compression: txo ib los ntawm ib qho yooj yim "txiav tawm ob peb bytes ntawm cov ntaub ntawv kawg. Txawm li cas los xij, qhov no tseem yog cov ntaub ntawv poob, txawm tias ib qho me me, uas koj yuav tau muab tso rau.

Hmoov zoo, lawv yog qhov tsis pom qhov muag kiag li. Nyob rau hauv lub spoiler (txij li thaum habrastorage tsis tuaj yeem tuav cov ntaub ntawv loj), tus nyeem ntawv tuaj yeem ntsuas qhov sib txawv ntawm qhov muag thiab lawv qhov hnyav, tau los ntawm kev rho tawm cov txiaj ntsig ntawm cov khoom hloov pauv los ntawm tus thawj: tus thawj, nrog cov ntaub ntawv hauv, qhov txawv (qhov duller xim, qhov sib txawv me me ntawm qhov thaiv).

Es tsis txhob ib tug xaus

Xav txog tag nrho cov teeb meem no, yuav lub hard drive lossis upload txhua yam rau hauv huab yuav zoo li kev daws teeb meem yooj yim dua rau qhov teeb meem. Tab sis txawm tias peb nyob hauv lub sijhawm zoo tam sim no, tsis muaj kev lees paub tias tag kis nws tseem tuaj yeem mus online thiab xa tag nrho koj cov ntaub ntawv ntxiv rau qhov chaw. Los yog mus rau tom khw thiab yuav koj tus kheej lwm txhiab terabyte hard drive. Tab sis koj tuaj yeem siv cov tsev uas twb muaj lawm.

-> GitHub

Tau qhov twg los: www.hab.com

Ntxiv ib saib