OttoCoddo

8 ms·

1.
by OttoCoddo 2y ago
With the right code, NTFS is not much slower than ext4, for example. Nearly 3% for tar.gz and more than 30% for a heavily multi-threaded use like Pack. https://forum.lazarus.freepascal.org/index.php/topic,66281.m...
2.
by OttoCoddo 2y ago
SQLite can be faster than FileSystem for small files. For big files, it can do more than 1 GB/s. On Pack [1], I benchmarked these speeds, and you can go very fast. It can be even 2X faster than tar [2]. In my opinion, SQLite can be fas
3.
by OttoCoddo 2y ago
Thank you for the notes. I am well aware of the levels and Pack uses custom configuration to match its inner design. Maybe more level come, or maybe not. But to be clear, Pack supports any valid Zstandard content, and this levels we are dis
4.
by OttoCoddo 2y ago
Then `--press=hard` would be the choice for you.
5.
by OttoCoddo 2y ago
Sane choice, albeit delayed, as Zstandard has been the leading standard algorithm in the field for quite some time. I tested most of them for developing Pack [1] and Zstandard looked like the best alternative to the good old DEFLATE. The di
6.
by OttoCoddo 2y ago
Readers may find Pack interesting: https://news.ycombinator.com/item?id=39793805
7.
by OttoCoddo 2y ago
Thank you for the new numbers. Sure, it can be different on different machines, especially full systems. For me on Linux and ext4, Pack finishes the Linux code base at just 0.96 s. Anyway, I do not expect an order of magnitude difference be
8.
by OttoCoddo 2y ago
Easy to build using this document: https://pack.ac/source Each binary has its own build script that you can use for yourself. Binaries are used for static builds and to ease future needs. https://github.com/
9.
by OttoCoddo 2y ago
Yes, I am. Pack can hold millions of files with no problem. One field in which it shines is that, aside from being fast at processing large amounts of data, it can process many small files much faster than similar tools or even many popular
10.
by OttoCoddo 2y ago
It makes me happy. It looks clean and pseudocode-like. It helps readers from around the world with different languages understand.
11.
by OttoCoddo 2y ago
You got a point. Although with that that option comes a great cost: We will lose portability, speed and even reliability. Portability: Receiver (or future you) needs to know what you used, and what version even. Speed: If you want to do the
12.
by OttoCoddo 2y ago
What are the parameters you gave to the CLI program? This issue seems interesting, as these files on Windows 11 were tested countless times. To be clear, you can run Pack as: `pack.exe ./linux-master/`
13.
by OttoCoddo 2y ago
You can do that with Pack: `pack -i ./test.pack --include=/a/file.txt` or a couple files and folders at once: `pack -i ./test.pack --include=/a/file.txt --include=/a/folder/` Use `--list` to get
14.
by OttoCoddo 2y ago
Pack binary? Can you tell what machine and what steps? Build.sh can be used for Windows too, using MSYS2 UCRT64.
15.
by OttoCoddo 2y ago
It is not disabled, as you think. It is secured by the internal code in Pack. Almost all parts of Pack are multithreaded.
16.
by OttoCoddo 2y ago
Thank you! Exposure comes from enthusiasts like you. I did not want to focus the point on speed, or say, "Look, others are bad". They are great; my point was, "Look what we can do if we update our design and code". Pack
17.
by OttoCoddo 2y ago
I am happy to hear that, and I really appreciate your interest. Did you compile it for yourself? Any problem or steps you used, I will be happy to hear, o at pack.ac or GitHub, as it is hard to follow the building here. As a reminder, Pack
18.
by OttoCoddo 2y ago
Did you compile it for yourself? Any problem or steps you used, I will be happy to hear, o at pack.ac or GitHub, as it is hard to follow the building here. I should prepare more documents on how to build it. I suspect that there is a proble
19.
by OttoCoddo 2y ago
You are one of the bravest. And you know that, using SQLite as the base storage, rules out many of the security problems we can face. Anyone needing to reimplement Pack, can do it, very easily, if not easier than implementing ZIP, IF they u
20.
by OttoCoddo 2y ago
I answered this question here: https://news.ycombinator.com/item?id=39801083 If that is not enough, let me know. 7-Zip with the ZSTD patch is good too, but Pack is much faster at handling many files. Testing packing the Lin
21.
by OttoCoddo 2y ago
Thank you very much! I guess you are friend with colors ;)
22.
by OttoCoddo 2y ago
Indeed. Benchmarking Pack as a file system was fun. It is near 10X faster to let you iterate all the files compared to what I get from NTFS (warm with all caching on for both solutions). Someday, it can be used as a virtual drive. I leave i
23.
by OttoCoddo 2y ago
I answered this question here: https://news.ycombinator.com/item?id=39801083 If that is not enough, let me know.
24.
by OttoCoddo 2y ago
You may like to read https://pack.ac/note/pack and test it for yourself.
25.
by OttoCoddo 2y ago
Hello, and thank you for the notes. Unfortunately, your points seem to be mostly wrong, so let me clarify them a little. Do not worry; many people misunderstand SQLite and its abilities. - Single implementation: Sure. Working with SQLite co
26.
by OttoCoddo 2y ago
As far as I know, PAQ strives to give the best compression for archival purposes. Pack, on the other hand, tries to give the best compression, while keeping speed as instantaneous as possible. You should try them for yourself.
27.
by OttoCoddo 2y ago
Hell yeah indeed! You can find me on the forum too if you liked to talk Pascal: https://forum.lazarus.freepascal.org/index.php/topic,66281.0...
28.
by OttoCoddo 2y ago
Sorry. Site is very new, and custom made, and needs to be worked on mobile.
29.
by OttoCoddo 2y ago
I just do not want to follow Wirth's law.
30.
by OttoCoddo 2y ago
It looks clean and pseudocode-like. It helps readers from around the world with different languages understand. FreePascal compiler is very good too.

More ›