5 ms·
Wc in D: 712 Characters Without a Single Branch
- brian_herman__ 7y agoI find D easier to grok coming from a Java/Python background.
- jordigh 7y agoI feel the same way too. It's fairly easy to translate Python into D, and you get all of that compile-time goodness to go with it (static type checking and compile-time function execution). And it's pretty fast! I bet we could optimise this D wc to match GNU's wc without too many crazy tricks.
- vips7L 7y agoMy only issue is that it's far more complicated than Java and for a language that fills that same niche it's hard for me to justify putting my resources into learning it even though I do like a lot of the features.
- bachmeier 7y agoIt's certainly possible to write more complicated code in a language like D than in Java. I personally find the verbosity of Java combined with jamming everything into a single approach makes Java hard to read.
- forgotpwd16 7y agoIn what way is more complicated than Java?
- zojirushibottle 7y agowell, java isn't really complicated. it's just verbose. but my understanding is that d has many of the features of c and modern cpp. that alone makes it more complicated than java...
- gallier2 7y agoYou can also write verbose code in D. The language doesn't prohibit it.
- Scarbutt 7y agoJava is actually a simple language.
- WalterBright 7y agoJava is indeed a simpler language, much simpler. Unfortunately, my experience with Java is it is too simple - I had to write way too much boilerplate over and over.
- crazypython 7y agoD is designed to be able to do 100% of everything C++ can do. "Alias this" is intended to replace C++'s implicit conversion through multiple inheritance pattern and its "function that returns a value with implicit conversion" pattern. "Mixin templates" are a way to do CRTP.
- catacombs 7y agoD is an amazing language. But what eventually put me off is the lack of documentation for installing and linking libraries in the compilers.
- WalterBright 7y agoThis is a bit perplexing. Libraries don't need to be installed, you simply put them in a directory, add a path to it in the command to the compiler, and list the library on the command to the compiler. Just as you would for C and C++. Other languages may need libraries to be installed, but not D.
- sorokod 7y agoJust like in Java
- AsusFan 7y agoI've only toyed with D a bit. IMHO, if you come from a typical OO programming language background (which to me includes C++, Java, Python...), the majority of D will immediately feel familiar, and to a great extent, obvious. The syntax is familiar and the ideas are familiar. You don't have to learn anything new (not immediately anyway) like you do in Rust (where you basically need to learn EVERYTHING new). Where this idyllic scenario starts falling apart with when you start actually using it for anything half-serious. Some of the bits feel extremely unintuitive and the documentation is difficult to navigate. There are few examples and the tutorial is a bit spartan. For example, I needed a deque-like container (double-ended queue), but it took me ages to figure out that a) the language actually has one and b) how to use the bloody thing. There is also a bit of schizophrenia going on, with the "new" ideas and the "old" ideas clashing in some places. For example, they claim that you can run D without a GC (the new), but apparently a good chunk of the stdlib requires the GC (the old), so you're stuck. I find this all to be unfortunate because D, to me, feels like it could be a better, saner C++.
- earenndil 7y ago> There is also a bit of schizophrenia going on, with the "new" ideas and the "old" ideas clashing in some places. For example, they claim that you can run D without a GC (the new), but apparently a good chunk of the stdlib requires the GC (the old), so you're stuck. AFAIK this is somewhat intentional; they don't want to make any hard compatibility breaks, so there's a long deprecation period for any 'old' idea. There's also a lack of manpower to renovate libraries; e.g. there's no good xml library. Regarding GC, it's IMO not a huge problem. The GC is really not a problem for most applications, and for those where it is, you can simply avoid GC allocations in inner loops (GC only runs when you allocate from it).
- aldacron 7y ago> they claim that you can run D without a GC (the new), but apparently a good chunk of the stdlib requires the GC (the old), so you're stuck. The intent isn't to turn off the GC completely (though GC-averse folks assume that it is). The `@nogc` function attribute is intended to be applied where you need it. Then you can guarantee that in that function's call stack, no language features that require the GC will be used. The standard library has been retrofitted to eliminate use of the GC where it isn't needed and provide alternatives where possible (such as a function that takes a buffer as an argument alongside one that allocates). There may still be places where it can be trimmed down even more, but it will never be fully `@nogc` compatible. D is meant to be used with the GC, but provides the means to avoid allocations, turn collections on/off (`GC.disable/enable`) and command line options for profiling GC usage and affecting its behavior. Anyone who wants to turn off the GC completely is going beyond the primary intended use case and is of course going to run into bumps with the standard library. Much of it is still usable, though. See https://dlang.org/blog/the-gc-series/ https://dlang.org/blog/the-gc-series/
- bestouff 7y agoThere's also a wc in rust (of course) with more code (120 lines) but quite more efficient: https://medium.com/@martinmroz/beating-c-with-120-lines-of-rust-wc-a0db679fe920 https://medium.com/@martinmroz/beating-c-with-120-lines-of-r...
- ses1984 7y agoIn the blog post you linked, a library for parallelism is used.
- kbenson 7y agoIt's an additional two lines of code when they add it (and probably one more to pull it in above), and only happens at the end of the actual work, after they've matched C's performance and beat C's memory footprint. Using the parallelism library at the very end hardly invalidates the rest of the exercise.
- andrepd 7y ago"without a single branch" I thought it meant actually a branchless version of wc. Turns out it's just no explicit if statements.
- jnordwick 7y agoIf you were careful, it seems pretty plausible to be able to use some indexing tricks and CMOV to write a jump/branchless version of wc. You are basically counting newlines and runs of whitespace.
- zerr 7y agoCMOV aside, I remember it was proved MOV itself is turing-complete.
- orangse 7y agohttps://github.com/xoreaxeaxeax/movfuscator https://github.com/xoreaxeaxeax/movfuscator posting for anyone wondering
- agumonkey 7y agoslightly related, does embedding arithmetic in mov instructions go faster than explicit ALU operations ?
- Filligree 7y agoDepending on the arithmetic, it seems that yes it can! I've noticed gcc using LEA instructions for arithmetic of the form (x * a + b), where 'a' and 'b' fit with the instruction.
- jnordwick 7y agoUsing LEA (load effective address) for calculation seems to be pretty common in most programs for both gcc and llvm. You basically get smaller code, and it used to schedule them better across more execution ports. Not sure if the CPU can fuse the ADD+MUL now.
- pixelbeat__ 7y agoA couple of points with comparing coreutils, * recompiling with -march=native can give significant wins over more generic binaries provided by linux distros. * parallel processing helps with bigger files, and it's easy enough to leverage the existing wc binary to process in parallel. Both points are discussed at: https://www.pixelbeat.org/docs/unix-parallel-tools.html https://www.pixelbeat.org/docs/unix-parallel-tools.html
- forgotpwd16 7y agoI'm wavering between D and Rust for which one could be used as a better alternative to C++. (Though I don't see C++ getting replaced anytime soon.) Even if Rust is getting most of the attention, D also seems to be a strong candidate.
- jordigh 7y agoD is fun! It's so easy to get started. Rust has an initial learning curve that just doesn't doesn't seem to fun to me, but I guess I'll have to actually scale it some day.
- Cogitri 7y agoI've been using Rust for a year or so and while it's very nice once you get used to it I recently started working in D since I feel _so_ much more productive in it thanks to the GC while it still feels powerful and is fast where it matters (you can avoid the GC at performance critical places). The lack of libraries is a little annoying at times but thanks to dstep it's somewhat easy to use C stuff.
- WalterBright 7y agoD is also getting an Ownership/Borrowing system: https://github.com/dlang/dmd/pull/10747 https://github.com/dlang/dmd/pull/10747
- tastyminerals 7y agoRust is definitely more difficult but it has it benefits, I guess? For me as a Python guy D was just easier both concept wise and syntax wise. I could write a relatively complex algorithm after 2 weeks of reading a D programming book with just standard ops. And it was definitely faster. Maybe not as fast as C but I felt efficient. Personally, I liked that D does pray FP like Scala while also being a multiparadigm language. Aaand it is definitely more readable.
- tastyminerals 7y ago*does not pray FP
- greggyb 7y agoPerhaps I'm just being pedantic or maybe I am misunderstanding. The author claims to be IO bound toward the end. But they are comparing to two versions that are faster. It is my understanding that IO-bound means that the IO subsystem is the thing which limits run time of the program. But the author clearly demonstrates that the IO subsystem of their machine is capable of supporting faster wc binaries. So what am I missing here?
- cormacrelf 7y agoYou are pretty much right. It isn't IO bound. If your definition said IO must be the only thing limiting the time, then few programs would be IO bound, except trivial ones like "count arriving packets". In a packet counter, your "implementation" would not affect wall clock time at all until packets could arrive faster than 3GHz, or if you figured out a way to make `count++` run slower than packets could arrive. Usually IO bound means 'kinda like that packet counter'. There are problems with being exactly like that packet counter (e.g. are you using the IO subsystem inefficiently, like reading one char at a time?), but it has the property that speeding up IO speeds up the program, and speeding up your code doesn't speed up the program (much, or at all). You're right that it isn't a useful comparison between existing programs. It is useful to compare your program's CPU performance to theoretical limits on wall time. When your `wc` implementation approaches the speed of reading a file and doing nothing with it, then you can say it's IO bound. For this reason, there are very few single-file-reading programs that could be described as IO bound. It's common in networking where networks go much slower, and in filesystem traversal (e.g. ripgrep) but not for plain file reading.
- greggyb 7y agoThanks for the thorough response. This aligns with what I had in my head, but it's nice to see a clearer explication and also confirmation that I'm not way out in left field with how I understand things.
- Negitivefrags 7y agoPeople incorrectly use the term “IO bound” or “bottlenecked by IO” without thinking about it all the time. Like they will talk about how their web app is IO bound because the DB query takes 1 second while their slow ruby code only takes 300ms after it gets the result from the DB back. Well guess what, making the web app twice as fast still cuts 150ms off the response time, and it still means you can do twice as many requests on the same server. In order to be able to say that something is “bound” by something else, you have to have some kind of concurrency going on. One task has to be doing all it’s work in the time that it’s waiting for more work to arrive from another.
- ape4 7y agoFairly elegant code. Using a library that splits the line into words makes it pretty simple.
- thom 7y agoGave myself a quick D lesson just to understand the approach to flags here (Yes.keepTerminator rather than just a meaningless bool). Turns out D lets you define a template with any args you like, in this case taking a string for a name of a Flag type, which in turn contains an enum with 'yes' and 'no' boolean values. This means that you can only use the right type of Flag, with a matching name, and its yes/no value. But the syntax is a bit icky (Flag!"keepTerminator) and so _another_ nice feature of D appears to be that you can intercept field dispatch in a struct. And so the 'Yes' struct does this, captures the 'keepTerminator' as a string, and creates the correct type of flag. For whatever reason I found all this rather cute (and apologies to any actual D programmers if I've misread this whole situation).
- acehreli 7y agoSpot on! :) With the help of opDispatch (the catch-all member function temlate), it's possible to drop the string from the use site. (I don't know a way of dropping it from the type name.) import std.stdio; import std.typecons; import std.string; // This type's opDispatch removes the need for string in the flag name. struct FlagFromBool { auto opDispatch(string flagName)(bool value) { mixin (format!q{ return value ? Yes.%s : No.%s; }(flagName, flagName)); } } // A convenience function to remove the need for empty struct construction parenthesis. auto flagFromBool() { return FlagFromBool(); } // Unfortunately, the type name still requires string flag names: void bar(Flag!"foo" flag) { writeln("called with ", flag); } void main() { // However, the expressions don't need a string: bar(flagFromBool.foo(false)); bar(flagFromBool.foo(true)); }
- biotronic 7y agoBoth your convenience function and the quotes in the type name can be removed via the use of static opDispatch: struct FlagImpl(string name) { bool value; alias value this; } struct Flag { alias opDispatch(string name) = FlagImpl!name; } struct Yes { static auto opDispatch(string name)() { return FlagImpl!name(true); } } struct No { static auto opDispatch(string name)() { return FlagImpl!name(false); } } void fun(Flag.foo a) {} // Look ma, no quotes! unittest { fun(Yes.foo); fun(Flag.foo(true)); }
- deleted 7y ago[deleted]
- bjarneh 7y agoEvery time I see a post with some D source, I think the language looks great; but for some reason I never try to learn it...
- WalterBright 7y agoThat's ok, one of these days you'll try it and then wonder what took you so long :-)
- bachmeier 7y agoStart playing with no setup: https://run.dlang.io/ https://run.dlang.io/
- dig1 7y agoI believe I'm missing something here or my day was too long, but in Clojure this can be squeezed in 13 lines and 435 characters keeping things fairly readable (for Clojure & Lisp developers ;)). (defn wc [^String file] (with-open [rdr (clojure.java.io/reader file)] (apply (partial printf "%d %d %d\n") (reduce (fn [[nl nw nb] ^String ln] (let [words (count (.split ln "[ ]+")) bytes (alength (.getBytes ln "UTF-8"))] [(inc nl) (+ nw words) (+ nb bytes)])) [0 0 0] (line-seq rdr))))) (defn -main [& args] (wc (first args))) I haven't tested how fast it is, but startup time can be optimized by compiling it with GraalVM.
- deleted 7y ago[deleted]
- tazjin 7y agoNeat! I wonder if the character decoding & regex usage has noticeable performance impact. My Common Lisp version was sped up somewhat by switching from a character stream to a byte stream: https://git.tazj.in/tree/fun/wcl/wc.lisp https://git.tazj.in/tree/fun/wcl/wc.lisp You can try this one via Nix with: nix-build -E '(import (builtins.fetchGit "https://git.tazj.in") {}).fun.wcl'
- patrec 7y agoIt's been a while since I last wrote CL but I think your program can produce any counts between zero and the correct one – you need to use eql instead of eq if you want this to work in standard common lisp.
- monadic2 7y agoThis is ridiculous: of course there are branches, but you don’t explicitly write them. This is purely aesthetic. Edit: i simply wish the author illustrated why this is good or desirable—conditionals are not difficult to read.
- coldtea 7y agoThe aesthetic aspect is insignificant, it's about the semantics -- and thus reasoning about the code and other such properties. So nothing ridiculous about it. It's like Haskell code "of course has" anything C has, as underneath the both run assembly instructions full of gotos and state manipulation, you just "don't explicitly write it".
- egdod 7y agoBranches aren’t hard to reason about though. The only* reason anyone cares about branches is that branch misprediction is expensive.
- BoiledCabbage 7y agoI think branches are actually harder to reason about, we're just used to doing it since all of us have done it for so long. However I do believe that branches are less straightforward than "linear" code and add to mental complexity on larger projects. Of course there is no data to back this up, but I think one of the next trends in programming beyond the adoption of functional styles, immutable data, "functional core / imperative shell" will be abstracting away from explicit conditional/branching logic in higher level code. Obviously people can come up with pedantic/extreme cases where the abstraction does nothing to hide the complexity, or even makes things more complex but im not taking about that. I mean more simple abstractions like what was used in OP, or a filter() abstracting over a while and if combo. I'm convinced based on personal experience that it makes for cleaner code and will become more widely adopted over the years as people explore it.
- monadic2 7y ago
- cestith 7y agoI'm no D expert but this seems to assume one file to count given as one argument on the command line. The wc in coreutils takes any number of arguments or will happily count STDIN.