12 ms·

Impending kOS

• bshimmin 12y ago
  Here's the text editor they're talking about: http://www.kparc.com/edit.k http://www.kparc.com/edit.k The code is, well, not the easiest to understand.
  • outside1234 12y ago
    That's an understatement. If we are allowed code like this, I can write an editor in ONE line of JavaScript. :)
    • deleted 12y ago
      [deleted]
    • TheOsiris 12y ago
      I think he does a better job than uglifyjs can
    • beagle3 12y ago
      Note that his lines are < 60 chars. Can you write this editor in one line of less than 6000 chars of javascript (without a "textarea")?
      • calinet6 12y ago
        It does not matter. This is not a programming language; languages are designed for human comprehension. Human comprehension is an irreducible part of the whole paradigm of programming, being necessary to understand the nearly always complex logic required to describe anything of value. This is gibberish. It fails at the one task of a programming language: to translate something a human can understand into something a computer can understand, with as little confusion as possible.
        • dang 12y ago
          Any language you don't know is gibberish—hence the proverb, "it's all Greek to me." Please don't dismiss the unfamiliar just because it's strange.
        • beagle3 12y ago
          No, it is just not a language for the masses who have gotten used to all programming language syntax being minor variations on the Algol/Pascal/C theme. Much like math and physics, this is an extremely terse notation that is immensely useful to those who practice it, and looks like gibberish to those who have not attempted to study it. You could give the same comment about an article in any area of math whose notation you are unfamiliar with -- and you would be just as equally wrong.
        • deleted 12y ago
          [deleted]
          • serf 12y ago
            Well, he's right in a code-commenting way. If you have a sudoku solver that's entirely commented by the phrase 'sudoku', that's generally an indicator of code that is optimized for the computer rather than the reader--regardless of the language used.
        • avmich 12y ago
          You owe to yourself to read the Iverson's Turing lecture, "Notation as a tool of thought". You can find it. Apparently APL family can claim relationship to math notation - which is not a small claim. Lisp family does it too - in different way.
      • sxp 12y ago
        Someone skilled in writing compact JS probably can. People write amazing programs in 1k & 4k of JS: http://www.google.com/webhp?#q=javascript+4k http://www.google.com/webhp?#q=javascript+4k
        • beagle3 12y ago
          Yes. There is, however, a fundamental difference between codegolfed K and codegolf in essentially any other language: Codegolfed k uses 1-letter variable names, and drops white spaces (which, unfortunately, is kind of idiomatic). But everything else is idiomatic even for the non-golfers. Codegolfed JS or C much more often than not is non-idiomatic in order to save all that space. e.g., compare the K Sudoku solution from http://kparc.com/z/game.k http://kparc.com/z/game.k to any other language, and you'll see that the K code is almost a generic solver (the 3, 9 and 10 stand for box size, board size and number of options respectively while setting up the constraints for the solver), whereas every other short solution (that usually come about twice as long to 10 times as long) is extremely specific to the standard sudoku table. That's not random - idiomatic K solutions tend to be more generic than other languages. The syntax and semantics gently push you towards more general and more efficient solutions compared to most languages, in that the more general (and/or efficient) solution is usually shorter and simpler to code than a specific case. I know it sounds unbelievable. But you might believe it when you see example after example and realize that it's not just a "one trick pony". And still, most people surround it with a SEP field. I recommend spending the time needed to grok it instead. [Or APL or J; my personal preference is K, but to each his own]
      • sz4kerto 12y ago
        Why do you exclude textarea? It's a high-level functionality available in JS. That q code uses the high-level functionality of q. What's the difference?
        • beagle3 12y ago
          No, it's part of webkit/blink/gecko, it's not part of JS. But that's not why I exclude textarea: I exclude textarea for the same reason dictionaries define something without using the same word and base form. And the argument is generally meaningless: You may just as well call emacs and say you are using "the operating system's high level functionality".
      • kemayo 12y ago
        Does that k code implement cursor-movement / scrolling / etc itself, or is it relying on some sort of system functionality for it?
    • Jupe 12y ago
      Open a new tab and enter this (sans quotes) into the address bar: data:text/html, <html contenteditable> Done :)
  • zokier 12y ago
    Here is the four line version: http://www.kparc.com/$/edit.k http://www.kparc.com/$/edit.k (lines beginning with / are comments)
    • xdissent 12y ago
      I think the first line imports view.k: http://www.kparc.com/$/view.k http://www.kparc.com/$/view.k
    • kemayo 12y ago
      Since I don't know how to read K, and it's not really very friendly to amateurs who're just scanning a program without a reference open... is this equivalent to a bare C program implementing an editor with no libraries, or is it relying on stuff the system provides?
      • nitrogen 12y ago
        I see cz cx and cv in the four-line version, which seem like they are referring to Ctrl-Z, Ctrl-X, etc. If that is the case, then there is a lot of behind the scenes support stuff to get from those four lines of code to a window on the screen or a buffer in a terminal.
    • RickHull 12y ago
      Obviously bug free! Regarding intelligibility / comprehensibility: it's somewhere between brainfuck and Perl's worst examples of line noise
  • tlrobinson 12y ago
    Could someone who can parse (reverse engineer?) this write up an explanation? (if you exist)
    • geocar 12y ago
      I'll give this a shot. I'll try to explain what's in my mind as I read it as well. First, get out the reference manual: http://kparc.com/k.txt http://kparc.com/k.txt and we'll do the first couple lines. The sequence that goes f x applies x to f. this f is unary. The sequence that goes x f y applies x and y to f. this f is binary (and just labelled verb). Some things (adverbs) go f a x and apply f in some special way to x. Last hint: You read this code from left to right (like english). Do not scan it. Now let's dive in: c::a$"\n" This says: c is a view of where a is a newline. That is, if "a" contains a file, then c is the offsets of each newline. A view is a concept in k that is unusual in other languages: When "a" gets updated, then "c" will automatically contain the new values. This is similar to how a cell in Excel can refer to other cells and reflect whatever the value of those cells happens to be. b::0,1+c This says: b is a view of zero join one plus c. That is, where "c" contains the offsets of each newline, 1+c would contain the beginning of each new line. We joint zero to the beginning because of course, the beginning of the file is also the beginning of a line. d::(#c),|/-':b This says: d is the view of the (count of c) joined with the max reduce, of each pairs difference of b. That sounds like a lot, but "each pairs difference of b" (where b is the position of all the new lines) is also the length of each line, and the max reduce of that is the longest line. You might say that "d" is the dimensions of the file. i::x,j-b x:b'j This says: i is a view of x (?) joined with j (?) minus b (the offset of the beginning of each line) applied to each x which is defined as the bin of j in b. j hasn't been defined yet, but you can see that x is defined later in the definition, and used to the left. This is because K (like APL) executes from right to left just like other programming languages. you write a(b(c(d()))) in C and you're executing the rightmost code first (d) then applying that leftwards. We can do this with anything including definitions. The other string thing is that we know that b is the offset of the beginning of each line, and yet we're applying something to it. This is because k does not distinguish between function application and array indexing. Think of all the times you write in JavaScript x.map(function(a) {return a[whatever] }) when you'd really just like to write x[whatever] -- k let's you do this. It's a very powerful concept. On that subject of binning: b'j is going to find the index of the value of b that is smaller or equal to j. Since we remember that b is the offset of the beginning of each line, then if j is an offset in the file, then this will tell us which line it is on(!) But we don't understand what j is yet; it's the next definition: j::*|k This says: j is a view of the last (first reverse) of k. We don't know what k is yet. f:"" This says: f is defined as an empty string. g::a$f This says: g is a view of the offsets of f (an empty string) in a. Initially this will be null, but other code will assign to f later making g a value. Next line. s::i&s|i-w-2 This is very straightforward; & is and, and | is or. While excel doesn't let us use a cell in it's own definition, k does: It means the old value of s. So this is literally: the new view of s is i and the old view of s or i minus w (?) minus 2. We don't know what w is yet. S:{s::0|x&d-w} This is a function (lambda). x is the first argument. If we called S[5] it would be the same as setting s to zero or 5 and d (dimensions) minus w. Double-colon changes meaning here; it no longer means view, but set-global. px:{S s+w*x} This requires some knowledge of g/z: http://kparc.com/z.txt http://kparc.com/z.txt Note that px is defining a callback for when the pageup/pagedown routines are called. x will be 1 if we page down and -1 if we page up. It may now become clear that S is a setter of s that checks it somehow. When we understand what w is (later in the program) it will be absolutely clear, but pageup/pagedown are changing s by negative w when pageup, and positive w when pagedown. wx:{S s+4*x} Consulting the g/z documentation, we can see this has to do with wheels. Note we modify s again relative to 4 times x; x is -1 when wheelup and 1 when wheeldown. It becomes clear that s is modified by the pageup/pagedown and the mouse wheel. cc:{9'*k_a} Again: in the documentation, 9' stashes something in the clipboard. cc is a function that takes the first slice of offsets k (we still don't understand) in a (the file). The g/z documentation says that cc is the callback for control-C. This is expected as control-C traditionally copies things to the clipboard. Since the slice of offsets k in a are being saved in the clipboard, we may guess at this point that k will contain the current selection. This process is time consuming, but it is to be expected: Learning english took a while at first, and often required consulting various dictionaries. Eventually you got better at it and could read somewhat quickly. I don't know if you want to try the next few lines yourself to see if you can get a feel for it, or if you want to try one of the less dense examples: * http://www.kparc.com/$/view.k http://www.kparc.com/$/view.k * http://www.kparc.com/$/edit.k http://www.kparc.com/$/edit.k ... or if you want me to keep going like this, or if you want to ask a few questions. What are your thoughts?
• tmikaeld 12y ago
  Um, so what happened?
  • geocar 12y ago
    It's not done yet. This summer, Pierre and I got kOS to boot directly into g (the graphical interface; formally called z) with ISR, keymap, modesetting, basic filesystem, etc weighing in around 100 lines of C. That was pretty exciting. Could probably be done with less with some deeper changes to Arthur's code, but it's still very useful to run k under Linux. Oleg made a silly little game in kOS. Arthur and Oleg did some performance benchmarks staging k against q (current kdb+), Postgres, some "popular RDBMS" (that I can't name), and MongoDB. It was impressive that k is so much faster than q, but it also really underscores the cost of the wrong data structure (and how hard it is to get the right one with SQL or MongoDB).
    • keithpeter 12y ago
      Quote from OA "Whitney sent Oleg and Pierre some of the C code he was working on, and notes on a problem he didn’t know how to solve. They emailed back a solution, coded in his style." Did Pierre and Oleg think their solution out in standard code first and then make it Whitney-like, or did they find themselves thinking in Whitneyese straight away? I imagine their teacher may have noticed Whitney tendencies and that is what led to the original encouragement to make contact.
      • geocar 12y ago
        I do not think they code in what you call "standard code", but I don't know how "straight away" it was either. Writing in a dense fashion facilitates thinking about the solution.
        • keithpeter 12y ago
          "Writing in a dense fashion facilitates thinking about the solution." What I was thinking, thanks
    • 12y ago
• zokier 12y ago
  Googled around, Kuro5hin (that's a name I haven't seen for some time) has a tutorial for K from 2002: http://www.kuro5hin.org/story/2002/11/14/22741/791 http://www.kuro5hin.org/story/2002/11/14/22741/791 The download link at http://www.kparc.com/ http://www.kparc.com/ asks for password, so I'm not sure whats going on with that.
  • sohagan857 12y ago
    You have to have a personal invite to download the code atm. The whole article is telling you it is not available yet, but "if coming".
    • bulatb 12y ago
      Wow, the hipster language prize has been awarded. Other languages go home.
    • kencausey 12y ago
      I may be mistaken but I believe zokier is referring to a download link for k whereas you are referring to a download for kOS. No?
      • fa 12y ago
        Right. The part of the page that says "download (then $chmod +x k)" and linking to http://kparc.com/download/k http://kparc.com/download/k is password-protected. I just want to follow along some of this tutorial material. I'll try to do this with Kona.
• TheOsiris 12y ago
  > “It is a lot easier to find your errors in four lines of code than in four hundred.” Looking at his code on http://www.kparc.com/edit.k http://www.kparc.com/edit.k I'd like to disagree with that statement
  • wyc 12y ago
    If I posted four lines of Chinese or Sanskrit, it's likely that native English speakers would disagree that they had much meaning either. However, this doesn't mean that those lines are inherently devoid of meaning or difficult to parse.
    • uberalex 12y ago
      I think that the issue is that just measuring simplicity in terms of number of lines is a bad metric. You can have extremely complex expressions in a single statement that are at least as hard to read and debug as an equivalent, much longer piece of code that employs temporary variables and single-purpose statements.
      • geocar 12y ago
        I disagree fundamentally. I have noticed every page I scroll causes a comprehensive loss of around 90%, so in reading something that is 10 pagefuls long, I might only be able to produce a tiny part of the program. Your milage may vary. I find not scrolling, and just moving my eyes, I rapidly absorb the program, and I find most bugs just by reading the code. This practice is absolutely impossible for me if I have to scroll very far and made difficult by scrolling at all. It is for this reason that I find simply counting the actual words to be an excellent estimate of complexity. By the way: There are several temporary variables in that code; c:: creates a view called "c" which automatically updates whenever the dependent variables on the right side change.
        • dang 12y ago
          Yes, the research literature on software development has consistently found that code size is the best measurement of complexity and predictor of error rates. (Sorry I don't have citations handy but we've discussed this many times on HN, and there's a recent study in the book "Making Software" that adds to it.) What's interesting is how strongly this goes against what most people think they know about good programming and clear code.
• jeffreyrogers 12y ago
  > kOS is coming. Nothing will be the same afterwards. There seems to be this strange idea going around that if we just get the right tool, everything else is going to change forever. I see this a lot with people trying to create IDEs that let non-programmers create programs without really knowing how to code. But the thing is, most people just don't have anything worth coding. The problem isn't that the tools don't exist. They do, even if they aren't perfect. It's that making something that matters isn't an easy thing to do. And no tool can change that.
  • dkarapetyan 12y ago
    Not knowing how to code is not really an issue because we can and do teach people how to code the same way we teach people how to count and do basic algebra. What we can't do well is teach people how to solve novel problems that they haven't encountered before.
    • eggy 12y ago
      The reason k, q or other array languages like J (which I use) help, not teach, people to solve novel problems is that they give you abstractions to quickly play with in order to tackle novel problems. My take on this whole thing is that these array languages (APL, k, J, q) are a perfect fit for things that are essentially arrays - big data, images, etc... And to those who shy away from the seemingly 'noisy' syntax, I point them to mathematics. Until you understand the sigma symbols and others, it is noise too. Why are all the new languages getting hot on 'vectorization' - Julia and company - because GPUs and data are all array-based. To have the array as your lingua franca puts you in a good position for all sorts of attacks on modern computing needs. And for those who cannot live with the syntax, and say how can you create a dsl - you can. Here is a J example of computing averages: +/%# You can set this up as: avg=: +/%# or tally =: # divide =: % sumall =: +/ How's that for a dsl? Which allows this now: avg=: sumall divide tally avg 2 4 6 4 EDIT: I forgot to add that these operations, as others in J or APL, work on scalars, vectors and arrays without any special typing or handling. See this presentation for at least the first 5 minutes to see J in action with explanation: http://www.infoq.com/presentations/j-language http://www.infoq.com/presentations/j-language The conciseness allows you to start seeing the small patterns in the small expressions just like mathematics, hence bringing clarity and speed of abstraction to your concept manipulations.
• radicalbyte 12y ago
  Interesting article. Found this (http://queue.acm.org/detail.cfm?id=1531242 http://queue.acm.org/detail.cfm?id=1531242, submitted here https://news.ycombinator.com/item?id=8476120 https://news.ycombinator.com/item?id=8476120) interview with Arthur Whitney (from 2009) which is also really interesting.
• kiyoto 12y ago
  >Kdb+ has sharp elbows. No shit. I used to work as a quant, and while I was an okay quant and mediocre trader at best, I survived for three years in the industry because of my kdb+ proficiency: the firm I was at spent a couple of million dollars on kdb+ only to find out that most people could not wrap their heads around kdb+ let alone debug it effectively. My (former) colleagues were definitely smart people. In many ways, they were way smarter than myself. But I somehow could get a much better handle of kdb+'s idiosyncrasies, and my ability to stare at dense k/q code (usually no more than a dozen lines) and figure out what's wrong with it earned me the reputation as the "q guy" - and some level of job security. The firm eventually phased out kdb+ completely after my boss and I left (the two proponents of kdb+).
  • masklinn 12y ago
    Why were you a proponent of kdb+?
    • kiyoto 12y ago
      For several reasons, some more legitimate than others. 1. kdb+ was (and maybe is) a good solution to the problem that we had: doing complex data manipulation/simple statistical calculations against billions of rows of time series data. Hadoop is the term du jour for data processing, but truth of the matter is that finance doesn't have really huge data. At best, it's a couple of terabytes, and most of the time, you are working with a small subset of it. Running KDB+ on a beefy server or two would usually do the job (rather well). 2. Maybe because I studied math, but I find k/q's vectorial/functional sematics appealing. I think the syntax is horrible, but the semantics is very neat. 3. Finally, because it helped me keep my job. It was rather amazing to me that all these Ph.D. statisticians that I worked with couldn't bring themselves to learn kdb+ effectively. Apparently this stuff can be very hard for even the smartest people (or maybe they thought it was such a niche skill with a low ROI).
      • mahyarm 12y ago
        It almost sounds like kdb needs an alternative syntax that is more human readable.
• zokier 12y ago
  I wonder what K would look like with bit reader-friendly syntax. I tried running a program that supposedly "will take a K expression and produce its English translation" (http://kx.com/a/k/examples/read.k http://kx.com/a/k/examples/read.k), but either it doesn't work with kdb+/q or I can't figure out how to use it. Does anyone have some example output, or advise?
• viksit 12y ago
  The last line of TFA reads like the beginning of some sort of movie in the drama/thriller category. "kOS is coming. Nothing will be the same afterwards." That's what irritated me most. What I'd like to understand is - what led the author to this particular conclusion? Is it the fact that this language is super expressive and concise? Is it that it routinely [1] outperforms its C counterparts even if it ultimately translates to C? Is the Z graphical interface so superior that it'll blow the pants off Cocoa and Quartz and X.org or Wayland or what have you? Why would one rewrite emacs or vim on it? I don't want some basic 4 line text editor - I would like to be productive. Why would Mozilla spend energy porting firefox to it? Or Google, chrome? Or bash? Simply talking about the history of K/kdb+ and how brilliant its creator is simply doesn't help the reader understand why they should be excited about it. If that was the intention of this article, then the real points to make should've started after that line. That would've been much more interesting. [1] - No pun intended, of course
  • geocar 12y ago
    What you're actually seeing is testimony: people saying they are seeing something amazing, and they aren't very good at explaining what they saw. Btw: k doesn't translate to C. It's actually a quite simple interpreter. The fact that it outperforms other languages so easily should be saying more about those languages than it should be saying anything about k.
    • burntsushi 12y ago
      Well, does there exist a technical analysis written by someone who understands k that explains why it is so much faster than X languages?
      • geocar 12y ago
        As I said, I think this is the wrong question. The real question is why is X language so slow? This is not intended to be glib: but I do not think I can put it more simply than that. X language is slow because it uses lots of library code that it doesn't actually use (to get a friendly interface), it has a lot of redundancy (because of the wrong abstraction level), and because it wastes memory (in order to have an API that links well with others). Or it's slow because it thinks B-Trees are really cool. Or because it has the wrong intrinsics. I don't know. But I am convinced this is the discussion we need to be having.
• ah- 12y ago
  k/q really doesn't have to be this unreadable, that's just Arthurs style. Here's some code in C by him for comparison: http://kx.com/q/cs107/a.c http://kx.com/q/cs107/a.c
  • sohagan857 12y ago
    Q isn't unreadable. Its implemented in standard English words.http://code.kx.com/wiki/Reference http://code.kx.com/wiki/Reference K is considered unreadable by many.
    • ah- 12y ago
      I was more referring to the almost exclusive use of single letter variable names. If one would use at least short words to name things, the implementation would be a whole lot more understandable.
  • cheez 12y ago
    If that's true, why aren't there DSLs that compile to k?
  • ANTSANTS 12y ago
    Good lord, I need a drink now.
  • nemo1618 12y ago
    I tried cleaning it up a bit: https://gist.github.com/lukechampine/f54fce8fd756254cefb2 https://gist.github.com/lukechampine/f54fce8fd756254cefb2 But the actual meaning of the program is still lost on me. I can only guess it has something to do with parsing files (note the checks for curly braces). Feeding it its own source code produces some output, but I have no idea what it actually modified.
    • Torn 12y ago
      The original is 404ing, do you have a mirror?
• lmm 12y ago
  There's a contradiction I always see in these pieces: they talk a lot about the importance of using the right data structure. But these languages get their incredible conciseness by not giving you any choice about your data structures; their array type is hardcoded into the language, and if you want to use something else then your code balloons.
  • beagle3 12y ago
    K basically has 3 data shapes: atom (int, float, char, date, symbol, ...) list (one dimensional array of atoms, dicts, flips or lists) dict (a map from one list to another) There's also a flip, which exchanges the first two indexes applied to an item (so, e.g., it effectively transposes a list of lists) but it is just sugar (both syntactic and semantic). You can trust Whitney that all of these are properly implemented, including appends. It's not often that you actually need more. I've discovered this after using K for a while, and going back to python. Back in my pre-K (ha!) C++ and Python day, I had an awful lot of classes everywhere. After using K for a while, my Python and C both have much much fewer (structs in C more often than python, as C is missing python's dict). And the code has gotten much shorter and more efficient. Arguably, more readable as well. And I've essentially dropped C++ for C, because the extra complexity is just not worth it.
• manish_gill 12y ago
  The language looks fascinating. Check out Kona, an open source implementation: https://github.com/kevinlawler/kona https://github.com/kevinlawler/kona
• sz4kerto 12y ago
  The same story told again -- it's aim is to generate this magical atmosphere around a fast db engine and language that's deliberately obfuscated to make people who work in it feel smart, so they try to spread that it's the best. KDB/Q is a nice tool, not the holy grail how they put it. And it's not fast because they know something better - it's because it lacks almost any safety measure.
  • sohagan857 12y ago
    Ridiculous comment. Have you at least tried q? K? Kdb+?
    • avmich 12y ago
      Yes, interesting. APL was meant to be used for teaching programming - with it's exceptional clarity (sic). Java took the place... J engine is claimed to not have memory leaks - that's from very early versions and being written in C. It's true that it could be hard to think about the problems "vectorally", but still - I agree, the comment sounds ridiculous.
    • sz4kerto 12y ago
      I've worked with and later run the kdb/q stack for a couple of years at a financial services firm. I certainly enjoyed it but I've also seen the downsides of it, and according to my experience it does not worth the investment, and the biggest reason behind its adoption is A.W.'s relations with the financial industry. By the way, I really don't want to go into anything personal but your comments represent the attitude around kdb well.
      • qfan 12y ago
        It is a fun environment and the community is very friendly and smart too. Its widespread adoption in finance is because it is a proven tool for processing the huge datasets that other systems cannot. If you have a business case for that, clearly it can be a worthwhile investment. Other industries are beginning to see value in leveraging their data sets and that's probably why after being in the business for 20+ years kx continues to expand their customer base; note they now have pharma and energy customers. Yes, there are difficulties involved in managing huge data sets, but IMO no other tools does it better than kdb+.
• rsync 12y ago
  I'm interested in a kOS with a "god says ..." program built in ...
  • serf 12y ago
    get Terry Davis on the job. Is he opposed to porting his holy code to other esoteric platforms?
• rgbrgb 12y ago
  Kind of curious to play around with that text editor. Any chance of K running on OSX?
  • ah- 12y ago
    You can get kdb+ for free from here: http://kx.com/software-download.php http://kx.com/software-download.php However, they've been working on a new version of k that's not publicly available yet and I suppose the kparc stuff requires that.
• robfig 12y ago
  So K is a general purpose programming language? If the claims are true, why don't they submit some entries to the Computer Language Benchmarks Game?
  • tinco 12y ago
    It's proprietary.
    • xorcist 12y ago
      <sarcasm>Yeah, that's why.</sarcasm>
      • igouy 12y ago
        Perhaps they asked if their K programs were wanted: http://benchmarksgame.alioth.debian.org/play.html#languagex http://benchmarksgame.alioth.debian.org/play.html#languagex
  • igouy 12y ago
    Perhaps they asked if their K programs were wanted: http://benchmarksgame.alioth.debian.org/play.html#languagex http://benchmarksgame.alioth.debian.org/play.html#languagex
    • icsa 12y ago
      See https://news.ycombinator.com/item?id=8477308 https://news.ycombinator.com/item?id=8477308
      • igouy 12y ago
        And…? I seem to remember being asked about one of those array languages and saying no thanks.
• anonbanker 12y ago
  A language/kernel/db/ui to watch. Reading the comments here, it seems the language separates the wheat from the chaff; the average brogrammer won't he able to handle this, but many of us are very interested in exploring. An OS this small is incredibly exciting to me.
• jwatte 12y ago
  Every ten years, some company comes around and claims that their functional/data flow/columnar/meta shortish is orders of magnitude better. Usually, the one thing they have going for them is that they focus on only a small subproblem. That lets them be small. Every demo is one of no edge cases, no exceptions, and no I/O errors. (Or they cram all that into some "standard" library.) The real challenge is that, 99% of the time, requirements and integration is what kills you, not raw performance. For the cases where performance (or formal correctness, or whatever) matters, the main challenge is usually to convince the market that it's worth paying for, and then finding the right developer project match.
  • avmich 12y ago
    This is all false for APL. First, APL is right there in the history with Fortran and Lisp - and its "every ten years" became irrelevant already when it was used to analyze IBM 360 hardware and found some bugs which were fixed in time for shipping. Second, these languages are truly for very wide ranges of problems - it's the industry curse that APL isn't used more widely; I guess the reason is it's harder to learn. But you can do can use it for really everything. Who'd thing JavaScript would be the language to write Linux emulator? So it's less wonder that k is used for OS writing.
• mamcx 12y ago
  So, I read about the fast interpreter and small language. How do something like this? "Whitney’s strategy was to implement a core of the language – including the bits everyone thought most difficult, the operators and nested arrays – and use that to implement the rest of the language. The core was to be written in self-expanding C. As far as I know, the kdb+ interpreter is built the same way. Unlike the tall skinny C programs in the textbooks, the code for this interpreter spills sideways across the page. It certainly doesn’t look like C." This mean that the code is very unreadable C? Like a kind of code-golf? How replicate it for built a speedy interpreter? And what if I use lua or python instead?
  • JulianMorrison 12y ago
    There are two opposite schools of readable C. The mainstream says: readable C has function and variable and type names that express meaning, so a function is read like a narrative with verbs, adjectives and nouns. The fact that this narrative scrolls over pages, is unimportant. The APL/K/J school says: readable C has functions, variables, and types named with single letters, so that the totality of a function is short enough to fit in one glance - preferably, on one line that does not need a scrollbar. The function does exactly what it says, no more and no less; its intent is thus completely clear. To name it descriptively would ruin the ability to grasp the whole thing as a gestalt.
    • mamcx 12y ago
      Ok, so APL/K/J is for people that read compressed code.. But that how relate to how build a fast interpreter? Faster than C?
• duckingtest 12y ago
  This seems to be a very fun language to write in, in the same way optimizing assembly code in the ancient times was fun; the age of skilled artisans. Unfortunately (for programmers - great for the rest) now is the time of a factory worker. Due to that, I don't think this language and its platform has a long future. Its continued survival is rather a testament to excellency of marketing (as evidenced by this article!) rather than actual merits.
  • avmich 12y ago
    I think the opposite :) - marketing of R is light years ahead. Language is very obscure, but so good that from time to time you're going to hear wonderful stories from new enlightened. And with the rise of parallel programming we're going to actually switch more to more vector way of describing and solving problems. Just like Lisp ideas are spreading everywhere in modern languages, APL ideas are also fruitful.
• ndesaulniers 12y ago
  I seriously thought that the article, and some of the code examples people are posting [0] was a joke! I need to go rethink my career. [0] http://www.kparc.com/$/edit.k http://www.kparc.com/$/edit.k
  • avmich 12y ago
    That's certainly good - we should learn languages which are different enough to justify learning. Your code in whatever language you work will benefit from that rethinking.
• sumanthvepa 12y ago
  Worked at Morgan Stanley's fixed income desk many years ago as my first job after grad school, programming A+. It was a decidedly acquired taste. The language and style were utterly alien to anyone coming from a background in C (and at that time an early version of Java.) While A+ was fast for manipulating arrays (much of what the trading floor needed), it seemed that doing GUI development with A+ was really pointless. The speed of the language didn't matter in GUI interactions, and it was very hard do understand other people's code.
• lafar6502 12y ago
  I don't buy this story. Maybe kdb is fast and great, but the article attempts to describe it as a work of a genius, better than anything else because it's 100 lines of code, doing its own memory management and running on bare metal. But in fact many other programs do their own memory management and can run on bare metal. JVM or .Net do their own mem management, all database servers too, and many, many others. So what's left on the table? 500 lines of C code that's meant to change the world? I really doubt it (or, these guys arent using line breaks). We've read so many similar stories about Lisp and how it's one language that can do everything in 5 lines of code or how you can build an entire OS and all applications in Lisp, but where's that OS now?
  • icsa 12y ago
    What's left on the table is the efficient use of resources. When I mentioned to a friend that the current version of K5 was a binary < 100KB, his response was "I don't believe it. I can't write HelloWorld in less than 100 KB!". Access to more resources does not mean that one should be wasteful. K benefits from a dedication to avoiding waste and duplication and efficient use of mathematical concepts. The Fundamental New Computing Technologies at VPRI has similar goals (an entire end-user system including "Office" apps in 40 KLOC or less). Being able to prototype a multi-proc map/reduce algorithm in k with 1000 procs on a laptop with 8 GB RAM is quite nice. > 500 lines of C code that's meant to change the world? I really doubt it (or, these guys arent using line breaks). There are line breaks, undoubtedly. That said, I'm sure the code is concise - much like k code. K3 was 1200 lines of code and included the language, windows(GUI), database, IPC, REPL (w/ simple debugger), FFI and OS interaction. The Windows executable was 320 KB.
• jamesfisher 12y ago
  Does a good formal introduction exist for K, or Q, or APL, or J, or any other languages in this family? Something with, you know, a syntax definition at least, and any kind of formal definition of the semantics. The closest I could find is this [1] but "The model is expressed in SHARP APL", so from the start, it's circular. [1] http://www.jsoftware.com/papers/APLSyntaxSemantics.htm http://www.jsoftware.com/papers/APLSyntaxSemantics.htm
  • avmich 12y ago
    For J, IMO, jsoftware.com has good resources. That includes vocabulary (the way to specify the language), a few textbooks (JforC, J Primer), essays, examples of short code... And J forums are pretty helpful. Coming back to the question, for J its vocabulary on jsoftware.com is a good resource.
  • eggy 12y ago
    It's not formal, but it is helpful to watch these series of videos by Martin Saurer on J: https://www.youtube.com/watch?v=VSJpJt3c11c https://www.youtube.com/watch?v=VSJpJt3c11c It goes form solving some Euler problems to a full-blown web app in J.
• esya 12y ago
  Hey Geo, if you read this, congrats :) Tristan
• svan99 12y ago
  I love K/Q and is using it in my startup. Thanks a lot for Kx's recent freeing up the 32bit version. To use APL-like languages I have to really shift the way of designing/modeling things. Most importantly K may not be best for lots of developers working on the same thing. Object oriented languages will fit better in that case. K projects often only involve one or two developers who model things in vector thinking (column based thinking in data domain), know exactly what to do and how to do it. Vector thinking is not suitable for all problems, but works really well if it does. Btw, new 3.2 version of kdb appears to be even faster than before. It also improves websocket integration and JSON data conversion. Very nice to integrate with nodejs/Qt. We also use Forth which I think is a really another way of shifting mind.
  • icsa 12y ago
    What kind of computation is done at your startup? I've used k/q for trading, graph analytics and computer vision (scene reconstruction).
• scottlocklin 12y ago
  The APL family was developed to think about math and linear algebra in particular. Iverson's "Notation as a tool of thought" f'rinstance: http://www.jsoftware.com/papers/tot.htm http://www.jsoftware.com/papers/tot.htm If you've worked with such things for your day job, exposure to an APL language is mind blowing in the same way as exposure to Lisp is. You'll rapidly find out that an awful lot of the numerics world is an ad-hoc reinvention of an APL language. Leading thinkers in the numerics world have noticed. Have a look at the Tensor type inside Torch7, or -idx- class in Lush (the same thing): they are, in fact, a sort of APL with a more conventional, aka painfully wordy, notation. Writing an editor in an array language seems crazy, but then, writing a parallel processing system in a language that was designed to run applications in your web browser also seems crazy. If people had stuck with APL style languages, well, databases, particularly distributed databases (Kx and 1010, both K based systems, scale to Pentascale, and have for a long time), would suck less, as would CUDA programming. Their revival could make life easier in these problem domains.
• tsmith 12y ago
  Sounds great! Where's the documentation on IPC / mutexes / threading? > todo > files, procs, tcp/ip, usb, .. Oh.
  • icsa 12y ago
    E.g. - (k4/q) http://code.kx.com/wiki/Reference/hopen http://code.kx.com/wiki/Reference/hopen shows how to open a file/proc. The rest of the wiki is quite useful including a references and tutorials for q. k5 uses operators instead of words like q. Btw, being able map/reduce w/ 1000 procs on a 8GB Linux vm (using k5) is both useful and fun.
• anonu 12y ago
  I hate to love KDB because its a very expensive closed platform. But if you understand some of the concepts and how easy it is to achieve those concepts with a few lines of q code you can do some brilliant things. Yes, KDB is a great data store and provides very quick methods for crunching that data with its vector-based approach. However, whats really impressed me with KDB is that you can do so much more with it. In some banks it has effectively become the messaging middleware for connecting hundreds of disparate data sources. In addition to passing messages you get the data storage and analytics tools for free...
  • sohagan857 12y ago
    Very true. But remember recently they released a completely free version of it. Downloads can be obtained from kx.com.
• agarttha 12y ago
  joke: search jqk on google images