8 ms·

The Array Cast – A podcast about the array programming languages

162 points by srpeck 5y ago

not_knuth 5y ago
This looks interesting. Can't wait to give it a listen! Has anyone already done so? Reviews?
- rscho 5y ago
  It's good. They've got representatives of all the major array language communities among the hosts (APL, J, K/Q) plus a C++ guy.
  - __vim__ 5y ago
    No fortran guy?
    - rscho 5y ago
      I think the goal in bringing in a C++ programmer was to provide an outsider view, but no, no Fortran guy. I don't think Fortran is an array language in that arrays are not the only datastructures in Fortran, though?
      - slver 5y ago
        Is an array language a language where you only have arrays? That honestly sounds a bit odd.
        mjaniczek 5y ago
        You could consider Pandas / numpy an array language, even though it's really a library for Python. The exposed API is IMHO what's important.
        Wezl 5y ago
        I agree, but you have to take extensibility into account. You either have an extensible language and extend it[0], you make it an array language to the core, or you just have a few slightly convenient functions (not a language). (I don't have enough experience to judge which category Pandas / numpy are in) [0]: https://github.com/phantomics/april https://github.com/phantomics/april
        klibertp 5y ago
        No, an array language is one in which all the built-in operations are applicable to arrays. Take addition as an example: 4 + 4 8 In array languages the same operator can be used for arrays; or equivalently you can say that the example above sums two arrays of length 1. In J, you can do this and expect it to work: 4 4 4 + 2 2 2 6 6 6 This is true for all the built-ins, and many user defined operations (as long as you don't fiddle with so called rank of the verb you're defining). Numpy is close to that, but there's still a distinction between arrays and scalars, while in array langauges that distinction is often blurred: 4 + 1 2 3 5 6 7 Edit: in J, you can have atoms, or scalars, but you need to box them: (3;4;5) ┌─┬─┬─┐ │3│4│5│ └─┴─┴─┘ but then you can't do anything with them until you unbox them again: (3;4;5) + 3 |domain error | (3;4;5) +3 (+&4) each (3;4;5) ┌─┬─┬─┐ │7│8│9│ └─┴─┴─┘ (Examples straight from J REPL)
        slver 5y ago
        It seems almost as if it'd be more useful to not explicitly expose operators as applicable to arrays, but implement SIMD optimization for operator expressions in .map(function) and something like .binaryMap(right, function) ex. [1, 2, 3].binaryMap([10, 10, 10], (a, b) => a * b) // Produces [10, 20, 30] This would be easier to optimize when compiled because the expressions can be simplified and mapped to the right SIMD instructions.
        dzaima 5y ago
        That just removes a tiny tiny bit of dynamic dispatch overhead. Which is still needed anyways, as array languages can often dynamically switch between 1-bit, 8-bit, 16-bit, 32-bit integer (and 64-bit float) arrays, depending on the elements, completely transparently to the user.
        slver 5y ago
        Most compilers can inline statically defined closures in these contexts. And tracing JITs do this even when the closure is not define statically (but is stable). It's more about allowing the SIMD goodness without the ambiguity and restrictions of "scalar operators work on arrays" implemented naively.
        mlochbaum 5y ago
        That's the case in APL and J. K uses nested lists to represent arrays, and has non-lists (atoms). But the convention is that an n-times nested list is considered an n-dimensional array so even an atom is an array, with 0 dimensions. There's a page on the various approaches to arrays in the APL family at https://aplwiki.com/wiki/Array_model https://aplwiki.com/wiki/Array_model .
        rklmno 5y ago
        Mostly yes. Numbers (and character) are implemented as arrays with 0 dimensions. Text would be an array of characters with 1 dimension (the number of characters), and generally speaking the dimension of an array is a one dimensional list of non-negative integers. Many array languages also include an array type which is approximately the same as a C pointer to an array, with a bit of jargon thrown in to distinguish a reference to an array from the array itself. Something like an SQL table in an array language would be implemented as a list of columns (rather than as a list of rows) and a corresponding list of column labels. This has some interesting benefits. That said, functions in array language are typically not arrays (though they presumably would have a textual representation). So... not everything is an array.
        rscho 5y ago
        From the J docs: https://www.jsoftware.com/help/dictionary/dx005.htm https://www.jsoftware.com/help/dictionary/dx005.htm nub=: (i.@# = i.~) # ] 5!:2 <'nub' +-------------------+-+-+ |+--------+-+------+|#|]| ||+--+-+-+|=|+--+-+|| | | |||i.|@|#|| ||i.|~||| | | ||+--+-+-+| |+--+-+|| | | |+--------+-+------+| | | +-------------------+-+-+ ... so J functions have an array representation, at least.
        rklmno 5y ago
        Yes, each J function has several array representations.
      - Banana699 5y ago
        Fortran isn't an array language at all, really. Maybe some features of the array languages style are super-imposed atop it with parallizing extensions, dialects, or whatnot; but it's style has always been and still is to write explicit loops and mutate state left and right. John Backus in his famous 1977 turing award speech explicitly named it a representative of the "fat weak" languages he talked of and said it was a thin veneer over assembly, he based his fictional FP language on APL instead. Maybe grandparent meant it has the same _application_ as array languages, in that its only surviving kingdom is scientific computing where devouring gargantuan arrays of numerics is the only thing that matters, unlike C or C++ that are much more widely used. Maybe that's why it has a long history of being parallized with various tools and runtimes _despite_ its inherent imperativeness. (I don't remember where I heard this, but somebody wrote an analyzer to analyze some Fortran programmes in the 90s and found that over 80%/90% of Fortran programmes are spent doing variations of map, filter and reduce. So it's an extremely imperative language against its intended use. Maybe I will post a link in an edit when I find the source of the claim)
        Bostonian 5y ago
        Fortran has had built-in array operations since the Fortran 90 standard. If X and Y are scalars or arrays of the same shape, you can X+Y, X*Y, exp(X), sin(X) etc. You can define your own elemental functions that act on both scalars and arrays of any rank. I still write loops when programming in Fortran 95 but less often than in Fortran 77. So I think modern Fortran is an array language.
- RojerGS 5y ago
  I listened to the first episode and it was really interesting to listen to really experienced programmers share the things they like the most about array programming and the array-oriented languages they are most familiar with.
- BlanketLogic 5y ago
  An interesting episode. Recommended. One of the presenters is a novice with c++ background and the rest all are experts in their respective array programming languages.
- kieckerjan 5y ago
  Nice podcast. Definitely made me want to check out APL. Again. (Disclaimer: I go through this phase every couple of years.)
mjburgess 5y ago
Surprising highly production values -- programming podcasts are typically extremely poor auido with very little prep going into to them.
- slver 5y ago
  These days having good video and audio quality doesn't require a lot of money. It just requires couple of hours educating yourself online what you need and how to use it.
enriquto 5y ago
A bit sad that this interesting content is not available for audio/video impaired readers.
- slver 5y ago
  Or patience impaired. It's a whole hour.
- jasode 5y ago
  Fyi if you weren't aware ... most podcasts don't have text of the audio because high-quality (accurate) transcription of podcasts costs money. Example rates: https://www.google.com/search?q=podcast+transcription+service+prices https://www.google.com/search?q=podcast+transcription+servic... So this thread's podcast of 52 minutes of a complex technical topic with multiple speakers could cost ~$200. A programming-related podcast is already a niche topic with a tiny audience and an Array Languages podcast is an even tinier subset of that so the cost might not be justified. I suppose podcasts could be uploaded to Youtube and let their speech-to-text algorithm do an auto-transcribe. However, the A.I. algorithm is not good at tech topics with industry jargon/acronyms and the resultant transcription will be inaccurate.
  - kwhitefoot 5y ago
    Presumably there was a script or at least a summary. Why not publish that as well as any slides used?
    - bidirectional 5y ago
      Why would there be? It's a recorded conversation between a group of people. Some of them may have some rough notes but maybe not even that.
    - abrudz 5y ago
      There was no script or summary, nor any slides. It was a completely organic conversation.
  - 5y ago
nulldata 5y ago
Great first episode. I'd love to hear Aaron Hsu on this!
FemmeAndroid 5y ago
This is very exciting. I’ve been really inspired by the passion Conor Hoekstra has for APL and J. His other podcast (Algorithms + Data Structures = Programming) is a lot of fun, and his YouTube videos are very educational, but I’ve really wanted something like this where he can interact with other experts outside of the C++ world. Instant subscribe from me.
chewxy 5y ago
this seems like it's very very niche (and very specific to the niche that I am interested in!)
- bidirectional 5y ago
  True, but I also feel like array programming is seen as less prevalent in industry than it really is due to a lack of online community (e.g. Haskell as a language probably has an order of magnitude more content online, let alone the functional paradigm). In any financial centre there are hundreds of (often very well-paying) jobs using these languages (well mainly k and its ilk).
an1sotropy 5y ago
I had thought of APL as something from computing pre-history, with its bizarro custom keyboard, but I learned that APL and other array languages are apparently alive and well. Will subscribe to the podcast. Two quotes the hosts brought up stuck with me: (at 15:05) "A language that doesn't change the way you think is not a language worth learning". From Alan Perlis [1], and his Epigrams in Programming (#19) [2] (at 16:49) "it is a privilege to learn a language/ a journey into the immediate". From poet Marilyn Hacker [3]; totally captivating idea, even if not not about programming languages [4] [1] https://en.wikipedia.org/wiki/Alan_Perlis https://en.wikipedia.org/wiki/Alan_Perlis [2] https://cpsc.yale.edu/epigrams-programming https://cpsc.yale.edu/epigrams-programming [3] https://poets.org/academy-american-poets/winner/prizes/james-laughlin-award/presentation-piece https://poets.org/academy-american-poets/winner/prizes/james... [4] https://www.enotes.com/topics/marilyn-hacker/critical-essays https://www.enotes.com/topics/marilyn-hacker/critical-essays
- bidirectional 5y ago
  k (and the closely related q) is the main language used in industry, particularly at investment banks and hedge funds. It can be a bit of a shock to realise there are people in London earning in excess of £1000/day (pretty good for London) working in a language where well-written code looks like this[1]: us:{$[#i:&{(y~*K)&"*"~\*x}':x;@[x;i;:[;,"_"]];x]} It's like discovering a whole different world of software development. Also I don't use that example to disparage k, I have come to appreciate the array language way-of-working. It just looks very alien. [1] Real example found in a random script on https://nsl.com/ https://nsl.com/: http://nsl.com/k9/sql.k http://nsl.com/k9/sql.k
  - jiggawatts 5y ago
    What the actual f? It's like someone threw up the noise that modems make during initial connection onto an electric typewriter from the 1960s, and then explained their intention using quotes from a Lovecraft novel.
    - rscho 5y ago
zanethomas 5y ago
https://github.com/abrudz/ngn-apl https://github.com/abrudz/ngn-apl
deleted 5y ago
[deleted]
skipcave 5y ago
Boxing in J sep=:10#.^:_1] NB. Separate digits ea=:&.> NB Perform on each sep ea 23 7534 322 7 24756 │2 3│7 5 3 4│3 2 2│7│2 4 7 5 6│ #ea n NB. How many? │2│4│3│1│5│ +/ea n NB. Sum │5│19│7│7│24│ */ea n NB. Product │6│420│12│7│1680│ /:~ ea n NB. Sort │2 3│3 4 5 7│2 2 3│7│2 4 5 6 7│

162 points by srpeck 5y ago

not_knuth 5y ago

This looks interesting. Can't wait to give it a listen! Has anyone already done so? Reviews?

rscho 5y ago
It's good. They've got representatives of all the major array language communities among the hosts (APL, J, K/Q) plus a C++ guy.
- __vim__ 5y ago
  No fortran guy?
  - rscho 5y ago
    I think the goal in bringing in a C++ programmer was to provide an outsider view, but no, no Fortran guy. I don't think Fortran is an array language in that arrays are not the only datastructures in Fortran, though?
    - slver 5y ago
      Is an array language a language where you only have arrays? That honestly sounds a bit odd.
      - mjaniczek 5y ago
        You could consider Pandas / numpy an array language, even though it's really a library for Python. The exposed API is IMHO what's important.
        Wezl 5y ago
        I agree, but you have to take extensibility into account. You either have an extensible language and extend it[0], you make it an array language to the core, or you just have a few slightly convenient functions (not a language). (I don't have enough experience to judge which category Pandas / numpy are in) [0]: https://github.com/phantomics/april https://github.com/phantomics/april
      - klibertp 5y ago
        No, an array language is one in which all the built-in operations are applicable to arrays. Take addition as an example: 4 + 4 8 In array languages the same operator can be used for arrays; or equivalently you can say that the example above sums two arrays of length 1. In J, you can do this and expect it to work: 4 4 4 + 2 2 2 6 6 6 This is true for all the built-ins, and many user defined operations (as long as you don't fiddle with so called rank of the verb you're defining). Numpy is close to that, but there's still a distinction between arrays and scalars, while in array langauges that distinction is often blurred: 4 + 1 2 3 5 6 7 Edit: in J, you can have atoms, or scalars, but you need to box them: (3;4;5) ┌─┬─┬─┐ │3│4│5│ └─┴─┴─┘ but then you can't do anything with them until you unbox them again: (3;4;5) + 3 |domain error | (3;4;5) +3 (+&4) each (3;4;5) ┌─┬─┬─┐ │7│8│9│ └─┴─┴─┘ (Examples straight from J REPL)
        slver 5y ago
        It seems almost as if it'd be more useful to not explicitly expose operators as applicable to arrays, but implement SIMD optimization for operator expressions in .map(function) and something like .binaryMap(right, function) ex. [1, 2, 3].binaryMap([10, 10, 10], (a, b) => a * b) // Produces [10, 20, 30] This would be easier to optimize when compiled because the expressions can be simplified and mapped to the right SIMD instructions.
        dzaima 5y ago
        That just removes a tiny tiny bit of dynamic dispatch overhead. Which is still needed anyways, as array languages can often dynamically switch between 1-bit, 8-bit, 16-bit, 32-bit integer (and 64-bit float) arrays, depending on the elements, completely transparently to the user.
        slver 5y ago
        Most compilers can inline statically defined closures in these contexts. And tracing JITs do this even when the closure is not define statically (but is stable). It's more about allowing the SIMD goodness without the ambiguity and restrictions of "scalar operators work on arrays" implemented naively.
      - mlochbaum 5y ago
        That's the case in APL and J. K uses nested lists to represent arrays, and has non-lists (atoms). But the convention is that an n-times nested list is considered an n-dimensional array so even an atom is an array, with 0 dimensions. There's a page on the various approaches to arrays in the APL family at https://aplwiki.com/wiki/Array_model https://aplwiki.com/wiki/Array_model .
      - rklmno 5y ago
        Mostly yes. Numbers (and character) are implemented as arrays with 0 dimensions. Text would be an array of characters with 1 dimension (the number of characters), and generally speaking the dimension of an array is a one dimensional list of non-negative integers. Many array languages also include an array type which is approximately the same as a C pointer to an array, with a bit of jargon thrown in to distinguish a reference to an array from the array itself. Something like an SQL table in an array language would be implemented as a list of columns (rather than as a list of rows) and a corresponding list of column labels. This has some interesting benefits. That said, functions in array language are typically not arrays (though they presumably would have a textual representation). So... not everything is an array.
        rscho 5y ago
        From the J docs: https://www.jsoftware.com/help/dictionary/dx005.htm https://www.jsoftware.com/help/dictionary/dx005.htm nub=: (i.@# = i.~) # ] 5!:2 <'nub' +-------------------+-+-+ |+--------+-+------+|#|]| ||+--+-+-+|=|+--+-+|| | | |||i.|@|#|| ||i.|~||| | | ||+--+-+-+| |+--+-+|| | | |+--------+-+------+| | | +-------------------+-+-+ ... so J functions have an array representation, at least.
        rklmno 5y ago
        Yes, each J function has several array representations.
    - Banana699 5y ago
      Fortran isn't an array language at all, really. Maybe some features of the array languages style are super-imposed atop it with parallizing extensions, dialects, or whatnot; but it's style has always been and still is to write explicit loops and mutate state left and right. John Backus in his famous 1977 turing award speech explicitly named it a representative of the "fat weak" languages he talked of and said it was a thin veneer over assembly, he based his fictional FP language on APL instead. Maybe grandparent meant it has the same _application_ as array languages, in that its only surviving kingdom is scientific computing where devouring gargantuan arrays of numerics is the only thing that matters, unlike C or C++ that are much more widely used. Maybe that's why it has a long history of being parallized with various tools and runtimes _despite_ its inherent imperativeness. (I don't remember where I heard this, but somebody wrote an analyzer to analyze some Fortran programmes in the 90s and found that over 80%/90% of Fortran programmes are spent doing variations of map, filter and reduce. So it's an extremely imperative language against its intended use. Maybe I will post a link in an edit when I find the source of the claim)
      - Bostonian 5y ago
        Fortran has had built-in array operations since the Fortran 90 standard. If X and Y are scalars or arrays of the same shape, you can X+Y, X*Y, exp(X), sin(X) etc. You can define your own elemental functions that act on both scalars and arrays of any rank. I still write loops when programming in Fortran 95 but less often than in Fortran 77. So I think modern Fortran is an array language.
RojerGS 5y ago
I listened to the first episode and it was really interesting to listen to really experienced programmers share the things they like the most about array programming and the array-oriented languages they are most familiar with.
BlanketLogic 5y ago
An interesting episode. Recommended. One of the presenters is a novice with c++ background and the rest all are experts in their respective array programming languages.
kieckerjan 5y ago
Nice podcast. Definitely made me want to check out APL. Again. (Disclaimer: I go through this phase every couple of years.)

mjburgess 5y ago

Surprising highly production values -- programming podcasts are typically extremely poor auido with very little prep going into to them.

slver 5y ago
These days having good video and audio quality doesn't require a lot of money. It just requires couple of hours educating yourself online what you need and how to use it.

enriquto 5y ago

A bit sad that this interesting content is not available for audio/video impaired readers.

slver 5y ago
Or patience impaired. It's a whole hour.
jasode 5y ago
Fyi if you weren't aware ... most podcasts don't have text of the audio because high-quality (accurate) transcription of podcasts costs money. Example rates: https://www.google.com/search?q=podcast+transcription+service+prices https://www.google.com/search?q=podcast+transcription+servic... So this thread's podcast of 52 minutes of a complex technical topic with multiple speakers could cost ~$200. A programming-related podcast is already a niche topic with a tiny audience and an Array Languages podcast is an even tinier subset of that so the cost might not be justified. I suppose podcasts could be uploaded to Youtube and let their speech-to-text algorithm do an auto-transcribe. However, the A.I. algorithm is not good at tech topics with industry jargon/acronyms and the resultant transcription will be inaccurate.
- kwhitefoot 5y ago
  Presumably there was a script or at least a summary. Why not publish that as well as any slides used?
  - bidirectional 5y ago
    Why would there be? It's a recorded conversation between a group of people. Some of them may have some rough notes but maybe not even that.
  - abrudz 5y ago
    There was no script or summary, nor any slides. It was a completely organic conversation.
- 5y ago

nulldata 5y ago

Great first episode. I'd love to hear Aaron Hsu on this!

FemmeAndroid 5y ago

This is very exciting. I’ve been really inspired by the passion Conor Hoekstra has for APL and J. His other podcast (Algorithms + Data Structures = Programming) is a lot of fun, and his YouTube videos are very educational, but I’ve really wanted something like this where he can interact with other experts outside of the C++ world. Instant subscribe from me.

chewxy 5y ago

this seems like it's very very niche (and very specific to the niche that I am interested in!)

bidirectional 5y ago
True, but I also feel like array programming is seen as less prevalent in industry than it really is due to a lack of online community (e.g. Haskell as a language probably has an order of magnitude more content online, let alone the functional paradigm). In any financial centre there are hundreds of (often very well-paying) jobs using these languages (well mainly k and its ilk).

an1sotropy 5y ago

I had thought of APL as something from computing pre-history, with its bizarro custom keyboard, but I learned that APL and other array languages are apparently alive and well. Will subscribe to the podcast. Two quotes the hosts brought up stuck with me: (at 15:05) "A language that doesn't change the way you think is not a language worth learning". From Alan Perlis [1], and his Epigrams in Programming (#19) [2] (at 16:49) "it is a privilege to learn a language/ a journey into the immediate". From poet Marilyn Hacker [3]; totally captivating idea, even if not not about programming languages [4] [1] https://en.wikipedia.org/wiki/Alan_Perlis https://en.wikipedia.org/wiki/Alan_Perlis [2] https://cpsc.yale.edu/epigrams-programming https://cpsc.yale.edu/epigrams-programming [3] https://poets.org/academy-american-poets/winner/prizes/james-laughlin-award/presentation-piece https://poets.org/academy-american-poets/winner/prizes/james... [4] https://www.enotes.com/topics/marilyn-hacker/critical-essays https://www.enotes.com/topics/marilyn-hacker/critical-essays

bidirectional 5y ago
k (and the closely related q) is the main language used in industry, particularly at investment banks and hedge funds. It can be a bit of a shock to realise there are people in London earning in excess of £1000/day (pretty good for London) working in a language where well-written code looks like this[1]: us:{$[#i:&{(y~*K)&"*"~\*x}':x;@[x;i;:[;,"_"]];x]} It's like discovering a whole different world of software development. Also I don't use that example to disparage k, I have come to appreciate the array language way-of-working. It just looks very alien. [1] Real example found in a random script on https://nsl.com/ https://nsl.com/: http://nsl.com/k9/sql.k http://nsl.com/k9/sql.k
- jiggawatts 5y ago
  What the actual f? It's like someone threw up the noise that modems make during initial connection onto an electric typewriter from the 1960s, and then explained their intention using quotes from a Lovecraft novel.
  - rscho 5y ago

zanethomas 5y ago

https://github.com/abrudz/ngn-apl https://github.com/abrudz/ngn-apl

deleted 5y ago

[deleted]

skipcave 5y ago

Boxing in J sep=:10#.^:_1] NB. Separate digits ea=:&.> NB Perform on each sep ea 23 7534 322 7 24756 │2 3│7 5 3 4│3 2 2│7│2 4 7 5 6│ #ea n NB. How many? │2│4│3│1│5│ +/ea n NB. Sum │5│19│7│7│24│ */ea n NB. Product │6│420│12│7│1680│ /:~ ea n NB. Sort │2 3│3 4 5 7│2 2 3│7│2 4 5 6 7│