5 ms·

Well, maybe one day...

neilk 13y ago
Please do this. I've dreamed of something like this for a long time.
zackmorris 13y ago
You are just a small step away from the concepts in Subtext: http://www.subtext-lang.org http://www.subtext-lang.org I highly recommend watching the screencasts, especially the demo: http://www.subtext-lang.org/demo1.html http://www.subtext-lang.org/demo1.html I have wanted a graphical interface for functional programming in environments like Excel for a long time. In my experience, most of the pseudo-programming that has dominated software engineering (object-oriented/MVC/message-based, patterns, best practices) goes away when you can model something directly. Nobody has been able to successfully apply proven engineering concepts from, say, electrical engineering or VLSI to software engineering. I think it's because the leap from imperative code is too long. But nobody has successfully made functional code approachable either, because too often it's contaminated by syntactic sugar that's too symbolic, various handcuffs that make even the simplest tasks arduous (poor handling of global state), or learning curves that can't be scaled when it comes to interacting with real-world time-based data (monads etc). For the most part Excel has none of these issues. You can stick an imperative macro in the middle of your functional code and it works just fine, as it should in any other functional language (as long as you follow certain rules like, no globals). Then you can write another macro to load CSV data into your spreadsheet, or even spit it out to a folder that has a shell script watching it to transform the data again. I think it's more productive to work in that unix-like manner (of a bunch of small tools that do one thing well, pipelined together), than it would be to try to integrate notions like dynamic data into Excel. But the achilles heal of Excel was always obfuscation. We need to get away from language when we're thinking about functional programming, and focus more on data and transformations. Slate looks like a great start, kudos!
- bchallenor 13y ago
  That Subtext screencast is fascinating - thanks for posting the link. Reminds me of the Bret Victor talks. I agree with your other points - this is something I'd love to build (or see built). But do you think it could displace Excel?
  - zackmorris 13y ago
    I'm not sure if anything will displace Excel (or spreadsheets in general). But what I'd like to see is a generalization beyond columns/rows/sheets for relating data of various dimensions. The underlying data type would probably be a list (so lisp) but have options to enable order, array access, key->value access, etc. Cell references would work more like in programming languages and operate over ranges that aren't just two dimensional. So a spreadsheet would be a subset of this. So a set of columns with array access, a set of rows with array access, a notation like column-row for cell references, and all of the operators like $ for absolute references, etc. I guess the underlying data structure could be a relational database but I'd prefer something human-readable like JSON with a thin notation over it for expressing relationships between structure members. You could use something like the Subtext editor to work on it but maybe if the syntax was easy enough you could write it manually. I think the real power of this would be if relationships could be executables (like how Excel macros work but with stdin/stdout/stderr and black box scripts written in any language). Imagine being able to call sed/awk or really high level tools like Go/Erlang and Matlab/Octave per-cell the way we use SUM in Excel..
    - mrcsparker 13y ago
      I actually worked on this exact solution a few months ago. I translated excel into a javascript data structure, translating the formulas into javascript functions. The data structure could be saved to json, and them reloaded back into javascript. The cool thing is that changing a value (a2 = 3) automatically updates dependent values in the rest of the data structure. The data structure can be referenced much like the spreadsheet - it is a tree with branches for pages, rows, columns. There is a simple faster csv like API that makes it easy to go page by page, row by row. Also, cells can be looked up explicitly. The code is proprietary, which is a shame. It was written as a library to quickly reverse engineer large, complex xlsx files. The trick in making it work is just having the right data structure. Sorry if I am rambling or any misspellings. I am very sick right now, drowsy from medication, and trying to type thus on my phone.
- crrashby 13y ago
  Interesting stuff. Yeah whats great about excel is there is zero learning needed to get started. Anyone can start typing numbers in the cells then move on to more complex tasks. The problem is, although each step towards building a model in Excel is quite simple the end results can be very unwieldy.
- andrewflnr 13y ago
  ...nobody has successfully made functional code approachable I'm not totally sure what to make of this statement. I think of functional programming as an extension of the expression-based computation almost everyone (in the first world anyway) learns in pre-algebra. What could be more approachable? Yet I always have trouble recommending a specific functional language to someone who's curious. I'd say scheme if sexprs weren't so intimidating, ocaml if the syntax weren't so annoying, Haskell if it weren't, well, Haskell (not that I've really used Haskell myself). So for now I just tell people to learn Python and show them map() and closures. Is there a semi-practical functional language with C-ish call syntax, especially with dynamic typing? And don't tell me Javascript, that's still mostly imperative. Rust seems close (closer than Python at least). I might start recommending that when it stabilizes.
  - PeterisP 13y ago
    I use Haskell, but for many data manipulation problems it is still clearer to do them in Excel, mostly because of the bigger emphasis on the data itself, easier visualisation and eyeball-verification intermediate results, and simpler data-garbage-fixing in case a data point "raises an exception".
nazka 13y ago
I think you will really love this video : http://worrydream.com/dbx/ http://worrydream.com/dbx/ It's a conference about programming languages in general. It's interesting, with cool materials, and fun to watch. Also if you need to have more ideas, you can take a look at the Unreal Engine and Cryengine. They built similar tools to allow artists to code graphically scripts for video games. Cool idea anyway.

neilk 13y ago

Please do this. I've dreamed of something like this for a long time.

You are just a small step away from the concepts in Subtext: http://www.subtext-lang.org http://www.subtext-lang.org I highly recommend watching the screencasts, especially the demo: http://www.subtext-lang.org/demo1.html http://www.subtext-lang.org/demo1.html I have wanted a graphical interface for functional programming in environments like Excel for a long time. In my experience, most of the pseudo-programming that has dominated software engineering (object-oriented/MVC/message-based, patterns, best practices) goes away when you can model something directly. Nobody has been able to successfully apply proven engineering concepts from, say, electrical engineering or VLSI to software engineering. I think it's because the leap from imperative code is too long. But nobody has successfully made functional code approachable either, because too often it's contaminated by syntactic sugar that's too symbolic, various handcuffs that make even the simplest tasks arduous (poor handling of global state), or learning curves that can't be scaled when it comes to interacting with real-world time-based data (monads etc). For the most part Excel has none of these issues. You can stick an imperative macro in the middle of your functional code and it works just fine, as it should in any other functional language (as long as you follow certain rules like, no globals). Then you can write another macro to load CSV data into your spreadsheet, or even spit it out to a folder that has a shell script watching it to transform the data again. I think it's more productive to work in that unix-like manner (of a bunch of small tools that do one thing well, pipelined together), than it would be to try to integrate notions like dynamic data into Excel. But the achilles heal of Excel was always obfuscation. We need to get away from language when we're thinking about functional programming, and focus more on data and transformations. Slate looks like a great start, kudos!

bchallenor 13y ago
That Subtext screencast is fascinating - thanks for posting the link. Reminds me of the Bret Victor talks. I agree with your other points - this is something I'd love to build (or see built). But do you think it could displace Excel?
- zackmorris 13y ago
  I'm not sure if anything will displace Excel (or spreadsheets in general). But what I'd like to see is a generalization beyond columns/rows/sheets for relating data of various dimensions. The underlying data type would probably be a list (so lisp) but have options to enable order, array access, key->value access, etc. Cell references would work more like in programming languages and operate over ranges that aren't just two dimensional. So a spreadsheet would be a subset of this. So a set of columns with array access, a set of rows with array access, a notation like column-row for cell references, and all of the operators like $ for absolute references, etc. I guess the underlying data structure could be a relational database but I'd prefer something human-readable like JSON with a thin notation over it for expressing relationships between structure members. You could use something like the Subtext editor to work on it but maybe if the syntax was easy enough you could write it manually. I think the real power of this would be if relationships could be executables (like how Excel macros work but with stdin/stdout/stderr and black box scripts written in any language). Imagine being able to call sed/awk or really high level tools like Go/Erlang and Matlab/Octave per-cell the way we use SUM in Excel..
  - mrcsparker 13y ago
    I actually worked on this exact solution a few months ago. I translated excel into a javascript data structure, translating the formulas into javascript functions. The data structure could be saved to json, and them reloaded back into javascript. The cool thing is that changing a value (a2 = 3) automatically updates dependent values in the rest of the data structure. The data structure can be referenced much like the spreadsheet - it is a tree with branches for pages, rows, columns. There is a simple faster csv like API that makes it easy to go page by page, row by row. Also, cells can be looked up explicitly. The code is proprietary, which is a shame. It was written as a library to quickly reverse engineer large, complex xlsx files. The trick in making it work is just having the right data structure. Sorry if I am rambling or any misspellings. I am very sick right now, drowsy from medication, and trying to type thus on my phone.
crrashby 13y ago
Interesting stuff. Yeah whats great about excel is there is zero learning needed to get started. Anyone can start typing numbers in the cells then move on to more complex tasks. The problem is, although each step towards building a model in Excel is quite simple the end results can be very unwieldy.
andrewflnr 13y ago
...nobody has successfully made functional code approachable I'm not totally sure what to make of this statement. I think of functional programming as an extension of the expression-based computation almost everyone (in the first world anyway) learns in pre-algebra. What could be more approachable? Yet I always have trouble recommending a specific functional language to someone who's curious. I'd say scheme if sexprs weren't so intimidating, ocaml if the syntax weren't so annoying, Haskell if it weren't, well, Haskell (not that I've really used Haskell myself). So for now I just tell people to learn Python and show them map() and closures. Is there a semi-practical functional language with C-ish call syntax, especially with dynamic typing? And don't tell me Javascript, that's still mostly imperative. Rust seems close (closer than Python at least). I might start recommending that when it stabilizes.
- PeterisP 13y ago
  I use Haskell, but for many data manipulation problems it is still clearer to do them in Excel, mostly because of the bigger emphasis on the data itself, easier visualisation and eyeball-verification intermediate results, and simpler data-garbage-fixing in case a data point "raises an exception".

nazka 13y ago

I think you will really love this video : http://worrydream.com/dbx/ http://worrydream.com/dbx/ It's a conference about programming languages in general. It's interesting, with cool materials, and fun to watch. Also if you need to have more ideas, you can take a look at the Unreal Engine and Cryengine. They built similar tools to allow artists to code graphically scripts for video games. Cool idea anyway.