Would Spreadsheets with Lisps be much more powerful?

Dear sarvasvkulpati,

I appreciate your question! It demonstrates a curiosity about the nature of computing itself. Some will surely say that this idea is unreasonable or not practical. But, if this is an idea that speaks to you, I encourage you to try your hand at making a prototype of your idea! This is the hacker way. And being in college you have time, peers who are equally enthusiastic, energy, and the help of professors. Regardless of the practicality, all interesting progress comes from people who ask questions like this, and see what comes of trying to make something interesting.

In college I had a peer who convinced us to all use Gentoo Linux because it was the most hardcore Linux. I bricked my computer three times trying to use it and ultimately switched away from using Gentoo but learned a ton about Linux that I use to this day on a less hardcore distro. I still use Vim even though it's more complicated than other editors. I also went through a period where I tried learning every programming language I could find to see which one was "the best". I ultimately found a few I liked the most. None of these were efficient at first, but eventually led to highly cool results.

Ask for help if you get stuck. I bet something interesting will come of it, even if you just learn what doesn't work.

Something something "The reasonable man adapts himself to the world: the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man."

Your idea is really cool. Stay hungry and awesome. Cheers to you.

You may find SIAG (Scheme in a grid) interesting. It explores some of the same ideas: https://siag.nu/siag/

There's also been some work along these lines in Excel itself, although using the Excel formula language rather than Scheme. Excel now has both let and lambda, as well as a way to work with composite data structures in cells.

https://www.microsoft.com/en-us/research/blog/lambda-the-ult...

https://support.microsoft.com/en-us/office/let-function-3484...

https://exceljet.net/dynamic-array-formulas-in-excel

In general, I think your idea is sound. Almost since the beginning, Spreadsheets have had two major ways of operating on data - the formula language, which is automatic, declarative, and functional in nature, and commands (like on a menu), which are generally imperative and have to be explicitly invoked (or invoked through a separate macro language). Both have their places, but the functional language of formulas is nice for the same reason functional languages in general are appealing. There's also an easier ramp for users new to programming to progress from "=A1+B1" to more advanced sorts of operations (lookupus, etc.) without having to switch to a formal "programming language".

However, aside from additional standard functions, the formula language half (in Excel, at least) was largely static since the 1980s - limited essentially to scalar data types, and no internal way for creating abstractions. This severely limited the range of what you could do with formulas, when compared to more complete programming languages, and forced a lot of functionality to be done in (more brittle) macros.

So the direction you're going - broadening the power of what can be done with a spreadsheet language - is a useful one, and there's prior art to justify that the idea makes sense.

Excel has a new function called LAMBDA, which makes Excel Turing-Complete. (https://www.microsoft.com/en-us/research/blog/lambda-the-ult...) If you haven’t gotten there yet in your CS classes, Turing-Complete means you can compute anything that is computable, so provably as powerful as any other language.

Turing-Complete doesn’t say anything about convenience though, there are always reasons to play with embedding your favorite language into your other environments, so don’t let prior work stop you from playing and exploring! You will earn yourself many benefits by trying to do it yourself.

Excel, BTW, was fairly powerful as a programming environment even before the LAMBDA function, there have been multiple examples of crazy applications written in an Excel spreadsheet, video playback, games, ray tracing, etc. Here’s a ray tracing example: https://news.ycombinator.com/item?id=20764825

Good question! Smartass answer: No, because spreadsheets in programs like Excel are Turing-complete, which means any program you could write in a Lispy spreadsheet, you can write in Excel.

Useful answer: In practical terms, probably, for a very small subset of users. The kinds of users who are into programming and also Lisp, and also use spreadsheets a lot. For these four users, adding their favourite programming paradigm would help them to implement cooler things than they would otherwise do.

About 40 years ago —

"An advanced object-oriented spreadsheet package for the Smalltalk-80 system

Asp™ is an analytic spreadsheet package that provides access to the full power of the Smalltalk-80™ programming environment. Developed by Xerox Special Information Systems (XSIS) as part of The Analyst™ Workstation System, ASP is now available as a separate package for the Smalltalk-80 environment.

Unlike other spreadsheet packages, ASP is designed for the analysis of more than just numbers and labels. Vectors, points, collections of data, text, charts, files, and images are just a few types of objects that ASP can manipulate and display.

While ASP provides a large array of predefined functions, it is also extensible. By adding new methods to the Smalltalk-80 environment, ASP can be instantly expanded to match user needs, without requiring extensive knowledge of either ASP or the Smalltalk-80 system."

http://www.bitsavers.org/pdf/xerox/xsis/XSIS_Smalltalk_Produ...

"Object-oriented spreadsheets: the analytic spreadsheet package"

https://dl.acm.org/doi/abs/10.1145/960112.28737

I haven’t used it myself, but I believe Simple Emacs Spreadsheets lets you write formulas in emacs lisp.

https://www.gnu.org/software/emacs/manual/html_node/ses/inde...

Perhaps even more powerful would be spreadsheets + Prolog. Since Prolog's main strength is finding answers for you, I imagine putting that on grid would have very interesting results.

See Kenny Tilton's Cells project, which more or less does the inverse.

https://github.com/kennytilton/cells

i've sometimes thought that APL could mesh really well with spreadsheets.

https://www.youtube.com/watch?v=v7Mt0GYHU9A https://www.youtube.com/watch?v=UltnvW83_CQ

The problem with regular programming is that you can see the code but not the data, and the problem with spreadsheets is that you can see the data but not the code.

You might be on to something with the “apply” idea. What if there was a split screen where you could see both code/operations and data and could mix them together with all the transparency of a spreadsheet? macros would just exploit code as data?

For a tangent that might spark interesting ideas, check out this structure editor project in Racket (a lisp) https://github.com/disconcision/fructure

One advantage of using lisp is it would make it easier to develop complex routines. You could even import external code, so you could write/debug some hairy algos externally and then use them in the spreadsheet. Imagine having the power of macsyma and doing pdes in your spreadsheet. Or being able to do advanced vector operations on columns.

Though I am not as fond of them a stack language like forth or postscript could also be the computing paradigm of your spreadsheet.

Or if you are feeling more perverse, there’s always TECO

I'm building a language (https://tablam.org) that, hopefully, could become the base for excel/access alternative.

lisp is not the better fir for excel, to see why, check this:

    "The memory models that underlie programming languages"

http://canonical.org/~kragen/memory-models/

Among them, array/relational could be a better fit (semantic and operational). Of course you also need to add some form of reactivity (or actors?) to mimic the immediacy of excel formulas.

---

To confirm this intuition, check how much pandas/numpy/R lang/apl/Julia/Sql people intersect with the usage of excel.

This means that languages/tools are closer in spirit than lisp.

Also, syntax wise, concatenative langs/array langs/forth langs/sql like is more nice for the terseness of excel usage.

For a syntax in a spreadsheet you would want something that is short and familiar but you can certainly toy with idea.

I remember when the geeky engineers had their calculators in reverse Polish notation:

https://en.m.wikipedia.org/wiki/Reverse_Polish_notation

Yes, just as "sharks with frikin lasers" are much more powerful.

But seriously, no. It is cult cargo thinking" to believe that merely juxtaposing two things will lead to something "more powerful. OTOH it is often the first step toward a new and useful idea. So all the magic is in the "how" of combining things.

More powerful? Yeah, sure. But honestly, many companies have made more powerful versions of Excel over the years and most of them have gone to the wall.

If you think of power in a different way, that of empowering users who typically have little or no technology experience, things get a bit more interesting. Anything you come up with has to be instantly obviously more useful in at least one way. Google Docs and Zoho are still viable products despite from a CS perspective being significantly less powerful.

Like many answers you’ve received, it comes down to the difference between technology, product design and commercial concerns.

The only product to ever supplant a world-wide dominant spreadsheet is... Excel.

Given you have a finite amount of time, the relatively obvious answer is - this isn't a good idea.

Excel is made by Microsoft. Microsoft is strong in languages and tooling around languages. If there was something genuinely useful to add, they'd have done it.

Now, obviously you can't think like this about everything or you'd never do anything. But, you have finite time so pick something better, I'd say. Maybe pick an area where you have some deep insight. Or some area where the existing products stink. Excel is really freaking good given the large number of use cases they have to support.

I was at OPLSS[0] in 2018 and Enzo Alda gave a talk/demo on ZenSheets, his in-progress reactive spreadsheet thing where you could write lambdas in the cells, call them in formulas from other cells, etc. That talk wasn't recorded, but a similar demo can be found here.[1]

[0]: https://www.cs.uoregon.edu/research/summerschool/summer18

[1]: https://www.youtube.com/watch?v=mJa0_gKE6xo

Look, yes and no: I want to ignore Turing completeness and the tarpit, forget about lambda, and talk about intermediate results. I write excel so as to write out intermediates, so to calculate f(n) = f(n-1)+.. I’d write N lines. If you could just write a function to “skip to the end”, there’d be value in seeing all the values in between, in some sort of debugger view / virtual sheet. If that makes sense.

If I define (square n) and (twice f) to calculate a quartic function, what would I see in a single cell, when I’m not looking at the formula view? That’s what I wonder.

Every time someone mentions extended spreadsheets I like to bring up Resolver One. In the mid 00s it was a spreadsheet interface in .Net that generated python modules under the hood running on IronPython. As a side effect you could import sections of one spreadsheet into another spreadsheet, go behind the scenes and edit the Python to connect to a DB or run arbitrary code, generate GUIs using the .Net facilities, etc. I thought it was neat. Unfortunately they never got traction and as far as I know they were aquihired by a financial company that used it internally.

If you're interested in this topic, you should look into emacs (a text editor that uses a Lisp-like language for customization) and it's accompanying org-mode (something that helps you take notes in emacs, and also has a built in spreadsheet), which allows for using emacs lisp for defining functions: https://orgmode.org/worg/org-tutorials/org-spreadsheet-lisp-...

Good luck!

Of course non-pure Lisps and JavaScript pretty much only differ in syntax, so there is no "new interaction possible".

They are more powerful than the basic formula languages in most spreadsheet software, which is why spreadsheet software usually offers the ability to write custom functions using traditional programming languages.

Yes. People who know how to program in matlab and python, but also have to manually open up excel / csv files need this. Loading up excel is slow, writing a python script where you have to search the docs to find the functions you want is slow too. A programmable and fast spreadsheet is needed.

Interesting idea. I like spreadsheets and a new way to program them would be interesting.

In your sample it seems like problematic entries cause the sheet to crash or become non-responsive. Probably a good idea to handle errors gracefully. If you can't evaluate, just give an error message.

Try to put this into a spreadsheet: https://mitpress.mit.edu/books/software-design-flexibility

You might find something useful here: https://people.eecs.berkeley.edu/~bh/pdf/ssch24.pdf

Excel formulas are already functional. They’re just composed with Sexpressions. I’m not convinced semantics of one is any better than the semantics of another. Ultimately that boils down to personal preference.

The answer is quite likely to be yes. But then when you consider who is the primary typical user of a spreadsheet and ask yourself "are they likely to learn Lisp?" then the answer is no.

Sarvasv,

You should explore this further! Once you are past the basic syntax, I think there are world of possibilities that can be explored.

It would be cool if each cell is an s expression on its own. - but are cells the best way to show s expressions?

from the other side, check out 'cells' for an interesting implementation of spreadsheet-like reactive updates within common lisp https://github.com/kennytilton/cells/wiki

Is lisp more powerful than JS? Not really; you can write lisp in JS. But that's a correct answer to a wrong question. Lisp is fun, but IMO it's wrapped in undeserved mythology. Maybe I'm just jaded, but if you move parens one word to the right and sprinkle commas between tokens, it's downright readable to a c programmer. That said, homoiconicity is powerful and spreadsheet-based metaprogramming could be super cool.

What I haven't seen is a maturation of spreadsheet-based programming. Code and data are mingled in cells, and excel (and clones I've used) hide the code and display the data in an opinionated manner. That's a huge UI problem, IMO. But, mingling code and data can be very fun! Perhaps lisp could be cool for that, but I'd actually prefer a concatenative array language to really take advantage of the spreadsheet UI. So, go read up on Forth and APL, and bake their unholy offspring into a spreadsheet. Nothing bad can come of this.

Well, I don't think "more powerful" is a good way to think about it. A spreadsheet is essentially a functional programming paradigm. One programming thingie is not going to be "more powerful" than another. That said, one could be cooler or more interesting than another one, at least to you.

My feeling is that it doesn't have to make sense, and you don't really need to have any justification. If you want to make a spreadsheet system with Lisp, I'd definitely do it. You learn the most when you do your own projects, and interest is the fuel.

Also, we're bad at determining beforehand what actually is important, so maybe you work on something as a toy, but later it (or some piece of knowledge developed while working on it) is crucial for something else later.

Mark your post as "Show HN". There's no way I will comment on your side project after I was tricked into reading about it.