Obviously this is asking for subjective opinions and experience among the crowd.
In one or another Arc (or Bel) intro essay, there's this notion that "a true hacker's-dream language" would be a new Lisp other than Common Lisp which is there accused of various suckages, and apparently also other than all the numerous Scheme variants (at least in the "no such corset of 'hygienic' macros, plz" sense). Perusing the various search results on 'CL vs Scheme / Racket' had me personally more convinced of CL for heavy-duty "real-world", non-"write-yourself-a-Foo" projects, though (wrt. perf / compilers / library ecosystem).
The last time I got hooked on pg's persuasive Lisp essays and got into it for a short while (cooking up an SSR templating engine somewhere in the PHP/ASP/Handlebars sense for Hunchentoot as a starter exercise) was in 2008. Real-world projects lured me away soon enough then and I didn't really miss it tbh. That short episode of first Lisping to me was just "kinda alright" rather than the-most-hackery-experience ever. Then ca. 2011 unleashed the novelties (for me) of WebGL, NodeJS, Golang and more, and those were enough "hackers delight" playgrounds for me for years to come.
But revisiting the essays (by now it might be nostalgia, like re-reading/viewing one's certain personal all-time / youth-time favourite novels, comics, movies once per decade or so, I guess =) I'm once again lured by the promise of "the language at the end of the power spectrum" against which every other one is just some shade of "Blub". Then again, I went into almost every language I ever got into from this sort of promising sense of power potential, including Haskell, C, and even Go (pre-generics) in its own ways of "power through simplicity, and then just code your code-gens for those rare meta-needs". (Zig, too, but though enjoyable, that kept changing itself up frequently enough back then to move me back to more settled grounds.)
You see the one sticky situation I keep ending up in is that I'm, in pg's words, always "painting myself into a corner" all too soon, hacking away iteratively and "dirtily" (in the make-it-work, then make-it-fast, then make-it-pretty/correct/maintainably-architected spirit — but I don't really get to the last part and only some of the 2nd if at all —we're talking green-field personal side projects here—, but anytime I start with the last part, it goes nowhere fast, nay slowly actually). I can paint myself into a corner in QBasic as well as in Haskell and anywhere in between (and lest I totally burn out my natural curiosity toward tech to learn to make things, I have to keep reminding myself that they probably aren't to be blamed as I'm the constant factor in all these experiences). And I keep picking up that "building stuff On Lisp" is maybe The One basis where that's either least possible / likely, or not a particularly impeding problem / hindrance due to "ever molding the tech basis / language itself around and toward the problem you're tackling".
So, what's your hot take on this question, your personal opinion, your experience report? In 2024, some ~30 years after On Lisp and some ~20 years after those essays, all the other languages have kept picking up nearly everything once-cutting-edge about Lisp, other than the "parensy pesky" that pg insists is Lisp's real power (convincingly so, at least from-a-distance, I might add), that whole melange of "sexpr (non)syntax / code-is-data / code-is-the-AST / read-time+compile-time+run-time macros in the same (non)language".
(And then, do we start purity-spiraling in a fever dream of a one-true-Lisp without the "suckages" of the existing and in-widespread-use ones? Probably not, eh?)
At the same time, the state of the art in programming languages has advanced since PG wrote those essays. Even many of the languages which he was presumably referring to as "Blub" are much better now than they were.
A bigger issue is: As one gets better and better at computer programming, at some point the specific implementation language starts to matter less and less. (It never completely ceases to matter; but, less and less.) In one of his essays, PG contrasts "applying Bayesian filtering" with "applying if statements", which describes the matter quite aptly. At some point, language constructs such as "if statements" really do melt away.
For this reason, I reject claims that using a certain programming language will make one "1000 times more productive" (or similar) out of hand. The big problems in software are not at the level of the language at all.
This may sound false to newer programmers who have just discovered a fantastic programming language which allows them to work much faster than they could before. To such ones: I've been in your shoes. I've tried many languages, perhaps even the one you are enamoured with now. The benefits don't scale indefinitely. At some point, as you apply your fantastic new language to bigger problems, you will find yourself wrestling with the same issues as you used to with your old language, perhaps in a different form.
I personally don't understand why anyone who has had the first-hand experience of writing software this way would willingly give up on this fun. Yet, I also understand that personal impressions can be deceiving. Just because I love chocolate and every single person I have ever met also loves it, it doesn't mean that there aren't people who genuinely hate it.
Sure, tons of other programming languages have borrowed features from Lisps and continue to do so, but there exists an emergent category of programs that are assuredly more difficult to build using non-Lispy languages. Check out, for example, hyperfiddle/electric.
My biggest personal regret is that I discovered Lisp too late in my professional life. For a very long time I have had the same experience you're describing — starting projects with high promise but then hitting practical roadblocks. With the caveat that none of them were in Lisp languages.
Today, I don't even think about it — if I have to build something, I'll grab a Lisp dialect. I don't consider Lisps to be different languages; I can relatively easily switch between Clojure, Clojurescript, Clojure-Dart, Fennel, Common Lisp, Elisp, etc. I can't say the same thing even about Javascript and Typescript. Darn, even when I have to switch between different JS libraries, it feels like I have to deal with some mental overhead. That doesn't happen with Lisps.
At the end of the day, there are always trade-offs, whatever tools you choose and whichever language(s) you pick. Nothing is ever perfect. For me, if something isn't working, then maybe I need to find a Lisp better suited for the job. And usually, there are always options to choose from.
The other advantages are less true because dynamic languages are prolific and powerful. But also you never really grok JavaScript or Python until you study Lisp. It’s common to see what’s essentially C or Java written in python syntax.
And let’s be fair nobody in the 90s understood C++ including pg.
Some benefits of Lisp that are less commonly reported:
Lisp let's you imagine a way of doing something, and then just implement that with nothing getting in your way. You can go from something half-assed to a whole-ass in no time. The type system lets you develop quickly and optimize when it's time. You can run interpreted, compiled, and even optimize the rendered machine code.
Image-based computing let's you skip the database for a lot of things as you can just read data into an image at startup and have, essentially, a fast in-memory DB.
Language-oriented programming allows you to extend Lisp with nouns and verbs related to your problem domain. So you can express the solution in terms domain experts already understand.
The hugely underappreciated condition system is far more than a way to do error handling, but can respond to events and do flow control.
Modernizing Common Lisp is something everyone does. We each curate our own distribution from Quicklisp and develop in that. More coordination there would be welcome, but we're not writing in your dad's Lisp.
A kitchen-sink, multi-paradigm language is ideal compared to a more opinionated system in which there are official paved paths to do things. You get to spend more time figuring out what needs to be done than figuring out the official way to do it. If there is a downside to that, it's that you have to take apart enough flashlights to figure out how things work and can't just NPM somebody's afternoon project over and hope it works.
When you have a perfect spec and you just need to implement it efficiently, think C. When you're not totally sure what you're doing, want to explore the problem space and not paint yourself into a corner, think Lisp. Actually, think Lisp anyway because specs are always wrong, so you're always exploring. Lisp makes for living systems that grow and change organically. C is for making complicated but dead automata.
Lisp makes metaprogramming easy via macros. It could be argued meta programming can be done in any modern language, but because lisp’s code is a data structure, metaprogramming in lisp is as easy as changing a value in an array, hash, or dictionary (whatever data structure you’re familiar with, it’s that easy), as opposed to regexing strings, escaping correctly, and other finicky and error-prone hijinks other (non homoiconic) languages necessitate.
The other thing lisp does well is its REPL. I’m not so familiar with it but AFAIK it’s like an interpreter and debugger combined into one, giving a very nice developer experience.
I chose common lisp. It's very expressive which means my development velocity was very high with minimal boilerplate. I love being able to build functions up from the inside out in the REPL, and the DX of doing this in emacs is awesome because of the mature support built into the editor.
The library ecosystem in lisp is not stellar, and I think this is lisp's biggest shortcoming. With that said, there's still robust libraries for threading, networking, JSON fuckery, and (importantly for my use case) statistical analysis.
Lisp is great and the SBCL compiler produces fast code.
The other thing that pg keeps hammering on about is Lisp Macros which I am not well versed enough to properly say how good of a feature they are. The alternatives most other languages have are much less powerful, but at the same time it seems that macros are very... abstract-y?
Modern software design thinking says that abstractions should be avoided when possible because they make the code harder to grasp for people not familiar with the abstraction on top of making the code too rigid and hard to modify once new requirements come in.
The alternatives to Lisp Macros like codegen, method annotations, Rust Macros. Are all much less powerful and usually harder to create, but maybe that is a good thing.
Consider Lisp and Haskell. When a Lisp user looks at Haskell, they know they're looking down. No macros? How can you get anything done in Haskell? But when a Haskell user looks at Lisp, they also know they're looking down. No decent type system? How can you get anything done in Lisp?
You have two languages, both looking down at each other, and both knowing why they're looking down. Something is wrong.
I claim that what's wrong is the idea that languages can be ranked on a single-dimensional axis labeled "power". To those who think it can, I ask, power for what? For writing programs? Which programs? And don't say "general programs". I've never written a general program in my life. I've written a bunch of specific ones, though.
What I actually care about is power for writing the specific program I'm trying to write. (Why would I care about power for writing the programs that I'm not trying to write?)
So the really useful question is, what makes it hard to write the program that I'm trying to write? (I don't mean what problem the program is trying to solve. I mean, what things make it hard to write a program that solves that problem.) This may be a variety of things - team skills, algorithms, memory management, thread management, and so on. You can think of those things as a vector in a multidimensional space. Then, think of languages as a tree in that multidimensional space. Pick the language that is the branch that reaches the furthest in the direction of the vector that is the things that make it hard to write the program. (Oh, and libraries. The availability of libraries can make a huge difference.)
Or, to state it without the metaphor: Pick the language that makes it easiest to write the program you're trying to write.
For example, say you're Google, and you're writing programs that have to have large numbers of threads, and have lots of dynamic memory, and will be long-lived and worked on by a large number of people who have to be brought up to speed on the existing code. Go is a language that addresses that set of problems. Neither Lisp nor Haskell would work as well. They may be "more powerful" in some sense, but they're not more powerful for that program.
1: https://en.wikipedia.org/wiki/OODA_loop To a certain extent this is tooling, but CL, at least (I can't speak for other members of the Lisp family) was designed with a peculiar combination of dynamism and efficiency in mind that nobody else does. It's in many ways more dynamic than Python and yet easier to write an efficient compiler for, which is quite the feat.
The downside is that it's a difficult tool to become fluid in and so it's difficult to find experienced hackers, so scaling the team is going to be slower than with other, more popular languages.
> And if so, _which_ Lisp and why?
Are there any other production-ready Lisps except of CL?
Julia is not as elegant as Lisp and doesn't offer as many powerful language constructs. But it is a very cleanly designed and well thought-out language that is fun to work in. I especially love its metaprogramming abilities (heavily drawn from Lisp), which include "real" macros.
In the end, the reason I use Julia over Lisp is the community and the ecosystem. As an ecological modeller, Julia is something my colleagues have at least heard of and that I can convince a few to use (instead of R and Netlogo). I fear that I could never convince anybody to use or contribute to a software written CL. Also, the Julia standard library is just very well put together and there are more packages available for the things that I need to do, compared to CL.
I still love CL as a language and sometimes toy with the idea of going back to it, but for the moment, practicality beats purity.
A few years ago I had a choice to learn lisp or smalltalk. It was a coin flip but I went for smalltalk
I've never seen lisp in production, but if I had I'm sure I'd be like, ok, we use lisp here.
Lisp is just a tool, and some people with find it ergonomic and powerful, and others will find it awkward and cumbersome. Use the tools that make you happy and productive and don't worry too much about trying to find the best in an ambiguous space.
not all of them have CL's set of features: excellent long-live REPL, image-based development, compiling to binaries, the condition system, the interactive debugger and excellent developper tools (resumable stack frames, never re-run a long operation from zero [1]), the whole language at compile-time, stable syntax, extensible syntax, stable ecosystem, fast implementations, numeric tower, first-class interop to Java (LispWorks, ABCL), commercial support, mobile platforms delivery (LispWorks)…
New CL implementations get created (CLASP, CL on C++ with LLVM or NPT, in C) and CL compilers and libraries evolve too [2]. SBCL is now callable as a shared library, it got a SIMD module, rumors say a bear party after ELS 2024 escalated and that they are adding coroutines soon. ECL is targetting WASM, all other major implementations had new releases.
New libraries unlock new possibilities (Coalton: Haskell-like typing on top of CL), companies still pick CL today (CL is all the rage in quantum computing?),
the community creates more resources and more tools such as, last but not least, new editor plugins o/ https://lispcookbook.github.io/cl-cookbook/editor-support.ht... (VSCode, Pulsar, Jetbrains, Sublime, Jupyter… Lem of course etc)
[1]: https://www.youtube.com/watch?v=jBBS4FeY7XM
[2]: https://lisp-journey.gitlab.io/blog/these-years-in-common-li...
But that isn't very powerful compared to a problem domain expert working in a language they like. If someone doesn't like lisp then they shouldn't use it and they'll probably do well.
The ideal Lisp for me would be one with an integrated IDE that comes with "batteries included", yet is smaller and more elegant than Racket. I don't care whether cross-compiled or interpreted, it should allow me to write native GUI applications with all the bells and whistles and one-click deploy them on all major platforms, including mobile and browsers. Basically, something like what REALbasic used to be when it was still affordable shareware, but for more platforms and with Lisp as the language.
About the "top end of the power spectrum", after discovering the array languages (of which for me APL is the one that clicks), I have come to believe that they are at the top end, higher than the lisps. I was disappointed when I found out that those languages existed before I was born, that I studied CS and thought I knew about all the programming paradigms, but I had still missed those.
Caveat I have never written Lisp professionally, tho I do use function generating functions when I can. E.g. an API with 20 methods that all do the same basic thing with little variations. I will write one big function generating function that takes parameters for all the variations and then just assign the dynamic output to the expected API name (in Python). I have a go test suite where it runs the AST over a directory and extracts all the functions with a certain name and generate a list of tests then all my components run the dame test suite but calling different implementations for service calls. Or test generators where you specify parameters like expected output and input and implementation details and get an array of callable tests in nodeJS. So my code isn't trivial for outsiders to read but easier than a very DSL macrofied lisp would be.
We have used these macros to save a lot of time and money (multiple developer salaries). One example is by implementing a DSL for numeric code that allowed researchers to write in Python-like syntax, but would get transformed into a DAG for longer-running jobs that could be evaluated much more efficiently + in parallel.
We could not make something nearly as performant in Python – we spent hundreds of developer hours trying, because we already had a lot of working code and were trying to make it faster/cheaper to run. Nobody wants to rewrite a large, working codebase. Yes, we tried numba, pypy, custom C extensions etc. – in the end a Julia prototype written in one week significantly outperformed all of them. Three years later we still use Julia for all our numeric code.
Writing in LISP can benefit from, I believe, a kind of psychological stance of wabi-sabi (non-perfectionism), for (barring exotic LISPs with sophisticated type systems), it will always be more challenging to refactor and get things just right than with languages like Rust.
Given the pressure of writing code for firms that can demand we are perfect or pay the price [being on call, etc.] (ie working for perfectionistic firms with perfectionistic customers — it’s a recursive problem) also can push us towards languages with powerful compilers.
Then again, maybe we need to invert things — use the languages we love that are not “perfect” but are so easy to love; for perhaps that love can just as recursively emanate outwards as would-be perfectionism…
Unless you already have a rare Lisp job, or you're retired, spend your time on things more employable.
> Perusing the various search results on 'CL vs Scheme / Racket' had me personally more convinced of CL for heavy-duty "real-world",
CL has more wins on this, but Scheme has, on at least one occasion, been used for large, important, rock-solid stuff, and super-productively. It comes down to the individual programmer(s).
> all the other languages have kept picking up nearly everything once-cutting-edge about Lisp
Much of the relative value of Lisps nowadays is the communities. They're generally made up of people who would be doing this even if software weren't a well-paying job. And there's relatively little BS to wade through (like explodes in all directions around popular employable platforms), since most BS follows the money.
An underrated Hickey talk is his Effective Programs one: https://github.com/matthiasn/talk-transcripts/blob/master/Hi... It's another entry in my personal (and useless time-wasting) fascination with wondering what could have been if CL won in the 90s. No need for Java to come around and "get C++ programmers half-way to CL", with excruciatingly slow followups year after year that finally by Java 8 and after got us something (when also using additional things like a giant IDE with good refactoring tools and simple code generation (no manually writing getters/setters or managing a giant list of imports), Spring, JRebel, and sometimes AOP frameworks) actually not too unpleasing compared to a CL experience. But most things were always there in the CL experience in the 90s, especially if you could shell out for the commercial systems, it really is just Java playing catch-up. And then you get in that talk and elsewhere a story about how non-technical decision making didn't even give Lisp systems a chance in production, so as a last resort Lisp was put on top of the JVM which was accepted. Clojure is a great achievement, and ABCL came too late, but it's kind of a shame either has to exist. Still, the thousands upon thousands of man-years have been spent, and because of that Lisp's advantages are fewer.
However the Lisp world itself hasn't been static since the 90s either, and so the counter-advantages of vastly larger ecosystems have grown fewer as well. For example, it took until as late as 2010, but finally quicklisp was made and the problem of libraries and finding/getting them was dramatically lessened. For comparison, Perl's CPAN is from the 90s, Python's PyPI is from 2003, Maven 2004... Lisp was late. But the last 10 years of Lisp seem to be going a lot better than the 10 before, and while there are still problems, they don't really look like uniquely Lisp problems, and supposedly unique ones like supposedly not being able to do infix arithmetic are addressed with slow education to the contrary (while I'm here might as well link for the nth time, the solution available since 1993: https://github.com/quil-lang/cmu-infix). It was such education years ago from a HN comment that made me give CL another look and then never look back at Clojure/Scheme/etc. (though Java is still my preferred breadwinner should I find myself suddenly needing more income). (The comment was simple, just showcasing that yes in fact CL has typing support, the compiler is available at runtime, and with SBCL the compiler can report not just about wrong types / typos but dead code elimination and failures to apply optimizations. Cool stuff. Made me realize I never knew CL at all. Even CLOS, the OOP system, had some pleasant surprises when I finally got around to learning it and seeing it's far more than what other langs offer.) Meanwhile the advantages are still there.
I don't think lisps strengths can be matched by piecemeal feature adoption. The GC, the REPL, the ability to reload code in a running process, the macro system, the minimal syntax - it _all_ comes together and it shapes your development process.
Case Study: My finest work was written in Clojure, I'm pretty sure I wouldn't have a clue how to achieve the same in Go, Rust or whatever else the broad programmer base is typically interested in.
The single most challenging project I've ever worked in, domain-wise, had so much churn in the spec because we laid the track as we went. We changed data storage products (Postgres -> Datomic), we frequently re-wrote the UI (back when ReactJS was popular and Redux etc were being proposed). We re-architected mid-way to use event sourcing as we needed to be able to recreate the system state at historical points in time. And so on. A lot of dysfunction as well. But Clojure was probably the biggest reason we managed to limp along. I can't imagine having to r-earchitect so furiously while pleasing a borrow-checker which tends to make refactoring much more extensive.
Recommendation: If you want to give it an honest shake, but you're afraid (as I read it) of painting yourself into a corner, consider Clojure. Clojure is a fringe language, like all Lisps, but rides on top of two massive ecosystems: the JVM (Clojure) and Node (ClojureJS). Additionally, if Emacs/(Neo)vim is not your thing, there's an IntelliJ plugin, Cursive, and it all works _beautifully_.
Additionally, the community, and its designer, have many interesting takes on software design which I found very educational.
Where I think Lisp shines ?
* When things are _complex_ or projects drift, revise and change often. The ability to be functional, to build custom DSLs, border-line custom interpreters with way less effort pays off.
* If I had to teach programming to _complete_ newcomers, gun to my head
* When valuing _genuinely_ different perspectives. Picking up Lisp offers more than yet another C-like language
* When you have limited mental capacity for more stuff, i.e. you have a family life and a complex, demanding job already - maybe pick something that won't drag you through borrow-checker nonsense or category theory.
Observation, why programming in the large tend to suck: Why normal languages kind of suck. --
Essentially, enterprise code tends to require a lot of lines, those lines can have bugs. People want to reuse code (write less code, really), so they start writing abstractions, leveraging the many different (flawed) tools their language provides. These are typically a _lot_ harder than Lisps function composition and macros (say, Rust macros, Python metaclasses+inheritance), this means complexity goes through the roof. Over time requirements shift and abstractions become less suited, now you're fighting on two fronts. I would say you either need Lisps ability to write terse code (macros being the unique feature), or you need a _good_ code generation tool. And we seem to have neither in most projects.
Conversely: where is Lisp less useful ? --
The well-paved roads where a popular language has a set of finely developed libraries for solving your issues. But once your domain itself, your business logic, gets complex, there won't be a NPM, PyPI, or crates.io to save you.
---
[Disclaimer: Below is mostly unsubstantiated, unscientific, personal, and perhaps even somewhat incoherent opinion.]
This might sound jaded, but the software industry and its incentives are far more perverse than they were 30, even 10 years ago. By and large, writing new, efficient, and/or interesting programs is not valued culturally among software engineers as a trade group. The number of software engineers "on the market" is at record highs, and the vast majority of them speedran any% a college degree or bootcamp and are looking for their entry-level six-figure Senior Software Engineer role straight away. The mere suggestion of taking a step back to look at language offerings, like Lisp and the paradigms it enables, is not helpful toward that pursuit. And even if someone does take that step back to take in what has been available for 7 decades, the average employer, too, won't care whatsoever. Lisp might provide a plethora of benefits, but the incentive structure of modern day industrial software engineering has all but eliminated the realization of those benefits in practice.
Maybe there's some comparison to the coal/oil industry vs nuclear to be made. If you can go out in a limb and agree that nuclear is better in almost all ways, why do we continue to depend on and bolster the coal and oil industry? Why is the market for coal/oil bigger? Why do we keep hearing news articles about this and that nuclear plant getting shuttered? At this point, it's almost nothing to do with the actual intrinsic pros and cons of each energy source, but rather the culture, politics, and infrastructure in place to support each one.
Lisp was born out of an idea that we can think about programs symbolically, and the Lisp of today carries that ethos in many ways. Even ignoring general debate about functional vs OO vs whatever, the idea that programs ought to first and foremost express ideas declaratively—and that we should have macros to let the programmer carry that torch into their domain of expertise—just hasn't caught on. In contrast, mechanical, line-by-line thinking still dominates every popular general purpose language (and I argue, will continue to).
If your goal is to write a program that does something new and reliably, then Lisp is still ahead in my opinion. It's still, to me, the most efficient programmer's programming language (similar to how jazz might be called the musician's music) that is accessible and supported today. It's still an excellent choice for building many kinds of products.
If you want a job in which you'll move slop around the internet superhighway, you will find one faster by writing Python.