https://stackoverflow.com/questions/36416532/lisp-in-ai-self-modifying-code-and-domain-specific-levels-of-abstraction
I was wondering if this is useful for AI that can modify (improve) its own code. Am I missing something?
The first thing that comes to mind is the general idea that for any kind of continually improving system (the obvious analogy would be an evolutionary algorithm[0] of some sort) you need a "fitness function" that dictates whether or not the system is, in fact, improving. And defining the fitness function and the representation that maps from your code to the thing you're trying to improve, is often the hardest part.
FWIW, people have explored the intersection of evolutionary algorithms and code generation under the rubric of "Genetic Programming"[1][2][3]. This isn't necessarily the only (or best) such approach, but it's one that has been explored quite extensively and where there is a large body of literature to review. You might find some of that stuff interesting, vis-a-vis this overall topic.
[0]: https://en.wikipedia.org/wiki/Evolutionary_algorithm
[1]: https://en.wikipedia.org/wiki/Genetic_programming
Currently fashionable machine learning approaches don't generate general purpose code but instead determine the coefficients for a matrix equation. Notably this kind of "program" is guaranteed to finish, which limits what it can do algorithmically, see
https://en.wikipedia.org/wiki/BlooP_and_FlooP
but that kind of program can run efficiently on a GPU or similar device because it is a bunch of repetitive math.
There are some older algorithms like
https://en.wikipedia.org/wiki/C4.5_algorithm
which "learn" rules that look more like ordinary code.
Lisp implementations typically give running programs access to the compiler, but that has nothing to do with macros, and then again, other languages do as well.