I wonder if implementing a garbage collector at system level in the kernel an letting those languages using that instead of each providing its own garbage collector might result in better performance.
You’d have to tighten what such processes can do with memory to make that not be a gaping security hole.
Certainly, anything ‘unsafe’ would have to be forbidden.
If you think that’s worth it and try to close that hole, you’ll find out that is harder than the relatively simple “ordinary processes cannot access kernel memory at all” that we have now, increasing the risk of having a bug that opens up the security hole.
Also, running a single garbage collector gives up the option of using different ones or ones with different settings for different processes. For short-running ones, for example, not doing any garbage collection ever, leaving it to the OS to release all memory at program exit time can be the optimal approach.
Other processes might be better of with a collector with low latency, even if that means giving up some throughput, or vice versa.
So no, running a kernel garbage collector for use by non-kernel processes isn’t a good idea.
Providing one as a library might be, but in practice, people cannot agree on what the API for such a garbage collection library should be because there are too many differences between languages to make that work for every language. For example, some languages allow internal pointers to objects to prevent those objects from being garbage collected, but others require keeping around a pointer to the start of the object. Also, some languages will produce relatively little garbage (for example by having GC-ed classes and non-GC-ed structs and having a culture of using structs where possible), others may make everything a GC-ed object, including small integers and IEEE floats.