Can LLM's generate a sudoku that has not been created before?

or can a LLM generate a sudoku which it has not been trained with?

LLM's are neural networks that are trained with a massive amount of data. Their training data can contain all the publicly available sudokus that in the internet.

So, can it really be creative enough to come up with any sudoku that it doesn't even see before?

I would expect that they could not because they are not trained with. Because they are just next word predicter.