Is testing problem solving morally wrong since it is not algorithmic?

I don't agree that it's not algorithmic. In the most generic sense of "problem solving" it's a binary search to find the issue. You test the most likely and continue to narrow down from there.

When you have specific examples for problem solving as in solving a problem with a specific thing those can very much be algorithmic. Simply look at any troubleshooting guides that many industries use to train their technicians with. After a while in a given area experience can often solve a problem faster than the guide procedure but that does not invalidate the procedure in the guide.

George Pólya would disagree. Problem solving does not qualify as strictly algorithmic, because it describes an approach to defining and attacking problems combined with domain expertise and experience rather than a series of steps to apply to every problem. But a person can learn problem solving skills and practice to get better at it.

I don't see the moral issue. Determining if someone performs better or worse at a task necessary for a job means making a practical decision, not a moral judgment.

Can people do it better than other people?

Can you be taught to do it better?

Then it's testable. It may be quite subjective in evaluation, but it's testable.

Simply following a process is not the only thing to be learned! Computers can do that.

If the only things we expect students to do are algorithmic we might as well train them to do nothing.