Deep learning enabled GUI automation tools?

Question

I was wondering, have anybody found really good tools, potentially cross-platform for GUI automation but which leverages image detection from Computer vision models, say convolutional neural networks?How about open-source alternatives?

RMPR · Accepted Answer

Not really a tool, but Python is widely used for deep learning so, you can combine Pytorch, Tensorflow, [insert your DL framework here] with Pyautogui[1] to achieve exactly what you're asking. If you feel Pyautogui is too much "manual", I built a kind of frontend for it [2].
[1]: https://github.com/asweigart/pyautogui
[2]: https://github.com/rmpr/atbswp