HACKER Q&A
📣 oulipo

An executable-Markdown for laptop automation


I was looking at someone here posting an automation framework for osX (https://github.com/elyase/screenium), and wondering the following, but perhaps someone knows an existing solution for that?

It would be a kind of open-source, text-based (eg recipes are .md with instructions) version of KeyboardMaestro!

I'd love to see such an "open automation" format (could even be more general than pure software, could also automate your IoT or whatever, through extensions)

eg you could have a file "Type my bank login password" for bank websites which doesn't let you use keyboard input but force you to click on stuff, like a self-documented script using .md with code

    # Type my bank login password
    
    ## Trigger
    ```trigger:hotkey
    key: cmd+l
    filter: frontmost-app=Chrome and chrome.tab.url=~mybank.com/login
    ```
    
    ## Deps
    ```ensure-deps
    shell-runner>=1.*
    screen-ocr>=1.*
    python-runner>=1.*
    ```
    Ensure that my system has the proper extensions for the framework, to run all tasks
    
    ## What it does
    This automation lets me input my password in a "click-only" input for my lousy bank UI
    
    ```run:shell /bin/sh:capture-output=password
    echo $(op --vault personal --site mybank)
    ```
    (the above runs the shell script and captures the output as a "password" variable I can use in other scripts below)
    
    ```run:screen-ocr:capture-output=ocr-result
    window:chrome
    ```
    
    ...go on scripting using typescript/python to locate the numbers in the ocr-result


  👤 not_your_vase Accepted Answer ✓
Look at Jupyter Notebooks. It has a bit different goal, but the end result is pretty much what you describe.

If you are not familiar with it: it's a special kind of IDE primarily for Python (though not exclusively), that allows embedding markdown documentation in the middle of the code. Also, code is split into steps that can be repeated... it's easier to try (or watch some videos), then explaining.

But it results in the output you just described.