(How does the app work? Think of the application as a file indexing system: you specify a folder on your local hard drive and the program will analyze the files (might take up to 30 minutes) and store metadata for it in a search-optimized datastructure.)
From what I understand most applications also separate frontend and backend layers and let the processes communicate via message passing over some form of channel and protocol. The solutions I found include Unix Sockets with gRPC (https://docs.microsoft.com/en-us/aspnet/core/grpc/interprocess?view=aspnetcore-6.0), local HTTP Servers with a REST API or gRPC, even JSON over stdin/stdout (https://github.com/xi-editor/xi-editor). Solutions for datastorage include custom file formats, sqlite databases or bundled mongodb instances into the main application.
I figured out the backend part with analysis running multithreaded and writing into a sqlite database. For the IPC I spawn a gRPC Server which clients connect to over HTTP/2. That way I compile a platform agnostic backend and platform specific frontends. It all works okish but I feel like it's unnecessarily complicated and there could be a common strategy I am missing.
*TL;DR: What are modern architectures for desktop applications equivalent to a microservice architecture with backend/frontend separation, message queueing, pub/sub, authentication, ...?*
In general it sounds like you're working too hard to introduce a strict process separation between the UI and the backend. A desktop application won't have multiple clients connecting to the backend, so you can run everything in one process. That's the use case for which SQLite is designed.
Your UI framework may introduce some level of process separation anyway: if the UI is a WebView or Electron, then it's going to run the renderer in its own process. So there's no need to go crazy with splitting up into tiny little services.
Event handlers, model/view, independent-ish subsystems, frontend/backend are some typical patterns in desktop applications. Some of it depends on your framework - much of the architecture will be defined by the framework. The default should be doing things in the canonical way for the framework.
But it's very unusual to split it over an RPC/IPC boundary - it's usually just linked in.
Xi had specific design goals of allowing multiple independently developed front ends. There are use-cases for this type of development but it's largely not worth the effort (YAGNI).
The typical application architecture is going to depend on your use-cases. I.e a db app is going structured different from a game or a CAD program.
Communication between threads could be as simple as global variables + a mutex lock.
Two problems to solve:
- Communication from UI thread to backend thread
- Communication from backend thread to UI thread
For communicating to the backend thread, I like using event flags but other people like using mailboxes. Both work. With event flags, you set a bit (for instance, backend_thread.start = True) and if the backend thread is sleeping, it gets woken up by the OS scheduler.
UI frameworks often provide a function to run your code in the UI thread, so your indexer could use that to periodically update the UI. For example you could have an Update() function that reads the global variables defining the state of the indexer and updates every UI element to match, and then the indexer periodically schedules Update to run on the main thread when it's doing its work.
In my work there's typically a state machine that coordinates UI + backend. It might start with state = BACKEND_STOPPED. Then when you press the start button, transitions to state = BACKEND_RUNNING and notifies the backend thread using an event flag. When the backend thread finishes, it can transition back to BACKEND_STOPPED and schedule an update to the UI.
----
Another pattern I've seen is using the database for communication. You might have a "jobs" table in sqlite with indexing jobs to perform. The front-end adds "jobs" to the table and notifies the backend thread to wake up. The back-end processes jobs and updates their status and notifies the UI thread when the status changes.
I do like the mature pattern of having a number of CLI programs that can be run, and an additional GUI that can run them as well, for non-expert or infrequent users. Have a log that shows what commands are called by the GUI. This gives powerful automation potential for free. Look at Handbrake GUI wrappers for technical inspiration, if not UX design.
Then you can do whatever IPC you like, can be as simple as reading from stdin/out or maybe Unix sockets before resorting to TCP/IP. If work has to be done outside of the user session, write a service/systemd module definition with one of the CLIs.
Re: Data, as others mentioned sqlite can stretch a lot farther than was commonly believed until a short while ago. Try before resorting to servers.
> running multithreaded and writing into a sqlite database.
sqlite doesn't support multithreaded writes, so you'll have to use a queue or something like that. Qt/Gtk ships with that as well.
I suggest you considers structuring the reusable pieces as a library and then reaping the benefits of strong typing and absurdly low latency when building the non reusable UI.
Akka might be interesting if you were a team but you really can just use regular old method calls.
The architecture for the “backend” isn’t different from a web or other app, the difference being that you don’t need to expose a api/rpc surface of any kind that isn’t regular methods.
Set up some good interfaces and/or view models. This will depend heavily on which front end. An imgui app will have very different preferences from a databound ui and so on. Very hard to make recommendations about this part without knowing what frameworks will be used.
But don’t pretend a desktop app is a web app.
Within the main process I actually have it call a Go process to handle all the heavy lifting that goes on in DataStation like database queries, HTTP requests, etc. Communication between the Nodejs main process and the Go process just happens in temp files. All long-term data is stored in SQLite.
Happy to talk more about it if you want! My email is in my profile.
In the mobile space, MVVM is pretty popular.
definitely not on the desktop, no
They communicated over IPC.
This was design this way on purpose, because the back end processes were existing applications. Rather than re writing them entirely, the developers added IPC communication.
Also, they believed it was better for a separate process to crash and just restart rather than showing it to the user and confusing them (the person delivering products to a grocery store).
The whole thing was a convoluted mess.
My advice to you is start simple. You don’t need multiple processes unless there’s a damn good reason. Everything should be a single process.
But again, separating front & backend for a desktop application, is kinda weird.
What’s your frontend used for? If the progress bar is everything then maybe you should just build a command line app.