What's the next evolution of the GUI OS?

Insufferable answer: Emacs does this to some degree by overlaying "minor-modes" of function on top of one another, even when visiting the same file.

e.g. You have a TSV file which you might want to align visually and edit multiple fields simultaneously, so you enable `tsv-mode', and then you enable `multiple-cursors` and you have the desired editing functionality on that file.

But to answer your first question: I genuinely envision my ideal future desktop as being some smart-glasses kind of thing, where I can type by thinking. Right now, what my mind wants and what my hands want are often aligned, but my mind wanders. If I can type at the speed that I think, I have the impression that this will greatly increase my overall focus and mind-discipline.

(I can also envision employers forcing their workers to install intrusive mind-type monitoring apps, to ensure that they're actually working and not just drooling into space, but that's perhaps a separate discussion.)

I need a separate window.

My WFH environment I use every day has three screens stitched together for 7680x1440. On the left third is my primary remote desktop app that connects me to work with a window for my old desktop PC. The middle third has 320pixels reserved on the right for context-sensitve notes, then splits the rest in half with the left half being my timesheeting/ticketing app and the right half being a decent plain text editor as a scratch pad / diary combo. On top of that I have a set of tabbed windows (using Groupy) from various apps like a file manager, browser, Slack, email, or command prompt. The right third has the left 320 pixels reserved for a music player (Winamp), VOIP softphone, and clipboard manager. The rest of that portion is another set of tabbed windows usually at least including a second remote desktop session to my VDI and another browser window. My browser and command prompt are tabbed. My file manager is dual-pane and tabbed with a preview pane.

What you're describing sounds like what was tried and failed with Windows 8.

Does the desktop contain programs or documents? You switch from one to other other half way through your second paragraph.

This sounds a bit like a concept that was popular in the 90s, I think it was called document-centric computing. Microsoft's implementation was called OLE and Apple's/IBMs was called OpenDoc.

The idea in these systems was that you don't really work with apps anymore, just generic document canvases into which you can drop arbitrary components. There is also a generic "compound" document format which the components can share in a hierarchical manner, and controls/menus/toolbars appear and disappear on demand.

This vision was somewhat impure on Windows, in that there was no generic document app - instead you were expected to use one of the Office apps as the canvas and then embed objects into those documents. OpenDoc in contrast had a sort of blank canvas system iirc, and if you wanted to work on a word processor document then you started a generic document and dropped a rich text area into it. A bit closer to visual programming than the app-centric model.

Suffice it to say, this model didn't win out. Embedding complex objects into compound documents turned out to have a lot of practical problems, OpenDoc got de-staffed when Jobs came back to Apple and OLE was steadily de-emphasized in Windows development. It was never that popular anyway, and was seen mostly as an exotic feature you had to do if you wanted your app to be embeddable in Word or Excel. The technology lived on somewhat in the form of OCX controls, which were for a time Microsoft's solution for shared libraries that you could use in a visual form designer and across languages.

You could try again but you'd probably want to study those earlier systems to learn why they didn't take off, and bring a new spin to the idea.

One trend I'd like to see is more dense UIs. You can tell a lot of web and desktop GUIs are designed by developers with wide monitors because when you go to use those apps on a laptop with a small screen they are hard to use and you lose context from scrolling so much.

I think GUI has evolved too far, what I want is devolution, not back to what came before, but back to other basics.

We're in a terrible state, some odd and ugly place between native and online, where even native apps are picking the worst from online and online is denied access from the best of native.

I want an operating system GUI that dictates the available widgets to the program, and hands me control of their presentation, I don't want applications, online nor native to be able to dictate their own look, only for them to present their available functionality to the OS, and have the OS compose the screens for me, probably guided by my predefined preferences, sort of how HTML could have been if CSS hadn't ruined the day, sort of how native winforms programs used to be done.

A concrete example is things that can save a file, I don't want the program to dictate the gui for this, I want it to keep the OS updated "I have these documents open which have been changed since last save" or "There are these things which can be printed", and have the OS dictate GUI for activating that functionality.. It used to LOOK that way, for a lot of things, you'd go to the file menubar, and there you would open/save/print your file, this was easy to learn and very consistent..

Currently, the most consistent interface experience on my computer is bash in xterm, in elinks, websites look pretty consistent, if they're well designed.

But if I open firefox, chrome, gimp and kate, nothing looks or feels the same.. Then I open blender and a fifth kind of interface appears. No thanks.

So I sort of agree with you that it'd be nice if we did away with the window metaphor to some degree.. But I was also no fan of the way Mac had the "god bar" at top (which I believe Ubuntu did too) terrible idea..

For now, the least crappy desktop experience for me is i3, but it does not change anything fundamental, it's still program windows, free to do whatever they want.

But, for some things, the most futuristic version of the environment is the one that does not need a screen.. The tasks that are silently done for you without needing you to transfer any significant amount of information between you and the machine, or being able to do so on another level than a screen (brain machine interfaces and whatnot)

I think, long term, apps will disappear and we will get APIs and algorithms that are controlled by an AI engine. Apps are data silos that prevent an AI from doing the things it could.

You should be able to tell the AI engine something like "as soon as the weather forecast predicts snow, order a snow blower from Amazon". The AI can only do this if it has access to weather data and an API from Amazon it can control to get you the snow blower. It should also be able to check your calendar, because you can't accept the delivery if you are on vacation.

Apps are silos that prevents the AI from doing these things. At best the AI could use the equivalent of screen scraping to use existing apps, but eventually all apps should be replaced by APIs that just extend the capabilities of the AI engine.

And what about GUIs? I think they will be pretty irrelevant. The AI engine should be able to render your data any way you want. Think Dall-E. There will certainly be designs to chose from, but you should be able to customize them any way you want ("remove that italic button, I never need it").

I like this idea, but as a designer I foresee challenges in optimizing what is visible as you add new "functions" when needed. for example, editing a doc, then editing an image to add to the doc. Traditionally each function would live in an app with gui optimized to that function (Word vs Photoshop). Getting these functions to coexist when both are needed would get difficult to display, as image editing has a screen full of tools usually. Not saying it cant be done! I recall Microsoft had a similar idea with OLE (object linking and embedding), where the menus and tools would change depending on what the user clicks on within what is currently displayed. (See https://en.m.wikipedia.org/wiki/Object_Linking_and_Embedding )

I think OS and their GUI's are in a fine place. People have been using file/window based computing for many decades now and it would be hard for any large change to that paradigm to take root.

Most everyone here on HN should be considered a power user and are in the minority of personal computing users statistically. A good idea is to give people options to configure things how they like so power users can feel more powerful. I think the reality is most people fool themselves with any perceived productivity benefit on window/screen management - its just a comfort feeling more than productivity. If people truly wanted to be more productive they'd spend less time on internet forums :D

Frankly, I would like to see better use of multiple desktops. The idea would be to link windows to a primary application - like browser tabs relating to what your working on in your IDE along with terminal screens, etc. Then swap around.

Honestly, I still wish for alot of the aspects of the 1990s tried to sell like being able to develop as easily as on a NeXT workstation (without objective-c) or the entire concepts MSFT was trying to do with Active Desktop.

We see it all the time where people get excited about whatever screen is on XYZ show until you learn the reality of the screens being "Hollywood OS". Like most of the screens seen on mac computers during the 1990s, it was clear and visually functional for whatever they portrayed - like ever seen a usenet browser like shown in Mission Impossible or The Saint? Neither have I. But it would have made it a hell of a lot more user friendly. And usenet still exists!

Just saying, there is a time when less is more, but other times you need it to be just a smidge more than it is today.

I am writing half an OS, the front half. It displays a full GUI in the browser. The goal is native distribution over a network with full decentralization.

For example in Windows you can drag and drop file(s) between windows to move those file(s) between directories. In my application you can achieve all the same interaction which may move those file(s) to a different directory, different personal device, or different user. The experience is the same whether the application is running on Linux, Windows, or OSX.

In this application when you double click a file, in the browser to execute it, the OS default application associated with that file type opens just like in the native OS. If that file is on another personal device or another user it is copied to the local device for execution.

In order to minimize clutter I have a button to minimize all windows to a tray, but that is hardly original. Soon I will start working on a terminal emulator (also displaying in the browser) for remote terminal access to personal devices.

I know many people don’t like interfaces written in JavaScript but it is portable to things like Electron and Tauri. The test automation executes in less than 10 seconds on just the local device or just under 2 minutes for full peer to peer testing. Because this is all JavaScript there is a bunch of low level concerns I don’t have to reinvent, like accessibility, but the socket based decentralization does require new transmission handling and some original security work.

So you're saying you want to go back to MDI interfaces? I mean, we could, but it seems to have decisively lost the mindshare battle. I haven't seen any applications use it in ages.

Do you want to get rid of windows or 'applications'?

One problem with current systems, and even more so with mobile, is that it is app-centric, and your data is locked into apps, rather than being something you can manipulate directly and independently.

I had a similar idea in the 90s/2000s which I called 'data-centric' (which now means something different, in gamedev).

The idea was that instead of a world of apps, which contained the data items you cared about (emails, documents, etc), or even files, that were tied to applications, you'd have more granular data items and collections of items, and ways to organise and present them. And then functions that operate on different types of data item, or collections thereof.

An 'application' would be just a collection of functions, and completely customizable by just adding new functions or removing ones you didn't like, at least at the UI level.

There have been a number of projects roughly related to this idea I think, but things like OLE don't count IMHO. That was less like 'object' embedding , and more 'entire app embedding', and in any case 'embedding' is not really the point. None of these projects have succeeded, even though I think the idea still has merit.

I can/could envisage many problems, eg data type classification/standards, incompatibility, standards bloat, etc, but the latest AI techniques might be a way to work through some of those issues.

Microsoft did this in the early 90s . It was called OLE (Object Linking and Embedding). It was a nightmare of different UIs for different purposes all fighting over the part they edited

You need the right OS underneath. Ideally something based on persistent objects like OS/400 and with the networking aspect of Plan9 so that local and remote resources can be treated the same.

The GUI would need a similarly smart approach and given an OO persistance layer I'd say that applications would just be objects that composed other objects - the document object would be your wordprocessor and it would be good at arranging and presenting other types of objects into 2d representations at A4 size.

The web browser would presumably just be a superset of the document object or might not even be needed at all as a web page is just another kind of document.

You'd still need a "fresh sheet of paper" concept but also a history so you could go back and search for what you were looking at/working on previously. Also different contexts - like all the stuff you need for one bit of work and another context for some other kind or play/work. I don't think windows are the worst way of doing this and I'm certain that you'll end up wanting to look at different things side by side or whatever and that will become windows by another name but there might be subtle improvements.

I don't think that individual windows are a problem - people generally focus the best with a clearly delineated area to put a problem on.

What I would like to get rid of is the constraint of the limited screen area. There were some attempts, but I would really love to see a set of AR goggles that would work as an "unlimited screen", allowing me to open windows and position them anywhere in the "space" around me. The windows would remain in the spot where I dragged them to, and I could get them "closer" or "further away" as needed.

Combine this with a rotating char with a built-in keyboard and a pointing device, and this is my ideal workstation. Note that the goggles wouldn't be VR - I want to be able to see my environment behind (or "through", as per configuration) the windows, so I can still interact with people around me.

I think you just need a bigger monitor. There are a lot of situations where multiple windows are the way to go. A designer needs to have the style guide PDF open while working with Photoshop, or an accountant needs to have its banking website on sight alongside the ERP. It might be possible for small tasks though.

https://news.ycombinator.com/item?id=33409558

This sounds a bit like what you are after.

“One example that I use often — the Photos app on Mac has a built-in editing interface, but also supports extensions that allow third-party apps (e.g. Affinity Photo and Pixelmator) to take over that whole UI and present their editing UI inside the Photos app.”

A headless OS came to mind immediately when I was reading your post. Let me expand: an OS which does provide a collection of components that can be used to build a UI but not the concept of windows and to make it connect to the hardware it provide the connecting functionality in a headless manner. App store transform into component store. Would be interesting to work on something like this.

Maybe. I suggest reading About Face : on Interaction Design if you’re interested in this topic OP.

The Affinity Suite has a nifty feature where, if you own the other packages, you seamlessly can edit a photo from Photo in Publisher for instance. Or a design in Designer from Photo etc. nothing loads, it just works.

Another example:

When I open two .doc files, I can only have two separate windows or tabs. But I don't need those windows or tabs. What I actually need is just two open files on the screen. And some Word functions. From UI perspective, it can be a "function bar" on top of those two files.

If I want to open an image file and Photoshop, currently I can only have a separate Photoshop window. What I actually need is just an open image file on the screen. And some Photoshop functions - a "function bar" on top of that file.

So the visual difference is that you now have 3 different open files and some "function bars" - all in one workspace. And you can arrange or scale your files in any way you want. And you can arrange "function bars" on the screen or hide them.

Coincidence? Automatically Organize Workspaces with Stage Manager

https://www.apple.com/newsroom/2022/10/macos-ventura-is-now-...

Let's forget about OLE/OpenDoc implemetaions, since they really limit our view. I'm talking about the interface only. The change in the interface that removes borders around files and programs. So that you work on two different programs (technically), but on the screen you see them as one (visually)! We can implement this solution even with current Windows/MacOS. All we need to do is just remove those borders and make the "windows" completely transparent. Now, you don't have any windows or borders. You just have different files stacked on the screen. And different "function bars" on top of them (from different programs/apps).

> What if we remove windows completely? Think about it - when you open a program, you don't actually need a separate window. What you need is functionality! So what if we remove windows and keep only the functionality. For example, you open a .doc file, but there is no separate window. You stay in the same "Desktop", but now you have an open file and functions to work with this file. And if you need some other functions, you open another program - and other functions appear on the desktop. So now you have two different programs working as one! And you don't need to arrange different programs and files on the screen. Because now you can have different programs and files working simultaneously as one.

This idea of a "universal container" as strong similarities to Apple/IBMs "project pink" and OpenDoc [1] in the 90s that went on to become Taligent [2] that went on to become... not very much. There was also Microsoft's roughly concurrent efforts with OLE [3] and Windows NT Cairo - OLE was a stab at what might be described as componentising application functionality at the UI level, and Cairo had (iirc) the ambitious goal of replacing (augmenting?) the windows file system with a semantic data store with OLE/ActiveX as the UI layer.

OLE did kind of work, and actually made it into products like Word and Excel - though whether that was a good thing is debatable. The rest of it basically got canned, probably because it was too difficult to implement and/or sell. But I'm sure the influence lives on to some extent. I'm not aware of any serious attempts to go down this route again.

I still have some OpenDoc SDK beta CDs somewhere. It might be interesting to get them out and have a look.

Personally I think that "application" as a discrete concept provides a useful conceptual boundary [4] - although I will admit that this may just be that my brain is by now hard-wired for it.

[1] https://en.wikipedia.org/wiki/OpenDoc

[2] https://en.wikipedia.org/wiki/Taligent

[3] https://en.wikipedia.org/wiki/Object_Linking_and_Embedding

[4] Both for people using the application and for the grubby activity of selling it to them. Got to have a "thing" to sell.

I hope it's not transparent iPads like every sci-fi movie uses /s.

An object oriented desktop instead of an app oriented desktop sounds like a good idea to me. Show me things and groups of things instead of apps. A person, a message, a video.

It's possible what you describe could be achieved by AI compositing. I.e. The AI creates a central view of what it believes is the most relevant info for your current task. This doesn't require any changes to the programs themselves.

Longer term, I think voice + AI assistants can help with this. In programming speak, our UX will become more declarative and less imperative. Combined with AR/VR interfaces, this will see a big improvement in productivity.

Back in the day, we used to have several monitors when developing web applications, mainly to mimic laptop use and desktop use. I thought that the dynamic resizing features in modern UI development would negate the need for these, but apparently not. Future of UI, or rather HCI, has to be voice or gesture. First company to make an intuitive interface and UX for the internet that everyone can use for their TV will be onto a winner.

When it comes to programs that work with files (such as Word or Adobe Illustrator), the window is just a border around the file. It doesn't have any other purpose. I think of it as barriers to the workflow. I have to arrange windows on the screen. What I actually want is to arrange files. And have functionality from different programs - all on the same screen (stacked like those function bars in Word or Adobe Illustrator).

I don't go back and forth between windows, I open them all fully across multiple monitors. The next evolution is right there: "unlimited" monitors using VR. But VR headsets are just not that comfortable yet. Maybe in a few years it will be more comfortable to use a VR based solution than sitting in front of 3 high end monitors

Have you played with Plan9 the operating system and its windowing system?

or at least the Acme editor, which is also available for Mac and various Unix systems like Linux, FreeBSD etc.

https://github.com/9fans/plan9port

Sounds quite similar to a terminal, no?

This question smells like product research…but it’s intriguing so I’m bookmarking.

Fascinating responses thus far.

Another example:

Suppose you work on a research and you have 10 different .doc files. Currently, you need to open 10 separate windows/tabs. Can we have 10 different files all stacked together in one window/tab? I think it will dramatically improve productivity, since you don't need to arrange windows or move back and forth between them. You just have 10 files in 1 window and you can arrange them any way you want.

There should be some functions to work with those files. Currently, in Word you can only have function on the top. But I more prefer the way its made in Adobe Illustrator, where you can arrange functions in any way you want - on the left, right, top, bottom.

Just imagine - all 10 files stacked around the screen and some function bars on top of them. That's how I see the future OS.

Okay, browsers, games, Excel, Calculator, and other type of programs obviously need some "window". But Calculator can be actualy made as a "function bar" that works with any other program.

A universal Undo API, with graphical indication of which actions can be added to the undo stack (e.g. file move), and which cannot (e.g. send email).

Sounds like an IDE. Yet what are the disadvantages of using an IDE?