ChatGPT et al. and NPCs
So, I imagine that in a few years when playing future video games similar to RDR2, NPCs would be able to answer in a non-fixed way, always within the context of the video game, perhaps even their voices would be also generated by AI. I even think players will be able to interact with NPCs via voice (so no more “press square to Dismiss, triangle to Antagonize”).
I read this and thought, "no way, the interactions could alter what actions the player expects the character to perform later in the story. how could you filter to a discrete in-game action given a bunch of LLM convo? a fully realized ai character would just feel disjointed and confusing"
Then I realized, the pre-made set of in-game actions could be filtered to the most likely one if the game _asks_ the same ai character with all of their conversation history which they'd do next!
This stuff is gonna be weird for players and game designers! Imagine a set of under-the-hood interview questions designed to figure out where an ai character's head is at in order to choose which content comes next haha
People in this thread are pointing out difficulties and I'm getting an overall dismissive tone. As an ML researcher myself, I'm often skeptical and reserved about AI hype.
That being said, the difficulties anticipated here are going to be resolved, or at least they will work like hell to make it happen. Not just because it will make for great video games (although wasn't that one of OpenAI's original objectives? I can't keep track of changes to their stated mission...) But because they are interesting and HARD research questions. And they lie directly on the currently charted path to better understanding of AGI.
I'm hopeful for video games but still worried about everything else that will come out of it.
The problem with GPT3 / LLM is how uncontrollable they are. Imagine making a detective game - you could prompt AI's with some information to give to the player on questions, but they also might just give complete made up answers. They answer with whatever works - whether it's true or not, which completely screws up a detective game.
Definitely possible to make sandbox style games with this in mind though. And there are a ton of possible other applications - ingame AI assistants (like shown in the above prototypes), NPC shopkeepers
I totally believe this is the future of NPC/player interaction for more immersive and useful in-game assistance.
I worked on a small project for fun a couple of weekends ago.
I wanted to capture and mix the state of the game (League of Legends) with AI to generate snarky fun comments.
I came up with this [1], using the LoL client internal API, OpenAI's libs, with few external dependencies and a couple of hundred lines of Python.
[1] https://youtu.be/tdONZF9iktY
I tried having ChatGPT serve as a DM. Its propensity for warning the user carried over to the fantasy realm - if I proposed an action that was dangerous, ChatGPT would just warn me that it was a bad idea rather than explain the consequences.
I'm imagining a RDR2 bandit NPC who starts explaining the dangers and immorality of robbing trains whenever you talk to him.
It would all work great until I ask Dutch what his favorite flavor of pizza roll is. I think the work required to train period-accurate and dependable dialog models for these characters is... a bit much. Most people will probably end up wishing they could write down a list of quotes and have a human record them instead.
I think it's highly underestimated how much computational resource every interaction requires.
That being said, I think AI could be great at assisting generation of complex interactions and storylines and behavior with the human storyteller focusing on the artistical part of the narrative.
I don't think an AI freely writing lines within the game is a good idea if the developers want to be certain that it won't ever produce questionable text. But it might be useful to produce much larger quantities of fixed texts beforehand using an AI and maybe let the AI change the wording of the text in real time based on context.
> I even think players will be able to interact with NPCs via voice (so no more “press square to Dismiss, triangle to Antagonize”).
A button press will always be faster and more reliable. It's clear that outside of VR experiments with control schemes are not very popular (remember Kinect?). It would be a nice optional feature for accessibility, but if it's non-optional I'm not playing that game.
As amazing as this would be, I think game devs will shy away from AI NPCs simply for maintaining some continuity and storyline.
Oblivion was ahead of it's time in allowing a massive array of NPC actions. NPCs would take all sorts of random actions, with game altering effects. A vital quest character could end up in the dungeons for pickpocketing, and you would have no idea how to complete the quest.
And while some players loved it, the consensus seemed to be that this was disruptive to the storyline. Hence the more sanitized interactions in Skyrim.
Even with GPT3 without any fine-tuning you can come up with prompts that support interactive goal-directed or topic-constrained chat. You can build in trigger conditions that cause it to output something machine readable so you can move to a different place in your state machine, and you can pre-process input from the user to decide which prompt template to leverage. And that's just with off-the-shelf GPT3 today and a few lines of code. The tough part is figuring out what exactly to do that is going to feel good, make sense, and ultimately be preferred by users, since this interaction modality is so different from what most are used to building or using.
Once you can run this stuff locally, or it's cheap enough to run lots of queries remotely, we're going to see a lot of software (esp. games) start trickling this tech in to middleman a lot of IO. Today it's:
commandXbutton.onClick(cmdX)
cmdX(){ print(result) }
I imagine we'll end up seeing a lot of that change to something like:
chat.onVoice(x => runCommand(textToCommand(voiceToText(x))))
cmdX(){ print(stringToUserDialect(result)) }
This is technically possible now -- do users want it?
I've noodled around with ChatGPT as a source for game content. I think it's already good enough for _flavour_ - rather than repetitive NPC barks, you get more varied content. The issue is you'll always get jarring inconsistencies, no matter how much you try to prepend to a prompt. I've used it for diary entries and other left-over text, I've even used it for structured output like describing four members of some imagined family, or the layout of their house. But I wouldn't use it for anything touching core storyline content because you just can't guarantee it won't say something mad.
The AI-generated RadioShack catalog struck a weird chord with me:
https://tilde.zone/@ftrain/109436259129597431
I'm concerned about a future where we spend our time jacked into an AI-generated universe that is almost -- but not quite -- consistent and sensical. A dreamscape that becomes a nightmare.
I had a similar thought, what if the single player story of an open world game was unique and generated for every game download on the fly. This would be awesome and is probably doable today, since the compute would have to be done only once on the servers of the publisher and could use some cloud api → the gamer would just download a game, a unique game.
I've been thinking about this for a while. I think games like dwarf fortress that collects a lot of stats/data will really benefit from this in the future. Just feeding ChatGPT the existing lore, and letting it generate more with some additional data points.
Seems neat!
I can't find the segment of the interview now, but I'm sure Lex Friedman discussed exactly this in his interview with Todd Howard. And if I remember correctly Todd basically said to yeah that's something they're looking at doing.
Check out https://character.ai - cool product, and it basically already does the NPC aspect very effectively.
First encountering it around last year in September, it blew my mind.
I think back to some debates I’ve had over the years, on technical matters, political matters, office matters, and so on, and I wonder whether the level of argument reflected anything better than I could’ve received from GPT.
Doing exactly what you're suggesting.
Wouldn't this be too resource-intensive?
GPT will also make astroturfing, propaganda, and fake user bases easier to achieve for the unscrupulous.