Using AI to Engage With Your Players Before Your Game Even Exists

Introduction

Game marketing is so much more than just selling a finished product. Knowing who you’re building for and what will resonate most with that group from the start can help shape aspects like game mechanics or art style. But as an indie developer, finding people to give this feedback can be difficult, especially before having anything to show.

That’s when I came across an intriguing post about Artificial Societies (AS), a YC backed startup with a simple premise: Use AI to simulate real target demographic groups to provide accurate feedback.

If that’s the case, can we simulate an artificial society of Steam users and ask them if they’d purchase a new game? Could we ask this simulated group what they’d prefer instead or what it would take to get them to try a new game? I was able to hop on a call with co-founder and CEO James He from Artificial Societies to see if we could get some of these questions answered.

Valuable Feedback

Anyone who has used ChatGPT looking for feedback knows that there is a big problem with this approach:

My first question was in regards to this: “why is ChatGPT so agreeable and what makes societies different? Is it actually going to tell me my idea isn’t that great?”

ChatGPT is so sycophantic because big LLM providers want you to chat with their bot… But we are not incentivized to keep you around. We’ve done a lot of work into collecting how real people behave and how they think, and that’s why our personas are very [brutally honest].

James also mentioned how he practices his own content with AS and how it’s very satisfying after finally getting a good simulation score, since so much of the feedback isn’t positive. My next question was in regards to interacting with the simulation, to which he replied:

The personas are basically individual people who just react and judge. We also have an insight agent that goes around and analyzes those results and provides suggestions. You can also ask follow up questions to gauge different ideas based on what the persona said.

So now we know that Artificial Societies are able to give us meaningful feedback, and we are able to interact with the target demographic of our choice. Can AS also help determine who the target market is?

Sadly our product isn’t shaped in a way to make that easy to do, but you can just create 10 different groups and run the simulation across all 10 of them.

James mentioned that their current customers are mostly tech founders and corporate marketers and they haven’t looked into gaming specific use cases all that much. However, at the end of the day game marketing is just marketing:

You can use it to market a game, or an AI startup, or some beauty product. We source our behavioural data from people’s online social media behaviour. So for example if you are selling tractors to farmers, we’re going to struggle, we don’t know what farmers behave like. But if you are selling to university students who are playing games. Oh yeah we can probably do that very well because uni students are so active online.

What Works on Steam

Before giving AS a try, we need to establish some baselines to better gauge the value of the feedback. In this blog post on How To Market A Game (HTMAG), Zukalous explains how “Crafty Buildy Strategy Simulation Games” are the most popular amongst the Steam community:

I think the biggest reason indies struggle to find success on Steam is they are making the wrong type of game.

Nearly 19 000 games were published to Steam in 2024 so Zukalous can’t make a blog post on every single thing that does or does not work. This is why the idea of leveraging AI to simulate this market is so intriguing.

Trying Artificial Societies

I gave AS a try and built my artificial society from the prompt:

Steam users looking for a new video game to buy.

Since the company focus hasn’t been on game dev, my experiment had to use a more generic use case called “product proposition” but James mentioned that if Steam has an API (which it does) then they could do an integration for a more accurate simulation.

I then asked ChatGPT to create a prompt for me to give to AS based on what we know about “Crafty Buildy simulation” games.

The simulated users were intrigued by the concept and setting which corroborates what Zukalous has been preaching on HTMAG. Artificial Societies also showed me some of the simulated users’ concerns: “Around 25% question the game's uniqueness amidst many similar strategy games already available on the market.” and many of them wanted to see some gameplay or a trailer. Again all feedback that makes sense.

However, none of this feedback matters if we can’t trust it to be accurate. I did another experiment by taking some Steam games that we know to be successful. I asked ChatGPT to create a product proposition prompt for the games Factorio and Slipways and fed those into AS.

Both were received well by my society with Factorio scoring bigger in the “very high” intent to purchase, while Slipways seemed to have a less intense appeal, aligning with their real world performance. As a final litmus test I asked ChatGPT to come up with a game concept that would not do well.

This was the prompt it gave me:

A hyper-realistic Lawn Chair Arrangement Simulator where the only mechanic is dragging and rotating plastic lawn chairs in a backyard. There are no goals, no progression, and no feedback — the game ends after 15 minutes with a message thanking you for your time. The art style is intentionally bland, the soundtrack is a single looping elevator-music track, and the Ul only works with a joystick (mouse/keyboard are disabled).

The feedback was almost all negative which made sense but the concept was so bad it actually sparked interest as a meme which I think is realistic.

Criticisms

Since the use-case I chose was “product proposition” the feedback I got was more-so on the wording and style of the proposition itself and not as much on the concept I was pitching. It told me how to better improve the proposition by changing certain words as opposed to focussing on what I was describing about game mechanics or play style.

Another criticism was that the population I simulated still had many people in it titled “founder” or “ceo” and it wasn’t a population of strictly gamers, although this may help when gauging an idea at large.

Lastly, testing variations was tedious. When testing which title for a game worked best it would be nice to prompt all 5 at the same time and see how each one scored side by side. The majority of the UI is dots representing the population which I feel could be better utilized by something like these comparisons.

Conclusion

Artificial Societies seems to provide more valuable feedback on content than generic LLMs. I think ChatGPT is better for coming up with a broad list of ideas but AS is better at polishing those ideas into actual content. The scoring system of AS makes decisions more objective and easier to compare as opposed to a long list of pros and cons to sift through from an LLM.

After signing up for the AS trial, I found myself using it a lot more than I thought I would and even used AS to help write this article. Since it was made for that use case I found it a lot more helpful, and will definitely continue to test my content there before posting it.

Special thanks to James and Artificial Societies for taking the time, and as always thank you for reading!