How to become a video game superforecaster
You all want to become omniscient and omnipotent, right?
[The GameDiscoverCo game discovery newsletter is written by ‘how people find your game’ expert & company founder Simon Carless, and is a regular look at how people discover and buy video games in the 2020s.]
We have returned to your inbox, video game data and discovery nerds, to bring you the second free GameDiscoverCo newsletter of the week. This one’s about the traits we - and experts - claim you need, in order to think straight about the video game biz.
Meanwhile, thanks to all the new readers that signed up since Monday’s missive. It means a lot to us that we regularly get 10-12k actual readers of these pieces, given that this is actually a fairly niche B2B publication. (We’re hardly Tiger Beat here.)
[Care to get lots of extra PC and console game analysis and data? Yes? Now 25% off for the next 10 days, our GameDiscoverCo Plus paid sub has a data-rich weekly PC/console game trend analysis, interactive Steam ‘Hype’ charts, two eBooks, a member-only Discord & more.]
Can you, too, become a game ‘superforecaster’?
Recently, somebody dropped Tim Harford’s book The Data Detective (aka ‘How To Make The World Add Up’) in the Little Free Library outside my house. It’s subtitled ‘Ten Easy Rules To Make Sense Of Statistics’, and it’s a seriously great book on thinking critically about data.
While I recommend the whole thing, one section deals with Philip Tetlock’s Good Judgment project, which posed around 100 to 150 questions each year on geopolitical events to “enthusiastic amateurs” as well as self-professed experts, and saw who came out on top.
So it basically ranked the forecasters. And, it’s claimed by Harford: “there were a select group of people whose forecasts, while they were no means perfect, were vastly better than the dart-throwing chimp standard reached by the typical prognosticator. What is more, they got better over time rather than fading away as their luck changed.”
So what made them stand out? Interestingly, it wasn’t necessarily deep subject matter expertise, or intelligence. Though doubtless those help. It was the following - all of which I’ll explain how to apply to the video game space:
Training on basic statistics: This includes ‘how stats work’ in general. But Harford singles out ‘base rates’ - ie, the actual baseline numbers you’re basing your estimate on. For example, in Monday’s newsletter, we learned that about 15 games per month launch on Steam with 50,000 Steam wishlists or more. And we know about median wishlist conversion rates. Thus, you have a starting point for possible success for a game. Without it, you’re lost.
Keeping score: this is being honest about whether you were right or wrong, and iterating on it. A lot of game companies - especially those chasing big hits - will tend to ‘fire and forget’. But if you actually do a postmortem after each game launch, and try to improve tactics based on it? That can help a lot.
Update forecasts often as new information arrives: so you’ve discovered that - with a few exceptions - the Nintendo Switch market has become a bit oversaturated, and concentrating on PC-first games might be a better idea? That means you need to shift your portfolio approach based on that info.
‘An open-minded personality’: Harford notes that those “willing to abandon an old view in the light of fresh evidence” are some of the most successful forecasters. For example, we thought that PC DLC was a bit pointless, and that VR game sales weren’t going to be significant. But we were shown proof this was wrong, and now GameDiscoverCo is covering these topics more. Not saying we’re superforecasters. But we are definitely willing to change our minds.
Anyhow, we find this all particularly interesting because it somewhat goes against the idea of the successful practitioner as the single-minded iconoclast or the rebel disruptor. That approach works if you’re exactly in the right place at the right time.
But in today’s game market - which is shifting business models swiftly - it seems to me that ‘listening to new information, and changing your tactics accordingly’ is one of the most important traits you can have. If you only have one approach… well, even a stopped clock is right twice per day, we guess?
Follow-up: ‘derivative games’, meta-mechanics, and indies..
Following our piece last week on whether ‘derivative games’ are the way to hit it big, we had some really good and robust discussion - both via email and on the GameDiscoverCo Plus Discord. So wanted to highlight some of the most important ones.
Firstly, Elliot Colp smartly noted that this isn’t just about game ‘clones’: “I think a key part of Kyle Kukshtel's article that kicked this all off is that this isn't just about derivative games vs. "micro-genres," but about certain mechanics and experiences that are becoming increasingly ubiquitous in games regardless of genre.
You'd be hard-pressed to find a AAA release these days without a crafting system, a skill tree, and randomized loot drops for example. These meta-mechanics are being tuned to maximize repeat engagement, with increasing influence from mobile and casino games.
As they become more standard, and players start to expect them in every game, it does feel like it's making it increasingly difficult to sell a game that doesn't fit the mold or easily allow you to layer those meta-mechanics on top of it. (‘depth’).”
This is a subtlety I think I missed in our analysis, which dealt more with subgenres than ‘portable’ game features. And Kyle Kukshtel himself - he’s a Plus subscriber, folks - turned up on our Discord to add:
“One bigger area that I didn't get into is how AAA is the biggest employer of games people, all skilled to make Assassin’s Creed-likes. So those people skill up in making AC-likes, crosspollinate to other AAA companies, and all sort of converge on making the same thing, as the talent pool becomes more and more specialized towards making that very specific type of game.”
It’s true - and alongside the homogenization, this is now the ‘depth’ hill that we now have to climb in order to compete with the big guys. And it can be pretty formidable, given the amount of features some of these franchises have built over time.
Finally, we got an interesting email from a different angle, thanks to Ryan Nicoletti. Ryan points out: "1. The risk / opportunity cost [for solo indies] isn't just ‘not getting paid’... it's also not getting paid programmer wages… The people that can make games have extremely valuable skills. 2. If ‘mimetics’ must drive my dev decisions to even have a chance, then I'm trading off the thing I want to buy with all the money I'm giving up from not doing a ‘boring’ business job: freedom to make the games I want to see succeed.”
In many parts of the world*, that’s the issue with small indies having to do ‘fast follow’ games. It just doesn’t map well with why they wanted to make games in the first place. Which was.. to be creative, have fun, and make the kind of game they truly want to play?
[*The U.S. in particular often has a massive gap between ‘boring programmer’ wages and indie game dev incomes. This is less true if you work at an actual game company, but… still true.]
The game discovery news round-up..
Finishing off for this week - and indeed month - there’s a lot to get through in terms of game discovery and platform news. So pull up a scrollwheel (?), and we’ll get through it all together:
The late-breaking Nintendo Direct totally happened, but it was a third-party only Mini, so perhaps a little more muted than expected. Interesting to see the Monkey Island reboot “launching first for consoles on Nintendo Switch later this year”, though. This must be paid timed console exclusivity, right?
Remember we were talking about Battle Passes being a trend on PC and console too? Well: “Conan Exiles announces Age of Sorcery, replacing DLC with battle passes” - just sayin’. Amused by a reader comment, though: “Battle Passes solve so many issues for developers at once. None of those issues benefits the players.”
The folks at GameRefinery have noted that “mobile game studios are trying to bypass Google and Apple Store fees by selling certain in-game items through external web stores”. The list of F2P games doing it right now includes big titles like Game of Thrones: Conquest, Clash of Clans, Marvel Strike Force and Star Trek Fleet Command. We heard this might happen… and now it is!
Microlinks: Sony has a new inZone brand for PC/console gaming monitors and headphones; if you’re making narrative in games, you can submit for a Peabody Award now; there’s a (paid) Discord coverage bot that broadcasts when Twitch, Web and YouTube sources mention your games.
Over at Ars Technica, Kyle Orland had a detailed look at the new Epic Games Store review process, finding some big pluses and big minuses: “Limiting reviewers to a small, random sampling of active players means it would be difficult… to organize [a] review bombing campaign… On the downside, Epic's system also seems to lead to significant score inflation.”
Gotta love the person who did a comprehensive breakdown of the ‘not-E3’ Wholesome Direct in terms of themes (above). So yes, I too will be making a *consults notes* small town woodland farming game, featuring cutesy animals. And, huh: “games in the showcase will invite you to play as cat, but to own a dog.”
Some layoffs in platform-y companies this week: Unity reportedly cutting ‘hundreds’, though don’t forget they’re an ads company more than an engine company, and Niantic is cutting almost 100 employees and canceling projects - more on the internal game side than the ‘AR metaverse provider’ side, perhaps.
The Meta Quest VR game store has just added live A/B testing of creative (descriptions, trailer video, and cover art) to store pages. This is something that most other store fronts on PC and console definitely don’t allow - and totally should. (Android does, though, as does iOS!)
Helpful microlinks: journalists tell you how to get your game covered, a case study on using mailing lists to add lots of Steam wishlists, ‘12 tips for killer user acquisition operations’ on mobile shows how deep the rabbithole gets when it comes to discovery complexity there.
Battle of the app associations! If you thought the Coalition for App Fairness (CAF) was an Epic puppet attacking Apple (well - it started as one, but seems slightly beyond that now?), you may not be surprised to learn that ACT | The App Association may be.. an Apple puppet attacking Epic? Fun times.
Want to know how much game devs are paid? If from the UK/Ireland, you can fill in or (anyone can) read the results of an anon game biz salary survey (second tab has salary averages), thanks to Silent Games’ (and ex-No More Robots’) Sally Blake. Now, somebody do a quality job like this for other regions…
Microlinks: the virtual space Rec Room has hit 75 million lifetime users - and $1M in creator payouts for Q1; MrBeast talked about YouTube success & the algorithm at VidCon; here’s the top product localization languages, based on orders received at large (beyond games!) loc company Alconost.
Finally: we just spotted a new interview with former Kixeye CEO Will Harbin on his comeback. Which reminded us that the Facebook & mobile game dev (War Commander) was an ‘attitude outlier’, even in the San Francisco of the early ‘10s.
Exhibit A: Harbin telling the designer of Kixeye’s office to “make it more evil”: “The elevators open onto a lobby drenched in red light. Speakers overhead mutter boiler noises and bureaucratic murmurs - "Report to Lab B for testing," AOL dial-up sounds, "Ice cream social at 2:30 p.m." At the end of the lobby, an armed man in military fatigues sits at a desk, an iron chair at his side as if for an interrogation.” Is this… a discovery tactic?
Exhibit B? This ‘game dev recruitment video’, making fun of “competitors like Zynga (aka Janka), EA (aka EAARP), and Kabam (aka Kerpoop) at the time.” So yes, this was widely perceived to be dumb at the time. And… it’s still fascinatingly dumb today:
[We’re GameDiscoverCo, an agency based around one simple issue: how do players find, buy and enjoy your premium PC or console game? We run the newsletter you’re reading, and provide consulting services for publishers, funds, and other smart game industry folks.]