There has been a bit of pondering around the blogs I follow about the implications for AI. Heretical Gaming picked up on a post from the Battlefields and Warriors blog, both about how well or otherwise AI generates scenarios for particular periods and gaming styles. Having read the posts and subjected the generated scenarios to a little consideration (but not too much, real life keeps intervening and I’ve only got so many brain-cells), I suppose the answer is ‘All right, but not particularly brilliantly’. Or, in school terms, B-, could do better (possibly).
My pondering was interrupted by, of all organizations, The Bank of England, warning that AI companies were going the way of the dot.com bubble, which we all knew and loved from the early noughties. I have heard this from other sources as well, that basically the bosses for these companies will walk away from the collapse with millions of dollars in their pockets, while the suddenly redundant staff do not even get their final month’s paycheck. Welcome to the world of capitalism.
Mind you, that is not to say that AI is not with us to stay. A number of companies, I believe, operate almost exclusively online to this day and seem to turn in a dime or two, even as they avoid paying taxes on the same. The dot.com thing was a bubble, but the idea of online business remains. That suggests that the AI bubble might pop, but AI itself will remain, albeit a bit chastened.
On the other hand, I do fear there is something of a moral panic setting in about AI. It is a pain, as I know from my former colleagues who are still wage slaves in Higher Education, but it still can be outmaneuvered. I am still waiting for the first AI product that can detect the use of AI in an undergraduate essay, as well. These things often become a battle of wits and resources, as warfare often does.
Still, attempting to head back into the area of wargaming, I also read a piece by theologian Janet Soskice this week (for reasons which have nothing at all to do with wargaming, but bear with me). In Chapter 8 of her The Kindness of God: Metaphor, gender, and religious language (OUP, Oxford, 2008), she discusses Bakhtin’s discussion of the novels of Dostoevsky. Bakhtin argues that Dostoevsky created a new form of novel where the characters are not voiceless slaves of the author, but stand up to him, disagree with him, and even rebel against him. This is remarkably like characters in Biblical texts where various Israelites argue with God (starting with Abraham, but perhaps most notably Job). The point here is that the characters are not objects but subjects, that is, they are quite capable of doing their own thing.
I was reflecting then (ah, some wargaming content) on my recent campaigns, from the Very Mogul Civil War, the War of Stuart Succession, and, most recently, the ongoing 1600 – Something campaign in Europe. In all of these, the outcomes and moves were determined by dice and cards – initiative rolls, card draws to determine moves, GOOS rolls for reactions, and so on, let alone the randomness engendered by the wargames of the ensuing encounters. This gives, in my totally unbiased view, of course, a rather rich background to what was going on and the narrative trajectory of the campaigns.
So, the questions posed are, I suppose, twofold. Would an AI be able to come up with something like the back story to any of these campaigns, and would it be able to create a viable next move in the game?
I am not about to rush off and pose the question to a passing AI bot thing. They have, after all, been raiding my blog for training content for the last month or so (although that seems to have suddenly stopped, or are they just terminally bored by my deathless prose?). I have a vague idea as to how these LLM pseudo-AIs work, and I doubt if they would do a very good job unless I fed in the campaign rules and the diplomatic table, as well as perhaps the army lists and means of generating the random armies.
That seems to add up to a significant quantity of data to be input to an AI in order for it to generate the next move in my campaign. A LLM, after all, has to start from somewhere. I doubt if one would come up with the idea of wargaming a fictitious struggle for the crown at the end of the Tudor period anyway. That idea came from my reading of the history and some of the options available to the various parties. Now, an AI bot could do that, but most of the histories finish with what the outcome was, historically.
I suppose the result of this is that while I could load the campaign into an AI and get the next move, I may as well keep going with my dice, cards, and so on. It also keeps me off the computer screen and gets my imagination wondering as to how something came about, or what to do with an unexpected move.
And that brings us back to Dostoevsky, I suppose. As the WsuS indicated, sometimes in a campaign, the characters do the unexpected. Who imagined when I started off that the Spanish would win the throne after an English army mutinied because they did not like the ally of the main contender? The characters, with only vague sketches of their natures, certainly felt like they were running the show. An AI might have managed it, but I am not entirely sure how, nor whether it could have come up with a convincing reason why it happened. Maybe it could; as I said, I’m not about to rush off and try.
As with the dot.com bubble, and, indeed, with the moral panic about the ‘Google generation’, I imagine that the frenzy about AI will slowly settle down. It already has its uses, as online companies did in the noughties, but I suspect that too many people who are prophesying doom for the human race as a result, as well as some of the overenthusiastic commentators, have simply been reading too much bad science fiction.
A thought-provoking post. A few specific reactions:
ReplyDelete1 - What score would you give the average published scenario? Do the LLM-generated scenarios rate favourably, or at least competitively?
2 - There are detectors although none of them seem to be that good. There does not appear to be any *necessary* feature of LLM-generated text that would infallibly give it a way (both tests and human observers tend not to be good at spotting them on average; some people seem to mistake the fact that some LLM-generated errors are really obvious with all LLM-generated text containing obvious errors; and if a human does a single edit pass then a lot of those obvious mistakes are eliminated). The impact of LLM-generated content on academia seems to have been huge.
3. I will have to think about the 'subject' characters in Dostoevesky. My initial reaction is that that is impossible, the characters can't do that (i.e. you can't generate external agency in that way) - it is just a feature of Dostoevsky's genius that makes us sometimes believe that is possible. Not entirely unlike what an LLM is doing, as it happens.
4. My instinct is that a good LLM would do reasonably well at those campaign related tasks you set it without any additional data at all (i.e. from training data only); the 'viable next turn' would be fine, as long as it wasn't too dependent on highly specific game mechanics (i.e. of the situation can be briefly described in words, then it would be fine; if it needs lots of game -state specific data, then perhaps not).
5. Given the high level of procedural generation you use in your campaigns and scenario generation, then I don't think it likely that an AI would do better than you - but it would be at least possible that it would suprise you in a way you can't really do to yourself. Even if the mechanics push you in unexpected directions (which is what happened in the WSuS) then you knew that these things were in the range of probabilities.
P.S. I did ask an LLM to come up with a C17 campaign scenarion. It suggested (this is the summary): Summary: The Weser Toll Crisis (1622–1623) pits the Prince-Bishopric of Verden (Catholic ruler, mixed confession subjects) against the Free City of Bremen (Lutheran, fiercely autonomous) over control of tolls, pilotage, and new river works on the lower Weser just as the Thirty Years’ War heats up. Oldenburg (downstream marshes and the Blexen bank), Brunswick-Lüneburg (nearby Welf princes), and the Dutch Republic (convoy shelter and engineers) circle as “mediators” with troops or coin, while Dunkirker privateers threaten shipping.
Thank you for the comments. Most of them would require a whole blog post to unpack....
ReplyDelete1. I think the LLM-generated scenarios are competitive as they stand, although perhaps not too innovative.
2. I'm not sure how much effort has really gone into detection, but it has been a problem in universities. I suspect they will have to go back to doing viva voce examinations eventually....
3. Actually, from novelists I've heard speak, it is quite common for them to feel the characters have taken over. One I heard explained about one character, whom she had been putting unsuccessfully in compromising situations with the main one. After a while she looked at him and said, 'You're gay, aren't you?' And he was. Whether this was the character taking over, the author's subconscious, the logic of plot development or what, I don't know. But in WSuS it certainly felt that the characters were taking over and reaching their own solution....
4. I dare say that any LLM or sufficiently programmed computer can keep up with the campaign, but I've started to wonder about the 'decision points', where the aims, opportunities, uncertainties, and resources come together in a complex way. What would an LLM do?
5. A related point to that of decision points. Computers are good at campaign accountancy, but can an LLM really do something totally unexpected? If so, would it be within the envelope of probability that most wargames operate in? Certainly, my card and dice-based systems can produce the unexpected, but as you say, within the envelope.
PS: Nice idea for a mini-campaign, I think. Can the LLM actually give the opening moves of the players?
2. there was a great deal of effort in 2023 but without much success (OpenAI notoriously removed its own AI detector, citing its unreliability); the game seems to be up now, most discussion I am aware of is looking more at the possibility of some kind of digital watermarking rather than specific content detection.
ReplyDelete3. I have read this too, it is just quite difficult to work out what is actually going on. The feeling of something doesn't imply that that is what is actually happening at the object level. It is a fascinating topic but my strong instinct is that the meta-narrative of talking to one's own fictional character and then making 'discoveries' about them is a top-end example of humans imposing narrative forms on themselves.
4. I suppose the insight/surprise from LLMs is that given a sufficient description on any subject (and sufficient can often be surprisingly minimal), then next word prediction of the answer is of generally high accuracy. If there are some niche, smuggled or silent assumptions then that may reduce it, but it would reduce it for non-genre expert humans too. I recall Henry Hyde talking about one of his opponents who would explicitly calculate odds and so on whilst playing, while he himself would rely on his knowledge of military history to know the right sort of thing to do. Since that knowledge of military history is ultimately text based, then an LLM's output is highly likely to reflect the same sort of understanding. Although TBF you can force the LLM to take a more explicit odds approach too, providing the LLM is given the numbers that a human would use to make the same calculations.
5. I think it generally would be within that envelope - in the same way that most strategies and tactics are not wholly surprising - but this is not certain. Particularly at the strategy level, one could use an LLM to help explicit game-theoretic modelling of multi-actor crises and so on, given their aims/utility functions. Another potential advantage is the 'memory-less' nature of interacting with LLMs - they see the 'board' afresh rather than human players, particularly solo players, whose brains emphasize continuous narrative.
For 'Weser Toll', it came up with (for first moves, bearing in mind there is no structure to this game yet):
The Weser Toll Crisis (1622–23) is a fight over who sets and skims the Weser’s pilotage and anchorage dues on marshy ground between Bremen and Verden, with Oldenburg guarding its Blexen bank and the Welfs of Brunswick-Lüneburg angling to “hold” disputed posts for a fee. To make war actually happen, the Welf peacekeeping/escrow price is kept steep—so costly that both sides prefer to fight. That makes the war aims plain: Bremen seeks neutral pilotage under its rules, firm control of Vegesack, no princely garrison on the river, and Oldenburg at least neutral; Verden seeks a planted garrison on a downstream battery, a recognised cut of tolls, and the power to curb Bremen’s convoy regime. Each also aims to keep the Welfs from parking a “temporary” garrison that taxes the river forever.
First moves: Bremen convoys all traffic immediately, hires a Dutch engineer and a few hundred professionals, throws up a temporary battery at Vegesack, and buys Oldenburg’s goodwill with a fixed pilotage cut—while publicly rejecting Welf escrow at the quoted price. Verden sprints to finish the downstream battery, reads Imperial letters everywhere to frame its move as lawful, pre-buys bread and powder in nearby marsh villages, and offers Oldenburg a share of fees—also refusing Welf escrow as too dear. Oldenburg declares land neutrality, keeps patrol boats off its bank, and offers to hold pilotage money only on fixed, low-fee terms (which the Welfs reject). The Welfs keep their price high and assemble a regiment “to keep order,” pushing both sides toward a shooting decision. The Dutch quietly lend an engineer and a small guardship; Dunkirkers lurk for any stragglers, nudging Bremen into convoys and tightening the screws.
Again, much of this needs a longer treatment than a comment, but...
Delete3. I wonder if this is not an application of Michael Polanyi's argument that we know more than we can say. If that is true (and I lean to the idea that it is in some circumstances) it means the author might know stuff about the characters which is not immediately articulated or articulable. Secondly, it also seems to suggest that AI might never know as much as a human, although that might depend on your definition of knowledge, of course.
With the rest, I am wondering about having the AI as an opponent in a campaign. There would be practical issues to fix, in setting the 'envelope' with the mechanics, maps and so on. But I do wonder how it would 'decide' on a particular move, or could the human put it into a dilemma state where it couldn't, like you used to be able to do with (very) old chess programs.
The Weser Toll scenario is interesting. I'll look forward to your account of the whole campaign! It shows that AI can be useful, to say the least.