January 7th, 2023
The system I create for logging the choices of the players has proven to be of enormous value. I have been carrying out detailed analysis of the results and learning a great deal. More important, this system permits something never done before, as far as I know. I shall be able to track the behavior of my players and adjust the design in response to what I have learned. Theoretically, I could spend years polishing and improving the design. Perfection awaits.
Here’s one example of what I mean:
This is a graph of the average value of one of the global variables as generated by 14 different players over the course of the story. Note its odd shape. In an ideal world, we’d expect a nice clean line sloping upward from 0.0, but this one has a stair step structure. The initial slow rise is acceptable, as in the beginning of the story, the player doesn’t know much about the storyworld and so shouldn’t be given many opportunities to affect this variable. However, there’s a short period between turns 140 and 190 where the value shoots up, then changes little thereafter. This is clearly a flaw in the design. I must adjust the delta values to fix it.
But first, I decided to examine the data more closely and clean it up. It turns out that there was a lot of spurious data in the form of my own test play and players who played only a few moves before quitting. So I went through the data cleaning out that material and re-ran the entire analysis. This didn’t change things by much:
The differences between the two graphs all arise in the first 100 turns, because that’s where the short-termers showed up. Even so, it might be worthwhile for me to alter the analysis program to clean out the questionable data.
I’m still digesting all this data and I expect to be improving the analytical techniques.