Alejandro Ashes
AI Alchemy 8 min

Nobody Grades Their Own Work

An AI that drafts your chapter will tell you the chapter is fine. An AI that builds your game will tell you the build passes. Dev Diary 4 is about the four rules that keep the new Kindred Slayer game honest, why each one works on a manuscript too, and the bug all of them missed.

Nobody Grades Their Own Work

An AI agent that has just built something will tell you it works. It means it, in the sense that it checked the things it thought of, and those were the things it already had in mind when it built. The grade comes back generous every time.

This is the fourth dev diary on the new game. The first was about your novel as a map, the second about cutting the screen back, the third about the meter. This one covers how I check the work when the workers are machines and I am one person.

Every rule below started as a time the machine passed its own exam and the game was still wrong.

Rule One: Make Too Easy and Too Hard Into Numbers

A card game can be broken in a way no one notices from a single run. If a lazy strategy wins most games, the game is too easy. If nothing wins, it is too hard. If a player who reads the table does no better than one who plays blind, the skill the whole design promises does not exist.

So bots play it, thousands of seeded games on every push. A seed makes a game replayable, so any number that looks wrong can be reproduced exactly. A simple bot has to win within a stated band of the time. Careless play has to lose clearly. And a bot that reads the table has to beat one that plays blind by a clear margin. If any of these drifts, the build fails and nothing ships.

For your manuscript: decide what too easy and too hard look like before the draft exists, then measure them. If you write mysteries, pick the share of beta readers who should guess the culprit by the midpoint. If your book is a puzzle for the reader, a draft where nearly everyone solves it early, or nobody does, has failed, and a number tells you before a review does.

Rule Two: The Critic Reports and the Fixer Fixes

A critic agent reads the work and writes up what is wrong. It cannot change a line. A separate fixer agent takes the report and makes changes. The builder, the critic and the fixer are three different jobs, and the one that finds a problem never gets to quietly repair it and call the matter closed.

This keeps the findings honest. A critic that can edit will edit, and what it reports becomes a list of things it already took care of.

For your manuscript: let your editor, human or machine, hand you a report first. Read it as the author, decide what to change, and make the changes in a separate pass. An edit pass that rewrites while it judges teaches you nothing about what was wrong.

Rule Three: Tests Come From the Spec, Written by Someone Who Never Saw the Code

When an agent writes a test for code it just wrote, the test checks what the code does. If the code has a mistake, the test copies the mistake and goes green.

The core tests are written from the written rules of the game by agents who never saw the implementation. Each one asks what the rules say should happen and checks whether it does. When the two disagree, one of them is wrong, and the disagreement is where the bugs live.

For your manuscript: write down what the reader must know and feel after chapter three, working from your outline and before you reread the draft. Then check the draft against that list. A checklist made by reading the draft only confirms the draft.

Rule Four: A Stranger Judges From the Screen Alone

The last gate is a first-time-player agent. It receives screenshots and nothing else: no rules, no code, no notes. It has to say what the screen is asking of it. When it cannot work out what to do next, the screen fails, no matter how many checks passed.

This is the cheapest gate and it catches the most embarrassing mistakes, the ones that look fine to the person who built them.

For your manuscript: give a chapter to a reader who has not read the series bible, and mark the place where they stop understanding. Your own sense of what is clear has been ruined by knowing too much.

The Gates and the Cost

All of this is wired into a single command that runs every check a change triggers, 33 of them in total, across rules, saves, replays, balance and wording. A hook runs the same checks when an agent ends its turn, so no one has to remember. The longest pass takes about 25 minutes.

That is slow, and it is the best money in the project. A check that costs half an hour is cheap beside a player discovering the bug on launch day.

The Bug Every Gate Missed

Recently I was tapping a button on my phone over and over. It was a repeat action: each tap did one small thing, and a limited number of taps were available. On the last tap the button was replaced, in the same spot, by an exit button. My next quick tap landed on the exit, and I was thrown out of the screen.

Thirty-three gates passed. Not one of them tapped like a thumb.

The fix was small: for a moment after the repeat action, no other button answers a tap, so a hammering thumb cannot reach what replaces the thing it was hammering. Finding it took a person and a phone, which is why the build also tells me, after any screen change, to go and look at it myself.

The lesson fits next to the four rules. Bots catch drift, the critic catches what the builder missed, the stranger catches confusion, and a human playing the real thing catches the rest. Put your own hands on the finished work. Nobody else’s checks include yours.

For Your Own Draft

You do not need agents to use any of this. Pick your numbers before you write. Get your editor’s report before you touch the chapter. Write your checklist from the outline, hand the chapter to a cold reader, and keep one pass for reading the finished thing the way a reader will.

The machine will tell you it is good, and it will mean it. Ask someone else.

✦  Ballads of Eldritch Metal
—
Alejandro Ashes
0:00 0:00