Building an AI Pokémon Card Game With Generative Video

Pokémon TCG Live Attacks is a complete digital Pokémon Trading Card Game: 20,444 cards from 174 sets, playable against three tiers of computer opponent or online, in standard, quick, sealed, and draft formats, where every attack plays as a short generated video of that attack landing. This is the tour of what you can play, how a turn works, and what the AI is and is not allowed to touch.

The design principle underneath everything is covered in its own article: the rules engine decides every outcome before a generative model is involved, and the video is a rendering of a settled fact. This piece is the wider view, for anyone who wants to know what an AI Pokémon game actually consists of.

Every card, read from disk

The card pool is the community dataset normalised into one 17-megabyte file: 17,244 Pokémon, 2,806 Trainers, 394 Energy, across 17 series from Base Set in 1999 to the newest releases. It is read from disk at startup and never bundled, which keeps the game deterministic and immune to the public API's rate limits. The browser gets a compact index instead, one short row per card with single-letter field names, about 280 kilobytes gzipped, so the deck builder can search all 20,000 cards instantly and locally. Card text is deliberately left out of that index (it would triple its size), and a server-side text search fills the gap for the queries that need it.

Printed attack text is parsed into runnable structure rather than hand-coded. Two thirds of all 27,927 attacks resolve exactly; the rest deal printed damage and the interface says so, on the card, in the deck list, and in the attack panel. A deck carries a coverage score, “87 percent simulated”, so you know before you sit down how much of it plays as printed. A 1999 deck plays almost entirely as printed; a 2024 deck built on Abilities the engine does not simulate scores lower, and the builder says which cards are the reason.

Ways to play

Three computer opponents

The CPU is deterministic and touches no randomness of its own. The easy tier is a fixed six-step ladder: fill the bench, evolve, play Trainers in a fixed priority order, attach Energy to the active Pokémon, attack for the best damage, retreat only when free. The normal tier looks one move ahead: generate every legal action, apply each to a copy of the state, score the result with an evaluation in units of hit points, and take the best. The hard tier runs a small beam search over the rest of its own turn, so it will attach Energy, Gust of Wind a weak Pokémon into the active spot, and attack as one plan rather than three unrelated guesses, and it plays Trainers with intent: PlusPower only when the extra ten converts a miss into a knockout, a Potion only when the heal actually takes its Pokémon out of range.

const W = {
  prize: 420, energyOnPokemon: 0.3, energyAsset: 18, stage: 25,
  bench: [30, 28, 22, 8, 6], tempo: 22, handCard: 3, handPokemon: 2,
  threat: 0.6, knockoutNext: 0.55, knockoutDanger: 0.7,
  lastPokemon: 4000, deckLow: 25, deckEmpty: 1200,
};

The gym circuit wraps the tiers in eight rivals, easiest first, each a difficulty plus one of sixteen archetype decks the CPU knows how to play, with a personality that colours the announcer, a look that drives their portrait, and a badge for beating them. Names and titles echo the Kanto gyms without being anyone from the games.

How a turn works

1. Play from hand. A card that needs a target is a two-step gesture: select it, then click the Pokémon it applies to. Everything else acts on the first click. A half-finished gesture is invalidated when the turn or phase changes, in the same render pass, so the board and the selection can never disagree.
2. Read the attack panel. Each attack shows a live preview from the real engine: applied damage, whether weakness or resistance applies, how much HP the defender would have left, whether a knockout is certain, and a label if the effect is not simulated. Resting the pointer on an attack for a third of a second starts its clip generating, because reaching for an attack is a strong hint it is about to be used.
3. Declare. The engine resolves the attack completely, coin flips included, and parks the outcome. The board does not change yet.
4. Watch. Coins are revealed one at a time as an interactive beat while the clip generates underneath. Then the clip plays in a full-screen overlay: the attacker's portrait, the strike, and the defender's portrait standing, hurt, or fainted, exactly as the engine decided. If generation fails or the player's monthly video budget is spent, the two portraits play with a caption instead.
5. Apply. Only after the video does the engine apply the outcome and pass the turn. What the panel promised, what the clip showed, and what the board does are one computation.

While a clip generates, the wait is filled with b-roll: short clips of the board as it stands, the two sides squared up in the arena, made ahead of time and keyed by board position rather than by match, so the shot of Charizard facing Blastoise in the volcanic caldera is made once and played in every match that reaches it.

The match as a film

Every clip of a match, attacks, evolutions, and entrances, is stored against the action it belongs to, and because a replay is just a seed plus an action list, the engine can reconstruct every board state and tell each clip what it shows. The in-app reel plays them as one take: two stacked video elements, one playing while the next preloads, with the join between clips chosen from the board. A hard cut when the Pokémon that took the hit attacks next, because the last frame is the next clip's first frame; a dip to black with a title on a knockout or a switch; a short crossfade otherwise. A chip over each clip's opening seconds carries the beats since the last one, a Trainer played, a prize taken, a status condition, read straight from the engine's log, so it is true by construction.

The downloadable version is stitched on the server as one ffmpeg filter graph with the same joins, each clip's audio normalised and faded, title cards rendered from SVG, and an optional music bed. It starts building in the background the moment a replay is saved, so the download is usually ready before anyone asks. One thing that was tried and rejected is worth recording: trimming the held stills at each join read as a jitter, because the two clips' held frames are close but not pixel-identical and the hold is what lets the eye settle.

What the AI is allowed to touch

• Never the rules. Damage, coins, conditions, knockouts, prize cards, and legality are the engine's, and the engine is a pure function of the state and a seed.
• The portraits. An image-edit model makes a battle portrait, a hurt portrait, and a fainted portrait per Pokémon from its card art, under a fixed composition contract, cached for the life of the app.
• The choreography. A Gemini pass reads the whole card, illustration included, and writes how the attack should look, once per card and attack. It is optional: without credentials the prompts compose from deterministic tables, exactly as the game worked before the director existed. It has its own article.
• The clip. An image-to-video model, given both portraits as the first and last frame and a prompt built from the resolved outcome.

The generative layer is decoration, and the game is engineered so that it can be: everything the models produce can fail, be late, or be skipped, and the match is unaffected. The hard parts are the engine, the parser, the server-authoritative online mode, and the latency engineering that makes a minute of generation feel like nothing, each of which is written up separately. The full system is in the Pokémon TCG Live Attacks case study. If you want to read one file, ai.ts is the most fun: 870 lines of a card game thinking, with no randomness anywhere.