Qwen3 4b Instruct from 1% to 99% on wordle in 11 hours with a single 5090 with PPO.

Executive Summary

Hello everyone! I’ve previously worked on transformers and ppo for traditional rl environments but I wanted to see if my training would scale up to llm fine tuning (it does at least for wordle). I modified/rewrote my gridworld agents from mapox-trainer to load Qwen3 weights and created a new value a...