KAIST Introduction to Reinforcement Learning · Project ·
Korean 4-Ball Billiards: A Continuous, Deterministic, Sparse-Reward Benchmark Solved by Inference-Time Search
We cast Korean 4-ball billiards (sagu) as a continuous, deterministic, sparse-reward RL benchmark with a fast, exact physics simulator. Off-policy RL (SAC/TD3) beats PPO by ~2.5x, but the bare task plateaus below one point per inning even with more compute, a curriculum, or a learned reward model. Explicit geometry — a first-contact aim constraint plus four carom features — lifts SAC from 0.487 to 6.460 points/inning. The final jump comes from inference-time search: greedy depth-2 lookahead using the simulator as its own verifier turns a ~6-point policy into chains of up to 9,392 consecutive scoring shots at 99.8% per-shot success.






