Trained a Unitree G1 humanoid (29-DOF) in IsaacLab/Isaac Sim to throw a punch at a target sphere placed in front of it, comparing two competing control strategies: a pure-PPO policy trained from reward shaping alone, and an AMP (Adversarial Motion Priors) policy regularized against motion-capture punching reference data via a learned discriminator.
Built a custom command term that samples a punch target in the robot's base frame, converts it to world space relative to the robot's live pose, tracks timing windows for when a punch is expected to land, and scores hits using a position + velocity-projection check. Both policies were trained for 10k iterations, then resumed and extended by another 30k each, and evaluated head-to-head in the simulator.
A custom IsaacLab CommandTerm samples a target offset in the robot's base frame (forward/lateral/height ranges), then recomputes the world-frame target every step from the robot's current root pose and orientation — so the target always tracks correctly relative to the robot rather than going stale after a reset.
The command tracks a punch-action time window and an expected-hit time window per episode. A hit is registered when the hand's distance to the target drops below a threshold while its velocity is projected toward the target above a minimum speed — separating "on-time" hits from incidental contact.
Combines distance-to-target, hand-velocity, on-time-hit, and post-hit stability rewards with a punch-efficiency penalty that penalizes hand speed outside the action window — discouraging the policy from flailing the arm for free reward between punches.
The AMP policy is additionally regularized by a learned discriminator trained against motion-capture punching clips, blending task reward with a style reward via the RSL-RL AMP runner, so the resulting motion looks more natural than the PPO baseline's reward-only solution.
The punch target is rendered as a red sphere using IsaacLab's VisualizationMarkers — a purely visual, physics-independent marker — after an initial attempt using a kinematic RigidObject proved unreliable for per-step position updates inside PhysX.
The punch target appeared to spawn inside the robot no matter what coordinates were configured. Root cause: tensor[env_ids, 0].uniform_(...) with a tensor index returns a copy in PyTorch, not a view — so the in-place sample never actually wrote back to the target buffer, leaving it at zero. Fixed by sampling into a temporary tensor and assigning it back explicitly.
Tried moving a kinematic, collision-disabled sphere every step via write_root_pose_to_sim to use it as a visual marker — it never reliably tracked the target. Switched to IsaacLab's VisualizationMarkers, which updates the USD stage directly and sidesteps PhysX kinematic-body semantics entirely.
Computing the world-frame target once at episode reset used a robot pose that hadn't been refreshed yet after the reset event, baking in a wrong offset for the rest of the episode. Fixed by recomputing the world target from the robot's live pose every step instead of caching it.
The playback script loads a pickled copy of the training-time config rather than re-reading source files — so code and config fixes silently didn't apply when replaying old checkpoints. Learned to either force config overrides post-load or always pass --task to reload from source.