ReKep is a system for spatiotemporal constraint inference on robotic manipulation, encoding desired robotic behavior by representing manipulation tasks as constraints between the robot and its environment. Leveraging large visual models and vision-language models, ReKep generates keypoint-based constraints that can be optimized for multi-stage, field, dual-hand, and reactive behaviors without requiring specific task training or environmental modeling. Key advantages of ReKep include its versatility, lack of manual annotation, and the capability for real-time optimization by off-the-shelf solvers to produce robot actions.