PhysAvatar is an innovative framework that combines inverse rendering and inverse physics to automatically estimate the shape, appearance, and physical parameters of clothing from multi-view video data. It employs grid-aligned 4D Gaussian spacetime mesh tracking technology and a physics-based inverse renderer to estimate the intrinsic material properties. PhysAvatar integrates a physics simulator, using gradient-based optimization methods to principled estimate clothing physical parameters. These innovative capabilities enable PhysAvatar to render high-quality avatars of people wearing loose clothing in new viewpoints under motions and illuminations unseen during training.