This paper proposes a simple and effective personalized image restoration method, named Dual-Pivot Tuning. The method consists of two steps: 1) Fine-tuning a conditional generative model to leverage the conditional information in the encoder for personalization; 2) Fixing the generative model and adjusting the parameters of the encoder to adapt to the strengthened personalized prior. This can generate natural images that preserve both personalized facial features and image degradation attributes. Experiments demonstrate that compared to non-personalized methods, this method can generate higher-fidelity facial images.