MouSi is a multimodal visual language model designed to address the challenges faced by current large-scale visual language models (VLMs). It utilizes an integrated expert approach, synergistically combining the capabilities of individual visual encoders for tasks like image-text matching, OCR, and image segmentation. The model introduces a fusion network to unify the outputs from different visual experts and bridge the gap between image encoders and pre-trained LLMs. Furthermore, MouSi explores diverse position encoding schemes to effectively tackle the issues of position encoding redundancy and length limitations. Experimental results demonstrate that VLMs with multiple experts exhibit superior performance compared to isolated visual encoders, achieving significant performance gains as more experts are integrated.