MobiLlama is a small language model (SLM) designed for resource-constrained devices, aiming to provide accurate and lightweight solutions to meet the processing needs, energy efficiency, low memory usage, and responsiveness of devices. MobiLlama derives from larger models but reduces the cost of pre-training and deployment through a meticulously designed parameter sharing scheme.