Jais
13B Parameter Arabic-English Bilingual Isomorphic Model
CommonProductProductivityChatbotIntelligent Chat
This is a pre-trained bilingual large language model with 130 billion parameters, supporting Arabic and English. It is trained on a dataset of 72 billion Arabic tokens and 279 billion English/Code tokens. The Arabic data has been iterated 1.6 epochs compared to 1 epoch for English/Code, with a total of 395 billion tokens trained. The model is based on the Transformer decoder-specific architecture (GPT-3), utilizing the SwiGLU nonlinear activation function. It implements ALiBi positional embeddings, allowing extrapolation to long sequence lengths, providing enhanced context processing and model accuracy.
Jais Visit Over Time
Monthly Visits
19075321
Bounce Rate
45.07%
Page per Visit
5.5
Visit Duration
00:05:32