Qwen3-VL is the most powerful vision-language model in the Tongyi series. It adopts a Mixture of Experts (MoE) architecture and provides weights in the GGUF format, supporting efficient inference on devices such as CPUs and GPUs. The model has been comprehensively upgraded in text understanding, visual perception, spatial understanding, video processing, etc.

MultimodalGguf

Qwen3-VL-2B-Thinking is one of the most powerful vision-language models in the Qwen series. It uses GGUF format weights and supports efficient inference on devices such as CPUs, NVIDIA GPUs, and Apple Silicon. This model has excellent multimodal understanding and reasoning capabilities, especially enhancing visual perception, spatial understanding, and agent interaction functions.

MultimodalTransformers

Qwen3-VL is the most powerful visual language model in the Tongyi series. It has excellent text understanding and generation capabilities, in - depth visual perception and reasoning abilities, long - context support, strong spatial and video dynamic understanding abilities, and agent interaction capabilities. This repository provides GGUF - format weights, supporting efficient inference on devices such as CPUs and GPUs.

MultimodalTransformers

Qwen3-VL-2B-Instruct-GGUF is the GGUF quantized version of the multimodal vision-language model in the Tongyi Qianwen series. It has 2 billion parameters and supports the seamless integration of image understanding and text generation. It can run efficiently on devices such as CPUs and GPUs.

MultimodalTransformers

SecInt is a SmolLM2-360M model fine-tuned for real-time nginx security log classification, aiming to automatically detect security threats, errors, and normal traffic patterns in web server logs with an accuracy of over 99%, enabling real-time detection on the CPU.

Natural Language ProcessingTransformersMultiple Languages

VieNeu-TTS is the first Vietnamese text-to-speech model that can run on personal devices and has the ability of instant voice cloning. Fine-tuned based on NeuTTS Air, it can generate natural and realistic Vietnamese speech and has real-time performance on the CPU.

Audio ProcessingSafetensorsOther

This is a Modal Balance Quantization (MBQ) variant of DeepSeek-OCR. It reduces memory usage by applying 4-bit NF4 quantization to the visual encoder while maintaining the BF16 precision of the projector and language/decoder. It can be locally deployed on ordinary GPUs and used for rapid CPU experiments.

MultimodalTransformers

Svara-TTS is an open-source multilingual text-to-speech model for Indian languages, supporting 19 languages (18 Indian languages + Indian English). This model is built based on the Orpheus-style discrete audio token method, aiming to achieve clear, expressive, and low-latency speech synthesis on ordinary GPUs/CPUs.

Audio ProcessingTransformersMultiple Languages

This is the Q4_K_M quantized version of the DeepSeek-MoE-16B model from DeepSeek Company. It enables the MoE (Mixture of Experts) CPU offload function through the Rust binding of llama.cpp, significantly reducing video memory usage.

Natural Language ProcessingGgufMultiple Languages

This is the GGUF format version of the GPT-OSS 20B model based on WeOpenML, which first implements the innovative MoE CPU expert offloading technology. This technology achieves a 99.9% reduction in video memory while maintaining the full generation quality, and only requires 2MB of video memory to run a 20 billion parameter mixture of experts model.

Natural Language ProcessingSafetensorsMultiple Languages

This is the GGUF quantized version of the TheDrummer/Snowpiercer-15B-v3 model, offering a variety of quantization options, from high quality to low memory usage, and supporting efficient operation on both CPU and GPU. The model is quantized using llama.cpp and optimized for different hardware.

Natural Language ProcessingGguf

This is a professional quantized version of the Qwen3-Omni model with 31.7B parameters. It uses advanced INT8+FP16 mixed-precision quantization technology, reducing memory usage by over 50%. It supports intelligent GPU/CPU hybrid inference, enabling large multimodal models to run efficiently on consumer-grade hardware.

MultimodalSafetensorsMultiple Languages

ColModernVBERT is a late interaction version of ModernVBERT, specifically fine - tuned for the visual document retrieval task and is the best - performing model on this task. It is a compact vision - language encoder with 250 million parameters, achieving performance close to that of models with 10 times more parameters in visual document benchmark tests, while having a considerable inference speed on the CPU.

MultimodalSafetensorsEnglish

This model is a GGUF - formatted version converted from huihui - ai/Huihui - Tongyi - DeepResearch - 30B - A3B - abliterated, specifically optimized for llama.cpp and supports efficient CPU inference. The model uses the Q4_K_M quantization level, which significantly reduces the model size while maintaining good performance.

Natural Language ProcessingTransformersEnglish

A GGUF-quantized model fine-tuned based on Llama 3.1 8B Instruct, with a strong, decisive, and provocative AI persona, optimized for role-playing and creative writing scenarios, and supports local inference on CPU or GPU.

Natural Language ProcessingTransformers

A large language model optimized based on OpenAI's gpt-oss-120b model. It uses the NEO dataset to improve performance, supports multi-scenario applications, and can run on CPU, GPU, or in partial offloading mode. It has a 128k context length and a mixture of experts architecture with up to 128 experts.

Natural Language ProcessingGgufMultiple Languages

This is a coding/programming model based on the Qwen 3 platform, with complete reasoning ability and extremely fast speed. It can process more than 150 tokens per second on medium hardware and more than 50 tokens per second even when using only the CPU. It is a general-purpose coding model suitable for generating code blocks, ideating coding ideas, and quickly generating code drafts.

Natural Language ProcessingGgufMultiple Languages

Lucy is a lightweight self-network search model built on the 1.7 billion parameter Qwen3-1.7B, optimized for mobile devices and capable of efficient operation on CPUs.

Natural Language ProcessingTransformersEnglish

SmolLM3-3B-INT8-INT4 is a quantized version based on the HuggingFaceTB/SmolLM3-3B model. It uses torchao to implement 8-bit embedding, 8-bit dynamic activation, and 4-bit weight linear quantization. The model is converted to the ExecuTorch format and achieves high performance on the CPU backend through optimization, making it particularly suitable for mobile device deployment.

Natural Language ProcessingPytorch

This is a SPLADE sparse retrieval model based on BERT-Mini (11 million parameters), trained through knowledge distillation of a cross-encoder on the MSMARCO dataset. This model is 6 times smaller than the official splade-v3-distilbert but can achieve 85% of its performance and can process thousands of documents on a CPU.

Natural Language ProcessingSentence-transformersEnglish