Phi-3-mini-4k-instruct-onnx

Quantized ONNX model of Phi-3 Mini, supporting acceleration of inference on multiple hardware platforms

PremiumNewProductProgrammingNatural Language ProcessingGenerative AI Model

Phi-3 Mini is a lightweight, advanced open-source large-scale model constructed on synthetic data from Phi-2 and filtered website data, committed to providing high-quality, inference-intensive data. The model has undergone a rigorous enhancement process, combining supervised fine-tuning and direct preference optimization to ensure strict adherence to instructions and robust security measures. This repository provides an optimized ONNX version of Phi-3 Mini, which can be accelerated on CPUs and GPUs using ONNX Runtime, supporting servers, Windows, Linux, Mac, and various platforms, with the best precision configuration tailored for each platform. ONNX Runtime's DirectML support allows developers to achieve large-scale hardware acceleration on Windows devices with AMD, Intel, and NVIDIA GPUs.

- Supports acceleration of inference on multiple hardware platforms
including: \n - DirectML: for Windows devices with AMD
Intel
and NVIDIA GPUs
with int4 precision achieved through AWQ quantization\n - FP16 CUDA: for NVIDIA GPUs
with FP16 precision\n - Int4 CUDA: for NVIDIA GPUs
with int4 precision achieved through RTN quantization\n - Int4 CPU and mobile: with int4 precision achieved through RTN quantization
providing two versions for CPUs and mobile devices to balance latency and precision
- Offers a new Generate() API for ONNX Runtime
greatly simplifying the integration process of generative AI models in applications

- Business: Integration of Phi-3 Mini into various business applications to provide natural language processing capabilities
- Developers: Leverage the powerful generation capabilities of Phi-3 Mini to develop various language-related applications and services
such as conversational systems
Q&A systems
text generation

1. Integrate Phi-3 Mini into the intelligent assistant system of a business to provide customers with natural language interaction and generative services
2. Develop automatic text generation and creative assistance tools based on Phi-3 Mini to provide support for writers
content creators
and others
3. Utilize the inference capabilities of Phi-3 Mini to build data analysis and report generation systems

1. Go to the Hugging Face page to download the required ONNX model files
2. Install the ONNX Runtime and ONNX Runtime Generate() API-related software packages
3. Load the ONNX model file in the code
4. Use the ONNX Runtime Generate() API to set inference parameters
such as batch size

Visit

Phi-3-mini-4k-instruct-onnx Visit Over Time

Monthly Visits

20899836

Bounce Rate

46.04%

Page per Visit

5.2

Visit Duration

00:04:57

Phi-3-mini-4k-instruct-onnx Visit Trend

Phi-3-mini-4k-instruct-onnx Visit Geography

Phi-3-mini-4k-instruct-onnx Traffic Sources

Phi-3-mini-4k-instruct-onnx Alternatives

iFLYTEK Spark — Comprehensively benchmarks the AI large language model against GPT-4 Turbo.

ChineseSelection•Large Model•Natural Language Processing

492

Phi-3-mini-128k-instruct-onnx — Phi-3 Mini-128K-Instruct ONNX optimized model for inference acceleration

Programming•Natural Language Processing•Large Language Model

576

Meta Llama 3 — Meta's new generation of open-source large language model with excellent performance

GlobalTrending•Large Model•Open Source

4656

Qwen1.5-MoE-A2.7B — A large-scale MoE (Mixture of Experts) language model whose performance rivals that of 70 billion parameter models.

Programming•Natural Language Processing•Large Model

480

Zero & All Things Large Model Open Platform — An open platform that offers access to high-quality Zero & All Things large models via the API

ChineseSelection•Large Model•AI Model

3396

Llama 3 — A new generation of open-source large language model with excellent performance.

Productivity•Large Model•Open-Source

4740

Baichuan 3 — A large language model with over trillion parameters

ChineseSelection•Language model•Natural language processing

4380

Gemini 2.0 Flash Experimental — A high-performance AI model developed by Google DeepMind

InternationalSelection•Machine Learning•Natural Language Processing

144

DeepSeek-VL2-Tiny — Advanced Large-scale Mixture of Experts Visual Language Model

Image•Visual Language Model•Mixture of Experts

Megrez-3B-Omni — Open-source full-modal understanding model for edge deployment

Productivity•Full-modal understanding•Image recognition

Q-RWKV-6 32B Instruct Preview — The most powerful variant of the RWKV model, breaking multiple English benchmark tests.

Programming•Machine Learning•Natural Language Processing

CosyVoice Speech Generation Model 2.0-0.5B — Efficient, multilingual speech synthesis model

Music•Speech Synthesis•Artificial Intelligence

180

WePOINTS — WePOINTS project offers a unified framework for multimodal models.

Programming•Multimodal•Artificial Intelligence

tulu-3-sft-olmo-2-mixture — A large-scale multilingual text dataset.

Others•Multilingual•Text Dataset

Devin.ai — AI programming assistant that aids engineering teams in increasing efficiency.

InternationalSelection•AI programming assistant•code migration

120

OLMo-2-1124-7B-RM — A large language model for text generation and classification.

Productivity•Artificial Intelligence•Natural Language Processing

Project Astra — Explore the capabilities of a future general-purpose AI assistant.

InternationalSelection•Assistant•Natural Language Processing

174

InternVL2_5-38B — Advanced Multimodal Large Language Model Series

Image•Multimodal•Large Language Models

108

mwp_ReFT — A deep reinforcement learning-based model fine-tuning framework

Programming•Natural Language Processing•Deep Learning

Meta Llama 3.3 — A multilingual large pre-trained language model with 70 billion parameters.

Programming•Multilingual•Pre-trained Model

Florence-VL — Enhancement tool for visual language models, combining generative visual encoders and deep breadth fusion technology.

Programming•Visual Language Models•Multimodal Learning

ChatGPT Pro — Scalable access plan for cutting-edge AI models.

InternationalSelection•Machine Learning•Data Science

264

PaliGemma 2 — PaliGemma 2 is a powerful visual language model that is easy to fine-tune.

Productivity•Visual Language Model•Machine Learning

OLMo 2 1124 7B Preference Mixture — A large-scale textual dataset for preference mixture research.

Others•Natural Language Processing•Text Dataset

OLMo 2 1124 13B Preference Mixture — Large-scale multilingual preference mixture dataset

Others•Dataset•Multilingual

Best AI Websites & Tools