Phi-3-mini-4k-instruct-onnx

Quantized ONNX model of Phi-3 Mini, supporting acceleration of inference on multiple hardware platforms

PremiumNewProductProgrammingNatural Language ProcessingGenerative AI Model

Phi-3 Mini is a lightweight, advanced open-source large-scale model constructed on synthetic data from Phi-2 and filtered website data, committed to providing high-quality, inference-intensive data. The model has undergone a rigorous enhancement process, combining supervised fine-tuning and direct preference optimization to ensure strict adherence to instructions and robust security measures. This repository provides an optimized ONNX version of Phi-3 Mini, which can be accelerated on CPUs and GPUs using ONNX Runtime, supporting servers, Windows, Linux, Mac, and various platforms, with the best precision configuration tailored for each platform. ONNX Runtime's DirectML support allows developers to achieve large-scale hardware acceleration on Windows devices with AMD, Intel, and NVIDIA GPUs.

- Supports acceleration of inference on multiple hardware platforms
including: \n - DirectML: for Windows devices with AMD
Intel
and NVIDIA GPUs
with int4 precision achieved through AWQ quantization\n - FP16 CUDA: for NVIDIA GPUs
with FP16 precision\n - Int4 CUDA: for NVIDIA GPUs
with int4 precision achieved through RTN quantization\n - Int4 CPU and mobile: with int4 precision achieved through RTN quantization
providing two versions for CPUs and mobile devices to balance latency and precision
- Offers a new Generate() API for ONNX Runtime
greatly simplifying the integration process of generative AI models in applications

- Business: Integration of Phi-3 Mini into various business applications to provide natural language processing capabilities
- Developers: Leverage the powerful generation capabilities of Phi-3 Mini to develop various language-related applications and services
such as conversational systems
Q&A systems
text generation

1. Integrate Phi-3 Mini into the intelligent assistant system of a business to provide customers with natural language interaction and generative services
2. Develop automatic text generation and creative assistance tools based on Phi-3 Mini to provide support for writers
content creators
and others
3. Utilize the inference capabilities of Phi-3 Mini to build data analysis and report generation systems

1. Go to the Hugging Face page to download the required ONNX model files
2. Install the ONNX Runtime and ONNX Runtime Generate() API-related software packages
3. Load the ONNX model file in the code
4. Use the ONNX Runtime Generate() API to set inference parameters
such as batch size

Visit

Phi-3-mini-4k-instruct-onnx Visit Over Time

Monthly Visits

19075321

Bounce Rate

45.07%

Page per Visit

5.5

Visit Duration

00:05:32

Phi-3-mini-4k-instruct-onnx Visit Trend

Phi-3-mini-4k-instruct-onnx Visit Geography

Phi-3-mini-4k-instruct-onnx Traffic Sources

Phi-3-mini-4k-instruct-onnx Alternatives

iFLYTEK Spark — Comprehensively benchmarks the AI large language model against GPT-4 Turbo.

ChineseSelection•Large Model•Natural Language Processing

324

Phi-3-mini-128k-instruct-onnx — Phi-3 Mini-128K-Instruct ONNX optimized model for inference acceleration

Programming•Natural Language Processing•Large Language Model

564

Meta Llama 3 — Meta's new generation of open-source large language model with excellent performance

GlobalTrending•Large Model•Open Source

4494

Qwen1.5-MoE-A2.7B — A large-scale MoE (Mixture of Experts) language model whose performance rivals that of 70 billion parameter models.

Programming•Natural Language Processing•Large Model

480

Zero & All Things Large Model Open Platform — An open platform that offers access to high-quality Zero & All Things large models via the API

ChineseSelection•Large Model•AI Model

3294

Llama 3 — A new generation of open-source large language model with excellent performance.

Productivity•Large Model•Open-Source

4590

Baichuan 3 — A large language model with over trillion parameters

ChineseSelection•Language model•Natural language processing

4152

Dokko — Conversational AI platform for knowledge sharing

chatting•Chatbot•Knowledge Management

Qwen Turbo 1M Demo — The Qwen Turbo 1M Demo is a Hugging Face space provided by Qwen.

Productivity•Natural Language Processing•Machine Learning

Pixtral-Large-Instruct-2411 — A 124B-parameter multimodal large language model.

Productivity•Multimodal•Large Language Model

Ultravox.ai — Next-generation voice AI, creating AI voice agents for natural communication.

Programming•AI Voice•Natural Language Processing

156

LLaMA-Mesh — Unified 3D Mesh Generation with Language Models

Productivity•3D Modeling•Artificial Intelligence

RMBG-2.0 — A model designed for efficient separation of image foreground and background.

Image•Image Segmentation•Background Removal

186

PlayDialog — PlayDialog creates smooth and emotionally engaging dialogue experiences.

chatting•AI Voice Model•Natural Language Processing

Qwen2.5-Coder-0.5B — The 0.5B parameter code generation model in the Qwen2.5-Coder series.

Programming•Code Generation•Code Reasoning

Yujing — Yujing, an intelligent language processing platform

ChineseSelection•Natural Language Processing•Text Analysis

366

OuteTTS — An experimental text-to-speech model.

Productivity•Text-to-Speech•Speech Synthesis

708

Chat.com — An interactive dialogue AI model offering Q&A and text generation services

InternationalSelection•Dialogue Generation•Natural Language Processing

222

InkSight — Handwritten note digitization model that does not require specialized equipment.

Others•Handwriting Recognition•Digitization

132

askrepo — Utilizes LLM to read source code and provide answers to questions.

Programming•Code Analysis•LLM

MobileLLM-1B — A sub-billion parameter language model developed by Meta, suitable for device-side applications.

Programming•Language Model•Transformer

O1-Journey — O1 Journey: Strategic Progress Report Part One

Programming•Artificial Intelligence•Machine Learning

OpenHands on Daytona — AI assistant to enhance programming efficiency and collaboration

Programming•AI Programming Assistant•Code Automation

108

Tencent-Hunyuan-Large — Industry-leading open-source large mixture-of-experts model

Productivity•Artificial Intelligence•Natural Language Processing

234

PARTNR — Benchmarking for Multi-Agent Task Planning and Reasoning

Others•Multi-Agent•Natural Language Processing

MobileLLM-350M — An efficiently optimized language model with sub-billion parameters, specifically designed for device-side applications.