Phi-3-mini-4k-instruct-onnx

Quantized ONNX model of Phi-3 Mini, supporting acceleration of inference on multiple hardware platforms

PremiumNewProductProgrammingNatural Language ProcessingGenerative AI Model

Phi-3 Mini is a lightweight, advanced open-source large-scale model constructed on synthetic data from Phi-2 and filtered website data, committed to providing high-quality, inference-intensive data. The model has undergone a rigorous enhancement process, combining supervised fine-tuning and direct preference optimization to ensure strict adherence to instructions and robust security measures. This repository provides an optimized ONNX version of Phi-3 Mini, which can be accelerated on CPUs and GPUs using ONNX Runtime, supporting servers, Windows, Linux, Mac, and various platforms, with the best precision configuration tailored for each platform. ONNX Runtime's DirectML support allows developers to achieve large-scale hardware acceleration on Windows devices with AMD, Intel, and NVIDIA GPUs.

- Supports acceleration of inference on multiple hardware platforms
including: \n - DirectML: for Windows devices with AMD
Intel
and NVIDIA GPUs
with int4 precision achieved through AWQ quantization\n - FP16 CUDA: for NVIDIA GPUs
with FP16 precision\n - Int4 CUDA: for NVIDIA GPUs
with int4 precision achieved through RTN quantization\n - Int4 CPU and mobile: with int4 precision achieved through RTN quantization
providing two versions for CPUs and mobile devices to balance latency and precision
- Offers a new Generate() API for ONNX Runtime
greatly simplifying the integration process of generative AI models in applications

- Business: Integration of Phi-3 Mini into various business applications to provide natural language processing capabilities
- Developers: Leverage the powerful generation capabilities of Phi-3 Mini to develop various language-related applications and services
such as conversational systems
Q&A systems
text generation

1. Integrate Phi-3 Mini into the intelligent assistant system of a business to provide customers with natural language interaction and generative services
2. Develop automatic text generation and creative assistance tools based on Phi-3 Mini to provide support for writers
content creators
and others
3. Utilize the inference capabilities of Phi-3 Mini to build data analysis and report generation systems

1. Go to the Hugging Face page to download the required ONNX model files
2. Install the ONNX Runtime and ONNX Runtime Generate() API-related software packages
3. Load the ONNX model file in the code
4. Use the ONNX Runtime Generate() API to set inference parameters
such as batch size

Visit

Phi-3-mini-4k-instruct-onnx Visit Over Time

Monthly Visits

17788201

Bounce Rate

44.87%

Page per Visit

5.4

Visit Duration

00:05:32

Phi-3-mini-4k-instruct-onnx Visit Trend

Phi-3-mini-4k-instruct-onnx Visit Geography

Phi-3-mini-4k-instruct-onnx Traffic Sources

Phi-3-mini-4k-instruct-onnx Alternatives

iFLYTEK Spark — Comprehensively benchmarks the AI large language model against GPT-4 Turbo.

ChineseSelection•Large Model•Natural Language Processing

276

Phi-3-mini-128k-instruct-onnx — Phi-3 Mini-128K-Instruct ONNX optimized model for inference acceleration

Programming•Natural Language Processing•Large Language Model

558

Meta Llama 3 — Meta's new generation of open-source large language model with excellent performance

GlobalTrending•Large Model•Open Source

4452

Qwen1.5-MoE-A2.7B — A large-scale MoE (Mixture of Experts) language model whose performance rivals that of 70 billion parameter models.

Programming•Natural Language Processing•Large Model

480

Zero & All Things Large Model Open Platform — An open platform that offers access to high-quality Zero & All Things large models via the API

ChineseSelection•Large Model•AI Model

3198

Llama 3 — A new generation of open-source large language model with excellent performance.

Productivity•Large Model•Open-Source

4560

Baichuan 3 — A large language model with over trillion parameters

ChineseSelection•Language model•Natural language processing

3954

MobileLLM-350M — An efficiently optimized language model with sub-billion parameters, specifically designed for device-side applications.

Programming•Language Model•Transformer

xAI API — API for Grok series foundational models available for developers.

InternationalSelection•API•Developer Tools

Data Formulator — AI-powered data visualization tool

Productivity•Data Visualization•Natural Language Processing

Stagehand — An AI-driven web browsing framework focused on simplicity and scalability.

Programming•Automation•Web Browsing

ChatGPT Search — Get quick and timely answers through natural language searches.

InternationalSelection•Search•Natural Language Processing

180

LongRAG — Enhanced Retrieval-Augmented Generation Model for Long-Text Question Answering

Programming•Long-Text Question Answering•Retrieval-Augmented

The Open Interpreter Project — A new way for LLMs to execute code and complete computer tasks

Programming•Programming Automation•Artificial Intelligence

NotebookLlama — An open-source tool for building workflows from PDF to Podcast.

Programming•Natural Language Processing•Machine Learning

MaskGCT TTS Demo — Text-to-speech demonstration based on the MaskGCT model.

Others•Text-to-Speech•Deep Learning

318

llmc — A local inference command-line tool that converts natural language descriptions into executable shell commands.

Programming•Command-Line Tool•Natural Language Processing

Quantized Llama — An efficient, lightweight Quantized Llama model that enhances performance on mobile devices while reducing memory usage.

Productivity•Quantization•Mobile Devices

Precog by Ubik — An intelligent AI assistant that provides optimal solutions for model selection.

Programming•AI Assistant•Model Selection

YouGua — A browser plugin that allows users to program and control web content through natural language dialogue.

Programming•Browser Plugin•Natural Language Processing

270

GitHub to LLM Converter — Convert GitHub links into a format suitable for LLM.

Programming•GitHub•LLM

Manicode — Edit code libraries using natural language to improve programming efficiency.

InternationalSelection•AI Programming•Code Editing

CloudSoul — A DevOps agent for deploying cloud infrastructure using natural language.

Productivity•Cloud Infrastructure•DevOps

Reindeer AI — Large language model AI robot, replacing human customer service.

ChineseSelection•AI Customer Service•Natural Language Processing

Best AI Websites & Tools