Phi-3-mini-4k-instruct-onnx

Quantized ONNX model of Phi-3 Mini, supporting acceleration of inference on multiple hardware platforms

PremiumNewProductProgrammingNatural Language ProcessingGenerative AI Model

Phi-3 Mini is a lightweight, advanced open-source large-scale model constructed on synthetic data from Phi-2 and filtered website data, committed to providing high-quality, inference-intensive data. The model has undergone a rigorous enhancement process, combining supervised fine-tuning and direct preference optimization to ensure strict adherence to instructions and robust security measures. This repository provides an optimized ONNX version of Phi-3 Mini, which can be accelerated on CPUs and GPUs using ONNX Runtime, supporting servers, Windows, Linux, Mac, and various platforms, with the best precision configuration tailored for each platform. ONNX Runtime's DirectML support allows developers to achieve large-scale hardware acceleration on Windows devices with AMD, Intel, and NVIDIA GPUs.

- Supports acceleration of inference on multiple hardware platforms
including: \n - DirectML: for Windows devices with AMD
Intel
and NVIDIA GPUs
with int4 precision achieved through AWQ quantization\n - FP16 CUDA: for NVIDIA GPUs
with FP16 precision\n - Int4 CUDA: for NVIDIA GPUs
with int4 precision achieved through RTN quantization\n - Int4 CPU and mobile: with int4 precision achieved through RTN quantization
providing two versions for CPUs and mobile devices to balance latency and precision
- Offers a new Generate() API for ONNX Runtime
greatly simplifying the integration process of generative AI models in applications

- Business: Integration of Phi-3 Mini into various business applications to provide natural language processing capabilities
- Developers: Leverage the powerful generation capabilities of Phi-3 Mini to develop various language-related applications and services
such as conversational systems
Q&A systems
text generation

1. Integrate Phi-3 Mini into the intelligent assistant system of a business to provide customers with natural language interaction and generative services
2. Develop automatic text generation and creative assistance tools based on Phi-3 Mini to provide support for writers
content creators
and others
3. Utilize the inference capabilities of Phi-3 Mini to build data analysis and report generation systems

1. Go to the Hugging Face page to download the required ONNX model files
2. Install the ONNX Runtime and ONNX Runtime Generate() API-related software packages
3. Load the ONNX model file in the code
4. Use the ONNX Runtime Generate() API to set inference parameters
such as batch size

Visit

Phi-3-mini-4k-instruct-onnx Visit Over Time

Monthly Visits

17104189

Bounce Rate

44.67%

Page per Visit

5.5

Visit Duration

00:05:49

Phi-3-mini-4k-instruct-onnx Visit Trend

Phi-3-mini-4k-instruct-onnx Visit Geography

Phi-3-mini-4k-instruct-onnx Traffic Sources

Phi-3-mini-4k-instruct-onnx Alternatives

iFLYTEK Spark — Comprehensively benchmarks the AI large language model against GPT-4 Turbo.

ChineseSelection•Large Model•Natural Language Processing

Phi-3-mini-128k-instruct-onnx — Phi-3 Mini-128K-Instruct ONNX optimized model for inference acceleration

Programming•Natural Language Processing•Large Language Model

546

Meta Llama 3 — Meta's new generation of open-source large language model with excellent performance

GlobalTrending•Large Model•Open Source

4188

Qwen1.5-MoE-A2.7B — A large-scale MoE (Mixture of Experts) language model whose performance rivals that of 70 billion parameter models.

Programming•Natural Language Processing•Large Model

480

Zero & All Things Large Model Open Platform — An open platform that offers access to high-quality Zero & All Things large models via the API

ChineseSelection•Large Model•AI Model

3144

Llama 3 — A new generation of open-source large language model with excellent performance.

Productivity•Large Model•Open-Source

4260

Baichuan 3 — A large language model with over trillion parameters

ChineseSelection•Language model•Natural language processing

3630

OneGen — An efficient single-pass unified generation and retrieval framework suitable for large language models.

Programming•Natural Language Processing•Large Language Models

Copilot in Excel — Copilot in Excel: Unlocking data-driven decision-making.

Productivity•Data Analysis•Automation

Mistral-Small-Instruct-2409 — High-performance instruct-tuned AI model

Programming•Artificial Intelligence•Natural Language Processing

Qwen2.5-LLM — An open-source high-performance language model that supports multi-platform applications.

Productivity•Natural Language Processing•Machine Learning

Qwen2.5 — An open-source large language model that supports multilingual and specialized domain applications.

Productivity•Artificial Intelligence•Natural Language Processing

xAI — AI to Accelerate Human Scientific Discovery

AgentRE — A framework based on agents for relationship extraction in complex information environments.

Programming•Relationship Extraction•Natural Language Processing

Skywork-Reward-Llama-3.1-8B — An advanced reward model for text classification and preference assessment.

Programming•Machine Learning•Natural Language Processing

Question — An intelligent Q&A system that offers in-depth insights and answers.

InternationalSelection•Intelligent Q&A•Natural Language Processing

120

DataGemma RIG — A text model designed for retrieving and generating content that integrates statistical data.

Others•Text Generation•Natural Language Processing

assistant-ui-stockbroker — An AI-powered stockbroker assistant that provides professional investment advice.

Business•Artificial Intelligence•Financial Analysis

pixtral-12b-240910 — A multimodal large language model that supports understanding of both images and text.

Image•Multimodal•Image Processing

MemoRAG — A memory-based RAG framework for general-purpose applications

Programming•Natural Language Processing•Machine Learning

AI-Researcher — Artificial intelligence tool for generating research ideas and project management

Education•Artificial Intelligence•Research Assistance

138

Qihoo-T2X — Qihoo-T2X, an efficient diffusion transformer model aimed at text-to-any-task processing.

Programming•Text Processing•Diffusion Transformer

rStar — Enhances problem-solving capabilities of small language models through self-play mutual reasoning.