MedTrinity-25M

A large-scale multimodal medical dataset

CommonProductOthersMedical ImagingMultimodal

MedTrinity-25M is a large-scale multimodal dataset featuring multi-granular medical annotations. Developed by multiple authors, it aims to advance research in medical image and text processing. The dataset's construction involves steps such as data extraction and multi-granular text description generation, supporting various medical image analysis tasks, such as visual question answering (VQA) and pathology image analysis.

Data extraction: Extract key information from collected data
including metadata integration to generate rough titles
region-of-interest localization
and medical knowledge collection.
Multi-granular text description generation: Utilize this information to prompt large language models to generate fine-grained annotations.
Model training and evaluation: Provide scripts for model training and evaluation
supporting pre-training and fine-tuning on specific datasets.
Model library: Offer various pre-trained models
such as LLaVA-Med++
supporting fine-tuning on specific medical image analysis tasks.

MedTrinity-25M is primarily aimed at researchers and developers in the fields of medical image processing and natural language processing. It offers a rich collection of medical images and textual data
facilitating model training
algorithm testing
and the development of new methods.

Researchers utilized the MedTrinity-25M dataset to train a deep learning model capable of identifying lesions in medical images.
Developers leveraged the dataset to create a system for automatically generating medical image reports.
Educational institutions use MedTrinity-25M as a teaching resource to help students understand the complexities of medical image analysis.

1. Visit the GitHub page and clone the MedTrinity-25M dataset to your local machine.
2. Install the necessary packages and dependencies according to the quick start guide.
3. Download and install the base model LLaVA-Meta-Llama-3-8B-Instruct-FT-S2.
4. Follow the provided scripts for pre-training and fine-tuning the model.
5. Use the evaluation scripts to assess the performance of the trained model.

Visit

MedTrinity-25M Visit Over Time

Monthly Visits

494758773

Bounce Rate

37.69%

Page per Visit

5.7

Visit Duration

00:06:29

MedTrinity-25M Visit Trend

MedTrinity-25M Visit Geography

MedTrinity-25M Traffic Sources

MedTrinity-25M Alternatives

dolmino-mix-1124 — A high-quality dataset for the second phase of OLMo2 training.

Programming•Dataset•Natural Language Processing

Molmo — Advanced Multimodal AI Model Family

InternationalSelection•Multimodal•Image Recognition

312

pixtral-12b-240910 — A multimodal large language model that supports understanding of both images and text.

Image•Multimodal•Image Processing

120

West Lake AI Model — A multimodal model with high emotional and intellectual intelligence

ChineseSelection•Artificial Intelligence•Multimodal

282

FlashRAG — Efficient Toolkit for Retrieval-Augmented Generation Research

Programming•Natural Language Processing•Machine Learning

180

Gemini 1.5 Flash — A lightweight and high-performance AI model from Google, designed for large-scale, high-frequency tasks.

Productivity•Machine Learning•Multimodal

522

MNBVC — MNBVC is a massive Chinese corpus comparable to the 40T data used to train ChatGPT.

OpenSource•Natural Language Processing•Chinese Language Dataset

282

ml-ferret — End-to-end MLLM, enabling precise referencing and localization.

Programming•Machine Learning•Language Model

936

Distil-Whisper — A collection of models and datasets.

Productivity•Model•Dataset

588

Q-RWKV-6 32B Instruct Preview — The most powerful variant of the RWKV model, breaking multiple English benchmark tests.

Programming•Machine Learning•Natural Language Processing

CosyVoice Speech Generation Model 2.0-0.5B — Efficient, multilingual speech synthesis model

Music•Speech Synthesis•Artificial Intelligence

114

OLMo-2-1124-7B-RM — A large language model for text generation and classification.

Productivity•Artificial Intelligence•Natural Language Processing

InternVL2_5-1B — A large multimodal language model that supports image and text understanding.

Image•Multimodal•Large Language Model

MAmmoTH-VL — A Large-Scale Multimodal Reasoning and Instruction Tuning Platform

Others•Multimodal•Reasoning

InternVL2_5-38B — Advanced Multimodal Large Language Model Series

Image•Multimodal•Large Language Models

108

InternVL2_5-78B — Advanced multimodal large language model series

Image•Multimodal•Large Language Model

ChatGPT Pro — Scalable access plan for cutting-edge AI models.

InternationalSelection•Machine Learning•Data Science

252

PaliGemma 2 — PaliGemma 2 is a powerful visual language model that is easy to fine-tune.

Productivity•Visual Language Model•Machine Learning

OLMo 2 1124 7B Preference Mixture — A large-scale textual dataset for preference mixture research.

Others•Natural Language Processing•Text Dataset

OLMo 2 1124 13B Preference Mixture — Large-scale multilingual preference mixture dataset

Others•Dataset•Multilingual

Amazon Nova — Amazon Nova is Amazon's next-generation foundational model, offering cutting-edge intelligence and industry-leading cost-effectiveness.

InternationalSelection•AWS•Artificial Intelligence

138

OLMo-2-1124-7B-SFT — High-performance English text generation model

Productivity•Text Generation•Natural Language Processing

OLMo-2-1124-7B-DPO — An advanced text generation model supporting diverse task handling.