airllm

大规模中文自然语言处理语料 Large Scale Chinese Corpus for NLP

BELLE: Be Everyone's Large Language model Engine（开源中文对话大模型）

BISHENG is an open LLM devops platform for next generation Enterprise AI applications. Powerful and comprehensive features include: GenAI workflow, RAG, Agent, Unified model management, Evaluation, SFT, Dataset Management, Enterprise-level System Management, Observability and more.

A curated list of resources for Chinese NLP 中文自然语言处理相关资料

SD-Trainer. LoRA & Dreambooth training scripts & GUI use kohya-ss's trainer, for diffusion model.

Fengshenbang-LM(封神榜大模型)是IDEA研究院认知计算与自然语言研究中心主导的大模型开源体系，成为中文AIGC和认知智能的基础设施。

百度NLP：分词，词性标注，命名实体识别，词重要性

MNBVC(Massive Never-ending BT Vast Chinese corpus)超大规模中文语料集。对标chatGPT训练的40T数据。MNBVC数据集不但包括主流文化，也包括各个小众文化甚至火星文的数据。MNBVC数据集包括新闻、作文、小说、书籍、杂志、论文、台词、帖子、wiki、古诗、歌词、商品介绍、笑话、糗事、聊天记录等一切形式的纯文本中文数据。

Transfer Learning Library for Domain Adaptation, Task Adaptation, and Domain Generalization

fastNLP: A Modularized and Extensible NLP Framework. Currently still in incubation.