Simplified-Rating-and-Preference-RL

Control GenAI interactions with power, precision, and consistency using LLM-native Conversation Design paradigms

RewardBench: the first evaluation tool for reward models.

PromptInject is a framework that assembles prompts in a modular fashion to provide a quantitative analysis of the robustness of LLMs to adversarial prompt attacks. ? Best Paper Awards @ NeurIPS ML Safety Workshop 2022

Free and open source code of the https://tournesol.app platform. Meet the community on Discord https://discord.gg/WvcSG55Bf3

Code accompanying the paper Pretraining Language Models with Human Preferences

How to Make Safe AI? Let's Discuss! ?|?|?|?

[AAAI'25 Oral] "MIA-Tuner: Adapting Large Language Models as Pre-training Text Detector".

The MAGICAL benchmark suite for robust imitation learning (NeurIPS 2020)

A curated list of awesome academic research, books, code of ethics, data sets, institutes, maturity models, newsletters, principles, podcasts, reports, tools, regulations and standards related to Responsible, Trustworthy, and Human-Centered AI.

Directional Preference Alignment