Popular GitHub Repositories for Ai Safety

Secrets of RLHF in Large Language Models Part I: PPO

PromptInject is a framework that assembles prompts in a modular fashion to provide a quantitative analysis of the robustness of LLMs to adversarial prompt attacks. ? Best Paper Awards @ NeurIPS ML Safety Workshop 2022

[NeurIPS '23 Spotlight] Thought Cloning: Learning to Think while Acting by Imitating Human Thinking

RuLES: a benchmark for evaluating rule-following in language models

Code accompanying the paper Pretraining Language Models with Human Preferences

How to Make Safe AI? Let's Discuss! ?|?|?|?

Toolkits to create a human-in-the-loop approval layer to monitor and guide AI agents workflow in real-time.

Attack to induce LLMs within hallucinations

[ICLR'24 Spotlight] A language model (LM)-based emulation framework for identifying the risks of LM agents with tool use

BeaverTails is a collection of datasets designed to facilitate research on safety alignment in large language models (LLMs).

[CCS'24] SafeGen: Mitigating Unsafe Content Generation in Text-to-Image Models

Safety Score for Pre-Trained Language Models

[SafeAI'21] Feature Space Singularity for Out-of-Distribution Detection.

A curated list of awesome academic research, books, code of ethics, data sets, institutes, maturity models, newsletters, principles, podcasts, reports, tools, regulations and standards related to Responsible, Trustworthy, and Human-Centered AI.

Alpha principles for the ethical use of AI and Data Driven Technologies in Ontario | Proposition de principes pour une utilisation éthique des technologies axées sur les données en Ontario

Benchmark evaluating LLMs on their ability to create and resist disinformation. Includes comprehensive testing across major models (Claude, GPT-4, Gemini, Llama, etc.) with standardized evaluation metrics.

Repository accompanying the paper https://arxiv.org/abs/2407.14937

A curated list of references on the role of AI in safety-critical systems ??

In situ interactive widgets for responsible AI ?

LAWLIA is an open-source computational legal framework designed to revolutionize legal reasoning and analysis. It combines the power of large language models with a structured syntactical grammar to facilitate precise legal assessments, truth values, and verdicts. LAWLIA is the future of computational jurisprudence