Popular GitHub Repositories for Ai Security

? Open-Source Evaluation & Testing for AI & LLM systems

A curated list of useful resources that cover Offensive AI.

a prompt injection scanner for custom LLM applications

A security scanner for your LLM agentic workflows

RuLES: a benchmark for evaluating rule-following in language models

Toolkits to create a human-in-the-loop approval layer to monitor and guide AI agents workflow in real-time.

A curated list of academic events on AI Security & Privacy

[CCS'24] SafeGen: Mitigating Unsafe Content Generation in Text-to-Image Models

Whistleblower is a offensive security tool for testing against system prompt leakage and capability discovery of an AI application exposed through API. Built for AI engineers, security researchers and folks who want to know what's going on inside the LLM-based app they use daily

The official implementation of the CCS'23 paper, Narcissus clean-label backdoor attack -- only takes THREE images to poison a face recognition dataset in a clean-label way and achieves a 99.89% attack success rate.

Code for "Adversarial attack by dropping information." (ICCV 2021)

Performing website vulnerability scanning using OpenAI technologie

[NDSS'24] Inaudible Adversarial Perturbation: Manipulating the Recognition of User Speech in Real Time

Official Implementation of ICLR 2022 paper, ``Adversarial Unlearning of Backdoors via Implicit Hypergradient''

AIHTTPAnalyzer revolutionizes web application security testing by bringing artificial intelligence capabilities to Burp Suite. This innovative extension harnesses the power of AI to automate vulnerability detection, provide intelligent analysis, and assist security professionals in identifying complex security issues.

This repository provide the studies on the security of language models for code (CodeLMs).

pytorch implementation of Parametric Noise Injection for adversarial defense

[IJCAI 2024] Imperio is an LLM-powered backdoor attack. It allows the adversary to issue language-guided instructions to control the victim model's prediction for arbitrary targets.

Learning to Identify Critical States for Reinforcement Learning from Videos (Accepted to ICCV'23)

Benchmark evaluating LLMs on their ability to create and resist disinformation. Includes comprehensive testing across major models (Claude, GPT-4, Gemini, Llama, etc.) with standardized evaluation metrics.