Popular GitHub Repositories for Llm Security

Ready-to-run cloud templates for RAG, AI pipelines, and enterprise search with live data. ?Docker-friendly.?Always in sync with Sharepoint, Google Drive, S3, Kafka, PostgreSQL, real-time data APIs, and more.

? Open-Source Evaluation & Testing for AI & LLM systems

the LLM vulnerability scanner

[CCS'24] A dataset consists of 15,140 ChatGPT prompts from Reddit, Discord, websites, and open-source datasets (including 1,405 jailbreak prompts).

The Security Toolkit for LLM Interactions

A secure low code honeypot framework, leveraging AI for System Virtualization.

Papers and resources related to the security and privacy of LLMs ?

A powerful tool for automated LLM fuzzing. It is designed to help developers and security researchers identify and mitigate potential jailbreaks in their LLM APIs.

? Vigil ? Detect prompt injections, jailbreaks, and other potentially risky Large Language Model (LLM) inputs

A security scanner for your LLM agentic workflows

This repository provides implementation to formalize and benchmark Prompt Injection attacks and defenses

Toolkits to create a human-in-the-loop approval layer to monitor and guide AI agents workflow in real-time.

Experimental tools to backdoor large language models by re-writing their system prompts at a raw parameter level. This allows you to potentially execute offline remote code execution without running any actual code on the victim's machine or thwart LLM-based fraud/moderation systems.

AI-driven Threat modeling-as-a-Code (TaaC-AI)

The fastest && easiest LLM security guardrails for CX AI Agents and applications.

Whistleblower is a offensive security tool for testing against system prompt leakage and capability discovery of an AI application exposed through API. Built for AI engineers, security researchers and folks who want to know what's going on inside the LLM-based app they use daily

Ultra-fast, low latency LLM prompt injection/jailbreak detection ??

Framework for LLM evaluation, guardrails and security

This repository contains various attack against Large Language Models.

A benchmark for prompt injection detection systems.