Session 1: Introduction to LLMs

• Simon Razniewski (TU Dresden)

Motivates LLM security through real-world incidents, then gives an accessible introduction to LLM internals: tokenization, context windows, the transformer architecture, training stages (pre-training, fine-tuning, RLHF), prompt roles, and how generation works.

Session 2: LLM Weaknesses & Hacking

• Himanshu Beniwal (TU Dresden)

Introduces the main attack categories: prompt injection, jailbreaking techniques, training data extraction, hallucination, and supply-chain risks such as system prompt theft and model poisoning.

Hands-on: Breaking an LLM

• Paramita Mirza (TU Dresden)
• Siavash Ghiasvand

Participants attack a pre-built chatbot through staged challenges—role-play attacks, delimiter manipulation, and indirect injection—aiming to extract the hidden system prompt or bypass safety checks.

Results & Debrief The group compares which attacks succeeded and why, with discussion of what participants would do differently as developers.

Session 3: LLMs & Privacy

• Hermann Diebel-Fischer (TU Dresden)

Examines how LLMs expose sensitive information: training data memorization, PII leakage, differential privacy and its limits, GDPR requirements, and re-identification attacks.

Session 4: Guardrails & Defense Mechanisms

• Himanshu Beniwal (TU Dresden)

Surveys defensive tools—input validation, output filtering, alignment techniques, LLM-as-judge, and prompt firewalls—along with their limitations.

Session 5: Secure LLM Architecture & Design [Discussion] Discusses the design of secure LLM systems.

Tutorial: Break & Fix a RAG System

• Siavash Ghiasvand
• Paramita Mirza (TU Dresden)

Participants learn how to exploit a deliberately vulnerable RAG pipeline, then fix it with access control and output filtering.

Results & Wrap-up Groups present findings, facilitators distill a practical pre-shipping checklist, and the workshop closes with further reading and next steps.