Session 1: Introduction to LLMs

• Simon Razniewski (TU Dresden)

Motivates LLM security through real-world incidents, then gives an accessible introduction to LLM internals: tokenization, context windows, the transformer architecture, training stages (pre-training, fine-tuning, RLHF), prompt roles, and how generation works.

Session 2: LLM Weaknesses & Hacking

• Himanshu Beniwal (TU Dresden)

Introduces the main attack categories: prompt injection, jailbreaking techniques, training data extraction, hallucination, and supply-chain risks such as system prompt theft and model poisoning.

Hands-on: Breaking an LLM

• Siavash Ghiasvand (TU Dresden)
• Paramita Mirza (TU Dresden)

Participants attack a pre-built chatbot through staged challenges—role-play attacks, delimiter manipulation, and indirect injection—aiming to extract the hidden system prompt or bypass safety checks.

Results & Debrief The group compares which attacks succeeded and why, with discussion of what participants would do differently as developers.

Session 3: LLMs & Privacy

• Hermann Diebel-Fischer (TU Dresden)

Examines how LLMs expose sensitive information: training data memorization, PII leakage, differential privacy and its limits, GDPR requirements, and re-identification attacks.

Session 4: Guardrails & Defense Mechanisms Surveys defensive tools—input validation, output filtering, alignment techniques, LLM-as-judge, and prompt firewalls—along with their limitations.

Session 5: Secure LLM Architecture & Design Covers designing secure LLM systems: least privilege for agents, trust boundaries, access control for RAG, logging and auditing, and STRIDE-adapted threat modelling.

Hands-on: Break & Fix a RAG System

• Siavash Ghiasvand (TU Dresden)
• Paramita Mirza (TU Dresden)

Participants exploit a deliberately vulnerable RAG pipeline, then fix it with access control and output filtering, documenting their work in a short remediation note.

Results & Wrap-up Groups present findings, facilitators distill a practical pre-shipping checklist, and the workshop closes with further reading and next steps.