Agent Evaluation — Task Success Rate, Eval Harnesses, Benchmarks
How to evaluate AI agents: eval harnesses, task success rate, benchmarks like GAIA and Tau-Bench, LLM-as-a-judge — and how it differs from pure model quality.
Redaktion
AI Concepts
Grundbegriffe und Mechanik moderner Sprachmodelle und KI-Workflows.
-
- ki-konzepte Agenten
Agent Memory — Short-Term and Long-Term Memory for AI Agents
Short- vs. long-term memory in AI agents: context window, external memory stores and vector DBs — how agents keep context across sessions.
Redaktion - ki-konzepte Agenten
Building AI Agents — From Tool Use to Multi-Agent Systems
How AI agents work: from a single tool call through MCP, structured outputs and LangGraph to the question of when multi-agent setups actually pay off.
Redaktion - ki-konzepte LLM-Grundlagen
Embeddings Simply Explained
An embedding is a vector of numbers that places meaning in space. Why similar content sits close together and what embeddings are used for.
Redaktion - ki-konzepte Training & Fine-Tuning
Fine-Tuning Explained — When, How, With What
When fine-tuning pays off, which methods exist (SFT, DPO, LoRA, QLoRA), and what AMD vs. NVIDIA hardware actually means in practice.
Redaktion -
Function Calling / Tool Use
How an LLM uses tools: define a tool as a schema, the model picks the function and arguments, the result returns to the chat — the basis of every agent.
Redaktion - ki-konzepte LLM-Grundlagen
LLM Hallucinations — Causes and Remedies
Why LLMs confidently invent falsehoods, what types of hallucinations exist, and which remedies actually help — RAG, source enforcement, verification.
Redaktion - ki-konzepte Evaluation
Measuring LLM Quality — Evals, Benchmarks, Judges
How to tell whether an LLM system actually works: three evaluation layers from benchmarks to CI evals, plus pitfalls like Goodhart and judge bias.
Redaktion - ki-konzepte Prompting
Prompt Security — Injection, Leaking, Guardrails
How prompt injection, prompt leaking, and jailbreaks work — and which defenses (guardrails, spotlighting, sanitization) actually help.
Redaktion - ki-konzepte Prompting
Prompting Techniques — An Overview
Zero-shot, few-shot, chain-of-thought, tree of thoughts, ReAct & co. — when each prompting technique pays off and how they fit together.
Redaktion - ki-konzepte RAG
RAG Tuning and Balancing — Embeddings, Weights, Decay and MCP for Advanced Setups
How to steer retrieval on purpose: embedding choice, hybrid weights, reranker cascades, time decay, authority boost, MMR and MCP as a retrieval tool.
Redaktion -
Running LLMs locally — hardware, tools, models
How to run language models on your own hardware — VRAM requirements, tooling (Ollama, LM Studio, llama.cpp, vLLM) and which models fit which GPU.
Redaktion - ki-konzepte API-Nutzung
Structured Outputs
How Structured Outputs at Anthropic, OpenAI and Gemini enforce JSON schemas via constrained decoding — validated output for pipelines, not free text.
Redaktion - ki-konzepte LLM-Grundlagen
Temperature, Top-p and Sampling in LLMs
How an LLM picks the next token: temperature sharpens or flattens the probabilities, top-p and top-k limit the choice. Which setting for what.
Redaktion - ki-konzepte LLM-Grundlagen
The Context Window of LLMs — What Fits and What Doesn't
What the context window is, how big modern windows are, the lost-in-the-middle phenomenon, cost and latency, and the distinction from RAG and long-term memory.
Redaktion - ki-konzepte LLM-Grundlagen
Tokens and Tokenization in LLMs Explained
What a token is, how tokenizers split text, and why tokens drive cost, context window, and speed — with rules of thumb for estimating token counts.
Redaktion - ki-konzepte RAG
Understanding RAG — From Idea to Production System
How a RAG pipeline works: embedding, vector DB, retrieval, reranking, prompt — and which pitfalls show up in practice.
Redaktion - ki-konzepte API-Nutzung
Working with the LLM API — Streaming, Caching, Rate Limits
Practical API mechanics beyond pricing: streaming for UX, prompt caching against token cost, the Batch API for bulk jobs, rate limits without 429 drama.
Redaktion