In Transformer Machine Learning Models: Wie unterscheidet sich die Mehrkopf-Aufmerksamkeit von der normalen Aufmerksamkeit? Beschreiben Sie zunächst die Intuition und dann die mathematische Perspektive.
In a Transformer machine learning model, multi-head attention is a variation of the attention mechanism that allows the model to attend to different parts of the input sequence in parallel.

Intuition:
Normal attention works by computing a weighted sum of the values of the input sequence, where the weights are determined by the similarity between a query vector and each of the input vectors. Multi-head attention takes this idea one step further by allowing the model to compute multiple sets of attention weights, or "heads", in parallel, each focusing on different parts of the input sequence. The outputs of each head are then concatenated and transformed linearly to produce the final attention output. By attending to multiple parts of the input sequence simultaneously, multi-head attention can potentially capture more complex relationships between different parts of the input.

Mathematical perspective:
Formally, given a set of input vectors {v_1, v_2, ..., v_n} and a query vector q, the output of multi-head attention is computed as follows:

1. First, the input vectors are linearly transformed into a set of key vectors K = {k_1, k_2, ..., k_n} and a set of value vectors V = {v_1, v_2, ..., v_n} using learned weight matrices W_k and W_v:
K_i = W_k * v_i
V_i = W_v * v_i

2. The query vector q is also transformed using a learned weight matrix W_q:
Q = W_q * q

3. For each of h attention "heads", the query vector Q and key vectors K are used to compute a set of attention weights a^h = {a^h_1, a^h_2, ..., a^h_n}:
a^h_i = softmax((Q * K_i^T) / sqrt(d_k))
where d_k is the dimensionality of the key vectors, and the softmax function ensures that the attention weights sum to 1.

4. Each attention head produces a weighted sum of the value vectors V using the attention weights a^h:
h^h = sum(a^h_i * V_i)

5. The outputs of all the attention heads are concatenated into a single vector h^c, and linearly transformed using a learned weight matrix W_o to produce the final attention output:
h = W_o * h^c

By computing multiple sets of attention weights in parallel, multi-head attention allows the model to attend to different parts of the input sequence with different priorities, potentially capturing more complex relationships between different parts of the input.