MHACache class

Constructors

MHACache.empty(int numKvHeads)

Properties

hashCode → int
The hash code for this object.
no setterinherited
k → List<Tensor?>
Per-KV-head running K, shape [T_seen, headDim]. null until the first append.
final
numKvHeads → int
Number of KV-head slots stored. For standard MHA this equals the number of Q heads; for GQA (Llama / Mistral / Qwen) it is smaller than the Q-head count and each cached slot is shared by several Q heads via head-grouping.
final
runtimeType → Type
A representation of the runtime type of the object.
no setterinherited
seqLen → int
Length of the cached sequence so far (0 if empty). All KV heads share the same T, so we read from KV head 0.
no setter
v → List<Tensor?>
Per-KV-head running V, shape [T_seen, headDim]. null until the first append.
final

Methods

appendK(int kvHead, Tensor newK) → Tensor
Append [N_new, headDim] K and V for a single KV head. Returns the updated concatenated tensor (also stored in the cache).
appendV(int kvHead, Tensor newV) → Tensor
noSuchMethod(Invocation invocation) → dynamic
Invoked when a nonexistent method or property is accessed.
inherited
toString() → String
A string representation of this object.
inherited

Operators

operator ==(Object other) → bool
The equality operator.
inherited