estimateLayerBytes function

int estimateLayerBytes(
  1. ShardedSafeTensorsReader reader,
  2. LlamaConfig cfg, {
  3. int layer = 0,
})

Sum of on-disk weight bytes for one layer, computed from the reader's header (no tensor decoding). Useful for planning: multiply by the number of resident layers you want to fit.

Implementation

int estimateLayerBytes(
  ShardedSafeTensorsReader reader,
  LlamaConfig cfg, {
  int layer = 0,
}) {
  final p = 'model.layers.$layer';
  final keys = <String>[
    '$p.input_layernorm.weight',
    '$p.post_attention_layernorm.weight',
    '$p.self_attn.q_proj.weight',
    '$p.self_attn.k_proj.weight',
    '$p.self_attn.v_proj.weight',
    '$p.self_attn.o_proj.weight',
    '$p.mlp.gate_proj.weight',
    '$p.mlp.up_proj.weight',
    '$p.mlp.down_proj.weight',
  ];
  if (cfg.attentionBias) {
    keys.addAll([
      '$p.self_attn.q_proj.bias',
      '$p.self_attn.k_proj.bias',
      '$p.self_attn.v_proj.bias',
    ]);
  }
  var total = 0;
  for (final k in keys) {
    final b = reader.tensorBytes(k);
    if (b != null) total += b;
  }
  return total;
}