pullrequests Search Results · bucket language:JavaScript language:Java language:JavaScript language:Python language:Python
Filter by
1M results
Summary
Part 2 of 2 (TorchTitan TPU Engine SFT pad_to_length Support):
1. Bucket-Padded SFT Loss Adapter (prepare_tpu_loss_inputs):
- Adapts (model_output, micro_batch) in ...
askhat-g
jorgecuestaSummary
- extend the shared prefill CUDA Graph ladder with 1- and 2-token buckets
- centralize effective capture-cap and bucket resolution across model defaults and operator overrides ...
AkazaAkane
... the nearest captured bucket (3000 → 2816, 4500
→ 4096, 4607 → 4096). The decode path appends the ceiling when it is off-grid, and the CPU graph path unions it in;
prefill was the odd one out. Since ...
he-yufeng
prudhvimaharishi
polsala
ayam04... eagerly and syncs the host.
Modifications
- Capture the FlashInfer KDA prefill in breakable prefill graphs: each bucket keeps one RecurrentKDAPrefillWrapper per
layer, re-planned on device before ...
jit-kernel
mmangkad
Problem and change
Track confidence-head presence across load_weights calls; only remove an absent head at final post-load processing,
rather than after an unrelated first bucket. Preserve dummy ...
aoshen02
d4mation