statementSaid on 2026-11-05
During the backward pass, the MTP residual noise and the NTP residual noise meet with opposite signs and cancel each other out, leaving only the clean self-distillation signal; the result is 0 added memory, 0 impact on token throughput, and full compatibility with Fused kernels.
A statement by post author, said on 2026-11-05. Statements are as reported by the cited publishers — not verified by DecisionGateway.
בזמן ה-Backward pass(בקפרופ) קורה איזה ״נס״ מתמטי: רעש השארית של ה-MTP ורעש השארית של ה-NTP נפגשים עם סימנים הפוכים ופשוט מאפסים זה את זה!