statementSaid on 2026-11-05

During the backward pass, the MTP residual noise and the NTP residual noise meet with opposite signs and cancel each other out, leaving only the clean self-distillation signal; the result is 0 added memory, 0 impact on token throughput, and full compatibility with Fused kernels.

A statement by post author, said on 2026-11-05. Statements are as reported by the cited publishers — not verified by DecisionGateway.

בזמן ה-Backward pass(בקפרופ) קורה איזה ״נס״ מתמטי: רעש השארית של ה-MTP ורעש השארית של ה-NTP נפגשים עם סימנים הפוכים ופשוט מאפסים זה את זה!
mathyaiwithmikePublished 2026-11-05 Read source

Discussion

0/2000