statementSaid on 2026-10-19

The exponent does exactly that, and after normalization you get softmax, i.e. attention; capacity jumps to about 1.2 million memories, from 64 to a million without adding a single parameter.

A statement by post author, said on 2026-10-19. Statements are as reported by the cited publishers — not verified by DecisionGateway.

האקספוננט עושה בדיוק את זה, ואחרי נרמול קיבלתם סופטמקס, כלומר attention
mathyaiwithmikePublished 2026-10-19 Read source

Discussion

0/2000