statementSaid on 2026-10-19
Adding nonlinearity that silences somewhat-similar keys and keeps only the truly similar ones yields an MLP: one layer holds the keys, the second the values, and the activation decides who gets to speak.
A statement by post author, said on 2026-10-19. Statements are as reported by the cited publishers — not verified by DecisionGateway.
אז מוסיפים אי-לינאריות שמשתיקה את המפתחות ״דומים במידת מה" ומשאירה רק את הדומים באמת.