7 ms·
Learnable parameters on activations do exist, look up parametric activation functions.
by beckhamc 2y ago
Learnable parameters on activations do exist, look up parametric activation functions.
- quantadev 2y agoIf course they do exist. A parameterized activation function is the most obvious thing to try in NN design, and has certainly been invented/studied by 1000s of researchers.