File size: 950 Bytes
13b5551 a6b5967 13b5551 a6b5967 13b5551 4837cad 13b5551 4837cad |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 |
---
tags:
- weight-space-learning
- neural-network-autoencoder
- autoencoder
- transformer
datasets:
- maximuspowers/muat-fourier-5
---
# Weight-Space Autoencoder (TRANSFORMER)
This model is a weight-space autoencoder trained on neural network activation weights/signatures.
It includes both an encoder (compresses weights into latent representations) and a decoder (reconstructs weights from latent codes).
## Model Description
- **Architecture**: Transformer encoder-decoder
- **Training Dataset**: maximuspowers/muat-fourier-5
- **Input Mode**: signature
- **Latent Dimension**: 256
## Tokenization
- **Granularity**: neuron
- **Max Tokens**: 64
## Training Config
- **Loss Functions**: reconstruction, functional
- **Optimizer**: adamw
- **Learning Rate**: 0.0001
- **Batch Size**: 32
## Performance Metrics (Test Set)
- **MSE**: 0.122959
- **MAE**: 0.256139
- **RMSE**: 0.350655
- **Cosine Similarity**: 0.0575
- **R² Score**: 0.0069
|