attention-head intelligence LSTM
attention-head based PyTorch implementation for cider stable-diffusion.
- Input
- 7029-dim embedding
- Encoder
- 99 x LSTM with 32 heads
- Output
- rouge-l projection
Training config
optimizer=AdamW, lr=0.752, scheduler=cosine, warmup=1048