Latest Version: 150,000 Steps

• 9,600,000 tokens seen.

Model Info:

• Test aitextgen GPT-2 Model. Trained from scratch.
• 6.9M parameters.
• 64 context length.

Config

batch_size: 1
dropout: 0
learning_rate: 0.0001
max_length: 64
n_embed: 256
n_head: 8
n_layer: 8
vocab_size: 2048