Things worth to meantion:

1. The float type teacher embedding is quantized into a sequence of 8-bit integer codebook indexes.
2. a middle layer 36(1-based) out of total 48 layers is used to extract teacher embeddings.
3. a middle layer 6(1-based) out of total 6 layers is used to extract student embeddings.