<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->

flan-t5-base-opus-en-id-id-en

This model consist to be Translator in multimodal Indonesia and English only.

<!---This model was trained from scratch on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.3685
• Score: 35.0259
• Counts: [4617, 2627, 1550, 883]
• Totals: [7288, 6288, 5297, 4382]
• Precisions: [63.350713501646545, 41.777989821882954, 29.261846328110252, 20.150616157005935]
• Bp: 0.991
• Sys Len: 7288
• Ref Len: 7354
• Gen Len: 10.556 Learning Rate: 0.0004-->

Model Details

Model Description

• Model type: Language model
• Language(s) (NLP): English, Indonesian
• License: Apache 2.0

Usage

Using the Pytorch model

Running the model on a CPU

<details> <summary> Click to expand </summary>

from transformers import T5Tokenizer, T5ForConditionalGeneration
tokenizer = T5Tokenizer.from_pretrained("muvazana/flan-t5-base-opus-en-id-id-en")
model = T5ForConditionalGeneration.from_pretrained("muvazana/flan-t5-base-opus-en-id-id-en")
input_text = "translate English to Indonesia: How old are you?"
input_ids = tokenizer(input_text, return_tensors="pt").input_ids
outputs = model.generate(input_ids)
print(tokenizer.decode(outputs[0]))

</details>

Running the model on a GPU

<details> <summary> Click to expand </summary>

# pip install accelerate
from transformers import T5Tokenizer, T5ForConditionalGeneration
tokenizer = T5Tokenizer.from_pretrained("muvazana/flan-t5-base-opus-en-id-id-en")
model = T5ForConditionalGeneration.from_pretrained("muvazana/flan-t5-base-opus-en-id-id-en", device_map="auto")
input_text = "translate English to Indonesia: How old are you?"
input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to("cuda")
outputs = model.generate(input_ids)
print(tokenizer.decode(outputs[0]))

</details>

Running the model on a GPU using different precisions

FP16

<details> <summary> Click to expand </summary>

# pip install accelerate
import torch
from transformers import T5Tokenizer, T5ForConditionalGeneration
tokenizer = T5Tokenizer.from_pretrained("muvazana/flan-t5-base-opus-en-id-id-ene")
model = T5ForConditionalGeneration.from_pretrained("muvazana/flan-t5-base-opus-en-id-id-en", device_map="auto", torch_dtype=torch.float16)
input_text = "translate English to Indonesia: How old are you?"
input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to("cuda")
outputs = model.generate(input_ids)
print(tokenizer.decode(outputs[0]))

</details>

INT8

<details> <summary> Click to expand </summary>

# pip install bitsandbytes accelerate
from transformers import T5Tokenizer, T5ForConditionalGeneration
tokenizer = T5Tokenizer.from_pretrained("muvazana/flan-t5-base-opus-en-id-id-en")
model = T5ForConditionalGeneration.from_pretrained("muvazana/flan-t5-base-opus-en-id-id-en", device_map="auto", load_in_8bit=True)
input_text = "translate English to Indonesia: How old are you?"
input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to("cuda")
outputs = model.generate(input_ids)
print(tokenizer.decode(outputs[0]))

</details>

<!-- - Related Models: All FLAN-T5 Checkpoints

• Original Checkpoints: All Original FLAN-T5 Checkpoints
• Resources for more information:
  • Research paper
  • GitHub Repo
  • Hugging Face FLAN-T5 Docs (Similar to T5)

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.005
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 3-->

Training results

Framework versions

• Transformers 4.29.2
• Pytorch 2.0.1
• Datasets 2.13.1
• Tokenizers 0.13.3