<!DOCTYPE html> <html>
<head> <title>Emotions Classification with DistilBERT Base Fine-Tuned Model</title> </head>
<body>
<h1>Emotions Classification with DistilBERT Base Fine-Tuned Model</h1>
<h2>Model Overview</h2> <ul> <li><strong>Model Architecture:</strong> DistilBERT (a distilled version of BERT) is used as the base model for sentiment classification.</li> <li><strong>Number of Labels:</strong> The model is trained for a multi-class classification task with 9 labels representing different sentiments.</li> <li><strong>Training Data:</strong> The model is trained on Twitter emotions data, which includes tweet IDs, sentiments, and content.</li> </ul>
<h2>Usage</h2>
<h3>Loading the Model</h3> <p>You can load the fine-tuned model using the <code>transformers</code> library:</p>
<pre><code> from transformers import AutoModelForSequenceClassification
model = AutoModelForSequenceClassification.from_pretrained("Sentiment_Classification_DistilBertBase_FT") </code></pre>
<h3>Inference</h3> <p>You can use the loaded model for sentiment classification on your own text data:</p>
<pre><code> from transformers import AutoTokenizer import torch
tokenizer = AutoTokenizer.from_pretrained("distilbert-base-uncased")
text = "Layin n bed with a headache ughhhh...waitin o..." inputs = tokenizer(text, return_tensors="pt", padding=True, truncation=True) outputs = model(**inputs) logits = outputs.logits
To get the predicted sentiment label
predicted_label_id = torch.argmax(logits, dim=1).item() </code></pre>
<h3>Training</h3> <p>If you want to fine-tune the model further or train it on a different dataset, you can use the provided training script:</p>
<pre><code> from transformers import AutoModelForSequenceClassification, TrainingArguments, Trainer
model = AutoModelForSequenceClassification.from_pretrained( "distilbert-base-uncased", num_labels=9, # Number of sentiment labels id2label=id2label, # Mapping from label IDs to label names label2id=label2id, # Mapping from label names to label IDs )
training_args = TrainingArguments( output_dir="Sentiment_Classification_DistilBertBase_FT", learning_rate=2e-5, per_device_train_batch_size=16, per_device_eval_batch_size=16, num_train_epochs=2, weight_decay=0.01, evaluation_strategy="epoch", save_strategy="epoch", load_best_model_at_end=True, push_to_hub=True, # Push the trained model to the Hugging Face Model Hub )
trainer = Trainer( model=model, args=training_args, train_dataset=tokenized_ds["train"], # Your training dataset eval_dataset=tokenized_ds["test"], # Your evaluation dataset tokenizer=tokenizer, data_collator=data_collator, # Your data collator function compute_metrics=compute_metrics, # Your metrics function ) </code></pre>
<h2>Model Training and Fine-Tuning</h2> <p>The model was fine-tuned using the provided training arguments and the Twitter sentiment dataset. The training and evaluation strategies are set to perform evaluations after each epoch and save the best model. The model uses a learning rate of 2e-5 and a batch size of 16 for both training and evaluation.</p>
<h2>Performance Metrics</h2> <p>You can evaluate the model using appropriate performance metrics defined in your <code>compute_metrics</code> function, which can include accuracy, F1-score, precision, recall, etc.</p>
<h2>Acknowledgments</h2> <ul> <li>The base model "distilbert-base-uncased" is provided by Hugging Face Transformers.</li> <li>The Twitter sentiment dataset is used for fine-tuning. Please make sure to acknowledge the source of this dataset in your project.</li> </ul>
<h2>Author</h2> <p>This model and README were created by Mahendra Kharra.</p>
<p>For any questions or issues, please contact mahendrakharra@gmail.com.</p> <a href = "https://www.linkedin.com/in/mahendra-kharra/">Linkedin</a>
</body>
</html>
