GPT-J COVID-19 French News with 8-bit weights
This is a version of Cedille's GPT-J (fr-boris) with 6 billion parameters fine-tuned on COVID-19 French News dataset to generate French headlines related to COVID-19.
You can generate the model in colab or equivalent desktop gpu (e.g. single 1080Ti) as the model has 8-bit weights. Inspired by GPT-J 8bit.
Here's how to run it:
This model can be easily loaded using the GPTJForCausalLM functionality:
from transformers import GPTJForCausalLM
model = GPTJForCausalLM.from_pretrained("gustavecortal/gpt-j-fr-covid-news")
Remember, you have to Monkey-Patch the model before loading it (see Colab above).
One thousand AI-generated French headlines related to COVID-19
How not to be disoriented in a pandemic era when faced with an immense flow of information? This page features one thousand AI-generated French headlines related to COVID-19.
fr-boris
Boris is a 6B parameter autoregressive language model based on the GPT-J architecture and trained using the mesh-transformer-jax codebase.
Boris was trained on around 78B tokens of French text from the C4 dataset.
