causal-lm fr

GPT-J COVID-19 French News with 8-bit weights

This is a version of Cedille's GPT-J (fr-boris) with 6 billion parameters fine-tuned on COVID-19 French News dataset to generate French headlines related to COVID-19.

You can generate the model in colab or equivalent desktop gpu (e.g. single 1080Ti) as the model has 8-bit weights. Inspired by GPT-J 8bit.

Here's how to run it: colab

This model can be easily loaded using the GPTJForCausalLM functionality:

from transformers import GPTJForCausalLM
model = GPTJForCausalLM.from_pretrained("gustavecortal/gpt-j-fr-covid-news")

Remember, you have to Monkey-Patch the model before loading it (see Colab above).

One thousand AI-generated French headlines related to COVID-19

How not to be disoriented in a pandemic era when faced with an immense flow of information? This page features one thousand AI-generated French headlines related to COVID-19.

fr-boris

Boris is a 6B parameter autoregressive language model based on the GPT-J architecture and trained using the mesh-transformer-jax codebase.

Boris was trained on around 78B tokens of French text from the C4 dataset.

Links