Model Details
The model in this repository is fine-tuned based on the charades datasets with different human actions. Each model in the folder is a checkpoint for the fine-tuned process.
Model Metadata
id:
Unique identifier for each video- 0MK2C
subject:
Unique identifier for each subject in the dataset- DXDI
scene:
One of the 15 indoor scenes in the dataset- Stairs
quality:
Quality of the video judged by an annotator (7-point scale, 7 = high quality)- 7
relevance:
Relevance of the video to the script judged by an annotator (7-point scale, 7 = very relevant)- 7
verified:
Yes - if an annotator successfully verified that the video matches the script, else No- "Yes"
script:
The human-generated script used to generate the video- "A person is running up the stairs holding a pair of shoes. The person goes through a door."
objects:
List of objects identified in the video- ["doorway", "shoe", "stairs"]
descriptions:
List of descriptions by annotators watching the video- "A person runs up the stairs carrying a pair of shoes and opens a door."
actions:
Consists of actions - human actions in the video and timings - timing of the action happening in the video- actions: [{ action: "run up the stairs", timing: ["1.50", "8.80"] }]
- Developed by: ICT3104-Team06-2023
- Finetuned from Model: stable-diffusion-v1-4
- Stable Diffusion Model Repository: https://huggingface.co/YueMafighting/FollowYourPose_v1
- Training Data Source: https://prior.allenai.org/projects/charades
