Dataset Description

• Repository: https://github.com/amogiska/NLP_adv_squad
• Paper:

Dataset Summary

Adv-SQuAD is a new adversarial dataset created to address the limitations of the ELEC-TRA model in handling complex question structures, lexical and syntactic ambiguity, and distractor information. It contains approximately 50 hand-crafted examples that are representative of the challenging instances identified in the SQuAD dataset. The creation of Adv-SQuAD is an iterative and informed process based on the understanding of the model's weaknesses.

Example

Context:

"The Eiffel Tower is a wrought-iron lattice tower on the Champ de Mars in Paris, France. It was named after the engineer Gustave Eiffel, whose company designed and built the tower. Constructed from 1887 to 1889 as the entrance to the 1889 World's Fair, it was initially criticized by some of France's leading artists and intellectuals for its design. However, it has since become a global cultural icon and is one of the most recognizable structures in the world."

Question:

"Which event prompted the construction of the Eiffel Tower, and who was it named after?"

Answer:

"The construction of the Eiffel Tower was prompted by the 1889 World's Fair, and it was named after the engineer Gustave Eiffel."

In this example, the adversarial question requires multi-step reasoning as it asks for two separate pieces of information, and the context contains distractor information about the criticism the tower received.

Supported Tasks

Question Answering (SQuAD).

Languages

English

Dataset Structure

Data Instances

Dataset contains 50 instances

Data Fields