De UvA gebruikt cookies en vergelijkbare technieken voor het functioneren, meten en optimaliseren van de website. Die worden ook benut om bijvoorbeeld YouTube-video's te kunnen tonen en voor marketingdoeleinden. Dat betekent dat je internetgedrag kan worden gevolgd. Als je op ‘Accepteer alle cookies’ klikt, ga je hiermee akkoord.
Poor data quality is a problem for industrial and academic research resulting in immense costs and unreliable research findings. Many survey participants answer questionnaires carelessly, thereby reducing data quality. Techniques have been developed to detect such participants, but these suffer several shortcomings. This study proposes implementing qualitative control questions (QCCs), which need to be answered in open text format as a quality measure. Because the interpretation of open-ended answers is performed mostly manually during the time of writing, it was investigated whether the quality of a question-answer pair can be evaluated automatically by Gpt-3. A dataset was generated consisting of reasonable and unreasonable question and answer pairs.
One zero-shot model, receiving no training data, and two fine-tuned models, trained on 50 examples, were fitted. Model accuracies in correctly classifying the quality ranged from 85% to 92%. Results suggest that automatic classification of open-ended answers into high vs low quality is possible using Gpt-3. However, model classification accuracies are not high enough to propose full automatization. It is argued that researcher use it as supplemental tool and ensure the correctness of the results when applying QCCs.