PhD position - Understanding human biases during AI-aided systematic reviewing

Do you want to understand how and why humans interact with an AI to accelerate systematic reviews? If so, apply for this PhD position! This position is part of the bigger project “Transparent and Reproducible AI-aided Systematic Reviewing for the Social Sciences (TRASS)”.

Your job
The rapidly evolving field of AI offers promising solutions to the literature screening challenge using machine learning models, like active learning, and, very recently, large language models (LLMs). However, many of these AI-driven solutions emerge from tech companies that publish new and, hopefully, better models at an unprecedented rate. The rapidity of advancements in the field of AI outpaces meticulous scientific evaluations, leaving many methods unrefined and unproven. The challenge is twofold: keeping pace with these relentless innovations while collaboratively forging a comprehensive understanding of their implications.

In the evolving landscape of AI-aided systematic tools (like ASReview), we need to explore the nuanced complexities and potential biases that arise when human discernment is coupled with the predictive capabilities of AI, aiming to foster a synergistic relationship that enhances the efficacy and reliability of the review process.

Therefore, in the subproject for which we are looking a PhD candidate, we will investigate questions like:

• Which biases - such as position bias or authority bias - do humans display when reviewing abstracts?
• How does knowledge about these biases generalize to other contexts, such as in doing research and university work generally.
• Should we replace human screeners with an AI (as predicted by OpenAI), or should we keep a human in the loop?
• How do (or should) humans interact with AI-aided screening models spotlighting potential biases inherent in AI-assisted processes?
• How can we reduce biases by humans and AI?

You will be responsible for conducting several high-quality experiments where humans interact with an AI to address such questions. Addressing these questions is crucial in fostering a collaborative synergy between humans and AI models, and to spotlight and reduce potential biases, fostering a more transparent and equitable research process. Moreover, it seeks to delineate the roles and responsibilities of human researchers in an AI-assisted landscape, paving the way for a harmonious and productive human-AI collaboration in academic research.

Requirements:

We are looking for an enthusiastic colleague who meets multiple of the following requirements:

• You have (completed or almost completed) a Research Master’s degree in Psychology, Sociology, Methods & Statistics, Organisation Science, Economics, Management, Public Administration, or a similar study.
• You have the ambition to conduct excellent scientific research and are motivated to create an impact with your research.
• You have an affinity with transdisciplinary research.
• You have demonstrable affinity and experience with quantitative research such as (field/lab) experiments and/or questionnaires.
• You need an interest in systematic reviewing and AI-aided techniques, or have used software like ASReview, to accelerate the screening process.
• You have an affinity with or strong interest in machine learning / AI / NLP.
• You have a demonstrable affinity with open science, open-code and/or open-source software.
• You have experience with R (Markdown) or Python (Jupyter Notebooks).
• You are familiar with Github.
• You have excellent verbal and written communication skills in English (or Dutch), which you can prove with for instance your Bachelor's or Master's thesis.
• You are social, well-organised, communicative, and collaborative.
• You are interested in contributing to high-quality teaching, including supervising students, teaching lectures in data science and statistics.

Salary Benefits:

We offer:

• a collaborative, enthusiastic, and dedicated team that guids you in your research, teaching, and other tasks;
• high-quality PhD education, being part of a research school, following courses, presenting at international conferences, etc.;
• a position for 4-4,5 years (depending on the amount of teaching);
• a working week of 38 hours and a gross monthly salary between €2,770 and €3,539 in the case of full-time employment (salary scale P under the Collective Labour Agreement for Dutch Universities (CAO NU));
• 8% holiday pay and 8.3% year-end bonus;
• a pension scheme, partially paid parental leave and flexible terms of employment based on the CAO NU.

In addition to the terms of employment laid down in the CAO NU, Utrecht University has a number of schemes and facilities of its own for employees. This includes schemes facilitating professional development, leave schemes and schemes for sports and cultural activities, as well as discounts on software and other IT products. We also offer access to additional employee benefits through our Terms of Employment Options Model. In this way, we encourage our employees to continue to invest in their growth. For more information, please visit Working at Utrecht University.

36 - 40 hours per week

Padualaan 14