NLM IRP Seminar Schedule



Scheduled Seminars on Oct. 31, 2023

Donald Comeau
11 a.m.
Presentation Title
Measuring Fairness in PubMed Search
Virtual, see link below

Contact with questions about this seminar.


Along with the dramatic growth of interest in artificial intelligence and deep learning is a growth in questions about such algorithms. For example, are they fair? A machine learning algorithm is behind PubMed search's Best Match algorithm. NIST's AI Risk Management Framework points out that fairness needs to be regularly measured and tracked across changes in algorithms. We measured fairness in PubMed search in the areas of article language and journal ranking. We also modified the search algorithm by changing which clicks are used to score articles and adding a dense retrieval feature. We measure the effect on fairness resulting from these changes. We conclude with a discussion of the implementation and implications of a common suggestion for balancing fairness and relevance of search results.