NLM IRP Seminar Schedule

Abstract:

Along with the dramatic growth of interest in artificial intelligence and deep learning is a growth in questions about such algorithms. For example, are they fair? A machine learning algorithm is behind PubMed search's Best Match algorithm. NIST's AI Risk Management Framework points out that fairness needs to be regularly measured and tracked across changes in algorithms. We measured fairness in PubMed search in the areas of article language and journal ranking. We also modified the search algorithm by changing which clicks are used to score articles and adding a dense retrieval feature. We measure the effect on fairness resulting from these changes. We conclude with a discussion of the implementation and implications of a common suggestion for balancing fairness and relevance of search results.