Workshop Report: “Future of Information Retrieval Research in the Age of Generative AI”

The report linked to below was published by the Computing Community Consortium (CCC) from the  Computing Research Association (CRA).

Title

Future of Information Retrieval Research in the Age of Generative AI

Authors

James Allan
University of Massachusetts Amherst

Eunsol Choi
University of Texas at Austin
New York University

Daniel P. Lopresti
Lehigh University
CCC

Hamed Zamani
University of Massachusetts Amherst

Source

CCC via arXiv

Abstract

 Executive Summary

In the fast-evolving field of information retrieval (IR), the integration of generative AI technologies such as large language models (LLMs) is transforming how users search for and interact with information. Recognizing this paradigm shift at the intersection of IR and generative AI (IR-GenAI), a visioning workshop supported by the Computing Community Consortium (CCC) was held in July 2024 to discuss the future of IR in the age of generative AI. This workshop convened 44 experts in information retrieval, natural language processing, human-computer interaction, and artificial intelligence from academia, industry, and government to explore how generative AI can enhance IR and vice versa, and to identify the major challenges and opportunities in this rapidly advancing field.

Workshop Activities

The workshop began with an overview of previous discussions from a preliminary brainstorming session and other related workshops and included presentations from selected participants to ignite deeper discussions. To explore specific themes, eight breakout sessions were then formed to discuss various aspects of the topic. The workshop agenda can be found on the CCC Website 1.

Mechanisms for Effective Collaboration

In order to effectively collaborate on the discussion topics, we followed the IDEO’s brainstorming method (7 Simple Rules of Brainstorming, n.d.) based on the following seven mechanisms for effective collaboration: (1) defer judgments, (2) encourage wild ideas, (3) build on others’ ideas, (4) stay focused on topic, (5) one conversation at a time, (6) be visual when needed, and (7) discuss as many ideas as possible (focus on quantity of ideas).

Research Directions

This report outlines eight research directions for IR-GenAI systems with high intellectual merits and broader impact: (1) Evaluation challenges and needs in IR-GenAI; (2) Learning from implicit and explicit human feedback for solving complex problems that may require reasoning; (3) Understanding and modeling users for the evolving generative AI-powered information access systems; (4) Challenges and potential solutions to address or mitigate socio-technical issues raised by the new technologies in IR-GenAI; (5) Methods for developing personalized IR-GenAI systems; (6) Efficiency considerations when scaling compute, data, and human efforts in developing IR-GenAI methods; (7) The role of information retrieval in enhancing AI agents; and (8) Developing foundation models specifically for information access and discovery.

This report contains a summary of discussions as potentially important research topics and contains a list of recommendations for academics, industry practitioners, institutions, evaluation campaigns, and funding agencies.

Direct to Full Text Report
39 pages; PDF.

Filed under: Data Files, Funding, News, Patrons and Users