[dbjapan] NTCIR-17 FairWeb-1タスク参加案内（4/15締切）

日本データベース学会の皆さま　早稲田大学の酒井と申します。 NTCIR-17 FairWeb-1 (グループフェアネスを考慮したウェブ検索タスク) の参加案内を共有させていただきます。 タスク参加登録締切は4/15です。ご検討いただければ幸いです。 日本語の情報(日付等、改訂前のものですが)はこちら にございます。 https://waseda.box.com/webdb2022fairweb1 https://waseda.box.com/webdb2022fairweb1slides *** SECOND CALL FOR TASK PARTICIPATION: NTCIR-17 FairWeb-1 Task (Registration due April 15, 2023 AoE) http://sakailab.com/fairweb1/ # Overview Intro slide deck: https://waseda.box.com/fairweb1intro2023feb The FairWeb-1 task is a new English web search task that considers both relevance and group fairness. Each of our search topic seeks information about specific entity types: R (researher), M (movies), and Y (YouTube contents); accordingly, we have four topic types: R-topics (e.g., information retrieval researchers) M-topics (e.g., Daniel Craig 007 movies) Y-topics (e.g., Coldplay covers on YouTube) For evaluating group fairness, we consider the following attribute sets containing either ordinal or nominal groups: R-topics: h-index (ordinal) AND gender (nominal) M-topics: #ratings on IMDb (ordinal) AND geographic region (nominal) Y-topics: #subscribers of the YouTube account (ordinal) For R- and M-topics, we will evaluate intersectional groupfairness. For example, for each R-topic, we want the search engine result page (SERP) to contain information about researchers with varying levels of h-index (not just high h-index people), AND with different genders (not just "men"). # Task input/ouput INPUT: - a search topic (R, M, or Y) - attribute set(s) and target distribution(s) OUTPUT: - a TREC-style run (a SERP for each topic) # Evaluation method We will use the Group Fairness and Relevance (GFR) framework: please see the slide deck for details. # Timeline February 15, 2023             Pilot relevance assessments for the sample topics and a few pilot runs released February 1-March 10, 2023     Topic development March 15, 2023                Topics released April 15, 2023                Task registrations due May 16, 2023                  Run submissions due May 17-July 31, 2023          Entity annotations; runs evaluated August 1, 2023                Evaluation results and draft overview released September 1, 2023             Draft participant papers due November 1, 2023        Camera ready papers due December 2023           NTCIR-17@NII, Tokyo, Japan # Inquiries: fairweb1org(a)list.waseda.jp # Organisers: Sijie Tao, Nuo Chen, Tetsuya Sakai (Waseda University, Japan) Zhumin Chu (Tsinghua University, P.R.C.) Nicola Ferro (University of Padua, Italy) Maria Maistro (University of Copenhagen, Denmark) Ian Soboroff (NIST, USA) Hiromi Arai (RIKEN AIP, Japan) ### Professor Tetsuya Sakai (tetsuya(a)waseda.jp) Department of Computer Science and Engineering, Waseda University http://sakailab.com/tetsuya/