お邪魔いたします。Microsoft Research Asiaの酒井と申します。 NTCIRの新しい日本語情報アクセスタスクの参加案内をお送りいたします。 参加をご検討いただければ幸いです。どうぞよろしくお願いいたします。 ********** FIRST CALL FOR PARTICIPATION ********** NTCIR-9 1CLICK (ONE CLICK ACCESS) - A subtask of the INTENT task http://www.thuir.org/1click/ntcir9/ Task participants registration due: December 20, 2010 Final workshop meeting at NTCIR-9: December 6-9, 2010 [NII, Tokyo] ************************************************** TASK OUTLINE: "1CLICK" is a brand new information access challenge for NTCIR-9. The goal of this task is to realize the following scenario: the user enters a query and clicks on the search button - and his/her information need is immediately satisfied with the first system output that's displayed. No need to click any further - hence "once click access". Thus we step out of the "ranked list" and "document relevance" paradigm, and enable IR, QA and summarization communities to solve a common and important problem. As this is Round 1 of Once Click Access (“1CLICK”), only Japanese textual output of length up to X characters will be considered. The textual output may represent a Search Engine Result Page (i.e. titles and snippets for a list of retrieved documents), part of a single document, or any fragments of text. Participating systems are allowed to use any existing web pages as the knowledge source for producing the system output. INPUT TO THE SYSTEM: We will release 60 Japanese queries covering four query types (15 CELEBRITY, 15 LOCAL, 15 DEFINITION and 15 Question Answering) as input to the system. These queries have been manually selected from a Japanese mobile query log and the Yahoo! Chiebukuro (Japanse Yahoo! Answers) data. For each query type, we assume that the user is looking for certain types of factual information. OUTPUT FROM THE SYSTEM: We accept two types of "runs" (system output files): DESKTOP runs (“D-runs”) which use up to X=500 characters (not bytes) for each query; and MOBILE runs (“M-runs”) which use up to X=140 characters (not bytes) for each query. EVALUATION METHOD: We plan to design a nugget-based evaluation method. We will provide a web-based tool for comparing a system output with a list of nuggets. Thus nugget matches will be identified manually. Unlike traditional summarization and question answering evaluation, we plan to utilize the position of each matched nugget for computing evaluation metrics. That is, a nugget match found near the end of the system output will be discounted compared to one found near the beginning of another output. Redundancy will be penalized. The goal is to cover the most relevant nuggets within an X-character window, AND order them so as to minimize the amount of text the user has to read. We assume that the user reads from top to bottom, left to right. FOR MORE DETAILS: Please visit http://www.thuir.org/1click/ntcir9/ A link to the NTCIR-9 task particpants registration page is now available from this page. Please note that the registration deadline is December 20, 2010 (for all NTCIR-9 tasks). ORGANIZERS: Tetsuya Sakai (Microsoft Research Asia) Makoto Kato (Kyoto University) Youngin Song (Microsoft Research Asia) Ruihua Song (Microsoft Research Asia) Min Zhang (Tsinghua University) Yiqun Liu (Tsinghua University) Nick Craswell (Microsoft) QUESTIONS? Please contact Tetsuya Sakai tesakai@microsoft.com