日本データベース学会の皆様， （重複して受け取られた場合にはご容赦ください） 京都大学の加藤です． 情報アクセス技術（情報検索，データベース，自然言語処理等） に関わる共有タスクをNTCIR-13にて募集しております． タスク提案の締切【4月28日】が近づいて参りましたので， 再度ご案内させていただきます． NTCIR-12ではWeb検索，電子カルテ分析，音声検索，数式検索，ライフログ検 索，質問応答，対話文生成などのタスクが提案されました． ある問題を広く周知したい，評価方法を確立したい， 他の方と問題を共有したい方などはぜひご提案いただければ幸いでございます． どうぞよろしくお願いいたします． ============================================ NTCIR-13 FINAL CALL FOR TASK PROPOSALS (Task Proposal Due: April 28, 2016) ============================================ NTCIR (NII Testbeds and Community for Information access Research) is a sesquiannual series of evaluation conferences that mainly focuses on Asian language information access. The first NTCIR conference (NTCIR-1) took place in August/September 1999, and the latest NTCIR-12 conference will take place in June 2016. Research teams from all over the world participate in one or more of the NTCIR tasks to advance the state of the art and to learn from one another's experiences. As the tasks being run at NTCIR-12 (http://research.nii.ac.jp/ntcir/ntcir-12/index.html) are reaching the final stages, it is time to call for task proposals for the next NTCIR (NTCIR-13) which will start in June 2016 and conclude in December 2017. Task proposals will be reviewed by the NTCIR Program Committee, following the schedule below: IMPORTANT DATES: Apr 28, 2016 Task Proposals due May 27, 2016 Acceptance Notification Jun 10, 2016 Task Planning Sessions and Presentation at the NTCIR-12 conference NTCIR-13 Tentative Schedule: Dec 2016: Task Registration Due Jan 2017: Dataset Release Jan-May 2017: Dry Run Mar-July 2017: Formal Run Aug 2017: Evaluation Result Release Dec 2017: NTCIR-13 Conference & EVIA 2017 in NII, Tokyo, Japan Organizers of existing NTCIR-12 tasks (IMine, MedNLPDoc, MobileClick, SpokenQuery&Doc, Temporalia, MathIR, Lifelog, QALab, and STC) are required to submit a new proposal if they wish to continue them in NTCIR-13. We also welcome new task proposals within the broad research area of information access. New task organizers will be given many opportunities to consult and learn from senior PC members for task design and organization (see MENTORING). To organize an evaluation task is to identify important research problems, tackle them strategically by collaborating with other researchers (participants), build the necessary evaluation framework to advance the state of the art, and make an impact to the research community and to the future. PROPOSAL TYPES: We will accept two types of task proposals: - Proposal of a Core task: This is for fostering researches on a particular information access problem by providing researchers with a common ground for evaluation. New test collections and evaluation methods may be developed through the collaboration between task organizers (proposers) and task participants. At NTCIR-12, the core tasks were: IMine, MedNLPDoc, MobileClick, SpokenQuery&Doc, Temporalia, and MathIR. - Proposal of a Pilot task: This is recommended for organizers who proposed to focus on a novel information access problem and there are uncertainties in task designing and organization. It may focus on a sub-problem of an information access problem and may attract a smaller group of participating teams than core tasks. However, it may grow into a core challenging task in the next round of NTCIR. At NTCIR-12, the pilot tasks were: Lifelog, QALab, and STC. Organizers are expected to run their task mainly with their own fundings and to make the task as self-sustaining as possible. A part of the fund can be supported by NTCIR, which we call "seed funding". It is usually used for some limited purposes such as hiring relevance assessors. The amount of the seed funding allocated to each task varies depending on requirements and the total number of accepted tasks, but typical cases would be: around 1M JPY for a core task and around 0.5M JPY for a pilot task (note that the amount may be changed in NTCIR-13). Please submit your task proposal as a pdf file to "NTCIR-13 Task Proposal" track in the following EasyChair site by April 28, 2016 (UTC-12). https://easychair.org/conferences/?conf=ntcir12 MENTORING: We offer a mentoring program especially for those who propose a task for the first time. If you would like help in your task organization, you may ask for a mentor: a person who will help you organize a task usually via e-mail. A mentor can also familiarize you with the standards and schedules of NTCIR. He or she will also help you understand how to design tasks, test collections, and evaluation methodology. Mentors are volunteers who are familiar with successful proposals. To request a mentor, please include a “Request for mentoring” section in your proposal, which must include problems for which you need support (e.g. scheduling, design of task, evaluation methodology, etc.). Since this program just begins from NTCIR-13 and is still at a trial stage, we may not fully meet your requests. DIFFERENCE FROM THE PAST NTCIR TASK PROPOSAL: - We have a plan to send acceptance notification before the NTCIR-12 conference, unlike the previous rounds where it was sent after the conferences. - Organizers at NTCIR-12 may keep the task name (e.g. TASKNAME-2) as long as the task includes improvement based on the previous rounds. - Please describe the data with as much detail as possible so that we can help your data release after the proposal is accepted. In the past NTCIR, it took much time to create memorandums, which sometimes slowed down the task organization. - We offer a mentoring program. - The proposal should not exceed four pages in A4 single-column format. - We encourage applicants to - Emphasize real-world applications of the proposed task by using real-world data, real tasks, and addressing real-world problems. - Address a challenge in information access technology evaluation such as a large amount of assessments required for evaluation, privacy-preserving use of proprietary data, and live test with real users. TASK PROPOSAL FORMAT: The proposal should not exceed four pages in A4 single-column format. Main part - Task name and short name - Task type (core or pilot) - Abstract - Motivation - Methodology - Expected results Appendix - Names and contact information of the organizers - Prospective participants - Data to be used and/or constructed - Budget planning - Schedule - Request for mentoring (Include this section if you would like to apply for the mentoring program; otherwise, exclude this section) - Other notes REVIEW CRITERIA: - Importance of the task to the information access community and to the society - Timeliness of the task - Organizers’ commitment in ensuring a successful task - Financial sustainability (self-sustainable tasks are encouraged) - Soundness of the evaluation methodology - Language scope NTCIR-13 PROGRAM COMMITTEE Tat-Seng Chua (National University of Singapore, Singapore) Nicola Ferro (University of Padua, Italy) Kalervo Jarvelin (University of Tampere, Finland) Gareth Jones (Dublin City University, Ireland) Chin-Yew Lin (Microsoft Research Asia, China) Mandar Mitra (Indian Statistical Institute, India) Maarten de Rijke (University of Amsterdam, The Netherlands) Mark Sanderson (RMIT University, Australia) Ian Soboroff (NIST, United States) Yiqun Liu (Co-chair, Tsinghua University, China) Makoto P. Kato (Co-chair, Kyoto University, Japan) NTCIR GENERAL CHAIRS: Charles Clarke (University of Waterloo, Canada) Noriko Kando (NII, Japan) Tetsuya Sakai (Waseda University, Japan) ===