日本データベース学会の皆様，
（重複して受け取られた場合にはご容赦ください）
京都大学の加藤と申します．
モバイル検索などにおいて、検索ボタンを押した直後にユーザを満足させるタス
クのご案内です。奮ってご参加ください．
###################################################
CALL FOR TASK PARTICIPATION
NTCIR-10 1CLICK-2 Task:
One Click Access for English and Japanese
http://research.microsoft.com/en-us/people/tesakai/1click2.aspx
Test queries released: Aug 2012
Evaluation results released: Feb 2013
Final NTCIR-10 workshop: June 18-21 2013
###################################################
### OVERVIEW ###
Unlike traditional search engines
that return a ranked list of web pages,
One Click Access systems
return a short multi-document summary of webpages
and aim to satisfy the user immediately after
a click on the SEARCH button.
Systems must aim to return important pieces of information and
to minimise the amount of text the user has to read.
These requirements are particularly important in mobile environments.
1CLICK-1 at NTCIR-9 only considered a Japanese task,
but for this second round (1CLICK-2),
we are expanding our scope to English.
Moreover, while 1CLICK-1 considered only four query
types (CELEBRITY, LOCAL, DEFINITION and QA),
1CLICK-2 will use a more fine-grained taxonomy
comprising eight query types.
Evaluation will be based on
"information units" (iUnits),
also often referred to as "nuggets".
As in 1CLICK-1, we will use S-measure from
Sakai/Kato/Song CIKM'11
http://research.microsoft.com/en-us/people/tesakai/cikmfp0040-sakai.pdf
and will also consider developing new evaluation methods based on
Pavlu et al. WSDM'12
http://dx.doi.org/10.1145/2124295.2124343
### TASKS ###
Japanese Main Task
Given a Japanese query, return an X-character string
that contain important information (X=500,140).
This is basically the same as the 1CLICK-1 task.
English Main Task
Similar to the Japanese Main Task except that
query and the output are in English.
Query Classification Subtask (for Japanese and English)
Given a query, classify it into one of the eight known query types.
In addition to queries, the organisers will provide
query suggestion data and baseline web search results
to enhance repeatability and fairness, and to
encourage newcomers to participate.
### IMPORTANT DATES ###
April 2012 sample queries and nuggets released
Aug 2012 test queries, query suggestions and baseline runs released
Sept 2012 run submissions due
Oct 2012 nugget match evaluation begins
Jan 2013 nugget match evaluation ends
Feb 2013 evaluation results released
March 2013? draft papers due
May 2013? camera ready papers due
June 2013 NTCIR-10
### HOW TO PARTICIPATE ###
Task registration page will be available soon on the NTCIR-10 website
(see below).
We encourage new participants to have a look at the 1CLICK-1 overview paper
to familiarise themselves with the task:
http://research.nii.ac.jp/ntcir/workshop/OnlineProceedings9/NTCIR/01-NTCIR9…
Please note:
task participants must write a paper for the NTCIR-10 meeting (unrefereed)
and attend the meeting at NII, Tokyo in June 2013.
### ORGANISERS ###
Makoto Kato, Kyoto University, Japan
Takehiro Yamamoto, Kyoto University, Japan
Mayu Iwata, Osaka University, Japan
Matthew Ekstrand-Abueg, Northeastern University, USA
Shahzad Rajput, Northeastern University, USA
Virgil Pavlu, Northeastern University, USA
Zhicheng Dou, MSRA, China
Young-In Song, MSRA, China
Tetsuya Sakai, MSRA, China
## QUESTIONS? PLEASE CONTACT ##
ntcadm-1click at
nii dot ac dot jp
## LINKS ##
1CLICK-2 homepage:
http://research.microsoft.com/en-us/people/tesakai/1click2.aspx
1CLICK-1 homepage:
http://research.microsoft.com/en-us/people/tesakai/1click.aspx
NTCIR-10:
http://research.nii.ac.jp/ntcir/ntcir-10/index.html
NTCIR online proceedings:
http://research.nii.ac.jp/ntcir/publication1-en.html
日本データベース学会の皆様，
（重複して受け取られた場合にはご容赦ください）
京都大学の加藤と申します．
曖昧なクエリや不明快なクエリに秘められた検索意図を扱うタスクのご案内で
す。奮ってご参加ください．
###################################################
CALL FOR TASK PARTICIPATION
NTCIR-10 INTENT-2 Task:
English/Chinese/Japanese Subtopic Mining subtasks
and
Chinese/Japanese Document Ranking subtasks
http://research.microsoft.com/en-us/people/tesakai/intent2.aspx
Topics released: May 2012
Evaluation results released: Jan 2013
Final NTCIR-10 workshop: June 18-21 2013
###################################################
### OVERVIEW ###
The NTCIR INTENT task addresses the problem of
handling ambiguous and/or underspecified queries in Web search.
Does "office" mean a Microsoft product or a workplace? (ambiguity)
Does "harry potter" mean the books, the films or the character? (lack of
specificity)
Unlike traditional information retrieval where only relevance plays the
central role,
INTENT considers multiple possible intents for a given query and
aims to balance relevance and diversity.
The first round of INTENT at NTCIR-9 was concluded successfully,
thanks to great research contributions from
17 research groups from seven countries/regions.
At INTENT-1, it has been found that some successful systems
were relevance-oriented, while others were diversity-oriented.
NTCIR-10 will host the second round of INTENT (INTENT-2)
to further clarify and hopefully solve the challenges of
handling multiple intents.
### TASKS ###
SUBTOPIC MINING subtask:
Given a query, systems must return a ranked list of
"subtopic strings" that represent diverse intents.
In addition to Chinese and Japanese,
this time we are running an English Subtopic Mining subtask
as well. For this, we will syncronise with the TREC 2012 diversity task
and share the topic set.
DOCUMENT RANKNIG subtask:
Given a query, systems must return a selectively diversified
web search results. Unlike the TREC diversity task,
we take the importance of intents and
per-intent graded relevance into account when evaluating the
participating systems.
This time, we will include some "one item search" queries
in the topic set, which probably do not require diversification.
Can systems selectively and effectively diversify?
In addition to topics, the organisers will provide
query suggestion data and baseline web search results
to enhance repeatability and fairness, and to
encourage newcomers to participate.
### IMPORTANT DATES ###
May 2012 topics, query suggestions and baseline runs released
July 2012 all submissions due
Aug-Dec 2012 identify intents from subtopics -> per-intent relevance
assessments
Jan 2013 evaluation results released
March 2013? draft papers due
May 2013? camera ready papers due
June 18-21 2013 NTCIR-10
### HOW TO PARTICIPATE ###
Task registration page will be available soon on the NTCIR-10 website
(see below).
Participants who plan to index the document corpora
need to obtain the SogouT data for Chinese, ClueWeb09-JA for Japanese,
and ClueWeb09 for English.
How to obtain the data is described in
the INTENT-1 homepage (see below).
We encourage new participants to have a look at the INTENT-1 overview paper
to familiarise themselves with the task:
http://research.nii.ac.jp/ntcir/workshop/OnlineProceedings9/NTCIR/01-NTCIR9…
Please note:
task participants must write a paper for the NTCIR-10 meeting (unrefereed)
and attend the meeting at NII, Tokyo in June 2013.
### ORGANISERS ###
Tetsuya Sakai, MSRA
Min Zhang, Tsinghua University
Ruihua Song, MSRA
Yiqun Liu, Tsinghua University
Zhicheng Dou, MSRA
Makoto Kato, Kyoto University
Mayu Iwata, Osaka University
Takehiro Yamamoto, Kyoto University
## QUESTIONS? PLEASE CONTACT ##
ntcadm-intent at
nii dot ac dot jp
## LINKS ##
INTENT-2 homepage:
http://research.microsoft.com/en-us/people/tesakai/intent2.aspx
INTENT-1 homepage:
http://www.thuir.org/intent/ntcir9/
NTCIR-10:
http://research.nii.ac.jp/ntcir/ntcir-10/index.html
NTCIR online proceedings:
http://research.nii.ac.jp/ntcir/publication1-en.html
