Skip to content

FAQ

This page collects common questions about the NTCIR-19 ModelRetrieval task. Use the quick links or expand individual questions to read answers.

Getting Started

This page highlights Top Questions and a searchable list of all FAQs below.

All Questions

Top Questions

Getting Started

How do I register for the task?

Registration is handled through the NTCIR-19 website. Visit the registration page and follow the instructions.

NTCIR-19 Registration

What do I need to do after registration?

After registering, prepare and submit your run files for the subtask(s) you registered for by the submission deadlines. Important dates (tentative):

  • Normal submission: August 1, 2026
  • Paper (draft) due: September 1, 2026
  • Late submission: October 15, 2026
  • Camera-ready paper due: November 1, 2026
  • NTCIR-19 conference: December 8–10, 2026 (Tokyo)

The exact submission site and instructions will be announced later

What are the paper submission instructions?

Please follow the NTCIR paper guidelines: NTCIR-19 paper instructions.

Where is the dataset and how do I access it?

Links to the task data repositories are listed on the Dataset & Resources page. For Subtask A (Language Models) and Subtask B (Style Transfer), see the linked GitHub repositories on the Data page.

What is the required submission format?

Submissions must use TREC_EVAL format with up to 5 runs per participant. Each line uses the format:

topicID Q0 docID Rank Score RunID

Check the Task Definition page for examples.

When are submissions due?

Check the Schedule for the latest timeline.

Data & Evaluation

What is the Input and Output for Sub Task A?

Input: training and validation data for a downstream text classification task (JSONL format with text–label pairs). Output: a ranked list of candidate pre-trained BERT models ordered by their expected F1 score on the hidden test set (highest first).

What can I use to predict ranked list in Sub Task A?

Available resources include the training queries (with full train/validation/test and ground-truth model rankings) and the test queries (only train/validation; test is hidden). Participants may use candidate models (e.g., compute embeddings, evaluate on validation data, or fine-tune) to produce features or predicted scores. If you fine-tune or otherwise use candidate models on the provided data, describe this methodology clearly in your paper.

How was the groundtruth data created of for Subtask A?

For each query and candidate model pair, the model was fine-tuned on the task's training set, with early stopping based on validation performance. Final performance was measured on the held-out test set; candidate models were then ranked by test F1 score (descending). See the Task Definition page for full fine-tuning settings.

What is the Input and Output for Sub Task B?

Input: a query image representing a desired style. Output: a ranked list of candidate Image Style Transfer LoRA models. Each query has a single ground-truth model (the one used to generate images); higher rank for the ground-truth model improves MRR.

What can I use to predict ranked list in Sub Task B

You may use the query image and the training image–model pairs to build retrieval models or similarity measures that predict which LoRA model best matches the query style.

Contact

Who should I contact for questions not covered here?

Please contact the organizers listed on the Organizers page. For quick help, email modelretrieval1-organizers[at]googlegroups.com.