Guidelines

Review Instructions for VRST 2022


This information here is to provide guidance to reviewers for VRST 2022.  Specifically, those who will perform full reviews like the secondary (2AC) and external reviewers.

The primary (1AC) will examine all full reviews for quality. If the primary finds any reviews of poor quality (e.g., lacking details, missing reasons for rejection, etc.), they will ask reviewers to update their reviews. In extreme cases, the 1AC may remove poorly written or incomplete reviews and find replacement reviewers.

Our goal is to ensure the highest possible reviewing standards for VRST 2022 both for full papers and short papers. Please read these guidelines before reviewing and reach out to us if you are unclear about anything or need more information. The below information is a guideline for reviewing the paper regardless if it is a full paper or a short paper.

Paper length must be based on the weight of the contribution. Please do not instruct authors to change a full paper to a short paper, but to review it as a full paper. A short paper is written to be very focused on a topic. For short papers, please evaluate the originality, cutting-edge nature, and other sharp points of the paper rather than its completeness.

In terms of the actual review, Ken Hinckley offers great advice on reviewing:

Hinckley, K. (2016). So You’re a Program Committee Member Now: On Excellence in Reviews and Meta-Reviews and Championing Submitted Work That Has Merit.

We present a few of major points from Ken’s paper with minor changes below. Please see the link for a longer list.These guidelines are based on ISMAR’ 22 website.

  • Read papers with care and sympathy. Many hours of work — in some cases, years of work — have gone into research and writing this paper. Try reviewing as you would like others to review your own paper.
  • Short and/or content-free reviews that read like unsubstantiated opinions are insufficient and will be rejected by 1AC.
  • State specifically why the paper is “great”, “mediocre” or “bad.”
  • Clearly describe on what grounds the paper should be accepted (or rejected). Keep in mind that this may be someone’s first paper. Your constructive feedback will be appreciated.
  • Explicitly and clearly discuss the weaknesses and limitations in a positive and constructive manner. Specifically, don’t be insulting – be positive.
  • Your review should be a critique of what the authors have done and not what they should have done.
  • Do not reject a paper because of anything that can be fixed/addressed easily as authors will have the opportunity to do so for their camera ready version.
  • Avoid the fallacy of novelty. Specifically, do not simply reject papers because they replicated experiments.
  • When evaluating papers with human-subjects studies, it is important that the participant sample population be representative of the population for which the technology is being designed. For example, if the technology is being designed for a general population then the participant population should include equal gender representation and a wide range of ages. All papers with human-subject studies should report at a minimum demographic information including age, gender, and every possible hint of social and diversity representation. If this is not the case, reviewers should not automatically reject the paper but instead provide appropriate constructive critique and advice regarding general claims that do not use representative sample populations.
  • User studies are not required or appropriate for all papers. While the authors need to support their claims with evidence, that form of evidence can vary from paper to paper. In short, the work needs to have a validation but not necessarily with a user study.
  • Papers should be reviewed holistically, taking into account the limitations imposed by COVID-19. The limitations of COVID-19 now strongly differ depending on countries. Please kindly take into consideration of this point.
  • Please do not reject system papers simply because it is built using existing well-known techniques if it accomplishes new functionality. In such situations, judge the novelty and significance of the new functionality. Here are some references for how to evaluate different types of research:

 

Desk Rejections: The primary reviewer may, upon conferring with the secondary reviewer and the program chairs, recommend a paper be rejected without additional reviews. Desk rejects will only occur for papers that are clearly off-topic for VRST, if they are incomplete, if they violate rules of this Author’s Guide, or if they are discovered to have been published previously or to have been submitted concurrently at another conference or journal.

 

Thank you for your support and work to ensure the highest-quality VRST reviews. If you need more information, please do not hesitate to contact us: papers@vrst2022.org