Paper Reviewing Guidelines
This document guides participants in the VRST 2026 reviewing process and is directed toward those who perform full reviews of papers (i.e., secondary review coordinators (2AC), international program committee (IPC) members, and external reviewers) and meta-reviews (primary review coordinators, or 1AC).
This document incorporates the official VRST Reviewing Guidelines (https://vrst.acm.org/reviewing/) and extends them with the review process and policies specific to VRST 2026. Reviewers should follow this document; where it differs from the official guidelines (e.g., the rating scale), the VRST 2026 procedures apply.
These guidelines cover recommendations and best practices. Please read them carefully. To ensure the integrity of the review process, we ask that reviewers refrain from uploading papers to any AI system (e.g., ChatGPT, Claude AI, Gemini, etc.) for analysis, support, or guidance during the review period, nor write their reviews using these systems. See Section 7 for details.
Table of Contents
1. Review Model for VRST 2026
VRST 2026 follows a single-cycle review process without rebuttal.
Each submission will receive one primary coordinator (1AC) and one secondary coordinator (2AC), with two additional reviewers assigned as appropriate by the 1AC and 2AC. After all reviews are submitted, a discussion phase takes place among the reviewers and coordinators.
Possible outcomes are:
- Conditionally accepted as a VRST 2026 conference paper
- Rejected
Conditionally accepted papers must satisfy all required revisions before final acceptance. The 1AC serves as the shepherd and confirms that the conditions have been met within the shepherding period.
Desk rejections may occur at any stage for submissions that clearly violate formatting, anonymity, scope, or ethical requirements. If you believe a submission qualifies for desk rejection, please flag it promptly and contact the IPC through PCS anonymous email, and the IPC will inform the Program Chairs.
2. General Reviewing Principles
When reviewing for VRST 2026, reviewers are expected to follow the principles outlined below. These principles align with the official VRST Reviewing Guidelines.
Evaluate contributions on their own merits. A paper should be considered of sufficient quality if it provides clear and meaningful contributions to the field, whether in theory, methods, engineering, design, framework/taxonomy, and/or evaluation approaches. The primary criterion for judging a paper is whether the submission provides a strong contribution to the broader XR research community. Authors may present preliminary findings if they support the authors’ claims, although more complete findings are always welcome.
Do not evaluate based on how you would have conducted the research. Your review should be a critique of what the authors actually did, not an explanation of what they should have done. Consider the value of the contribution as presented by the authors. Almost every paper we review “could have done x, y, or z”—do not fall into this trap. We cannot reject papers because authors did not conduct their research the way we would have, or even the way it is typically done. Instead, judge whether the body of work as presented can stand on its own and is of sufficient quality and impact.
Read papers with care and sympathy. Many hours of work—in some cases, years of work—have gone into the research and writing of each submission. Keep in mind that this may be someone’s first paper. Focus on contributions, not on searching for hidden flaws. Try to avoid last-minute reviews.
Consider edge cases and emerging directions fairly. A submission may be at the boundary of traditional VRST topics or explore emerging directions. If it addresses a topic that is novel, timely, or underexplored—but relevant to VR, AR, MR, or the broader XR space—it should be reviewed thoughtfully and fairly. VRST welcomes work across the full spectrum of Extended Reality, including applied systems, interaction, perception, design, theory, and evaluation.
Do not evaluate a paper based on its length, but instead by its contribution. Papers can have any length between 4 and 9 double-column pages (excluding references) in Latex format, or up to 8,000 words including space for figures, tables, etc. in MS Word format. A paper must adequately describe the contribution made by the authors regardless of its length. A shorter paper that presents a focused, well-articulated contribution should not be penalized for its brevity, nor should a longer paper be favored simply because of its length.
User studies are not required or appropriate for all papers. While the authors need to support their claims with evidence, the form of that evidence can vary from paper to paper. In short, the work needs to have validation, but not necessarily through a user study.
Do not reject system papers simply because they are built using existing well-known techniques. If a system accomplishes new functionality, judge the novelty and significance of that new functionality. The following references provide useful guidance on how to evaluate different types of research:
- Daniel R. Olsen Jr. (2007): Evaluating User Interface Systems Research
- Kasper Hornbæk, Aske Mottelson, Jarrod Knibbe, and Daniel Vogel (2019): What Do We Mean by “Interaction”? An Analysis of 35 Years of CHI
- David Ledo, Steven Houben, Jo Vermeulen, Nicolai Marquardt, Lora Oehlberg, and Saul Greenberg (2018): Evaluation Strategies for HCI Toolkit Research
- James Fogarty (2017): Code and Contribution in Interactive Systems Research
3. Review Scores and Recommendations
VRST 2026 uses a 6-point rating scale. Note that the official VRST Reviewing Guidelines describe a 5-point scale used in previous editions; for VRST 2026, please use the 6-point scale below. Reviewers will provide a numerical rating together with a written justification. Scores must always be supported by clear reasoning in the public comments.
6 — Definitely accept: I would argue strongly for accepting this submission. Select this option if the paper is acceptable as-is (except for minor edits), with a strong contribution and clear merits for the XR research community.
5 — Probably accept: I would argue for accepting this submission. Select this option if the paper has a valid contribution and merits for the community. Some additional explanations or minor corrections are required.
4 — Rather accept: The paper has weaknesses, but the contributions outweigh the weaknesses. Select this option if the research is relevant, the topic is of value for the community, and the overall attitude toward this contribution is positive despite the identified weaknesses.
3 — Rather reject: The paper has contributions, but the weaknesses outweigh the contributions. Select this option if the research is relevant and the topic is of value for the community, but the overall attitude is negative because of the identified weaknesses.
2 — Probably reject: I would argue for rejecting this submission. Select this option if the research is relevant and the topic is of value for the community, but the research has several severe weaknesses.
1 — Definitely reject: I would argue strongly for rejecting this submission. Select this option if the contribution is not understandable and the paper has no recognizable merits, or it is entirely unclear what information the community gains from this submission.
We are aware that the decision can be subjective in many cases and that selecting between adjacent scores is often a judgment call. We hope this explanation removes some gray area in decision-making and helps align reviewers.
4. Writing a High-Quality Review
The following guidelines outline the content and key points of a high-quality review for VRST 2026. We believe that all paper reviews should be written with the same guidelines in mind.
Length and substance. A high-quality review should have about one page of well-considered commentary (at least 500 words), or more if warranted. Short and/or content-free reviews that read like unsubstantiated opinions are insufficient and may be returned by the 1AC for improvement. Always put yourself in the author’s position: what level of detailed feedback would you like to see for your own work? Make sure to write a detailed review whether or not you like the paper—short positive or negative reviews without justification do not provide adequate support for decisions about a paper during the discussion phase.
State the grounds for your recommendation. Reviewers must state specifically the reason(s) for the score they have selected. Clearly describe on what grounds the paper should be accepted or rejected.
Describe the contributions. Describe the contributions in the paper and why they are noteworthy or important to the community. Clearly and explicitly call out the strengths and utility of the work.
Discuss weaknesses constructively. Explicitly and clearly discuss the weaknesses and limitations in a positive and constructive manner. Be respectful—do not be insulting or dismissive.
Assess the work as presented. Your review should assess what the authors actually did and whether their methods are appropriate to support their claims. Consider how the authors’ arguments, results, and demonstrations fit into closely related work as well as the field as a whole.
Do not reject for easily fixable issues. Do not reject a paper because of anything that can be easily fixed or addressed (e.g., missing references, minor spell-checking, fuzzy statements, or lack of minor implementation details). A paper’s failure to justify or fully motivate certain decisions likely represents a correctable oversight, not an unequivocal sign of poorly conceived research. Clearly state the requirements for revision, as all conditionally accepted papers will be shepherded toward an improved version that meets them.
Do not reject for missing citations alone. Do not reject a paper because of a few missed citations. Furthermore, it is unacceptable for a reviewer to require additional citations for the sole purpose of influencing bibliometric measures of either an individual or a periodical.
Respect the value of replication. Do not simply reject papers because they replicated experiments. Constructive replication is a valued contribution to XR research, and a new interpretation or evaluation of previously published ideas can make a good paper. New work that performs similar research and finds consistent (or even conflicting) results is of value to the community and to science in general, and should be considered on its own merits.
Consider participant diversity in user studies. When evaluating papers that include human-subjects studies, it is important that the participant sample population is representative of the population for which the technology is being designed. All papers with human-subject studies should report demographic information at a minimum, including age, gender, and relevant aspects of social and diversity representation. If this is not the case, reviewers should not automatically reject the paper; instead, they should provide appropriate, constructive critique and advice regarding general claims that do not use representative sample populations.
We strongly recommend that you read the following short article, which provides constructive advice on reviewing:
Hinckley, K. (2016). So You’re a Program Committee Member Now: On Excellence in Reviews and Meta-Reviews and Championing Submitted Work That Has Merit. https://www.microsoft.com/en-us/research/wp-content/uploads/2016/10/Excellence-in-Reviews-MobileHCI-2015-Web-Site.pdf
Mind that your decisions affect VRST 2026’s public image. The program chairs are committed to ensuring the highest possible reviewing standards. The coordinators and/or program chairs may ask you to improve your review if they find the reasoning for your judgment unclear.
5. Prior Publication and Replication
Preprints. Papers that have been posted on preprint servers such as ArXiv should not be rejected on that basis alone, as authors may have done so to establish a timestamp for their work. However, the manuscript should not state anywhere that the submission is under review for VRST. Authors should also be careful not to publicly promote the submission on social media, websites, or other platforms during the review period.
Non-archival venues. Non-archival venues such as workshop presentations, posters, and demos do not count as prior publications. A VRST paper should not be rejected on the grounds that it overlaps with work developed independently that was published after the VRST submission was made during the review period. Work that an author could not have known about should not count against them.
Previously presented work. In some situations, a submission may build upon prior work that has been presented in a different format (e.g., poster or demo). As part of the Call for Papers, paper authors have been instructed to clarify such cases. If you suspect any issues, please contact the paper’s primary coordinator first.
Replication. Content appearing at VRST should be new and represent a meaningful contribution. However, constructive replication of other researchers’ work is a valued contribution to XR research. A new interpretation or evaluation of previously published ideas can make a good VRST paper. For future replications to be possible, submitted work must include sufficient information, and efforts to include well-organized supplementary material facilitating replication (such as software, analysis code, and data) should be rewarded.
6. Human-Subjects Research and Ethics
For papers involving human participants, authors are required to report whether ethical approval or informed consent was obtained, as appropriate. The lack of a clear statement on ethics review and participant consent in the paper should be raised constructively, but is not, by itself, grounds for rejection beyond the desk-reject stage. Reviewers should consider whether participant samples are appropriate for the claims made, and provide balanced feedback when limitations exist.
7. Use of AI Tools in Reviewing
To maintain the integrity of the review process, reviewers must not upload or input any content from submitted manuscripts into AI systems, including but not limited to large language models (LLMs) such as ChatGPT, Claude AI, Gemini, or similar tools. This applies regardless of whether the AI system is hosted locally or accessed via the web, and whether it is accessed directly or via an API. Under no circumstances should any content of a submitted manuscript be entered into an LLM.
As a reviewer, you are responsible for the content and accuracy of your reviews, including the references cited. This is especially relevant if you choose to use AI tools for editing or proofreading your own review text, as LLMs are prone to producing plausible-sounding but inaccurate content. Maintaining confidentiality is crucial in the review process; copying and pasting content from a manuscript into an LLM may pose risks to confidentiality. For further guidance, please refer to the ACM Policy on using AI tools in reviewing: https://www.acm.org/publications/policies/peer-review-faq.
8. Writing a Meta-Review (for 1ACs)
The following guidelines outline the content of a good meta-review. This section is for review coordinators (1AC) and explains the content that best supports a final decision.
Describe the primary contribution of the paper. Start by summarizing what the paper does and what contribution it makes.
Summarize the most significant strengths and weaknesses. The most critical points are often those highlighted by the majority of reviewers. There is no need to reiterate every single aspect—the individual reviews serve that purpose.
Explain the decision and supporting reasoning. Clearly articulate the rationale behind the recommendation. The authors will see the meta-review with the final decision, so be constructive and explain your reasoning thoroughly, especially when the decision is unfavorable.
For conditional acceptance, describe the specific conditions the authors must meet before the paper can be finally accepted. These conditions should be solvable within the shepherding period and should be written in a form that allows you to verify their fulfillment when the revised version is submitted.
For rejection, add constructive suggestions for improvement. Very often, the research or paper was not ready at the time of submission. Invite the authors to resubmit in a future venue if feasible.
Avoid including discussion-phase details or individual scores in the meta-review unless they are necessary to explain the decision.
9. Frequently Asked Questions (FAQs)
Should we reject papers that have been published on ArXiv?
No. Authors may have posted on ArXiv or a similar service to obtain a timestamp for their work. However, the manuscript should not indicate that the submission is under review for VRST, and the authors should not publicly promote the submission during the review period. If you believe a paper may constitute a violation of VRST policies, please raise any related concerns in your review and/or contact the Program Chairs.
Should we reject papers that have been presented before in a different format (e.g., poster or demo)?
In some situations, a submission may build upon prior work. Authors have been instructed to be proactive in clarifying such cases by uploading additional anonymous documents to the submission system. If you suspect any issues related to this point, please contact the paper’s primary coordinator first.
Should we reject papers that report user studies without ethics review or Institutional Review Board (IRB) approval?
Authors will indicate whether they followed the ethical guidelines of their affiliation and whether approval has been obtained, or explain why no approval was necessary, in the submission form. The lack of a clear statement on ethics review and participant consent in the paper should be raised, but is not grounds for rejection past the desk-reject stage.
Should papers without a user study be rejected?
No. User studies are not required or appropriate for all papers. The authors need to support their claims with evidence, but the form of evidence can vary from paper to paper (e.g., technical evaluations, demonstrations, or formal analyses). Judge whether the validation provided is appropriate for the claims made.
Thank you for your support and work to ensure the highest-quality VRST reviews.
VRST 2026 Program Chairs
- Kazuyuki Fujita, Tohoku University, Japan
- Tham Piumsomboon, University of Canterbury, New Zealand
- Kangsoo Kim, University of Calgary, Canada
- Takefumi Hiraki, University of Tsukuba, Japan / Cluster Metaverse Lab, Japan
Document History:
This document was prepared by the VRST 2026 Program Chairs, drawing on the official VRST Reviewing Guidelines and reviewing guidelines from ISMAR 2026 and CHI 2026.