The peer review system is bursting at the seams. Editors have an increasingly hard time to secure reviewers, and at the same time, it is nearly impossible for any individual reviewer to identify every potential issue in a manuscript. As a result, incomplete, erroneous, or even fabricated results continue to find their way into published papers. A solution might be found in the use of automated screening tools. Indeed, authors, reviewers, and journals are already experimenting with a wide variety of tools; from relatively simple and tailored “spellcheckers for statistics” to AI quality assessments generated by ChatGPT. However, we currently do not know if and how tools can best support traditional peer review, or, more generally, if we are using the right tools in the right way and at the right moment. In this talk, I will argue that we need a "future-proof validity framework for automated peer review". Such a framework would allow us to 1) systematically assess tool capability and validity, 2) define an optimal division of labor between tools and reviewers, and 3) strategically apply the right tools at the right moment in the research pipeline.