Editor
Jordan Anaya
The GRIMMER test: A method for testing the validity of reported measures of variability (2016)
Jordan Anaya
In this publication I extended granularity testing to variances, and as a result standard deviations and standard errors as well. Extending granularity testing to variances was nontrivial, and in fact wasn't thought possible.
Essentially I identified that variances of discrete data show a regular pattern, and only certain variances are consistent with certain means. At smaller sample sizes these patterns are not hard to see by eye, but at larger sample sizes programs were used to identify the patterns. I also described various properties of the patterns, and made a web application for testing whether variances/standard deviations/standard errors are mathematically possible.
While my code claimed these patterns exist, and the patterns had dramatic properties, it still wasn't clear whether there was a mathematical basis for the patterns and whether they would hold up at any sample size. However, a mathematical explanation for these patterns was recently posted: https://osf.io/rvgqk/. When looking at all the variances there is no rhyme or reason to them, but looking at all the variances which match a set of means reveals a simple step size. The question then becomes how to find the first variance in this set, and the authors of this preprint provide a mathematical proof for doing so.
The importance of being able to check whether standard deviations are mathematically possible can be debated. If a researcher is committing fraud it is likely just checking the means in a paper will be sufficient. However, if granularity testing is seen more as a type of "spellchecker", then obviously being able to check more statistics is valuable. In addition, given that some test statistics can be recalculated simply by knowing the means, standard deviations, and sample sizes, knowing the granularity of the means and standard deviations allows you to find the granularity of the test statistic.
A more recent use of the patterns identified in this publication is in reconstructing data. Reconstructing data can be computationally intensive, so it can be worthwhile to check if the reconstruction is even possible by looking at the granularity of the means and standard deviations. Indeed, my version of SPRITE does this, and the authors of CORVIDS are also now doing this.
While extending granularity testing to variances is a small step, it is a step nonetheless, so I am accepting my article.
A peeriodical is a lightweight virtual journal with you as the Editor-in-chief, giving you complete freedom in setting editorial policy to select the most interesting and useful manuscripts for your readers. The manuscripts you will evaluate and select are existing publications—preprints and papers. Thus, a peeriodical replicates all the functions of a traditional journal, including discovery, selection and certification, except publication itself.
Why set up a peeriodical? The traditional journal has changed remarkably little in centuries and many people feel that scientific publishing is stuck in a rut, subject to a corporatist drift, and is not serving science optimally. The advent of preprints in many fields beyond those served by the ArXiv is liberating the dissemination of research, but most other journal functions have not been replaced effectively. Now you—all researchers—have the opportunity to select and certify research according to your own criteria. We expect peeriodical subject matters and editorial policies to be extremely varied. Some peeriodicals may wish to target narrow domains, while others will adopt a generalist approach. Some peeriodicals will be inclusive, focusing on discovery, whereas others may aim to enforce stringent quality criteria, prioritising certification. The point is that all approaches are permitted and supported—we hope you will innovate! You can create multiple peeriodicals. It will be users and readers who decide which peeriodicals they find useful and interesting. Users can sign up to receive alerts from any peeriodical they wish.
A peeriodical has one or more editors. Anybody can set-up a peeriodical and either operate it alone or invite colleagues to form an editorial board or community. The editors can select "manuscripts"—existing papers or preprints—to consider, either spontaneously or through suggestions from other researchers, including of course the authors. Note that there is no obligation that the manuscript be recent; for instance, we expect that some peeriodicals could focus on underappreciated classics. After all, predictions about scientific impact are generally more accurate for the past than the future. If the editors wish, they can solicit reviews for the manuscript via the Peeriodicals interface. Reviews will be published and the referees will have the option of posting anonymously or signing their review. Editors may decide at any time to accept, reject or comment on the manuscript, taking into account the comments received. They may of course suggest improvements to the manuscript or underlying study. If they justify their decision, their editorial decision will also be published.
How will Peeriodicals fit into the publishing landscape? We see them as a space without entry barriers in which researchers can innovate and explore new approaches to scientific dissemination, in parallel to the traditional publishing industry. There are related and complementary initiatives, notably the overlay journals promoted by Tim Gowers, exemplified by Discrete Analysis, but also Science Open Collections, PLoS Channels, the APPRAISE initiative and Peer Community in... Each of these projects has their own specificities and goals. Nobody yet knows exactly what the future will look like, but we strongly believe that we are about to experience a period of rapid evolution in the dissemination of science and we hope that Peeriodicals will inspire and help you to share your imagination and expertise with the whole research community.
For those starting a peeriodical, you will discover that the hardest part is building up an audience. Unfortunately, we can't yet guarantee you the exposure you would get from a paper in a glamour journal. Reviews with scientific content will be mirrored on PubPeer, offering an audience through the PubPeer browser and Zotero extensions. However, it will be largely up to you to run your publicity, most likely through social media. We are on Twitter (@PEERIODICALS) and will of course help out as we can.
Get started now by requesting an invitation with the link in the top right menu.