Minimum information to report about a flow cytometry experiment on extracellular vesicles: Communication from the ISTH SSC subcommittee on vascular biology

The Extracellular Vesicle Flow Cytometry Working Group ( http://www.evflowcytometry.org ) is formed by members of the International Society for Extracellular Vesicles (ISEV), the International Society for Advancement of Cytometry (ISAC), and the International Society on Thrombosis and Haemostasis (ISTH). This working group of flow cytometry experts develops guidelines for best practices regarding flow cytometry detection of extracellular vesicles. To improve rigor and standardization, this working group published a framework outlining the minimal information to report about a flow cytometry experiment on extracellular vesicles (MIFlowCyt-EV) in the Journal of Extracellular Vesicles, the ISEV journal, in 2020. In parallel, an article explaining MIFlowCyt-EV was published in Cytometry Part A, one of the ISAC journals, and now will be introduced to the ISTH as an SSC Communication in the Journal of Thrombosis and Haemostasis. The goal of this SSC Communication is to explain why flow cytometry is becoming the instrument of choice to characterize single extracellular vesicles, the obstacles that have been identified and (mostly) overcome by developing procedures to calibrate flow cytometers, and the relevance of reporting minimal information to improve reliability and reproducibility of experiments in which flow cytometers are used for characterization of extracellular vesicles.     

Clinical validation of three-dimensional ultrasound for abdominal aortic aneurysm

ObjectiveThree-dimensional ultrasound (3D-US) examination is a relatively new modality that can be used for abdominal aortic aneurysm (AAA) surveillance, and may offer improved reproducibility over conventional two-dimensional ultrasound (2D-US) examination. The aim of this study was to evaluate the interoperator reproducibility of maximum anterior-to-posterior diameter by nonphysician ultrasound technicians in a typical vascular laboratory setting, on patients with infrarenal AAAs using 3D-US and 2D-US examination.MethodsA total of 134 consecutive patients with asymptomatic infrarenal AAAs were screened. Of the 134 patients, 28 (21%) were screen failures. From the remaining 106 patients, 3 (2.8%) had missing data and 13 (12.3%) had technically unacceptable image quality. As a result, 90 patients were included for final analysis. Ultrasound image acquisitions were performed during the single visit. The 2D-US images were evaluated at the time of examination by the respective ultrasound technicians who acquired them. All 3D-US images were evaluated offline by both ultrasound technicians after a wash-out period of at least 6 weeks.ResultsExcellent interoperator reproducibility was observed for measuring maximum diameter using 3D-US (intraclass correlation coefficient, 0.97), and good agreement among ultrasound technicians (mean difference, −0.08 mm; limits of agreement, −3.17; 3.00 mm). When using 3D-US examination, 74 of the 90 patients (82%) were estimated within 2 mm of interoperator variability. Of 90 patients, 52 (58%) were estimated to be within the same variability by 2D-US examination. Estimating AAA diameter using 3D-US was superior to 2D-US with respect to interoperator reproducibility.ConclusionsBoth 3D-US and 2D-US examination demonstrated good reproducibility among two vascular ultrasound technicians with superior agreement from 3D-US examination. The present results support the broader use of 3D-US in standard AAA surveillance programs.     

Quality and denoising in real‐time functional magnetic resonance imaging neurofeedback: A methods review

Neurofeedback training using real‐time functional magnetic resonance imaging (rtfMRI‐NF) allows subjects voluntary control of localised and distributed brain activity. It has sparked increased interest as a promising non‐invasive treatment option in neuropsychiatric and neurocognitive disorders, although its efficacy and clinical significance are yet to be determined. In this work, we present the first extensive review of acquisition, processing and quality control methods available to improve the quality of the neurofeedback signal. Furthermore, we investigate the state of denoising and quality control practices in 128 recently published rtfMRI‐NF studies. We found: (a) that less than a third of the studies reported implementing standard real‐time fMRI denoising steps, (b) significant room for improvement with regards to methods reporting and (c) the need for methodological studies quantifying and comparing the contribution of denoising steps to the neurofeedback signal quality. Advances in rtfMRI‐NF research depend on reproducibility of methods and results. Notably, a systematic effort is needed to build up evidence that disentangles the various mechanisms influencing neurofeedback effects. To this end, we recommend that future rtfMRI‐NF studies: (a) report implementation of a set of standard real‐time fMRI denoising steps according to a proposed COBIDAS‐style checklist ( https://osf.io/kjwhf/ ), (b) ensure the quality of the neurofeedback signal by calculating and reporting community‐informed quality metrics and applying offline control checks and (c) strive to adopt transparent principles in the form of methods and data sharing and support of open‐source rtfMRI‐NF software. Code and data for reproducibility, as well as an interactive environment to explore the study data, can be accessed at https://github.com/jsheunis/quality‐and‐denoising‐in‐rtfmri‐nf.     

Reproducibility in the absence of selective reporting: An illustration from large-scale brain asymmetry research

The problem of poor reproducibility of scientific findings has received much attention over recent years, in a variety of fields including psychology and neuroscience. The problem has been partly attributed to publication bias and unwanted practices such as p-hacking. Low statistical power in individual studies is also understood to be an important factor. In a recent multisite collaborative study, we mapped brain anatomical left–right asymmetries for regional measures of surface area and cortical thickness, in 99 MRI datasets from around the world, for a total of over 17,000 participants. In the present study, we revisited these hemispheric effects from the perspective of reproducibility. Within each dataset, we considered that an effect had been reproduced when it matched the meta-analytic effect from the 98 other datasets, in terms of effect direction and significance threshold. In this sense, the results within each dataset were viewed as coming from separate studies in an “ideal publishing environment,” that is, free from selective reporting and p hacking. We found an average reproducibility rate of 63.2% (SD = 22.9%, min = 22.2%, max = 97.0%). As expected, reproducibility was higher for larger effects and in larger datasets. Reproducibility was not obviously related to the age of participants, scanner field strength, FreeSurfer software version, cortical regional measurement reliability, or regional size. These findings constitute an empirical illustration of reproducibility in the absence of publication bias or p hacking, when assessing realistic biological effects in heterogeneous neuroscience data, and given typically-used sample sizes.     

Enhancing reproducibility in developmental EEG research: BIDS,cluster-based permutation tests,and effect sizes

Developmental research using electroencephalography (EEG) offers valuable insights in brain processes early in life, but at the same time, applying this sensitive technique to young children who are often non-compliant and have short attention spans comes with practical limitations. It is thus of particular importance to optimally use the limited resources to advance our understanding of development through reproducible and replicable research practices. Here, we describe methodological approaches that help maximize the reproducibility of developmental EEG research. We discuss how to transform EEG data into the standardized Brain Imaging Data Structure (BIDS) which organizes data according to the FAIR data sharing principles. We provide a tutorial on how to use cluster-based permutation testing to analyze developmental EEG data. This versatile test statistic solves the multiple comparison problem omnipresent in EEG analysis and thereby substantially decreases the risk of reporting false discoveries. Finally, we describe how to quantify effect sizes, in particular of cluster-based permutation results. Reporting effect sizes conveys a finding’s impact and robustness which in turn informs future research. To demonstrate these methodological approaches to data organization, analysis and report, we use a publicly accessible infant EEG dataset and provide a complete copy of the analysis code.     

Inter-rater reliability of the Swedish modified version of the Postural Assessment Scale for Stroke Patients (SwePASS) in the acute phase after stroke

Background: Before implementation of the new scale, the Swedish modified version of the Postural Assessment Scale for Stroke Patients (SwePASS), to clinical practice, it is fundamental to analyze its measurement properties.Objective: To examine the inter-rater reliability of the SwePASS in the acute phase after stroke.

Methods: Day 3 to day 7 after admission to a stroke unit, 64 persons with stroke were assessed twice, using the SwePASS, by two physiotherapists. Inter-rater reliability was determined using percentage-agreement and the rank-invariant method: relative position, relative concentration, and relative rank variance.

Results: The raters showed a percentage agreement of ≥75% in the assessments using the SwePASS. For 9 of the 12 items, the percentage agreement was >80%. For 8 of the 12 items, there was a statistically significant change in position, revealed in relative position values between 0.08 and 0.15. Three items had statistically significant positive relative concentration values between ?0.11 and 0.10. Except for a statistically significant negligible relative variance value of 0.01 for the items 1 and 8, there was no relative variance.

Conclusions: The SwePASS shows an acceptable inter-rater reliability, albeit with potential for improvement. The reliability can be improved by a consensus how to interpret the scale between the raters prior to implementation in the clinic.