An interview study with medical scribes on how their work may alleviate clinician burnout through delegated health IT tasks

ObjectivesTo understand how medical scribes’ work may contribute to alleviating clinician burnout attributable directly or indirectly to the use of health IT.Materials and MethodsQualitative analysis of semistructured interviews with 32 participants who had scribing experience in a variety of clinical settings.ResultsWe identified 7 categories of clinical tasks that clinicians commonly choose to offload to medical scribes, many of which involve delegated use of health IT. These range from notes-taking and computerized data entry to foraging, assembling, and tracking information scattered across multiple clinical information systems. Some common characteristics shared among these tasks include: (1) time-consuming to perform; (2) difficult to remember or keep track of; (3) disruptive to clinical workflow, clinicians’ cognitive processes, or patient–provider interactions; (4) perceived to be low-skill “clerical” work; and (5) deemed as adding no value to direct patient care.DiscussionThe fact that clinicians opt to “outsource” certain clinical tasks to medical scribes is a strong indication that performing these tasks is not perceived to be the best use of their time. Given that a vast majority of healthcare practices in the US do not have the luxury of affording medical scribes, the burden would inevitably fall onto clinicians’ shoulders, which could be a major source for clinician burnout.ConclusionsMedical scribes help to offload a substantial amount of burden from clinicians—particularly with tasks that involve onerous interactions with health IT. Developing a better understanding of medical scribes’ work provides useful insights into the sources of clinician burnout and potential solutions to it.     

Different roles with different goals: Designing to support shared situational awareness between patients and clinicians in the hospital

ObjectiveTeam situational awareness helps to ensure high-quality care and prevent errors in the complex hospital environment. Although extensive work has examined factors that contribute to breakdowns in situational awareness among clinicians, patients’ and caregivers’ roles have been neglected. To address this gap, we studied team-based situational awareness from the perspective of patients and their caregivers.Materials and MethodsWe utilized a mixed-methods approach, including card sorting and semi-structured interviews with hospitalized patients and their caregivers at a pediatric hospital and an adult hospital. We analyzed the results utilizing the situational awareness (SA) theoretical framework, which identifies 3 distinct stages: (1) perception of a signal, (2) comprehension of what the signal means, and (3) projection of what will happen as a result of the signal.ResultsA total of 28 patients and 19 caregivers across the 2 sites participated in the study. Our analysis uncovered how team SA helps patients and caregivers ensure that their values are heard, their autonomy is supported, and their clinical outcomes are the best possible. In addition, our participants described both barriers—such as challenges with communication—and enablers to facilitating shared SA in the hospital.DiscussionPatients and caregivers possess critical knowledge, expertise, and values required to ensure successful and accurate team SA. Therefore, hospitals need to incorporate tools that facilitate patients and caregivers as key team members for effective SA.ConclusionsElevating patients and caregivers from passive recipients to equal contributors and members of the healthcare team will improve SA and ensure the best possible outcomes.     

Electronic consultations and economies of scale: a qualitative study of clinician perspectives on scaling up e-consult delivery

ObjectiveTo explore Veterans Health Administration clinicians’ perspectives on the idea of redesigning electronic consultation (e-consult) delivery in line with a hub-and-spoke (centralized) model.Materials and MethodsWe conducted a qualitative study in VA New England Healthcare System (VISN 1). Semi-structured phone interviews were conducted with 35 primary care providers and 38 specialty care providers, including 13 clinical leaders, at 6 VISN 1 sites varying in size, specialist availability, and e-consult volume. Interviews included exploration of the hub-and-spoke (centralized) e-consult model as a system redesign option. Qualitative content analysis procedures were applied to identify and describe salient categories.ResultsParticipants saw several potential benefits to scaling up e-consult delivery from a decentralized model to a hub-and-spoke model, including expanded access to specialist expertise and increased timeliness of e-consult responses. Concerns included differences in resource availability and management styles between sites, anticipated disruption to working relationships, lack of incentives for central e-consultants, dedicated staff’s burnout and fatigue, technological challenges, and lack of motivation for change.DiscussionBased on a case study from one of the largest integrated healthcare systems in the United States, our work identifies novel concerns and offers insights for healthcare organizations contemplating a scale-up of their e-consult systems.ConclusionsScaling up e-consults in line with the hub-and-spoke model may help pave the way for a centralized and efficient approach to care delivery, but the success of this transformation will depend on healthcare systems’ ability to evaluate and address barriers to leveraging economies of scale for e-consults.     

Measuring time clinicians spend using EHRs in the inpatient setting: a national,mixed-methods study

ObjectiveTo understand hospitals’ use of EHR audit-log-based measures to address burden associated with inpatient EHR use.Materials and MethodsUsing mixed methods, we analyzed 2018 American Hospital Association Information Technology Supplement Survey data (n = 2864 hospitals; 64% response rate) to characterize measures used and provided by EHR vendors to track clinician time spent documenting. We interviewed staff from the top 3 EHR vendors that provided these measures. Multivariable analyses identified variation in use of the measures among hospitals with these 3 vendors.Results53% of hospitals reported using EHR data to track clinician time documenting, compared to 68% of the hospitals using the EHR from the top 3 vendors. Among hospitals with EHRs from these vendors, usage was significantly lower among rural hospitals and independent hospitals (P < .05). Two of these vendors provided measures of time spent doing specific tasks while the third measured an aggregate of auditable activities. Vendors varied in the underlying data used to create measures, measure specification, and data displays.DiscussionTools to track clinicians’ documentation time are becoming more available. The measures provided differ across vendors and disparities in use exist across hospitals. Increasing the specificity of standards underlying the data would support a common set of core measures making these measures more widely available.ConclusionAlthough half of US hospitals use measures of time spent in the EHR derived from EHR generated data, work remains to make such measures and analyses more broadly available to all hospitals and to increase its utility for national burden measurement.     

Enabling a learning healthcare system with automated computer protocols that produce replicable and personalized clinician actions

Clinical decision-making is based on knowledge, expertise, and authority, with clinicians approving almost every intervention—the starting point for delivery of “All the right care, but only the right care,” an unachieved healthcare quality improvement goal. Unaided clinicians suffer from human cognitive limitations and biases when decisions are based only on their training, expertise, and experience. Electronic health records (EHRs) could improve healthcare with robust decision-support tools that reduce unwarranted variation of clinician decisions and actions. Current EHRs, focused on results review, documentation, and accounting, are awkward, time-consuming, and contribute to clinician stress and burnout. Decision-support tools could reduce clinician burden and enable replicable clinician decisions and actions that personalize patient care. Most current clinical decision-support tools or aids lack detail and neither reduce burden nor enable replicable actions. Clinicians must provide subjective interpretation and missing logic, thus introducing personal biases and mindless, unwarranted, variation from evidence-based practice. Replicability occurs when different clinicians, with the same patient information and context, come to the same decision and action. We propose a feasible subset of therapeutic decision-support tools based on credible clinical outcome evidence: computer protocols leading to replicable clinician actions (eActions). eActions enable different clinicians to make consistent decisions and actions when faced with the same patient input data. eActions embrace good everyday decision-making informed by evidence, experience, EHR data, and individual patient status. eActions can reduce unwarranted variation, increase quality of clinical care and research, reduce EHR noise, and could enable a learning healthcare system.     

An architecture for research computing in health to support clinical and translational investigators with electronic patient data

ObjectiveObtaining electronic patient data, especially from electronic health record (EHR) systems, for clinical and translational research is difficult. Multiple research informatics systems exist but navigating the numerous applications can be challenging for scientists. This article describes Architecture for Research Computing in Health (ARCH), our institution’s approach for matching investigators with tools and services for obtaining electronic patient data.Materials and MethodsSupporting the spectrum of studies from populations to individuals, ARCH delivers a breadth of scientific functions—including but not limited to cohort discovery, electronic data capture, and multi-institutional data sharing—that manifest in specific systems—such as i2b2, REDCap, and PCORnet. Through a consultative process, ARCH staff align investigators with tools with respect to study design, data sources, and cost. Although most ARCH services are available free of charge, advanced engagements require fee for service.ResultsSince 2016 at Weill Cornell Medicine, ARCH has supported over 1200 unique investigators through more than 4177 consultations. Notably, ARCH infrastructure enabled critical coronavirus disease 2019 response activities for research and patient care.DiscussionARCH has provided a technical, regulatory, financial, and educational framework to support the biomedical research enterprise with electronic patient data. Collaboration among informaticians, biostatisticians, and clinicians has been critical to rapid generation and analysis of EHR data.ConclusionA suite of tools and services, ARCH helps match investigators with informatics systems to reduce time to science. ARCH has facilitated research at Weill Cornell Medicine and may provide a model for informatics and research leaders to support scientists elsewhere.     

Development and validation of prediction models for mechanical ventilation,renal replacement therapy,and readmission in COVID-19 patients

ObjectiveCoronavirus disease 2019 (COVID-19) patients are at risk for resource-intensive outcomes including mechanical ventilation (MV), renal replacement therapy (RRT), and readmission. Accurate outcome prognostication could facilitate hospital resource allocation. We develop and validate predictive models for each outcome using retrospective electronic health record data for COVID-19 patients treated between March 2 and May 6, 2020.Materials and MethodsFor each outcome, we trained 3 classes of prediction models using clinical data for a cohort of SARS-CoV-2 (severe acute respiratory syndrome coronavirus 2)–positive patients (n = 2256). Cross-validation was used to select the best-performing models per the areas under the receiver-operating characteristic and precision-recall curves. Models were validated using a held-out cohort (n = 855). We measured each model’s calibration and evaluated feature importances to interpret model output.ResultsThe predictive performance for our selected models on the held-out cohort was as follows: area under the receiver-operating characteristic curve—MV 0.743 (95% CI, 0.682-0.812), RRT 0.847 (95% CI, 0.772-0.936), readmission 0.871 (95% CI, 0.830-0.917); area under the precision-recall curve—MV 0.137 (95% CI, 0.047-0.175), RRT 0.325 (95% CI, 0.117-0.497), readmission 0.504 (95% CI, 0.388-0.604). Predictions were well calibrated, and the most important features within each model were consistent with clinical intuition.DiscussionOur models produce performant, well-calibrated, and interpretable predictions for COVID-19 patients at risk for the target outcomes. They demonstrate the potential to accurately estimate outcome prognosis in resource-constrained care sites managing COVID-19 patients.ConclusionsWe develop and validate prognostic models targeting MV, RRT, and readmission for hospitalized COVID-19 patients which produce accurate, interpretable predictions. Additional external validation studies are needed to further verify the generalizability of our results.     

Development of a “meta-model” to address missing data,predict patient-specific cancer survival and provide a foundation for clinical decision support

ObjectiveLike most real-world data, electronic health record (EHR)–derived data from oncology patients typically exhibits wide interpatient variability in terms of available data elements. This interpatient variability leads to missing data and can present critical challenges in developing and implementing predictive models to underlie clinical decision support for patient-specific oncology care. Here, we sought to develop a novel ensemble approach to addressing missing data that we term the “meta-model” and apply the meta-model to patient-specific cancer prognosis.Materials and MethodsUsing real-world data, we developed a suite of individual random survival forest models to predict survival in patients with advanced lung cancer, colorectal cancer, and breast cancer. Individual models varied by the predictor data used. We combined models for each cancer type into a meta-model that predicted survival for each patient using a weighted mean of the individual models for which the patient had all requisite predictors.ResultsThe meta-model significantly outperformed many of the individual models and performed similarly to the best performing individual models. Comparisons of the meta-model to a more traditional imputation-based method of addressing missing data supported the meta-model’s utility.ConclusionsWe developed a novel machine learning–based strategy to underlie clinical decision support and predict survival in cancer patients, despite missing data. The meta-model may more generally provide a tool for addressing missing data across a variety of clinical prediction problems. Moreover, the meta-model may address other challenges in clinical predictive modeling including model extensibility and integration of predictive algorithms trained across different institutions and datasets.     

A framework for making predictive models useful in practice

ObjectiveTo analyze the impact of factors in healthcare delivery on the net benefit of triggering an Advanced Care Planning (ACP) workflow based on predictions of 12-month mortality.Materials and MethodsWe built a predictive model of 12-month mortality using electronic health record data and evaluated the impact of healthcare delivery factors on the net benefit of triggering an ACP workflow based on the models’ predictions. Factors included nonclinical reasons that make ACP inappropriate: limited capacity for ACP, inability to follow up due to patient discharge, and availability of an outpatient workflow to follow up on missed cases. We also quantified the relative benefits of increasing capacity for inpatient ACP versus outpatient ACP.ResultsWork capacity constraints and discharge timing can significantly reduce the net benefit of triggering the ACP workflow based on a model’s predictions. However, the reduction can be mitigated by creating an outpatient ACP workflow. Given limited resources to either add capacity for inpatient ACP versus developing outpatient ACP capability, the latter is likely to provide more benefit to patient care.DiscussionThe benefit of using a predictive model for identifying patients for interventions is highly dependent on the capacity to execute the workflow triggered by the model. We provide a framework for quantifying the impact of healthcare delivery factors and work capacity constraints on achieved benefit.ConclusionAn analysis of the sensitivity of the net benefit realized by a predictive model triggered clinical workflow to various healthcare delivery factors is necessary for making predictive models useful in practice.     

Enabling realistic health data re-identification risk assessment through adversarial modeling

ObjectiveRe-identification risk methods for biomedical data often assume a worst case, in which attackers know all identifiable features (eg, age and race) about a subject. Yet, worst-case adversarial modeling can overestimate risk and induce heavy editing of shared data. The objective of this study is to introduce a framework for assessing the risk considering the attacker’s resources and capabilities.Materials and MethodsWe integrate 3 established risk measures (ie, prosecutor, journalist, and marketer risks) and compute re-identification probabilities for data subjects. This probability is dependent on an attacker’s capabilities (eg, ability to obtain external identified resources) and the subject’s decision on whether to reveal their participation in a dataset. We illustrate the framework through case studies using data from over 1 000 000 patients from Vanderbilt University Medical Center and show how re-identification risk changes when attackers are pragmatic and use 2 known resources for attack: (1) voter registration lists and (2) social media posts.ResultsOur framework illustrates that the risk is substantially smaller in the pragmatic scenarios than in the worst case. Our experiments yield a median worst-case risk of 0.987 (where 0 is least risky and 1 is most risky); however, the median reduction in risk was 90.1% in the voter registration scenario and 100% in the social media posts scenario. Notably, these observations hold true for a wide range of adversarial capabilities.ConclusionsThis research illustrates that re-identification risk is situationally dependent and that appropriate adversarial modeling may permit biomedical data sharing on a wider scale than is currently the case.     

A predictive model for next cycle start date that accounts for adherence in menstrual self-tracking

ObjectiveThe study sought to build predictive models of next menstrual cycle start date based on mobile health self-tracked cycle data. Because app users may skip tracking, disentangling physiological patterns of menstruation from tracking behaviors is necessary for the development of predictive models.Materials and MethodsWe use data from a popular menstrual tracker (186 000 menstruators with over 2 million tracked cycles) to learn a predictive model, which (1) accounts explicitly for self-tracking adherence; (2) updates predictions as a given cycle evolves, allowing for interpretable insight into how these predictions change over time; and (3) enables modeling of an individual''s cycle length history while incorporating population-level information.ResultsCompared with 5 baselines (mean, median, convolutional neural network, recurrent neural network, and long short-term memory network), the model yields better predictions and consistently outperforms them as the cycle evolves. The model also provides predictions of skipped tracking probabilities.DiscussionMobile health apps such as menstrual trackers provide a rich source of self-tracked observations, but these data have questionable reliability, as they hinge on user adherence to the app. By taking a machine learning approach to modeling self-tracked cycle lengths, we can separate true cycle behavior from user adherence, allowing for more informed predictions and insights into the underlying observed data structure.ConclusionsDisentangling physiological patterns of menstruation from adherence allows for accurate and informative predictions of menstrual cycle start date and is necessary for mobile tracking apps. The proposed predictive model can support app users in being more aware of their self-tracking behavior and in better understanding their cycle dynamics.     

SynTEG: a framework for temporal structured electronic health data simulation

ObjectiveSimulating electronic health record data offers an opportunity to resolve the tension between data sharing and patient privacy. Recent techniques based on generative adversarial networks have shown promise but neglect the temporal aspect of healthcare. We introduce a generative framework for simulating the trajectory of patients’ diagnoses and measures to evaluate utility and privacy.Materials and MethodsThe framework simulates date-stamped diagnosis sequences based on a 2-stage process that 1) sequentially extracts temporal patterns from clinical visits and 2) generates synthetic data conditioned on the learned patterns. We designed 3 utility measures to characterize the extent to which the framework maintains feature correlations and temporal patterns in clinical events. We evaluated the framework with billing codes, represented as phenome-wide association study codes (phecodes), from over 500 000 Vanderbilt University Medical Center electronic health records. We further assessed the privacy risks based on membership inference and attribute disclosure attacks.ResultsThe simulated temporal sequences exhibited similar characteristics to real sequences on the utility measures. Notably, diagnosis prediction models based on real versus synthetic temporal data exhibited an average relative difference in area under the ROC curve of 1.6% with standard deviation of 3.8% for 1276 phecodes. Additionally, the relative difference in the mean occurrence age and time between visits were 4.9% and 4.2%, respectively. The privacy risks in synthetic data, with respect to the membership and attribute inference were negligible.ConclusionThis investigation indicates that temporal diagnosis code sequences can be simulated in a manner that provides utility and respects privacy.     

Identifying and managing adverse environmental health effects: 5. Persistent organic pollutants

CONCERN AND AWARENESS IS GROWING about the health effects of exposures to environmental contaminants, including those found in food. Most primary care physicians lack knowledge and training in the clinical recognition and management of the health effects of environmental exposures. We have found that the use of a simple history-taking tool — the CH²OPD² mnemonic (Community, Home, Hobbies, Occupation, Personal habits, Diet and Drugs) — can help physicians identify patients at risk of such health effects. We present an illustrative case of a mother who is concerned about eating fish and wild game because her 7-year-old son has been found to have learning difficulties and she is planning another pregnancy. Potential exposures to persistent organic pollutants (POPs) and mercury are considered. The neurodevelopmental effects of POPs on the fetus are reviewed. We provide advice to limit a patient's exposure to these contaminants and discuss the relevance of these exposures to the learning difficulties of the 7-year-old child and to the planning of future pregnancies.     

Predicting hospitalization of COVID-19 positive patients using clinician-guided machine learning methods

ObjectivesThe coronavirus disease 2019 (COVID-19) is a resource-intensive global pandemic. It is important for healthcare systems to identify high-risk COVID-19-positive patients who need timely health care. This study was conducted to predict the hospitalization of older adults who have tested positive for COVID-19.MethodsWe screened all patients with COVID test records from 11 Mass General Brigham hospitals to identify the study population. A total of 1495 patients with age 65 and above from the outpatient setting were included in the final cohort, among which 459 patients were hospitalized. We conducted a clinician-guided, 3-stage feature selection, and phenotyping process using iterative combinations of literature review, clinician expert opinion, and electronic healthcare record data exploration. A list of 44 features, including temporal features, was generated from this process and used for model training. Four machine learning prediction models were developed, including regularized logistic regression, support vector machine, random forest, and neural network.ResultsAll 4 models achieved area under the receiver operating characteristic curve (AUC) greater than 0.80. Random forest achieved the best predictive performance (AUC = 0.83). Albumin, an index for nutritional status, was found to have the strongest association with hospitalization among COVID positive older adults.ConclusionsIn this study, we developed 4 machine learning models for predicting general hospitalization among COVID positive older adults. We identified important clinical factors associated with hospitalization and observed temporal patterns in our study cohort. Our modeling pipeline and algorithm could potentially be used to facilitate more accurate and efficient decision support for triaging COVID positive patients.     

Can we understand population healthcare needs using electronic medical records?

INTRODUCTIONThe identification of population-level healthcare needs using hospital electronic medical records (EMRs) is a promising approach for the evaluation and development of tailored healthcare services. Population segmentation based on healthcare needs may be possible using information on health and social service needs from EMRs. However, it is currently unknown if EMRs from restructured hospitals in Singapore provide information of sufficient quality for this purpose. We compared the inter-rater reliability between a population segment that was assigned prospectively and one that was assigned retrospectively based on EMR review.METHODS200 non-critical patients aged ≥ 55 years were prospectively evaluated by clinicians for their healthcare needs in the emergency department at Singapore General Hospital, Singapore. Trained clinician raters with no prior knowledge of these patients subsequently accessed the EMR up to the prospective rating date. A similar healthcare needs evaluation was conducted using the EMR. The inter-rater reliability between the two rating sets was evaluated using Cohen’s Kappa and the incidence of missing information was tabulated.RESULTSThe inter-rater reliability for the medical ‘global impression’ rating was 0.37 for doctors and 0.35 for nurses. The inter-rater reliability for the same variable, retrospectively rated by two doctors, was 0.75. Variables with a higher incidence of missing EMR information such as ‘social support in case of need’ and ‘patient activation’ had poorer inter-rater reliability.CONCLUSIONPre-existing EMR systems may not capture sufficient information for reliable determination of healthcare needs. Thus, we should consider integrating policy-relevant healthcare need variables into EMRs.     

A neuro-symbolic method for understanding free-text medical evidence

ObjectiveWe introduce Medical evidence Dependency (MD)–informed attention, a novel neuro-symbolic model for understanding free-text clinical trial publications with generalizability and interpretability.Materials and MethodsWe trained one head in the multi-head self-attention model to attend to the Medical evidence Ddependency (MD) and to pass linguistic and domain knowledge on to later layers (MD informed). This MD-informed attention model was integrated into BioBERT and tested on 2 public machine reading comprehension benchmarks for clinical trial publications: Evidence Inference 2.0 and PubMedQA. We also curated a small set of recently published articles reporting randomized controlled trials on COVID-19 (coronavirus disease 2019) following the Evidence Inference 2.0 guidelines to evaluate the model’s robustness to unseen data. ResultsThe integration of MD-informed attention head improves BioBERT substantially in both benchmark tasks—as large as an increase of +30% in the F1 score—and achieves the new state-of-the-art performance on the Evidence Inference 2.0. It achieves 84% and 82% in overall accuracy and F1 score, respectively, on the unseen COVID-19 data.Conclusions MD-informed attention empowers neural reading comprehension models with interpretability and generalizability via reusable domain knowledge. Its compositionality can benefit any transformer-based architecture for machine reading comprehension of free-text medical evidence.     

Deep propensity network using a sparse autoencoder for estimation of treatment effects

ObjectiveDrawing causal estimates from observational data is problematic, because datasets often contain underlying bias (eg, discrimination in treatment assignment). To examine causal effects, it is important to evaluate what-if scenarios—the so-called “counterfactuals.” We propose a novel deep learning architecture for propensity score matching and counterfactual prediction—the deep propensity network using a sparse autoencoder (DPN-SA)—to tackle the problems of high dimensionality, nonlinear/nonparallel treatment assignment, and residual confounding when estimating treatment effects.Materials and MethodsWe used 2 randomized prospective datasets, a semisynthetic one with nonlinear/nonparallel treatment selection bias and simulated counterfactual outcomes from the Infant Health and Development Program and a real-world dataset from the LaLonde’s employment training program. We compared different configurations of the DPN-SA against logistic regression and LASSO as well as deep counterfactual networks with propensity dropout (DCN-PD). Models’ performances were assessed in terms of average treatment effects, mean squared error in precision on effect’s heterogeneity, and average treatment effect on the treated, over multiple training/test runs.ResultsThe DPN-SA outperformed logistic regression and LASSO by 36%–63%, and DCN-PD by 6%–10% across all datasets. All deep learning architectures yielded average treatment effects close to the true ones with low variance. Results were also robust to noise-injection and addition of correlated variables. Code is publicly available at https://github.com/Shantanu48114860/DPN-SAz.Discussion and ConclusionDeep sparse autoencoders are particularly suited for treatment effect estimation studies using electronic health records because they can handle high-dimensional covariate sets, large sample sizes, and complex heterogeneity in treatment assignments.     

The mental health status of children who have been evacuated or migrated from rural areas in Fukushima prefecture after the Fukushima daiichi nuclear power station accident: results from the Fukushima health management survey

Introduction: We evaluated the mental health status of children residing in Kawauchi village (Kawauchi), Fukushima Prefecture, after the 2011 accident at the Fukushima Daiichi Nuclear Power Station, based on the children’s experience of the nuclear disaster. Methods: We conducted this cross-sectional study within the framework of the Fukushima Health Management Survey (FHMS); FHMS data on age, sex, exercise habits, sleeping times, experience of the nuclear disaster, and the “Strengths and Difficulties Questionnaire (SDQ)” scores for 156 children from Kawauchi in 2012 were collected. Groups with and without experience of the nuclear disaster — “nuclear disaster (+)” and “nuclear disaster (−)” — were also compared. Results: Our effective response was 93 (59.6%); the mean SDQ score was 11.4±6.8 among elementary school-aged participants and 12.4±6.8 among junior high school-aged ones. We statistically compared the Total Difficulties Scores (TDS) and sub-item scores of the SDQ between “elementary school” and “junior high school” or “nuclear disaster” (+) and (−). There was no significant difference between these items. Conclusions: We found indications of poor mental health among elementary and junior high school-aged children in the disaster area immediately following the accident, but no differences based on their experience of the nuclear disaster. These results indicate the possibility of triggering stress, separate to that from experiences related to the nuclear disaster, in children who lived in affected rural areas and were evacuated just after the nuclear disaster.