Learning to Discretize: Solving 1D Scalar Conservation Laws via Deep Reinforcement Learning

Conservation laws are considered to be fundamental laws of nature. It has broad applications in many fields, including physics, chemistry, biology, geology, and engineering. Solving the differential equations associated with conservation laws is a major branch in computational mathematics. The recent success of machine learning, especially deep learning in areas such as computer vision and natural language processing, has attracted a lot of attention from the community of computational mathematics and inspired many intriguing works in combining machine learning with traditional methods. In this paper, we are the first to view numerical PDE solvers as an MDP and to use (deep) RL to learn new solvers. As proof of concept, we focus on 1-dimensional scalar conservation laws. We deploy the machinery of deep reinforcement learning to train a policy network that can decide on how the numerical solutions should be approximated in a sequential and spatial-temporal adaptive manner. We will show that the problem of solving conservation laws can be naturally viewed as a sequential decision-making process, and the numerical schemes learned in such a way can easily enforce long-term accuracy. Furthermore, the learned policy network is carefully designed to determine a good local discrete approximation based on the current state of the solution, which essentially makes the proposed method a meta-learning approach. In other words, the proposed method is capable of learning how to discretize for a given situation mimicking human experts. Finally, we will provide details on how the policy network is trained, how well it performs compared with some state-of-the-art numerical solvers such as WENO schemes, and supervised learning based approach L3D and PINN, and how well it generalizes.     

Turboprop: improved PROPELLER imaging.

A variant of periodically rotated overlapping parallel lines with enhanced reconstruction (PROPELLER) MRI, called turboprop, is introduced. This method employs an oscillating readout gradient during each spin echo of the echo train to collect more lines of data per echo train, which reduces the minimum scan time, motion-related artifact, and specific absorption rate (SAR) while increasing sampling efficiency. It can be applied to conventional fast spin-echo (FSE) imaging; however, this article emphasizes its application in diffusion-weighted imaging (DWI). The method is described and compared with conventional PROPELLER imaging, and clinical images collected with this PROPELLER variant are shown.     

Correction of frequency and phase variations induced by eddy currents in localized spectroscopy with multiple echo times.

As a consequence of the time-varying magnetic field induced by eddy currents, frequency drifting occurs when the sampling window of localized spectroscopy continuously shifts. The frequency drifting and the concomitant phase variations can severely affect spectroscopy results when data are acquired with multiple echo times (TEs), such as in the measurement of glutamate (Glu) concentration using the TE-averaged method. Specifically, the averaged spectra are further broadened and distorted in the presence of residual eddy currents, and editing of the coupled spins of Glu C4 protons is affected, resulting in errors in the measured relative intensity ratio. Postacquisition correction using unsuppressed water as reference can effectively minimize this detrimental effect, as manifested by the significantly enhanced signal intensity. Also, it is demonstrated that the methyl signals of creatine (Cr) at 3.0 ppm and choline (Cho) at 3.2 ppm can be used as internal references in finding frequency and phase disparities between different TEs.     

166Ho-DOTMP radiation-absorbed dose estimation for skeletal targeted radiotherapy.

166Ho-1,4,7,10-tetraazacyclododecane-1,4,7,10-tetramethylene-phosphonate (DOTMP) is a tetraphosphonate molecule radiolabeled with 166Ho that localizes to bone surfaces. This study evaluated pharmacokinetics and radiation-absorbed dose to all organs from this beta-emitting radiopharmaceutical. METHODS: After two 1.1-GBq administrations of 166Ho-DOTMP, data from whole-body counting using a gamma-camera or uptake probe were assessed for reproducibility of whole-body retention in 12 patients with multiple myeloma. The radiation-absorbed dose to normal organs was estimated using MIRD methodology, applying residence times and S values for 166Ho. Marrow dose was estimated from measured activity retained after 18 h. The activity to deliver a therapeutic dose of 25 Gy to the marrow was determined. Methods based on region-of-interest (ROI) and whole-body clearance were evaluated to estimate kidney activity, because the radiotracer is rapidly excreted in the urine. The dose to the surface of the bladder wall was estimated using a dynamic bladder model. RESULTS: In clinical practice, gamma-camera methods were more reliable than uptake probe-based methods for whole-body counting. The intrapatient variability of dose calculations was less than 10% between the 2 tracer studies. Skeletal uptake of 166Ho-DOTMP varied from 19% to 39% (mean, 28%). The activity of 166Ho prescribed for therapy ranged from 38 to 67 GBq (1,030-1,810 mCi). After high-dose therapy, the estimates of absorbed dose to the kidney varied from 1.6 to 4 Gy using the whole-body clearance-based method and from 8.3 to 17.3 Gy using the ROI-based method. Bladder dose ranged from 10 to 20 Gy, bone surface dose ranged from 39 to 57 Gy, and doses to other organs were less than 2 Gy for all patients. Repetitive administration had no impact on tracer biodistribution, pharmacokinetics, or organ dose. CONCLUSION: Pharmacokinetics analysis validated gamma-camera whole-body counting of 166Ho as an appropriate approach to assess clearance and to estimate radiation-absorbed dose to normal organs except the kidneys. Quantitative gamma-camera imaging is difficult and requires scatter subtraction because of the multiple energy emissions of 166Ho. Kidney dose estimates were approximately 5-fold higher when the ROI-based method was used rather than the clearance-based model, and neither appeared reliable. In future clinical trials with 166Ho-DOTMP, we recommend that dose estimation based on the methods described here be used for all organs except the kidneys. Assumptions for the kidney dose require further evaluation.     

Etanercept decreases tumor necrosis factor-α activity in chronic wound fluid

High levels of tumor necrosis factor-alpha (TNF-alpha), a pro-inflammatory cytokine, are present in the wound fluid of chronic nonhealing wounds. This leads to increased inflammation, cytokine expression, and ultimately results in impaired wound healing and tissue destruction. Etanercept is a recombinant fusion protein that consists of the soluble TNF receptor (p75) linked to the Fc portion of human IgG1. It is an effective inhibitor of TNF-alpha and has been shown to provide rapid and sustained improvement in rheumatoid arthritis by acting as a soluble receptor binding TNF-alpha and preventing its proinflammatory activities. Therefore, the aim of this study was to determine whether Etanercept could inhibit TNF-alpha activity in chronic wound fluid. Wound fluid was collected from the venous leg ulcers of 16 different patients. The effect of Etanercept on TNF-alpha activity was evaluated using both a TNF-alpha bioassay and an enzyme-linked immunoassay. Etanercept was found to reduce the cytotoxic effect of chronic wound fluid on L929 fibroblasts by approximately 30% and neutralized TNF-alpha binding in the enzyme-linked immunoassay by up to 80%. Direct application of Etanercept to chronic wounds may therefore reduce the inflammatory activity of TNF-alpha, which could reduce the chronicity of venous leg ulcers and thus aid in the healing of these wounds.