Issue 
J. Space Weather Space Clim.
Volume 11, 2021



Article Number  9  
Number of page(s)  16  
DOI  https://doi.org/10.1051/swsc/2020073  
Published online  29 January 2021 
Research Article
Inferring depthdependent plasma motions from surface observations using the DeepVel neural network
^{1}
National Solar Observatory, 3665 Discovery Dr., Boulder, 80303 CO, USA
^{2}
University of Colorado, Astrophysical and Planetary Sciences, 2000 Colorado Avenue, Boulder, 80303 CO, USA
^{3}
Université de Montréal, Département de Physique, 1375 Avenue ThérèseLavoieRoux, Montréal, H2V 0B3 QC, Canada
^{4}
Environment & Climate Change Canada, Dorval, H9P 1J3 QC, Canada
^{5}
University of California, Space Sciences Laboratory, 7 Gauss Way, Berkeley, 94720 CA, USA
^{*} Corresponding author: btremblay@nso.edu
Received:
5
June
2020
Accepted:
26
November
2020
Coverage of plasma motions is limited to the lineofsight component at the Sun’s surface. Multiple tracking and inversion methods were developed to infer the transverse motions from observational data. Recently, the DeepVel neural network was trained with computations performed by numerical simulations of the solar photosphere to recover the missing transverse component at the surface and at two additional optical depths simultaneously from the surface white light intensity in the Quiet Sun. We argue that deep learning could provide additional spatial coverage to existing observations in the form of depthdependent synthetic observations, i.e. estimates generated through the emulation of numerical simulations. We trained different versions of DeepVel using slices from numerical simulations of both the Quiet Sun and Active Region at various optical and geometrical depths in the solar atmosphere, photosphere and upper convection zone to establish the upper and lower limits at which the neural network can generate reliable synthetic observations of plasma motions from surface intensitygrams. Flow fields inferred in the photosphere and low chromosphere τ ∈ [0.1, 1) are comparable to inversions performed at the surface (τ ≈ 1) and are deemed to be suitable for use as synthetic estimates in data assimilation processes and datadriven simulations. This upper limit extends closer to the transition region (τ ≈ 0.01) in the Quiet Sun, but not for Active Regions. Subsurface flows inferred from surface intensitygrams fail to capture the smallscale features of turbulent convective motions as depth crosses a few hundred kilometers. We suggest that these reconstructions could be used as first estimates of a model’s velocity vector in data assimilation processes to nowcast and forecast short term solar activity and space weather.
Key words: active region / chromosphere / convection zone / deep learning / granulation / photosphere / velocity fields
© Tremblay et al., Published by EDP Sciences 2021
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
1 Introduction
Prediction of space weather critically depends on how we can use observations to numerically simulate the physical processes in the solar photosphere and above. Datainspired numerical models of the Sun, which are built upon our best understanding of radiative magnetohydrodynamics (MHD), have produced increasingly realistic simulations of the Quiet Sun (e.g., Stein & Nordlund, 1998; Vögler et al., 2005; Abbett, 2007) and of Active Regions (e.g., Rempel & Cheung, 2014; Jiang et al., 2016). Recent models even generate energy releases consistent with flaring events (Cheung et al., 2018). Datadriven simulations or the implementation of more complex methods like data assimilation could further bridge the gap between MHD models of the upper convection zone to corona and observations, as suggested in Abbett & Fisher (2010), and implemented by e.g. Fisher et al. (2015) and Hayashi et al. (2018) for the solar atmosphere. Data assimilation is defined as adjusting the initial condition of a model (not its parameters!) to minimize errors between model predictions and satellite/groundbased observations within a window of time (i.e., the assimilation window; Bouttier & Courtier, 2002). Generated from the new initial condition is an improved representation of a given epoch of the Sun and from which to validate the physics encompassed by the model within the assimilation window (i.e., nowcasting) and to forecast short term solar activity (e.g., the evolution of an active region and the onset of space weather events) beyond the assimilation window. Remaining errors between observations and the adjusted model predictions can be interpreted as a measure of the degree of realism of the simulation. In this sense, the assimilation of data within a time window containing a space weather event could improve our understanding of the physical mechanisms and physical conditions associated with its onset and, by extension, improve our ability to forecast future events.
Performing data assimilation in a MHD model is not without difficulties. Firstly, an estimate of the initial threedimensional model state (i.e., physical values at the beginning of the assimilation window) is required to begin the process. Secondly, the aforementioned simulations solve for physical quantities that, due to physical or instrumental limitations, cannot be derived from direct observations or can only be derived at select depths. One such model variable is the velocity vector. For example, Dopplergrams can only capture the lineofsight component of plasma motions at the Sun’s surface. Various methods have been developed to reconstruct the missing transverse component from tracking algorithms and observed intensitygrams (e.g., November & Simon, 1988; Potts et al., 2004; Rieutord et al., 2007) or magnetograms (e.g., Fisher & Welsch, 2008), or from magnetograms and physical principles (e.g., Longcope, 2004; Schuck, 2008; Kazachenko et al., 2014; Attie & Innes, 2015) in the photosphere. Recently, neural network computing has been used in conjunction with numerical models of solar granulation to be able to recover the transverse velocity vector in photospheric plasma of the Quiet Sun. The fullyconvolutional DeepVel neural network (Asensio Ramos et al., 2017) infers instantaneous transverse plasma motions at three simultaneous optical depths [τ] ≈ {1, 0.1, 0.01}, including the surface, photosphere and chromosphere, from pairs of intensitygrams. A comparison between intensitybased methods identified DeepVel as the method best suited to generate instantaneous synthetic observations of plasma motions at the Quiet Sun’s surface, i.e., plasma motions that emulate the physics of a numerical model but are made to look as though they were observed by a specific instrument (Tremblay et al., 2018). Beyond assimilation, plasma motions impact the amount of free energy in the solar corona that can then be released in the form of flaring events, e.g. through the transport by flows of magnetic energy from surface layers to the solar atmosphere or the shearing and twisting of field lines (Welsch, 2006). The shuffling of magnetic footpoints or the coherent twisting of magnetic loops by photospheric motions can lead to an accumulation of twist in coronal loops and subsequent reconnections (e.g., Parker, 1988).
We propose that reconstructions of the transverse velocity vector could benefit data assimilation in a MHD model by providing additional coverage of the state variables of the model, either as instantaneous synthetic observations or first estimates of the model state. Inferred plasma motions or the electric field computed from it (e.g., Kazachenko et al., 2014) could also be used as boundary conditions from which to drive MHD simulations of the solar atmosphere (Fisher et al., 2015). Alternatively, these could be combined with other reconstructions to estimate quantities such as the Poynting flux and study their evolution in the context of short term solar activity. In this paper, we aim to provide additional spatial coverage for the velocity vector by generating depthdependent reconstructions in the Quiet Sun (hereafter QS) and Active Regions (hereafter ARs) with the DeepVel neural network.
The data preprocessing and training of DeepVel are described in Section 2. In Section 3, continuum intensity and velocity fields computed by simulations are used to test the flows inferred above and below the surface. We conclude and discuss future work in Section 4.
2 Training sets and procedures
The DeepVel neural network builds an approximation through a series of transformations (i.e., convolutions) that relates the transverse velocity vector field [v_{t}(x, y, z, t_{i})] (i.e., the output) at time [t = t_{i}] to pairs of maps of the continuum intensity I_{c}(x, y, τ ≈ 1, t = {t_{i}, t_{i} + Δt}) at the surface (i.e., intensitygrams; the input), where [Δt] is the timestep between images. Training of DeepVel is performed using computations of the input and output quantities from numerical simulations (i.e., through supervised learning). We refer to Asensio Ramos et al. (2017) for a detailed description of the neural network architecture. Depths [z] are single or multiple optical depths [τ] or geometrical depths [d] or a combination of both. Geometrical depths d are measured from the radius at which an average optical depth of τ ≈ 1 is achieved (d = 0 km) towards the solar interior (i.e., d > 0 km below the surface). Optical depth is the natural scale for radiative transfer and by extension for derivations from observations. Fixed geometrical heights are difficult to derive in the context of radiative transfer, but they are the natural scale in simulation meshes and are thus used when training and testing DeepVel against synthetic data. The interpretation of geometrical heights when using real observational data as input is expected to be more challenging.
Outputs are dependent on the properties of the data presented during training. Different versions of DeepVel will thus need to be trained depending on the physics of the model used for training and the preprocessing of the inputs that will be used for execution. This extends to the horizontal pixel sizes Δx and Δy, and the cadence Δt. The number of output layers must also be modified according to the depths z to probe.
The training and validation sets for DeepVel consist of subimages I_{c}(x = [x_{i}, x_{i} + 50Δx], y = [y_{i}, y_{i} + 50Δy], τ ≈ 1, t = {t_{i}, t_{i} + Δt}) and v_{t}(x = [x_{i}, x_{i} + 50Δx], y = [y_{i}, y_{i} + 50Δy], z, t_{i}), where x_{i} and y_{i} are randomlyselected positions in the full fieldofview images (i.e., subimages of dimensions 50 by 50 pixels^{2} at each depth), z is a set of optical or geometrical depths, and t_{i} is a randomlyselected time within the sequence covered by the simulation data. Weights and biases in the network architecture are adjusted in an attempt to optimize the mean squared errors (i.e., the cost function) between the flows inferred by DeepVel and the training set data. Their values are updated after each training epoch only if the mean squared errors for the validation set are improved. The number of training epochs [n_{e}] refers to the number of times the training and validation sets are presented to the neural network. This process is repeated until the cost function has achieved convergence or no longer varies significantly. Trained convolutional neural networks are capable of generalizing their outputs for any input image dimensions. The test sets that are used to evaluate DeepVel post training feature full fieldofview images. Intensitygrams are normalized by the median of the continuum intensity over the time sequence covered by the dataset. Each component of the velocity vector is shifted by its minimum value in the dataset and is then normalized by the difference between its maximum and minimum values so that v_{x}, v_{y} ∈ [0, 1]. No data augmentation was performed for the training and validation sets, although it could further improve performance and generalization.
A version of DeepVel was trained in Tremblay et al. (2018) to generate synthetic velocity maps at τ ≈ 1 that are consistent with the spatial and temporal scales of level2 data from the Helioseismic Magnetic Imager (HMI: Schou et al., 2012) onboard the Solar Dynamics Observatory (SDO: Hoeksema et al., 2014). The synthetic training set, hereafter referred to as the STAGGER dataset, was derived from slices of the STAGGER^{1} magnetoconvection simulation of the QS (Stein, 2012; Stein & Nordlund, 2012). These slices cover a patch of granulation at select optical depths τ ≈ {1, 0.1, 0.01} within a fieldofview of dimensions L_{x} × L_{y} = 96.768 × 96.768 Mm^{2} that is sampled at a horizontal spatial resolution Δx = Δy = 96 km, downsampled by a factor two from the native resolution of the simulation (i.e., Δx = Δy = 48 km), and cadence Δt = 60 s. The STAGGER simulation data was convolved with the SDO/HMI point spread function (PSF) from Wachter et al. (2012) to emulate the instrument and was then resampled to the SDO/HMI spatial resolution near disk center (Δx = Δy ≈ 368 km pixel^{−1}) using nearestneighbor sampling. We refer to Tremblay et al. (2018) for a more detailed description of the preprocessing steps. A total of 2000 examples were presented to the network over 10 training epochs, with an additional 200 examples for the validation steps. The test set is comprised of a 30min sequence of full fieldofview images at all three optical depths. Although the analysis presented in this work was limited to τ ≈ 1, the neural network was actually trained to infer v_{t}(x, y, τ, t_{i}) at τ ≈ {1, 0.1, 0.01} simultaneously. Intensitygrams I_{c}(x, y, τ ≈{0.1, 0.01}, t_{i}) in the STAGGER dataset have undergone the reversal of the granulation pattern with respect to I_{c}(x, y, τ ≈ 1, t_{i}) (not shown) and are thus located at least a few hundred kilometers above the surface (e.g., Cheung et al., 2007). Optical depth τ ≈ 0.1 is close to the transition between the photosphere and chromosphere (e.g., Cranmer et al., 2007). This version of DeepVel is thus used to evaluate QS flow inversions in the photosphere and chromosphere from surface data.
Additional versions of DeepVel were trained to infer flows above and below the surface using the highresolution and highcadence MURaM simulation of solar granulation (hereafter the MURaMQS simulation; Vögler et al., 2005). A patch of QS is evolved within a volume V ≡ L_{x} × L_{y} × L_{z} = 6.144 × 6.144 × 4 Mm^{3} which spans from 2.3 Mm in the upper convection to 1.7 Mm above the surface. The fieldofview is ≈244 times smaller in area than in the STAGGER dataset, but it is much more detailed. MURaMQS achieves a pixelsize Δx = Δy ≡ 16 km per pixel in the horizontal direction, a value that is close to the Daniel K. Inouye Solar Telescope (DKIST: Rimmele et al., 2020) spatial resolution (Δx ≈ 20 km: Warner et al., 2018). Training was performed using simulation data at its native resolution and timestep Δt = 2 s. Slices were extracted at constant optical depths τ ≈ {1, 0.1, 0.01} and constant geometrical depths d = {0, 144, 560} km. Unlike for the STAGGER dataset, subimages cover an area that is smaller than the average granule. We therefore increase the size of the training and validation sets to 30,000 and 3000 patches each and perform 30 training epochs.
Current applications of DeepVel have been limited to the QS (e.g., Asensio Ramos et al., 2017; Tremblay et al., 2018). We introduce a generalization of the algorithm for ARs that was trained from computations of the MURaM simulation of a sunspot (hereafter MURaMARs; Rempel & Cheung, 2014) at optical depths τ ≈ {1, 0.1, 0.01, 0.001} and geometrical depth d = 1000 km. The sunspot, which is ≈25 Mm in diameter, is located at the center of the field of view which spans 98.304 by 98.304 Mm^{2}. The MURaMARs data shares the same spatial resolution as the raw STAGGER data (i.e., Δx = Δy = 96 km pixel^{−1}) and was resampled to the SDO/HMI spatial resolution following the same procedure (hereafter the ARs dataset). The cadence Δt = 45 s coincides nicely with the timestep between consecutive SDO/HMI level2 products (Hoeksema et al., 2014). The training and validation sets include 3600 and 900 examples each, with each subimage containing a random fraction of QS and ARs. The test set is comprised of a 30timestep sequence of full fieldofview images.
Computations were performed on a NVIDIAGTX 960 GPU and a NVIDIAGTX 1080 Ti GPU using the Keras library with the Tensorflow backend.
3 Results
3.1 Statistical metrics
Each version of the DeepVel neural network is tested against full fieldofview images that were generated by the same numerical simulation from which the training and validation sets were derived. These best case scenarios are used to identify the range of depths at which flow inversions can be performed reliably. Maps of the velocity fields are shown to provide visual aid for spatial features. Only patches (i.e., subfields) are displayed for clarity. Scatterplots are used to study the transverse velocity amplitudes within those patches. Metrics for the test sets are computed over the entire fieldofview. They include the root mean squared errors
(1)the spatially averaged absolute errors
(2)where 〈·〉 is the spatial averaging operator in the horizontal plane, and the spatially averaged relative errors
(3)between the heightdependent transverse velocity fields [v_{D,t}] inferred by DeepVel from simulation intensitygrams [I_{ref}] and the plasma motions [v_{ref,t}] computed by the STAGGER simulation, the MURaMQS simulation or the MURaMARs simulation. The correlation coefficient [C(v_{ref,t}, v_{D,t})] from Schrijver et al. (2006) is used as a measure of similarity between maps of the velocity
The global spatial distribution of the orientation of the velocity vectors are measured through the spatiallyaveraged normalized dot product
(5)where A = ±1 for parallel and antiparallel vectors and A = 0 for perpendicular vectors. To summarize, the ideal reconstruction of transverse velocity field satisfies: C(v_{ref,t}, v_{D,t}) = 1.0 and A(v_{ref,t}, v_{D,t}) = 1.0.
Knowledge of plasma velocity, v = (v_{t}, v_{z}), allows us to estimate the proxy for the vertical component of the magnetic energy Poyntingflux vector, [S_{z}]. Specifically, if we assume to have an ideal photospheric electric field, E, then from observed magnetic field vector B and derived velocity field vector v we can find cE = −v × B. The Poyntingflux vector
(6)measures the flow of electromagnetic energy where we estimate E and B. The vertical component of S measures the amount of energy flowing into and out of the Sun and depends upon the transverse components of both the electric field and the magnetic field:
Here, following e.g. Liu & Schuck (2012), Parnell & De Moortel (2012) and Welsch (2015), we conceptually divide the vertical Poynting flux into an “emergence” term with v_{z}, and a “shear” term with v_{t}. This distinction though, is not very precise, since both terms involve the emergence of magnetized plasma across the surface. If we ignore the “emergence” term, we could describe the shear component of the vertical Poynting flux S_{z} (Yeates et al., 2014; Welsch, 2015):
(8)where we set B_{t} = B_{ref,t}, B_{z} = B_{ref,z} and v_{t} = {v_{ref,t}, v_{D,t}}.
To compare the quality of S_{z} reconstruction we derive the ratio between the integrals of the unsigned shear components of the Poynting fluxes over the full fieldofview M
3.2 Flows above the surface of the Quiet Sun
We first expand upon the analysis presented in Tremblay et al. (2018) which focused on surface flow inversions but featured a version of the DeepVel neural network that was trained with the STAGGER dataset to perform inversions at τ ≈ {1, 0.1, 0.01} from intensitygrams. The vertical component of the plasma motions v_{ref,z}(τ), as computed by the STAGGER simulation and downsampled to the SDO/HMI resolution, is displayed as background in Figure 1 with downflows (v_{ref,z} < 0) outlining the granulation pattern at τ ≈ 1 (Fig. 1a) and the reverse granulation pattern at τ ≈ {0.1, 0.01} (Figs. 1d and 1g). The spatial distribution of plasma motions v_{D,t}(τ ≈ 1) inferred by DeepVel is consistent with v_{ref,t}(τ ≈ 1) from the STAGGER test set (Table 2). Diverging velocity vectors are produced at the center of granules (v_{ref,z}(τ ≈ 1) > 0) and converging vectors in the intergranular lanes (v_{ref,z}(τ ≈ 1) < 0; Fig. 1b), achieving A = 0.79 for the inferred angles. This behavior is akin to but does not satisfy the law of conservation of mass which is encompassed in the training data. Further analysis of Figure 1b revealed that downflow regions where v_{ref,z}(τ ≈ 1) < 0 are the largest sources of errors in the vector orientations (not shown). Downflows in the intergranular lanes result in spatiallyconfined, and thus not as well resolved, transverse flow structures that are more complex than the patterns produced by upflows in the center of granules and are thus harder to reproduce. Figure 1c suggests that there is a slight tendency to underestimate the inferred velocity amplitudes v_{D}(τ ≈ 1) despite the neural network’s ability to match fairly well their spatial distribution (see the coefficient of determination in Figure 1c and the correlation coefficient C in Table 2). This, in addition to the vector orientations, contributes to the underestimation of the total Poynting flux from v_{D,t} inside the field of view (Table 2). Similar conclusions are drawn for inversions higher in the photosphere (τ ≈ 0.1: Figs. 1e and 1f) and in the chromosphere (τ ≈ 0.01: Figs. 1h and 1i). A comparable level of quality is achieved by the convolutional neural network at those optical depths (see metrics in Table 2) because the structures in v_{ref,t}(τ ≈ {0.1, 0.01}, t_{i}) are spatially correlated with the reversed granulation pattern in I_{c}(τ ≈ {0.1, 0.01}, t_{i}) and, by extension, with v_{ref,t}(τ ≈ 1, t_{i}) and the input data I_{c}(τ ≈ 1, t_{i}). Table 1 quantifies the similarity between flow fields from the STAGGER simulation at the surface and at optical depths τ ≈ {0.1, 0.01}. Metrics C and A decrease with τ, but the velocity fields remain correlated enough with the surface data to allow for reliable inversions from the patterns in I_{c}(τ ≈ 1) (Table 2).
Fig. 1 Above the surface velocity inversions, STAGGER dataset. Patches of 50 by 50 pixels^{2} extracted from the 30minaveraged transverse velocity fields v_{D,t}(τ, t_{i}) (left column) computed by the STAGGER simulation and (middle column) inferred by DeepVel at optical depths (top) τ ≈ 1, (middle) τ ≈ 0.1 and (bottom) τ ≈ 0.01. The timeaveraged vertical velocity v_{ref,z}(τ ≈ {1, 0.1, 0.01}, t_{i}) computed by the STAGGER simulation and resampled to the SDO/HMI resolution is displayed as colored background. (Right column) Scatterplots comparing amplitudes v_{D,t} to v_{ref,t}. The black line represents an ideal solution (i.e., a coefficient of determination R^{2} = 1). 
Similarity between the reference velocity field at the surface v_{ref}(τ ≈ 1) and at various optical and geometrical depth above and below the surface, τ and d, respectively. Metrics C and A correspond to the correlation coefficient (Eq. (4)) and the spatiallyaveraged normalized dot product A (Eq. (5)), respectively, averaged over time.
Comparison between flow fields v_{D,t} and v_{ref,t} for the QS datasets (i.e., STAGGER and MURaMQS). RMSE is the root mean squared error (Eq. (1)), E_{abs} is the mean absolute error (Eq. (2)), E_{rel} is the mean relative error (Eq. (3)), C is the correlation coefficient (Eq. (4)), A is the spatiallyaveraged normalized dot product (Eq. (5)) and S is the ratio between the integrals of the unsigned shear components of the DeepVel and reference Poynting fluxes (Eq. (9)).
We then increase the spatial and temporal resolution of the training data and perform inversions using the MURaMQS dataset which features elaborate flow patterns. The subfield presented in Figure 2 is a closeup of the intersection between four granules at τ ≈ {1, 0.1, 0.01}, showcasing the complexity of the intergranular lanes and granule edges. This subfield is approximately 80 times smaller in area than the one featured in Figure 1, and each pixel is covering an area that is more than 500 times smaller. As the optical depth decreases and distance from the solar surface increases, the reference transverse velocity undergoes morphological changes which are more significant than that seen in Figure 1 and Table 1 at the same optical depths, most notably near granule edges. The scatterplot comparing the reference amplitude v_{ref,t}(τ ≈ 0.1) (resp. v_{ref,t}(τ ≈ 0.01)) to v_{ref,t}(τ ≈ 1) has a coefficient of determination of 0.82 (resp. 0.72) (not shown). The vertical component, displayed as background in Figure 2, undergoes similar changes. Yet, despite differences in patterns, spatial resolution, cadence and physics, the MURaMQStrained version of DeepVel achieves similar if not slightly improved metrics than the STAGGERtrained version in the photosphere and in the chromosphere (Table 2). The MURaMQS training set featured fifteen times the number of examples in the STAGGER training set. Three times as many training epochs were performed in anticipation of challenges related to the higher spatial resolution and cadence. Additionally, reference flows at τ ≈ 0.1, 0.01 appear to be smoother than at τ ≈ 1 (not shown). This results in poor metrics in Table 1 but also in flow patterns that, from the perspective of a neural network, are less complex than in v_{ref,t}(τ ≈ 1) and still look similar to the surface granulation pattern. As for the STAGGER dataset, performances decrease in downflow regions.
Fig. 2 Above the surface velocity inversions, MURaMQS dataset. Patches of 125 by 125 pixels^{2} extracted from the transverse velocity fields v_{D,t}(τ, t_{i}) (left column) computed by the MURaMQS simulation (Vögler et al., 2005) and (middle column) inferred by DeepVel at optical depths (top) τ ≈ 1, (middle) τ ≈ 0.1 and (bottom) τ ≈ 0.01. The vertical velocity v_{ref,z}(τ ≈ {1, 0.1, 0.01}, t_{i}) computed by the MURaMQS simulation is displayed as colored background with a symmetrical threshold. (Right column) Scatterplots comparing amplitudes v_{D,t} to v_{ref,t}. The black line represents the expected solution (i.e., a coefficient of determination R^{2} = 1). 
3.3 Subsurface flows in the Quiet Sun
Experiments with multipledepth inversions of flows reveal that spatial structures which are depthspecific (e.g., vortexlike structures in the upper convection zone; see Figs. 3d and 3g) can appear in the velocity inversions at other depths (e.g., the surface; not shown). These artifacts are the results of the minimization process and the network architecture which, for all intents and purposes, is simply a mapping function. The neural network’s cost function is the sum of the mean squared errors at all three depths. For a given architecture, its weights and biases are not adjusted to provide the best solution at each specific depth but rather the best overall solution at an ensemble of depths. This can explain why DeepVel favored v_{D,t}(τ ≈ {0.1, 0.01}, t) over v_{D,t}(τ ≈ 1, t) in Figure 1 and Table 2 as it best optimized the cost function. As discussed in Section 2, the number of depths inferred simultaneously by DeepVel can be changed through its architecture. For example, the original threedepth architecture can be replaced by three singledepthspecific architectures, potentially improving the results at each depth with the caveat being an increase in the number of neural networks to train and run. Fortunately, the training step only needs to be performed once and the execution of DeepVel takes less than a second. From this point forward, the flow inversions presented were computed using singledepth versions of DeepVel instead of the original threedepth version. Note that this approach could further improve the results from Figures 1 to 2.
Fig. 3 Below the surface velocity inversions, MURaMQS dataset. Patches of 125 by 125 pixels^{2} extracted from instantaneous transverse velocity field maps which were computed at geometrical depths (top) d = 0 km, (middle) d = 144 km, (bottom) d = 560 km by (left column) the MURaMQS simulation (Vögler et al., 2005) and (middle column) singlelayer output versions of DeepVel. The divergence of the transverse plasma motions (i.e., ∇_{t}·v_{t}) is displayed as colored background to emphasize the complexity of the structures that arise at increasing depths. (Right column) Scatterplots comparing amplitudes v_{D,t} to v_{ref,t} at each geometrical depth. The black line represents the expected solution (i.e., a coefficient of determination R^{2} = 1). 
Previous velocity field reconstructions were performed in a medium where plasma motions are ultimately dictated by the magnetic field. Transitioning to below the surface brings forth turbulent convectiondriven plasma motions. In addition, inversions presented in this Section are performed from data at a constant optical depth τ ≈ 1 to probe flows at a constant geometrical depth d (i.e., radius), with d = 0 defined as the radius at which 〈τ〉 ≈ 1 is achieved in the simulation. Figures 2a and 3a and Table 1 underline that noticeable differences arise between the velocity field associated with the DeepVel input data (v_{ref,t}(τ ≈ 1)) and v_{ref,t}(d = 0 km). The divergence of transverse velocity vectors is displayed as background in Figure 3, highlighting the complexity of transverse flow patterns. In particular, the highresolution MURaMQS simulation features vortexlike patterns between convective cells (Figs. 3d and 3g) which are more complex than that seen in the intergranular lanes above the solar surface (Fig. 2), i.e. where the neural network inversions have previously been the least accurate. Such patterns do not appear in surface intensitygrams. Scatterplots comparing local velocity amplitudes from Figure 2a (i.e., the surface flows) to those of Figures 3a, 3d and 3g reveal that similarity with surface flows varies significantly with d as the coefficient of determination decreases from 0.81 to 0.72 and 0.59 for geometrical depths of 0 km, 144 km and 560 km respectively (scatterplots not shown). This is supported by Table 1. In this context, subsurface flows prove to be challenging to capture at time t_{i} simply from maps of the continuum intensity I_{c}(τ ≈ 1, t = {t_{i}, t_{i+1}}), as demonstrated by the velocity vectors in Figures 3b, 3e and 3h and metrics in Table 2. Scatterplots 3c, 3f and 3i follow a positive trend but exhibit deviations from a linear relationship for larger velocity amplitudes, the latter being associated with the edges of convective cells. Metrics for v_{D,t}(d = 0 km) (Figs. 3b and 3c) are on par with those achieved for v_{D,t}(τ ≈ 1) (see Table 2). As we increase to d = 144 km inside the convection zone, metrics C and A decrease to 0.83 and 0.80 respectively (Figs. 3e and 3f). The neural network underperforms in comparison to previous inversions at distances from the surface that are greater than 144 km above the surface (Fig. 2), but remain of a similar quality to what was achieved at τ ≈ {1, 0.1, 0.01} with the STAGGER dataset in Figure 1 (see Table 2). Most notably, the complexity of inferred flow patterns between convective cells in Figure 3e is much less than that in the MURaMQS simulation (Fig. 3d). At geometrical depth d = 560 km, reference and inferred flows become weakly correlated (Table 2). The velocity field is devoid of structure between adjacent convective cells, only faintly outlining their edges (see divergence in Fig. 2h and Table 2). Nonetheless, coefficients C = 0.415 and A = 0.428 (Table 2) and R^{2} = 0.59 (Fig. 3i) improve upon the similarity between v_{ref,t}(d = 560 km) and v_{ref,t}(τ ≈ 1) (Table 1). Vögler et al. (2005) reported that the MURaMQS magnetic field lines transition from being spreadout at d = 0 to disrupted at d ≈ 300 km per the turbulent convective motions inside the Sun. We expect this decorrelation with the surface to be a determining factor in DeepVel’s ability to map intensitygrams to subsurface flows. The neural network is capable of distinguishing between different convective cells, but the level of detail captured quickly decreases as d increases.
Inputs in DeepVel consist of two consecutive intensitygrams at τ ≈ 1. As distance from the surface increases, so does the time delay for the plasma at the surface to travel to geometrical depth d and vice versa. The native timestep of the MURaMQS dataset (Δt = 2 s), also used by our implementation of DeepVel, is too short to account for such variations. Future experiments will be dedicated to varying the time gap between intensitygrams (i.e., the neural network inputs) as a function of the depth probed by DeepVel.
3.3.1 Flux emergences and sunspots
The MURaMARs dataset and the STAGGER dataset share the same spatial resolution. Hence MURaMARs intensitygrams can readily be used as input in the version of DeepVel used in Section 3.2, if the output velocities are multiplied by a factor of to account for the difference in cadence between the training and test sets. When doing so, the spatial distribution of the flows inferred in the QS are consistent with the reference flow (lower left corner of Figs. 4a and 4b, and Table 3 where B_{ref,z} < 100 G in the ARs dataset), with divergent vectors in the center of granules and convergent vectors in the intergranular lanes. Note that the length of the velocity vectors in Figure 4b was renormalized to match the length of the arrows in Figure 4a and focus on the spatial features. More specifically, for the same arrow length, the velocity amplitudes are twice as large in Figure 4a than in Figure 4b. The STAGGER simulation computes transverse velocity amplitudes of up to 3 km s^{−1}, whereas the MURaMARs simulation achieves velocities more than twice as large. For this reason, the STAGGERtrained version of DeepVel underestimates velocity amplitudes in the QS (see Fig. 4d for the scatterplot of the FOV covered by Fig. 4b). In the penumbra of the sunspot (i.e., the line joining the upper left and lower right corners of Figs. 4a and 4b), the velocities are significantly underestimated (Fig. 4d) and do not appear to be diverging away from the center of the sunspot. Inside the sunspot (i.e., the upper right corner of Figs. 4a and 4b), variations in the continuum intensity are mistakenly interpreted by the neural network as the edges of granular cells (Fig. 4b), and the vector orientations are poorly reconstructed (Table 3 where B_{ref,z} ≥ 100 G). This is justified by the neural network having been trained exclusively on examples of solar granulation. The generalization from the STAGGERtrained version of DeepVel to the MURaMARs simulation is thus limited to the QS spatial features.
Fig. 4 At the surface velocity inversions, MURaMARs dataset. Patches of 50 × 50 pixels^{2} extracted from 30timestepaveraged transverse velocity field maps which were computed at optical depth τ ≈ 1 by (a) the MURaMARs simulation (Rempel & Cheung, 2014), (b) the DeepVel neural network that generated Figure 1a, and (c) a version of the DeepVel neural network which was trained using the MURaMARs simulation resampled at the SDO/HMI spatial resolution (Δx ≈ 368 km). The vertical component of the velocity computed by the MURaMARs simulation is displayed as colored background. (Right column) Scatterplots comparing amplitudes v_{D,t} to v_{ref,t}. The black line represents the expected solution (i.e., a coefficient of determination R^{2} = 1). 
Comparison between flow fields v_{D,t} and v_{ref,t} for the ARs dataset. RMSE is the root mean squared error (Eq. (1)), E_{abs} is the mean absolute error (Eq. (2)), E_{rel} is the mean relative error (Eq. (3)), C is the correlation coefficient (Eq. (4)), A is the spatiallyaveraged normalized dot product (Eq. (5)) and S is the ratio between the integrals of the unsigned shear components of the DeepVel and reference Poynting fluxes (Eq. (9)).
Figure 4c showcases an improved timeaveraged velocity field that was obtained by training DeepVel with the ARs dataset. The spatial distribution of plasma motions where B_{ref,z} ≥ 100 G is now much more consistent with the MURaMARs test set (Table 3), most notably in the penumbra. However, velocity amplitudes remain overestimated inside the sunspot (Fig. 4d and errors in Table 3). Inversions in the QS are of similar quality to the results of Section 3.2. To sum up, this version of the neural network successfully generalizes the flow patterns at τ ≈ 1 for ARs and granulation. Estimates of the total Poynting flux inside the field of view are also more accurate, with the STAGGERtrained version significantly underestimating its value (Table 3).
Displayed in Figure 5 are examples of velocity fields reconstructed by the DeepVel neural network at optical depths τ ≈ 0.1, 0.01, 0.001 above the surface. Metric values C = 0.867, A = 0.785 and R^{2} = 0.79 for v_{D,t}(τ ≈ 0.1) (Table 3, Figs. 5b and 5c) are similar to those achieved in Section 3.2 at the same optical depth for the Quiet Sun (Table 2), thus extending the generalization potential of DeepVel for ARs from the surface to the base of the chromosphere. As for inversions at τ ≈ 1, loss of performance is traced back to regions where B_{ref,z} ≥ 100 G, with E_{rel} exceeding 200% and metrics C and A lowering to 0.639 and 0.410 respectively (Table 3). These errors contribute to the overestimation of the total Poynting flux within the fieldofview (Table 3). This further demonstrates the difficulty of capturing plasma motions inside magnetic structures such as sunspots simply from maps of the continuum intensity and advocates for the use of additional inputs in DeepVel (e.g., magnetograms and dopplergrams: Tremblay & Attie, 2020). Higher in the solar atmosphere of the MURaMARs simulation, flow patterns undergo significant morphological changes in weak field regions (B_{ref,z} < 100 G) with respect to surface flows v_{ref}(τ ≈ 1), as quantified in Table 1. This is apparent in the lower portion of the subfield presented in Figures 5a, 5d and 5g, both in terms of the vertical velocity v_{ref,z} presented as background and the arrow length and orientation of the transverse velocity vectors v_{ref,t}. This loss of similarity translates to a loss of performance for v_{D,t}(τ ≈ {0.01, 0.001}) in the same area (Table 3 and Figs. 5f and 5i). Training attempts at optical depths τ ≈ {0.01, 0.001} have only slightly improved the value of C in comparison to the aforementioned similarity with v_{ref}(τ ≈ 1), whereas values for the normalized dot product and coefficient of determination are about the same (see Tables 1 and 3). Scatterplots 5f and 5i show that larger velocities are significantly underestimated (see also arrow lengths in Figs. 5f and 5i). Despite the total Poynting flux being closer to its expected value than at any other optical depth, the remaining metrics suggest that this most likely due to a fortuitous cancellation of errors.
Fig. 5 Above the surface velocity inversions, MURaMARs dataset. (Left column) Patches of 50 by 50 pixels^{2} extracted from the 30minaveraged transverse velocity fields v_{D,t}(τ, t_{i}) inferred by DeepVel at optical depths (top) τ ≈ 0.1, (middle) τ ≈ 0.01 and (bottom) τ ≈ 0.001. The timeaveraged vertical velocity v_{ref,z}(τ ≈ {0.1, 0.01, 0.001}, t_{i}) computed by the MURaMARs simulation and resampled to the SDO/HMI resolution is displayed as colored background. (Right column) Scatterplots comparing amplitudes v_{D,t} to v_{ref,t}. The black line represents the expected solution (i.e., a coefficient of determination R^{2} = 1). 
Inversions at geometrical depth d = 1000 km in the upper convection zone (Fig. 6b), i.e. approximately 500 km deeper below the surface than previously probed with the MURaMQS dataset (Fig. 3h), fail to accurately capture the angle between horizontal components with A = 0.164 and, by extension, C = 0.258 (Eq. (4) and Table 3). Reconstructions are underperforming with respect to the similarity between v_{ref,t}(τ ≈ 1) and v_{ref,t}(d = 1000 km) (C = 0.492 and A = 0.438, Table 1 and Fig. 6a). Scatterplot 6c indicates a strong positive correlation between inferred and reference flow amplitudes with R^{2} = 0.83, corroborating that the main source of errors at d = 1000 km are the vector orientations. As noticed with previous inversions, larger velocities are underestimated and there is a tendency to overestimate lower amplitudes. This is demonstrated by the arrow lengths in Figures 6a and 6b, most notably in the upper right corner and near the center of the subfield (i.e., where there is a concentration of magnetic flux). Training DeepVel on flow amplitudes and angles rather than the transverse components themselves may improve the results. This feature engineering approach will be tested in the future.
Fig. 6 Below the surface velocity inversions, MURaMARs dataset. (a) Patch of 50 by 50 pixels^{2} extracted from the 30minaveraged transverse velocity field v_{D,t}(d, t_{i}) inferred by DeepVel at geometrical depth d = 1000 km. The timeaveraged vertical velocity v_{ref,z}(d = 1000 km, t_{i}) computed by the MURaMARs simulation and resampled to the SDO/HMI resolution is displayed as colored background. (Right column) Scatterplot comparing amplitudes v_{D,t} to v_{ref,t}. The black line represents the expected solution (i.e., a coefficient of determination R^{2} = 1). 
4 Conclusions & future work
We explored the use of deep learning tools and observed continuum intensity maps at the photosphere to derive horizontal plasma velocity vector below and above the photosphere. Specifically, using MURaM and STAGGER synthetic data, we trained the DeepVel neural network to infer horizontal plasma motions at various optical depths τ ≈ {0.001, 0.01, 0.1, 1} above the surface and geometrical depths d = {0, 144, 560, 1000} km below the surface (i.e., in the upper convection zone). We compare the properties of the DeepVel reconstructions and the actual synthetic velocity fields v_{D,t} and v_{ref,t} using statistical metrics and measures of similarity such as the Schrijver et al. (2006) correlation coefficient C (expected value of 1; Eq. (4)) and normalized dot product A (expected value of 1; Eq. (5)). Our findings are as follows:

QS flow inversions at the surface and in the solar atmosphere: Using the STAGGER and MURaMQS simulations, we find that the inversions at optical depths τ ≈ {0.1, 0.01} in the QS (i.e., near the base of the chromosphere and near the transition region, respectively) are comparable in quality to the inversions at τ ≈ 1 presented in Tremblay et al. (2018), with C(τ ≈ 0.1) = 0.867, A(τ ≈ 0.1) = 0.810, C(τ ≈ 0.01) = 0.852 and A(τ ≈ 0.01) = 0.791 achieved for the SDO/HMI resolution STAGGER dataset and C(τ ≈ 0.1) = 0.904, A(τ ≈ 0.1) = 0.850, C(τ ≈ 0.01) = 0.819 and A(τ ≈ 0.01) = 0.727 achieved for the highresolution MURaMQS dataset. The inferred flow fields are thus deemed suitable approximations for use as synthetic observations of the transverse velocity vector that reflect the physics of the carefullypreprocessed model data used to train DeepVel, if the latter is a valid representation of real observations. In particular, the STAGGERtrained version of the neural network could be implemented in a data reduction pipeline due to its fast computations, and it can readily accept SDO/HMI intensitygrams as input if the output velocities at τ ≈ {1, 0.1, 0.01} are multiplied by a factor of to account for the difference in cadence between SDO level2 data (i.e., 45 s) and the STAGGER data (i.e., 60 s). We refer to Tremblay et al. (2018) for an example of such application. Similarly, the MURaMQS version of DeepVel could be retrained to match the cadence of future DKIST data products and use the latter as inputs. Velocity amplitudes, which are typically underestimated by the method, can be adjusted by multiplying the vector field by a correction factor (i.e., a scalar) while still preserving its spatial properties. Additionally, training versions of the neural network to reconstruct singledepth velocity fields from surface intensitygrams will likely further minimize the remaining errors. This would increase the amount of time to train or execute trained neural networks by a factor equal to the number of depths to reconstruct. If implemented in a data reduction pipeline, the total execution time would still remain reasonably short. Future experiments will be dedicated to identifying an upper limit on the geometrical or optical depth at which the DeepVel neural network can produce reliable results from surface observations of the QS.

QS flow inversions below the surface: Inferred subsurface flows fail to capture the complexity of the turbulent convective motions at increasing depths from intensitygrams of the surface. Inversions at d = 144 km below the surface achieve metrics comparable to what was obtained above the surface (i.e., C = 0.827 and A = 0.796) and could potentially be used as synthetic observations, though details between convective cells are missed by DeepVel. At d = 560 km, the level of detail diminishes significantly, with metric values C = 0.415 and A = 0.428 suggesting that these reconstructions may be used as first estimates of the velocity vector at best. This behavior is consistent with the decorrelation observed in the MURaMQS simulation between magnetic field lines at the surface and at geometrical depths d ≥ 300 km as convection becomes too disruptive (Vögler et al., 2005). By extension, the spatial correlation between surface intensitygrams and subsurface plasma motions decreases significantly with depth inside the convection zone, much more so than at increasing height above the surface (see Table 1). Additionally, the timestep used between two consecutive intensitygrams does not account for how variations at the surface affect the probed depth and vice versa. Increasing the timestep Δt and/or the number of intensitygrams provided as input may improve the results. Furthermore, processing sequences of images with a recurrent neural network architecture would build a memory of how past information impacts the output (Chollet, 2017).

ARs flow inversions at the surface and in the solar atmosphere: Using the MURaMARs simulation, we find that inversions inside and around ARs at τ ≈ {1, 0.1} (i.e., in the photosphere and near the base of the chromosphere) are comparable to our QS inversions at the same optical depths (i.e., C(τ ≈ 1) = 0.905, A(τ ≈ 1) = 0.830, C(τ ≈ 0.1)=0.867 and A(τ ≈ 0.1) = 0.785), although the inside of the sunspot is an area in need of further improvement. As is the case for the STAGGERtrained version of DeepVel, the MURaMARstrained version can readily use SDO/HMI intensitygrams as input to generate synthetic transverse velocity estimates at τ ≈ {1, 0.1}. Note that due to differences in the training data, outputs generated by the two neural networks will differ in the QS, most notably in terms of velocity amplitudes. As we approach higher regions in the atmosphere, the velocity fields in the solar atmosphere of the MURaMARs simulation become less correlated with the surface flows and inversions from surface intensitygrams become less reliable, with metrics decreasing to C(τ ≈ 0.01) = 0.660, A(τ ≈ 0.01) = 0.585, C(τ ≈ 0.001) = 0.331 and A(τ ≈ 0.001) = 0.208.

ARs flow inversions below the surface: Inferred subsurface velocity fields at d = 1000 km follow the trend set by QS flow inversions, with performance decreasing significantly as a function of depth. Metrics C = 0.258 and A = 0.164 underperform in comparison to the simple similarity between flow patterns at τ ≈ 1 and at d = 1000 km, with velocity vector angles being the main source of errors. Simulation data at additional geometrical depths between 0 and 1000 km below the surface will be required to estimate up to which depth the neural network can reliably probe subsurface magnetic structures and flows from surface observations of ARs.
We conclude that the DeepVel neural network is capable of extrapolating reliable flow field approximations in the photosphere, the base of the chromosphere and upper convection zone simply from observed intensitygrams. The level of detail and, by extension, errors in reconstructions vary as a function of distance from the surface as flow patterns become decorrelated from surface intensitygrams. Based on our results and the simulation data, this trend appears to be more prominent in ARs than in the QS. Nonetheless, we believe that DeepVel synthetic data is ideal for (1) inferring depthdependent and datadriven estimates of transversevelocitydependent physical quantities, (2) generating boundary conditions for datadriven simulations of the solar atmosphere, and (3) generating internal conditions for data assimilation in MHD models by providing additional spatial coverage of the transverse velocity vector. We provided examples of depthdependent Poynting flux estimates that were derived from DeepVel transverse velocity maps and simulation magnetograms at the same depths. Such estimates are to be validated in the chromosphere with the advent of DKIST magnetograms and Dopplergrams. Data assimilation processes minimize errors between model predictions and observations within a given time window. We are now able to provide synthetic observations of the transverse velocity vector that may be included in the minimization process to guide more effectively the model towards a solution representative of a given epoch of the Sun. As suggested in Tremblay et al. (2018), the modeldependency of DeepVel flow fields may work as an advantage if the training set is emulating the same model in which the velocity vector is to be assimilated. This would also simplify the observation operator which projects model velocity vector into the observations space. We currently limited inversions to select depths to establish a range at which DeepVel can operate, but one could train versions of the neural network to generate a cube of transverse velocities at all depths in between. Errors as a function of depth are to be quantified and accounted for in the covariance matrix of observation errors (Bouttier & Courtier, 2002). An improved nowcasting step in data assimilation allows to more accurately invert other model state variables such as the magnetic field components which can be studied over the duration of assimilation window, providing insight on the QS and ARs and their influence on solar activity based on the physics encompassed by the model. This also translates in improved forecasts beyond the assimilation window of the evolution of the QS, ARs and, if the simulation allows for it, the occurrence of space weather events (e.g., flares in MURaM: Cheung et al., 2018). Additionally, an estimate of the velocity vector is required at the beginning of the assimilation window in order to initiate the data assimilation process. Hence, despite errors, DeepVel reconstructions may be used to that effect to assist in the convergence of the data assimilation method. Taking the longer view, rapid and automated reconstruction of subsurface flows from surface observations also has potential interest beyond data assimilation towards shortterm activity forecasting. For example, modelling of global surface magnetic flux transport has revealed the importance of accurately capturing the physical characteristics of emerging bipolar active regions in order to properly model their contribution to the global solar dipole (Jiang et al., 2015; Hathaway & Upton, 2016, also Petrovay et al., this volume). This is essential to dynamo modelbased the prediction of overall activity levels on decadal timescales (see, e.g., Labonville et al., 2019), and possibly to understand the onset of extended “Grand Minima” epochs of reduced activity (Nagy et al., 2017). Detecting active regions through internal flow perturbations they induce prior to their emergence at the photosphere may lead to improved determination of their ultimate dynamo efficiency. Current tests have revealed that when the DeepVel neural network is trained with the computations of a simulation of an sunspot, it is capable of distinguishing between the different depthdependent flows in the QS and ARs with varying level of detail. We speculate that if the future focus is shifted from the missing detailed spatial features to the successful detection of spatial structures such as the edges and center of granules and sunspots, this approach may be adapted for the prediction of the emergence of magnetic flux or of a sunspot prior to its occurrence at the surface. Instead of velocity maps, the outputs could be a measure of a spatiallyaveraged physical quantity or a simple binary signal for detection and nondetection. This is left as future work.
As we probed the upper convection zone, photosphere and chromosphere, the DeepVel neural network was subjected to a wide range of flow patterns and spatial scales (i.e., turbulence, intergranular lanes, granulation, mesogranulation, supergranulation). We were motivated by those structures to modify the DeepVel neural network architecture to that of a Unet (Ronneberger et al., 2015). This architecture allows to probe the dominant features not only at small scales (i.e., the pixel size) but also scales up the size of the subimages presented during training (DeepVelU: Tremblay & Attie, 2020; Tremblay et al., 2019). Our recent efforts have also focused on modifying the inputs of the neural network in order to accept a combination of consecutive intensitygrams (i.e., the default inputs), magnetograms and Dopplergrams (Tremblay & Attie, 2020). This is motivated by the magnetic induction equation which relates transverse plasma motions to the magnetic field and lineofsight plasma motions. Intensitybased methods are effective in inferring plasma motions in the Quiet Sun, however reconstructions in the Active Sun typically rely on magnetograms (e.g., Longcope, 2004; Schuck, 2005, 2006, 2008) or magnetograms and Dopplergrams (Kazachenko et al., 2014; Lumme et al., 2019; Fisher et al., 2020) to solve the magnetic induction equation. We expect that providing additional information to DeepVel regarding the physics will further improve the metrics in active regions and thus in Figure 4c and Table 3. DeepVel’s architecture and cost function could also be modified to ensure that inferred solutions are consistent with the magnetic induction equation using the framework of physicsinformed neural networks (PINNs: Raissi et al., 2019) which have recently been used to solve for the velocity vector in NavierStokes equations in the context of fluid mechanics (e.g., NSFnets: Jin et al., 2020). All of the above may help us improve flow depthdependent inversions. This will be explored in future work.
Finally, a similar method could be invoked to infer other physical quantities of interest that cannot yet be measured directly at the photosphere or anywhere else in the solar atmosphere, e.g. the electric field and the Poynting flux.
DeepVel codes, weights and biases used in this paper can be found in the following Github repository: https://github.com/tremblaybenoit/DeepVel_DeepVelU.
Acknowledgments
The authors would like to thank the anonymous referees for their constructive comments. The authors would also like to thank Dr. Matthias Rempel (NCAR) for providing the MURaM simulation data. The editor thanks Michael Kirk and an anonymous reviewer for their assistance in evaluating this paper.
Slices of the STAGGER simulation at constant optical depth are available for download: http://steinr.pa.msu.edu/~bob/96averages/.
References
 Abbett WP. 2007. The magnetic connection between the convection zone and corona in the Quiet Sun. Astrophys J 665: 1469–1488. https://doi.org/10.1086/519788. [NASA ADS] [CrossRef] [Google Scholar]
 Abbett WP, Fisher GH. 2010. Improving largescale convectionzonetocorona models. Mem Soc Astron Italiana 81: 721. [Google Scholar]
 Asensio Ramos A, Requerey IS, Vitas N. 2017. DeepVel: Deep learning for the estimation of horizontal velocities at the solar surface. A&A 604: A11. https://doi.org/10.1051/00046361/201730783. [NASA ADS] [CrossRef] [EDP Sciences] [Google Scholar]
 Attie R, Innes DE. 2015. Magnetic balltracking: Tracking the photospheric magnetic flux. A&A 574: A106. https://doi.org/10.1051/00046361/201424552. [NASA ADS] [CrossRef] [EDP Sciences] [Google Scholar]
 Bouttier F, Courtier P. 2002. Data assimilation concepts and methods. https://www.ecmwf.int/node/16928. [Google Scholar]
 Cheung MCM, Rempel M, Chintzoglou G, Chen F, Testa P, et al. 2018. A comprehensive threedimensional radiative magnetohydrodynamic simulation of a solar flare. Nat Astron 3: 160–166. https://doi.org/10.1038/s4155001806293. [NASA ADS] [CrossRef] [Google Scholar]
 Cheung MCM, Schüssler M, MorenoInsertis F. 2007. The origin of the reversed granulation in the solar photosphere. A&A 461: 1163–1171. https://doi.org/10.1051/00046361:20066390. [NASA ADS] [CrossRef] [EDP Sciences] [Google Scholar]
 Chollet F. 2017. Deep learning with Python, chap. 1, 2, 5, 7, 1–50, 19–143, 260–263, 1st edn, Manning Publications Co., Greenwich, CT, USA. ISBN 1617294438, 9781617294433. [Google Scholar]
 Cranmer SR, van Ballegooijen AA, Edgar RJ. 2007. Selfconsistent coronal heating and solar wind acceleration from anisotropic magnetohydrodynamic turbulence. Astrophys J Suppl Ser 171(2): 520–551. https://doi.org/10.1086/518001. [NASA ADS] [CrossRef] [Google Scholar]
 Fisher GH, Welsch BT. 2008. FLCT: A fast, Efficient method for performing local correlation tracking. In Subsurface and atmospheric influences on solar activity, Howe R, Komm RW, Balasubramaniam KS, Petrie GJD (Eds.), Vol. 383 of Astronomical Society of the Pacific Conference Series, Astronomical Society of the Pacific, San Francisco, CA, USA, 373 p. [Google Scholar]
 Fisher GH, Abbett WP, Bercik DJ, Kazachenko MD, Lynch BJ, et al. 2015. The Coronal Global Evolutionary Model: Using HMI vector magnetogram and doppler data to model the buildup of free magnetic energy in the solar corona. Space Weather 13: 369. https://doi.org/10.1002/2015SW001191. [NASA ADS] [CrossRef] [Google Scholar]
 Fisher GH, Kazachenko MD, Welsch BT, Sun X, Lumme E, Bercik DJ, DeRosa ML, Cheung MCM. 2020. The PDFI_SS electric field inversion software. Astrophys J Suppl Ser 248(1): 2. https://doi.org/10.3847/15384365/ab8303. [CrossRef] [Google Scholar]
 Hathaway DH, Upton LA. 2016. Predicting the amplitude and hemispheric asymmetry of solar cycle 25 with surface flux transport. J Geophys Res(Space Phys) 121(11): 10744–10753. https://doi.org/10.1002/2016JA023190. [CrossRef] [Google Scholar]
 Hayashi K, Feng X, Xiong M, Jiang C. 2018. An MHD simulation of solar active region 11158 driven with a timedependent electric field determined from HMI vector magnetic field measurement data. Astrophys J 855: 11. https://doi.org/10.3847/15384357/aaacd8. [NASA ADS] [CrossRef] [Google Scholar]
 Hoeksema JT, Liu Y, Hayashi K, Sun X, Schou J, et al. 2014. The helioseismic and magnetic imager (HMI) vector magnetic field pipeline: Overview and performance. Sol Phys 289: 3483–3530. https://doi.org/10.1007/s1120701405168. [NASA ADS] [CrossRef] [Google Scholar]
 Jiang C, Wu ST, Feng X, Hu Q. 2016. Datadriven magnetohydrodynamic modelling of a fluxemerging active region leading to solar eruption. Nat Commun 7: 11522. https://doi.org/10.1038/ncomms11522. [NASA ADS] [CrossRef] [Google Scholar]
 Jiang J, Cameron RH, Schüssler M. 2015. The cause of the weak solar cycle 24. Astrophys J Lett 808(1): L28. https://doi.org/10.1088/20418205/808/1/L28. [Google Scholar]
 Jin X, Cai S, Li H, Karniadakis GE. 2020. NSFnets (NavierStokes flow nets): Physicsinformed neural networks for the incompressible NavierStokes equations. J Comput Phys, 109951. ISSN 00219991. https://doi.org/10.1016/j.jcp.2020.109951. http://www.sciencedirect.com/science/article/pii/S0021999120307257. [Google Scholar]
 Kazachenko MD, Fisher GH, Welsch BT. 2014. A comprehensive method of estimating electric fields from vector magnetic field and doppler measurements. Astrophys J 795: 17. https://doi.org/10.1088/0004637X/795/1/17. [NASA ADS] [CrossRef] [Google Scholar]
 Labonville F, Charbonneau P, Lemerle A. 2019. A dynamobased forecast of solar cycle 25. Sol Phys 294(6): 82. https://doi.org/10.1007/s1120701914800. [CrossRef] [Google Scholar]
 Liu Y, Schuck PW. 2012. Magnetic energy and helicity in two emerging active regions in the Sun. Astrophys J 761(2): 105. https://doi.org/10.1088/0004637X/761/2/105. [CrossRef] [Google Scholar]
 Longcope DW. 2004. Inferring a photospheric velocity field from a sequence of vector magnetograms: The minimum energy fit. Astrophys J 612: 1181–1192. https://doi.org/10.1086/422579. [NASA ADS] [CrossRef] [Google Scholar]
 Lumme E, Kazachenko MD, Fisher GH, Welsch BT, Pomoell J, Kilpua EKJ. 2019. Probing the effect of cadence on the estimates of photospheric energy and helicity injections in eruptive active region NOAA AR 11158. Sol Phys 294(6): 84. https://doi.org/10.1007/s112070191475x. [CrossRef] [Google Scholar]
 Nagy M, Lemerle A, Labonville F, Petrovay K, Charbonneau P. 2017. The effect of “Rogue” active regions on the solar cycle. Sol Phys 292(11): 167. https://doi.org/10.1007/s1120701711940. [NASA ADS] [CrossRef] [Google Scholar]
 November LJ, Simon GW. 1988. Precise propermotion measurement of solar granulation. Astrophys J 333: 427–442. https://doi.org/10.1086/166758. [CrossRef] [Google Scholar]
 Parker EN. 1988. Nanoflares and the solar Xray corona. Astrophys J 330: 474. https://doi.org/10.1086/166485. [NASA ADS] [CrossRef] [Google Scholar]
 Parnell CE, De Moortel I. 2012. A contemporary view of coronal heating. Roy Soc Lond Philos Trans Ser A 370: 3217–3240. https://doi.org/10.1098/rsta.2012.0113. [Google Scholar]
 Potts HE, Barrett RK, Diver DA. 2004. Balltracking: An highly efficient method for tracking flow fields. A&A 424: 253–262. https://doi.org/10.1051/00046361:20035891. [NASA ADS] [CrossRef] [EDP Sciences] [Google Scholar]
 Raissi M, Perdikaris P, Karniadakis GE. 2019. Physicsinformed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J Comput Phys 378: 686–707. https://doi.org/10.1016/j.jcp.2018.10.045. [CrossRef] [Google Scholar]
 Rempel M, Cheung MCM. 2014. Numerical simulations of active region scale flux emergence: From spot formation to decay. Astrophys J 785: 90. https://doi.org/10.1088/0004637X/785/2/90. [NASA ADS] [CrossRef] [Google Scholar]
 Rieutord M, Roudier T, Roques S, Ducottet C. 2007. Tracking granules on the Sun’s surface and reconstructing velocity fields. I. The CST algorithm. A&A 471: 687–694. https://doi.org/10.1051/00046361:20066491. [NASA ADS] [CrossRef] [EDP Sciences] [Google Scholar]
 Rimmele TR, Warner M, Keil SL, Goode PR, Knölker M, et al. 2020. The Daniel K. Inouye solar telescope – observatory overview. Sol Phys 295(12): 172. https://doi.org/0.1007/s11207020017367. https://ui.adsabs.harvard.edu/abs/2020SoPh..295..172R. Provided by the SAO/NASA Astrophysics Data System. [CrossRef] [Google Scholar]
 Ronneberger O, Fischer P, Brox T. 2015. UNet: Convolutional networks for biomedical image segmentation. In: International conference on Medical image computing and computerassisted intervention, Springer, Cham, pp. 234–241. [Google Scholar]
 Schou J, Scherrer PH, Bush RI, Wachter R, Couvidat S, et al. 2012. Design and ground calibration of the helioseismic and magnetic imager (HMI) instrument on the solar dynamics observatory (SDO). Sol Phys 275: 229–259. https://doi.org/10.1007/s1120701198422. [NASA ADS] [CrossRef] [Google Scholar]
 Schrijver CJ, De Rosa ML, Metcalf TR, Liu Y, McTiernan J, Régnier S, Valori G, Wheatland MS, Wiegelmann T. 2006. Nonlinear forcefree modeling of coronal magnetic fields part I: A quantitative comparison of methods. Sol Phys 235: 161–190. https://doi.org/10.1007/s1120700600687. [NASA ADS] [CrossRef] [Google Scholar]
 Schuck PW. 2005. Local correlation tracking and the magnetic induction equation. Astrophys J Lett 632: L53–L56. https://doi.org/10.1086/497633. [CrossRef] [Google Scholar]
 Schuck PW. 2006. Tracking magnetic footpoints with the magnetic induction equation. Astrophys J 646: 1358–1391. https://doi.org/10.1086/505015. [CrossRef] [Google Scholar]
 Schuck PW. 2008. Tracking Vector magnetograms with the magnetic induction equation. Astrophys J 683: 1134–1152. https://doi.org/10.1086/589434. [CrossRef] [Google Scholar]
 Stein RF. 2012. Solar surface magnetoconvection. Living Rev Sol Phys 9: 4. https://doi.org/10.12942/lrsp20124. [CrossRef] [Google Scholar]
 Stein RF, Nordlund Å. 1998. Simulations of Solar Granulation. I. General Properties. Astrophys J 499: 914–933. https://doi.org/10.1086/305678. [NASA ADS] [CrossRef] [Google Scholar]
 Stein RF, Nordlund Å. 2012. On the formation of active regions. Astrophys J Lett 753: L13. https://doi.org/10.1088/20418205/753/1/L13. [NASA ADS] [CrossRef] [Google Scholar]
 Tremblay B, Attie R. 2020. Inferring plasma flows at granular and supergranular scales with a new architecture for the DeepVel neural network. Front Astron Space Sci 7: 25. https://doi.org/10.3389/fspas.2020.00025. [CrossRef] [Google Scholar]
 Tremblay B, Roudier T, Rieutord M, Vincent A. 2018. Reconstruction of horizontal plasma motions at the photosphere from intensitygrams: A comparison between DeepVel, LCT, FLCT, and CST. Sol Phys 293: 57. https://doi.org/10.1007/s1120701812767. [CrossRef] [Google Scholar]
 Tremblay B, Roudier T, Cossette JF, Attié R, Rieutord M, Vincent A. 2019. Neural network to emulate numerical simulations of the Sun and infer synthetic observations for data assimilation. In: Solar Heliospheric and INterplanetary Environment (SHINE 2019), 30 p. https://shinecon.org/Publications.php. [Google Scholar]
 Vögler A, Shelyag S, Schüssler M, Cattaneo F, Emonet T, Linde T. 2005. Simulations of magnetoconvection in the solar photosphere. Equations, methods, and results of the MURaM code. A&A 429: 335–351. https://doi.org/10.1051/00046361:20041507. [NASA ADS] [CrossRef] [EDP Sciences] [Google Scholar]
 Wachter R, Schou J, RabelloSoares MC, Miles JW, Duvall TL, Bush RI. 2012. Image quality of the helioseismic and magnetic imager (HMI) onboard the solar dynamics observatory (SDO). Sol Phys 275(1–2): 261–284. https://doi.org/10.1007/s1120701197096. [NASA ADS] [CrossRef] [Google Scholar]
 Warner M, Rimmele TR, Martinez Pillet V, Casini R, Berukoff S, et al. 2018. Construction update of the Daniel K. Inouye Solar Telescope project. In: Groundbased and Airborne Telescopes VII, Vol. 10700 of Society of PhotoOptical Instrumentation Engineers (SPIE) Conference Series, SPIE, Bellingham, WA, USA, 107000V p. https://doi.org/10.1117/12.2314212. [Google Scholar]
 Welsch BT. 2006. Magnetic flux cancellation and coronal magnetic energy. Astrophys J 638(2): 1101–1109. https://doi.org/10.1086/498638. [NASA ADS] [CrossRef] [Google Scholar]
 Welsch BT. 2015. The photospheric Poynting flux and coronal heating. Publ Astron Soc Jpn 67(2): 18. https://doi.org/10.1093/pasj/psu151. [CrossRef] [Google Scholar]
 Yeates AR, Bianchi F, Welsch BT, Bushby PJ. 2014. The coronal energy input from magnetic braiding. A&A 564: A131. https://doi.org/10.1051/00046361/201323276. [NASA ADS] [CrossRef] [EDP Sciences] [Google Scholar]
Cite this article as: Tremblay B, Cossette JF, Kazachenko MD, Charbonneau P & Vincent A. 2021. Inferring depthdependent plasma motions from surface observations using the DeepVel neural network. J. Space Weather Space Clim. 11, 9. https://doi.org/10.1051/swsc/2020073.
All Tables
Similarity between the reference velocity field at the surface v_{ref}(τ ≈ 1) and at various optical and geometrical depth above and below the surface, τ and d, respectively. Metrics C and A correspond to the correlation coefficient (Eq. (4)) and the spatiallyaveraged normalized dot product A (Eq. (5)), respectively, averaged over time.
Comparison between flow fields v_{D,t} and v_{ref,t} for the QS datasets (i.e., STAGGER and MURaMQS). RMSE is the root mean squared error (Eq. (1)), E_{abs} is the mean absolute error (Eq. (2)), E_{rel} is the mean relative error (Eq. (3)), C is the correlation coefficient (Eq. (4)), A is the spatiallyaveraged normalized dot product (Eq. (5)) and S is the ratio between the integrals of the unsigned shear components of the DeepVel and reference Poynting fluxes (Eq. (9)).
Comparison between flow fields v_{D,t} and v_{ref,t} for the ARs dataset. RMSE is the root mean squared error (Eq. (1)), E_{abs} is the mean absolute error (Eq. (2)), E_{rel} is the mean relative error (Eq. (3)), C is the correlation coefficient (Eq. (4)), A is the spatiallyaveraged normalized dot product (Eq. (5)) and S is the ratio between the integrals of the unsigned shear components of the DeepVel and reference Poynting fluxes (Eq. (9)).
All Figures
Fig. 1 Above the surface velocity inversions, STAGGER dataset. Patches of 50 by 50 pixels^{2} extracted from the 30minaveraged transverse velocity fields v_{D,t}(τ, t_{i}) (left column) computed by the STAGGER simulation and (middle column) inferred by DeepVel at optical depths (top) τ ≈ 1, (middle) τ ≈ 0.1 and (bottom) τ ≈ 0.01. The timeaveraged vertical velocity v_{ref,z}(τ ≈ {1, 0.1, 0.01}, t_{i}) computed by the STAGGER simulation and resampled to the SDO/HMI resolution is displayed as colored background. (Right column) Scatterplots comparing amplitudes v_{D,t} to v_{ref,t}. The black line represents an ideal solution (i.e., a coefficient of determination R^{2} = 1). 

In the text 
Fig. 2 Above the surface velocity inversions, MURaMQS dataset. Patches of 125 by 125 pixels^{2} extracted from the transverse velocity fields v_{D,t}(τ, t_{i}) (left column) computed by the MURaMQS simulation (Vögler et al., 2005) and (middle column) inferred by DeepVel at optical depths (top) τ ≈ 1, (middle) τ ≈ 0.1 and (bottom) τ ≈ 0.01. The vertical velocity v_{ref,z}(τ ≈ {1, 0.1, 0.01}, t_{i}) computed by the MURaMQS simulation is displayed as colored background with a symmetrical threshold. (Right column) Scatterplots comparing amplitudes v_{D,t} to v_{ref,t}. The black line represents the expected solution (i.e., a coefficient of determination R^{2} = 1). 

In the text 
Fig. 3 Below the surface velocity inversions, MURaMQS dataset. Patches of 125 by 125 pixels^{2} extracted from instantaneous transverse velocity field maps which were computed at geometrical depths (top) d = 0 km, (middle) d = 144 km, (bottom) d = 560 km by (left column) the MURaMQS simulation (Vögler et al., 2005) and (middle column) singlelayer output versions of DeepVel. The divergence of the transverse plasma motions (i.e., ∇_{t}·v_{t}) is displayed as colored background to emphasize the complexity of the structures that arise at increasing depths. (Right column) Scatterplots comparing amplitudes v_{D,t} to v_{ref,t} at each geometrical depth. The black line represents the expected solution (i.e., a coefficient of determination R^{2} = 1). 

In the text 
Fig. 4 At the surface velocity inversions, MURaMARs dataset. Patches of 50 × 50 pixels^{2} extracted from 30timestepaveraged transverse velocity field maps which were computed at optical depth τ ≈ 1 by (a) the MURaMARs simulation (Rempel & Cheung, 2014), (b) the DeepVel neural network that generated Figure 1a, and (c) a version of the DeepVel neural network which was trained using the MURaMARs simulation resampled at the SDO/HMI spatial resolution (Δx ≈ 368 km). The vertical component of the velocity computed by the MURaMARs simulation is displayed as colored background. (Right column) Scatterplots comparing amplitudes v_{D,t} to v_{ref,t}. The black line represents the expected solution (i.e., a coefficient of determination R^{2} = 1). 

In the text 
Fig. 5 Above the surface velocity inversions, MURaMARs dataset. (Left column) Patches of 50 by 50 pixels^{2} extracted from the 30minaveraged transverse velocity fields v_{D,t}(τ, t_{i}) inferred by DeepVel at optical depths (top) τ ≈ 0.1, (middle) τ ≈ 0.01 and (bottom) τ ≈ 0.001. The timeaveraged vertical velocity v_{ref,z}(τ ≈ {0.1, 0.01, 0.001}, t_{i}) computed by the MURaMARs simulation and resampled to the SDO/HMI resolution is displayed as colored background. (Right column) Scatterplots comparing amplitudes v_{D,t} to v_{ref,t}. The black line represents the expected solution (i.e., a coefficient of determination R^{2} = 1). 

In the text 
Fig. 6 Below the surface velocity inversions, MURaMARs dataset. (a) Patch of 50 by 50 pixels^{2} extracted from the 30minaveraged transverse velocity field v_{D,t}(d, t_{i}) inferred by DeepVel at geometrical depth d = 1000 km. The timeaveraged vertical velocity v_{ref,z}(d = 1000 km, t_{i}) computed by the MURaMARs simulation and resampled to the SDO/HMI resolution is displayed as colored background. (Right column) Scatterplot comparing amplitudes v_{D,t} to v_{ref,t}. The black line represents the expected solution (i.e., a coefficient of determination R^{2} = 1). 

In the text 
Current usage metrics show cumulative count of Article Views (fulltext article views including HTML views, PDF and ePub downloads, according to the available data) and Abstracts Views on Vision4Press platform.
Data correspond to usage on the plateform after 2015. The current usage metrics is available 4896 hours after online publication and is updated daily on week days.
Initial download of the metrics may take a while.