3D‐Var data assimilation using a variational autoencoder - Melinc, Boštjan; Zaplotnik, Žiga (Quarterly journal of the Royal Meteorological Society, 2024)
Data assimilation of atmospheric observations traditionally relies on variational and Kalman filter methods. Here, an alternative neural network data assimilation (NNDA) with variational autoencoder (VAE) is proposed. The three‐dimensional variational (3D‐Var) data assimilation cost function is utilised to determine the analysis that optimally fuses simulated observations and the encoded short‐range persistence forecast (background), accounting for their errors. The minimisation is performed in the reduced‐order latent space discovered by the VAE. The variational problem is autodifferentiable, simplifying the computation of the cost‐function gradient necessary for efficient minimisation. We demonstrate that the background‐error covariance (B) matrix measured and represented in the latent space is quasidiagonal. The background‐error covariances in the grid‐point space are flow‐dependent, evolving seasonally and depending on the current state of the atmosphere. Data assimilation experiments with a single temperature observation in the lower troposphere indicate that the B matrix describes both tropical and extratropical background‐error covariances simultaneously. Our study introduces a novel 3D‐Var‐based data assimilation (DA) technique which performs variational cost function minimisation in the reduced‐order latent space discovered by the variational autoencoder. In such latent space, the background‐error covariance matrix becomes quasi‐diagonal, leading to efficient minimisation. Single observation DA experiments reveal non‐isotropic and flow‐dependent analysis increments, which differ vastly between the midlatitudes and the Tropics, representing the distinct atmospheric dynamics between the two regions.