{ "cells": [ { "cell_type": "markdown", "id": "5c219948-bb3e-4474-a663-310f9c6426df", "metadata": {}, "source": [ "# Introduction" ] }, { "cell_type": "markdown", "id": "72fb0629-adf1-47f1-b648-47023671c7f2", "metadata": {}, "source": [ "archNEMESIS is an algorithm designed to 1) generate a forward model of the spectrum of a planetary atmosphere based on a set of input parameters, and 2) perform atmospheric retrievals, by which some of the input parameters are iteratively modified until the simulated spectrum best fits the measured one.\n", "\n", "The code can be divided in four high-level parts:\n", "\n", "- Reference classes: These classes include all the input information we require to model the electromagnetic spectrum of a planetary atmosphere, starting from the atmospheric and surface properties of the planet, to the geometry and specifics of the instrument response we want to simulate.\n", "\n", "- Model parameterisations: These are a set of parameters that are iteratively modified in our model to find the best fit between the modelled and measured spectrum. In atmospheric retrievals, these variables are more generally known as the state vector.\n", "\n", "- Forward Model: It refers to the set of functions that solve the radiative transfer equation and allow the calculation of the spectrum based on the model inputs and parameterisations.\n", "\n", "- Retrieval engine: It refers to the algorithm or methodology to solve the inverse problem – search for the set of model parameters that produce a best fit between the modelled and measured spectrum.\n", "\n", "The figure below shows a high-level schematic of the structure of archNEMESIS for a forward model simulation. In the following sub-sections, we describe each of these parts in more detail.\n", "\n", "![Optional alt text](../images/archNEMESIS_structure_v1.png)" ] }, { "cell_type": "markdown", "id": "a72ea71f-bef1-4c5b-b98c-a22d65a762c9", "metadata": {}, "source": [ "# Input files\n", "\n", "The information in archNEMESIS is generally read from a set of input files with a required specific format. There are two versions of the files that can be used, one using the existing standard NEMESIS input files, and another one specifically designed for using archNEMESIS." ] }, { "cell_type": "markdown", "id": "68739811-b047-4887-9bac-1fdda5b4ca55", "metadata": {}, "source": [ "## Standard NEMESIS files\n", "\n", "NEMESIS reads the information from a set of text files with a specific format. Here we list the format of such required files." ] }, { "cell_type": "raw", "id": "8e050210-cb54-4426-9c86-8b1e8c3abd60", "metadata": {}, "source": [ "\n", "\n", "
\n", "
\n", " .set file\n", " \n", " \n", "
\n", "
" ] }, { "cell_type": "raw", "id": "b90a77b7-5804-4fa1-a308-6d658b0eaae7", "metadata": {}, "source": [ "\n", "\n", "
\n", "
\n", " .inp file\n", " \n", " \n", "
\n", "
" ] }, { "cell_type": "raw", "id": "658c3e2b-3464-4e10-984d-0d48336dfc5d", "metadata": {}, "source": [ "\n", "\n", "
\n", "
\n", " .fla file\n", " \n", " \n", "
\n", "
" ] }, { "cell_type": "raw", "id": "b03b7bf8-1b30-4950-ad4c-a14a981e6bbb", "metadata": {}, "source": [ "\n", "\n", "
\n", "
\n", " .ref file\n", " \n", " \n", "
\n", "
" ] }, { "cell_type": "raw", "id": "31ac82e4-0a74-4971-b3a7-506032d42140", "metadata": {}, "source": [ "\n", "\n", "
\n", "
\n", " aerosol.ref file\n", " \n", " \n", "
\n", "
" ] }, { "cell_type": "raw", "id": "204ab986-cd05-4e79-b591-08961782efd3", "metadata": {}, "source": [ "\n", "\n", "
\n", "
\n", " parah2.ref file\n", " \n", " \n", "
\n", "
" ] }, { "cell_type": "raw", "id": "40ccc0fa-d178-4a2d-b63e-86e9b68ef03b", "metadata": {}, "source": [ "\n", "\n", "
\n", "
\n", " .spx file\n", " \n", " \n", "
\n", "
" ] }, { "cell_type": "raw", "id": "a0d62c2f-8851-4f5f-8b19-2645e519d6a4", "metadata": {}, "source": [ "\n", "\n", "
\n", "
\n", " .sha file\n", " \n", " \n", "
\n", "
" ] }, { "cell_type": "raw", "id": "4a7309bc-5a96-4ce7-8023-c03509f339b5", "metadata": {}, "source": [ "\n", "\n", "
\n", "
\n", " .fil file\n", " \n", " \n", "
\n", "
" ] }, { "cell_type": "raw", "id": "cd578fb1-834b-431c-964e-c0a284960bbf", "metadata": {}, "source": [ "\n", "\n", "
\n", "
\n", " .lls or .kls file\n", " \n", " \n", "
\n", "
" ] }, { "cell_type": "raw", "id": "b82e0f9d-d713-4086-875f-5edf714e3e19", "metadata": {}, "source": [ "\n", "\n", "
\n", "
\n", " .sol file\n", " \n", " \n", "
\n", "
" ] }, { "cell_type": "markdown", "id": "bbf6a47b-f8a5-4bcc-940b-d1e467ff4781", "metadata": {}, "source": [ "When using archNEMESIS to run a forward model or a retrieval using these inputs, the fastest and easiest way of reading all the inputs and properly feeding this into the reference classes is by using the *read_input_files* method.\n", "\n", "```python\n", "import archnemesis as ans\n", "Atmosphere,Measurement,Spectroscopy,Scatter,Stellar,Surface,CIA,Layer,Variables,Retrieval = ans.Files.read_input_files(runname)\n", "```\n" ] }, { "cell_type": "markdown", "id": "0301fb9c-32ba-4596-a3d4-124a4bed3a5a", "metadata": {}, "source": [ "## HDF5 archNEMESIS file" ] }, { "cell_type": "markdown", "id": "60ae7357-1f96-425f-9f0d-b9c7484a4c1c", "metadata": {}, "source": [ "In addition to the standard NEMESIS files, a specific version of the input files for archNEMESIS has been designed. The information is essentially the same, but the format of the files benefits from a tailored structure for this version. In particular, a forward model or retrieval in archNEMESIS can be run by reading information from a single HDF5 file. This file format allows the organisation of the data in a hierarchical manner using groups and datasets. This format is particularly useful for the structure of the inputs from archNEMESIS, since each of the reference classes corresponds to a unique group within the file, which includes all the information (i.e., datasets) for that particular class.\n", "\n", "All classes include *read_hdf5()* and *write_hdf5()* methods to easily read and write the archNEMESIS HDF5 file. In addition, these methods include an assessing function to check the validity of the information within the class (i.e., see if there are any conflicting flags). In addition, these methods write metadata to easily visualise the contents of the file (i.e., includes of explanation of each variable in the file, units, and selected flags). HDF5 files can be visualised by using specific software. If using Visual Studio Code, we recommend using the [H5Web](https://marketplace.visualstudio.com/items?itemName=h5web.vscode-h5web) extension to easily read the contents. Otherwise, HDF5 files can be explored using software such as [Panoply](https://www.giss.nasa.gov/tools/panoply/) or [HDFView](https://www.hdfgroup.org/download-hdfview/).\n", "\n", "In the case that the HDF5 file includes information about all the classes required to run a forward model or retrieval, then all inputs can be simultaneously read by using the function:\n", "\n", "```python\n", "import archnemesis as ans\n", "Atmosphere,Measurement,Spectroscopy,Scatter,Stellar,Surface,CIA,Layer,Variables,Retrieval,Telluric = ans.Files.read_input_files_hdf5(runname)\n", "```" ] }, { "cell_type": "markdown", "id": "551584be-b3aa-4ae7-b4ce-b6f9daba1b81", "metadata": {}, "source": [ "# Output files\n", "\n", "Similarly, once a retrieval or forward model simulations has been performed, the output information is written into a set of output files, either following the standard NEMESIS formats, or the archNEMESIS HDF5 one." ] }, { "cell_type": "markdown", "id": "e685d0b7-df49-43ca-ba32-bfa602dc6135", "metadata": {}, "source": [ "## Standard NEMESIS files\n", "\n", "NEMESIS writes the information from a retrieval simulation into a set of text files with a specific format. Here we list the format of such required files." ] }, { "cell_type": "raw", "id": "9578bb83-3f2a-4c07-8806-5e2f2632275d", "metadata": {}, "source": [ "\n", "\n", "
\n", "
\n", " .mre file\n", " \n", " \n", "
\n", "
" ] }, { "cell_type": "raw", "id": "e9daa50b-55e4-4789-bcc0-5e1c0fb01c51", "metadata": {}, "source": [ "\n", "\n", "
\n", "
\n", " .cov file\n", " \n", " \n", "
\n", "
" ] }, { "cell_type": "markdown", "id": "a8b2fe23-07da-468c-92b6-fe8d28e9207d", "metadata": {}, "source": [ "## HDF5 archNEMESIS file" ] }, { "cell_type": "markdown", "id": "5bafe66d-788a-4bc8-9310-b06ec080a142", "metadata": {}, "source": [ "If the retrieval with archNEMESIS is run using the HDF5 file format, then the outputs are written into the same file. In particular, the outputs will be written into the *Retrieval* section of the HDF5 file.\n", "\n", "The best fit to the spectrum can be easily read using the read_bestfit_hdf5() function stored in *Files.py*. In particular, this function will return the *Measurement* class from the file, but will also include an extra parameter called SPECMOD, containing the best fit modelled spectrum, which can be directly compared with the MEAS measured spectrum. \n", "\n", "The retrieved parameters can be easily read using the read_retparam_hdf5() function stored in *Files.py*. This function will return the prior parameters included in the .apr file, as well as the retrieved parameters for each model parameterisation." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.11" } }, "nbformat": 4, "nbformat_minor": 5 }