{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# RUBIX pipeline\n", "\n", "RUBIX is designed as a linear pipeline, where the individual functions are called and constructed as a pipeline. This allows as to execude the whole data transformation from a cosmological hydrodynamical simulation of a galaxy to an IFU cube in two lines of code. This notebook shows, how to execute the pipeline. To see, how the pipeline is execuded in small individual steps per individual function, we refer to the notebook `rubix_pipeline_stepwise.ipynb`.\n", "\n", "## How to use the Pipeline\n", "1) Define a `config`\n", "2) Setup the `pipeline yaml`\n", "3) Run the RUBIX pipeline\n", "4) Do science with the mock-data" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 1: Config\n", "\n", "The `config` contains all the information needed to run the pipeline. Those are run specfic configurations. Currently we just support Illustris as simulation, but extensions to other simulations (e.g. NIHAO) are planned.\n", "\n", "For the `config` you can choose the following options:\n", "- `pipeline`: you specify the name of the pipeline that is stored in the yaml file in rubix/config/pipeline_config.yml\n", "- `logger`: RUBIX has implemented a logger to report the user, what is happening during the pipeline execution and give warnings\n", "- `data - args - particle_type`: load only stars particle (\"particle_type\": [\"stars\"]) or only gas particle (\"particle_type\": [\"gas\"]) or both (\"particle_type\": [\"stars\",\"gas\"])\n", "- `data - args - simulation`: choose the Illustris simulation (e.g. \"simulation\": \"TNG50-1\")\n", "- `data - args - snapshot`: which time step of the simulation (99 for present day)\n", "- `data - args - save_data_path`: set the path to save the downloaded Illustris data\n", "- `data - load_galaxy_args - id`: define, which Illustris galaxy is downloaded\n", "- `data - load_galaxy_args - reuse`: if True, if in th esave_data_path directory a file for this galaxy id already exists, the downloading is skipped and the preexisting file is used\n", "- `data - subset`: only a defined number of stars/gas particles is used and stored for the pipeline. This may be helpful for quick testing\n", "- `simulation - name`: currently only IllustrisTNG is supported\n", "- `simulation - args - path`: where the data is stored and how the file will be named\n", "- `output_path`: where the hdf5 file is stored, which is then the input to the RUBIX pipeline\n", "- `telescope - name`: define the telescope instrument that is observing the simulation. Some telescopes are predefined, e.g. MUSE. If your instrument does not exist predefined, you can easily define your instrument in rubix/telescope/telescopes.yaml\n", "- `telescope - psf`: define the point spread function that is applied to the mock data\n", "- `telescope - lsf`: define the line spread function that is applied to the mock data\n", "- `telescope - noise`: define the noise that is applied to the mock data\n", "- `cosmology`: specify the cosmology you want to use, standard for RUBIX is \"PLANCK15\"\n", "- `galaxy - dist_z`: specify at which redshift the mock-galaxy is observed\n", "- `galaxy - rotation`: specify the orientation of the galaxy. You can set the types edge-on or face-on or specify the angles alpha, beta and gamma as rotations around x-, y- and z-axis\n", "- `ssp - template`: specify the simple stellar population lookup template to get the stellar spectrum for each stars particle. In RUBIX frequently \"BruzualCharlot2003\" is used." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#NBVAL_SKIP\n", "import matplotlib.pyplot as plt\n", "from rubix.core.pipeline import RubixPipeline \n", "import os\n", "config = {\n", " \"pipeline\":{\"name\": \"calc_ifu\"},\n", " \n", " \"logger\": {\n", " \"log_level\": \"DEBUG\",\n", " \"log_file_path\": None,\n", " \"format\": \"%(asctime)s - %(name)s - %(levelname)s - %(message)s\",\n", " },\n", " \"data\": {\n", " \"name\": \"IllustrisAPI\",\n", " \"args\": {\n", " \"api_key\": os.environ.get(\"ILLUSTRIS_API_KEY\"),\n", " \"particle_type\": [\"stars\"],\n", " \"simulation\": \"TNG50-1\",\n", " \"snapshot\": 99,\n", " \"save_data_path\": \"data\",\n", " },\n", " \n", " \"load_galaxy_args\": {\n", " \"id\": 14,\n", " \"reuse\": True,\n", " },\n", " \n", " \"subset\": {\n", " \"use_subset\": True,\n", " \"subset_size\": 1000,\n", " },\n", " },\n", " \"simulation\": {\n", " \"name\": \"IllustrisTNG\",\n", " \"args\": {\n", " \"path\": \"data/galaxy-id-14.hdf5\",\n", " },\n", " \n", " },\n", " \"output_path\": \"output\",\n", "\n", " \"telescope\":\n", " {\"name\": \"MUSE\",\n", " \"psf\": {\"name\": \"gaussian\", \"size\": 5, \"sigma\": 0.6},\n", " \"lsf\": {\"sigma\": 0.5},\n", " \"noise\": {\"signal_to_noise\": 1,\"noise_distribution\": \"normal\"},},\n", " \"cosmology\":\n", " {\"name\": \"PLANCK15\"},\n", " \n", " \"galaxy\":\n", " {\"dist_z\": 0.1,\n", " \"rotation\": {\"type\": \"edge-on\"},\n", " },\n", " \n", " \"ssp\": {\n", " \"template\": {\n", " \"name\": \"BruzualCharlot2003\"\n", " },\n", " }, \n", "}" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 2: Pipeline yaml\n", "\n", "To run the RUBIX pipeline, you need a yaml file (stored in `rubix/config/pipeline_config.yml`) that defines which functions are used during the execution of the pipeline. This shows the example pipeline yaml to compute a stellar IFU cube.\n", "\n", "```yaml\n", "calc_ifu:\n", " Transformers:\n", " rotate_galaxy:\n", " name: rotate_galaxy\n", " depends_on: null\n", " args: []\n", " kwargs:\n", " type: \"face-on\"\n", " filter_particles:\n", " name: filter_particles\n", " depends_on: rotate_galaxy\n", " args: []\n", " kwargs: {}\n", " spaxel_assignment:\n", " name: spaxel_assignment\n", " depends_on: filter_particles\n", " args: []\n", " kwargs: {}\n", "\n", " reshape_data:\n", " name: reshape_data\n", " depends_on: spaxel_assignment\n", " args: []\n", " kwargs: {}\n", "\n", " calculate_spectra:\n", " name: calculate_spectra\n", " depends_on: reshape_data\n", " args: []\n", " kwargs: {}\n", "\n", " scale_spectrum_by_mass:\n", " name: scale_spectrum_by_mass\n", " depends_on: calculate_spectra\n", " args: []\n", " kwargs: {}\n", " doppler_shift_and_resampling:\n", " name: doppler_shift_and_resampling\n", " depends_on: scale_spectrum_by_mass\n", " args: []\n", " kwargs: {}\n", " calculate_datacube:\n", " name: calculate_datacube\n", " depends_on: doppler_shift_and_resampling\n", " args: []\n", " kwargs: {}\n", " convolve_psf:\n", " name: convolve_psf\n", " depends_on: calculate_datacube\n", " args: []\n", " kwargs: {}\n", " convolve_lsf:\n", " name: convolve_lsf\n", " depends_on: convolve_psf\n", " args: []\n", " kwargs: {}\n", " apply_noise:\n", " name: apply_noise\n", " depends_on: convolve_lsf\n", " args: []\n", " kwargs: {}\n", "```\n", "\n", "Ther is one thing you have to know about the naming of the functions in this yaml: To use the functions inside the pipeline, the functions have to be called exactly the same as they are returned from the core module function!" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 3: Run the pipeline\n", "\n", "After defining the `config` and the `pipeline_config` you can simply run the whole pipeline by these two lines of code." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#NBVAL_SKIP\n", "pipe = RubixPipeline(config)\n", "\n", "rubixdata = pipe.run()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 4: Mock-data\n", "\n", "Now we have our final datacube and can use the mock-data to do science. Here we have a quick look in the optical wavelengthrange of the mock-datacube and show the spectra of a central spaxel and a spatial image." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#NBVAL_SKIP\n", "import jax.numpy as jnp\n", "\n", "wave = pipe.telescope.wave_seq\n", "# get the indices of the visible wavelengths of 4000-8000 Angstroms\n", "visible_indices = jnp.where((wave >= 4000) & (wave <= 8000))\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This is how you can access the spectrum of an individual spaxel, the wavelength can be accessed via `pipe.wave_seq`" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#NBVAL_SKIP\n", "wave = pipe.telescope.wave_seq\n", "\n", "spectra = rubixdata.stars.datacube # Spectra of all stars\n", "print(spectra.shape)\n", "\n", "plt.plot(wave, spectra[12,12,:])\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Plot a spacial image of the data cube" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#NBVAL_SKIP\n", "# get the spectra of the visible wavelengths from the ifu cube\n", "visible_spectra = rubixdata.stars.datacube[:, :, visible_indices[0]]\n", "#visible_spectra.shape\n", "\n", "# Sum up all spectra to create an image\n", "image = jnp.sum(visible_spectra, axis = 2)\n", "plt.imshow(image, origin=\"lower\", cmap=\"inferno\")\n", "plt.colorbar()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## DONE!\n", "\n", "Congratulations, you have sucessfully run the RUBIX pipeline to create your own mock-observed IFU datacube! Now enjoy playing around with the RUBIX pipeline and enjoy doing amazing science with RUBIX :)" ] } ], "metadata": { "kernelspec": { "display_name": "rubix", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.14" } }, "nbformat": 4, "nbformat_minor": 2 }