{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Create RUBIX data\n",
    "\n",
    "## The config\n",
    "\n",
    "The config contains all the information needed to run the pipeline. Those are run specfic configurations. Currently we just support Illustris as simulation, but extensions to other simulations (e.g. NIHAO) are planned.\n",
    "\n",
    "For the config you can choose the following options:\n",
    "- particle_type: load only stars particle (\"particle_type\": [\"stars\"]) or only gas particle (\"particle_type\": [\"gas\"]) or both (\"particle_type\": [\"stars\",\"gas\"])\n",
    "- simulation: choose the Illustris simulation (e.g. \"simulation\": \"TNG50-1\")\n",
    "- snapshot: which time step of the simulation (99 for present day)\n",
    "- save_data_path: set the path to save the downloaded Illustris data\n",
    "- load_galaxy_args - id: define, which Illustris galaxy is downloaded\n",
    "- load_galaxy_args - reuse: if True, if in th esave_data_path directory a file for this galaxy id already exists, the downloading is skipped and the preexisting file is used\n",
    "- subset: only a defined number of stars/gas particles is used and stored for the pipeline. This may be helpful for quick testing\n",
    "- simulation - name: currently only IllustrisTNG is supported\n",
    "- simulation - args - path: where the data is stored and how the file will be named\n",
    "- output_path: where the hdf5 file is stored, which is then the input to the RUBIX pipeline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# NBVAL_SKIP\n",
    "import os\n",
    "\n",
    "config = {\n",
    "    \"logger\": {\n",
    "        \"log_level\": \"DEBUG\",\n",
    "        \"log_file_path\": None,\n",
    "        \"format\": \"%(asctime)s - %(name)s - %(levelname)s - %(message)s\",\n",
    "    },\n",
    "    \"data\": {\n",
    "        \"name\": \"IllustrisAPI\",\n",
    "        \"args\": {\n",
    "            \"api_key\": os.environ.get(\"ILLUSTRIS_API_KEY\"),\n",
    "            \"particle_type\": [\"stars\",\"gas\"],\n",
    "            \"simulation\": \"TNG50-1\",\n",
    "            \"snapshot\": 99,\n",
    "            \"save_data_path\": \"data\",\n",
    "        },\n",
    "        \n",
    "        \"load_galaxy_args\": {\n",
    "        \"id\": 12,\n",
    "        \"reuse\": True,\n",
    "        },\n",
    "\n",
    "        \"subset\": {\n",
    "            \"use_subset\": True,\n",
    "            \"subset_size\": 1000,\n",
    "        },\n",
    "    },\n",
    "    \"simulation\": {\n",
    "        \"name\": \"IllustrisTNG\",\n",
    "        \"args\": {\n",
    "            \"path\": \"data/galaxy-id-12.hdf5\",\n",
    "        },\n",
    "    },\n",
    "    \"output_path\": \"output\",\n",
    "\n",
    "        \n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Convert data\n",
    "\n",
    "Convert the Data into Rubix Galaxy HDF5. This will make the call to the IllustrisAPI to download the data, and then convert it into the rubix hdf5 format using the input handler"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# NBVAL_SKIP\n",
    "from rubix.core.data import convert_to_rubix\n",
    "\n",
    "convert_to_rubix(config)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load data\n",
    "\n",
    "prepare_input loads the hdf5 file that was created and stored with the convert_to_rubix function. It loads the data into the rubixdata format and centers the particles. the rubixdata object has then the attributes stars and gas and both have then attributes with the relevant quantities for each particle. For example, if you want to access the coordinates of the stella rparticles, you can access them via rubixdata.stars.coords"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# NBVAL_SKIP\n",
    "from rubix.core.data import prepare_input\n",
    "\n",
    "rubixdata = prepare_input(config)\n",
    "\n",
    "#print, which attributes are available for rubixdata.stars\n",
    "attr = [attr for attr in dir(rubixdata.stars) if not attr.startswith('__')]\n",
    "print(attr)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To have not to call two individual function to have the data ready to be passed into the pipeline, you can just use the get_rubix_data(config) from the rubix.core.data module"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Overview over the hdf5 file structure"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# NBVAL_SKIP\n",
    "from rubix.utils import print_hdf5_file_structure\n",
    "\n",
    "print(print_hdf5_file_structure(\"output/rubix_galaxy.h5\"))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "rubix",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}