RUBIX pipeline in steps

RUBIX is designed as a linear pipeline, where the individual functions are called and constructed as a pipeline. This allows as to execude the whole data transformation from a cosmological hydrodynamical simulation of a galaxy to an IFU cube in two lines of code. To get a better sense, what is happening during the execution of the pipeline, this notebook splits the pipeline in small steps.

This notebook contains the functions that are called inside the rubix pipeline. To see, how the pipeline is execuded, we refer to the notebook rubix_pipeline_single_function.ipynb.

Step 1: Config

The config contains all the information needed to run the pipeline. Those are run specfic configurations. Currently we just support Illustris as simulation, but extensions to other simulations (e.g. NIHAO) are planned.

For the config you can choose the following options:

• pipeline: you specify the name of the pipeline that is stored in the yaml file in rubix/config/pipeline_config.yml

• logger: RUBIX has implemented a logger to report the user, what is happening during the pipeline execution and give warnings

• data - args - particle_type: load only stars particle (“particle_type”: [“stars”]) or only gas particle (“particle_type”: [“gas”]) or both (“particle_type”: [“stars”,”gas”])

• data - args - simulation: choose the Illustris simulation (e.g. “simulation”: “TNG50-1”)

• data - args - snapshot: which time step of the simulation (99 for present day)

• data - args - save_data_path: set the path to save the downloaded Illustris data

• data - load_galaxy_args - id: define, which Illustris galaxy is downloaded

• data - load_galaxy_args - reuse: if True, if in th esave_data_path directory a file for this galaxy id already exists, the downloading is skipped and the preexisting file is used

• data - subset: only a defined number of stars/gas particles is used and stored for the pipeline. This may be helpful for quick testing

• simulation - name: currently only IllustrisTNG is supported

• simulation - args - path: where the data is stored and how the file will be named

• output_path: where the hdf5 file is stored, which is then the input to the RUBIX pipeline

• telescope - name: define the telescope instrument that is observing the simulation. Some telescopes are predefined, e.g. MUSE. If your instrument does not exist predefined, you can easily define your instrument in rubix/telescope/telescopes.yaml

• telescope - psf: define the point spread function that is applied to the mock data

• telescope - lsf: define the line spread function that is applied to the mock data

• telescope - noise: define the noise that is applied to the mock data

• cosmology: specify the cosmology you want to use, standard for RUBIX is “PLANCK15”

• galaxy - dist_z: specify at which redshift the mock-galaxy is observed

• galaxy - rotation: specify the orientation of the galaxy. You can set the types edge-on or face-on or specify the angles alpha, beta and gamma as rotations around x-, y- and z-axis

• ssp - template: specify the simple stellar population lookup template to get the stellar spectrum for each stars particle. In RUBIX frequently “BruzualCharlot2003” is used.

# NBVAL_SKIP
import os
os.environ['SPS_HOME'] = '/home/annalena/sps_fsps'

# NBVAL_SKIP
import os
config = {
    "pipeline":{"name": "calc_ifu"},
    
    "logger": {
        "log_level": "DEBUG",
        "log_file_path": None,
        "format": "%(asctime)s - %(name)s - %(levelname)s - %(message)s",
    },
    "data": {
        "name": "IllustrisAPI",
        "args": {
            "api_key": os.environ.get("ILLUSTRIS_API_KEY"),
            "particle_type": ["stars"],
            "simulation": "TNG50-1",
            "snapshot": 99,
            "save_data_path": "data",
        },
        
        "load_galaxy_args": {
        "id": 12,
        "reuse": True,
        },

        "subset": {
            "use_subset": True,
            "subset_size": 1000,
        },
    },
    "simulation": {
        "name": "IllustrisTNG",
        "args": {
            "path": "data/galaxy-id-12.hdf5",
        },
    
    },
    "output_path": "output",

    "telescope":
        {"name": "MUSE",
         "psf": {"name": "gaussian", "size": 5, "sigma": 0.6},
         "lsf": {"sigma": 0.5},
         "noise": {"signal_to_noise": 1,"noise_distribution": "normal"},},
        
    "cosmology":
        {"name": "PLANCK15"},
        
    "galaxy":
        {"dist_z": 0.1,
         "rotation": {"type": "edge-on"},
        },
    "ssp": {
        "template": {
            "name": "BruzualCharlot2003"
        },
    },    
}

Step 2: RUBIX data format

First, we have to download the simulation data from the Illustris webpage and store it and transform it to the rubixdata format. The rubixdata format is a unige format for the pipeline. Any simulated galaxy can be transformed in the rubixdata format, which enables RUBIX to deal with all kind of cosmological hydrodynamical simulations of galaxies. For more deatails of this step, please have a look in the notebook create_rubix_data.ipynb.

# NBVAL_SKIP
from rubix.core.data import convert_to_rubix, prepare_input

convert_to_rubix(config) # Convert the config to rubix format and store in output_path folder
rubixdata = prepare_input(config) # Prepare the input for the pipeline

2025-11-10 17:15:54,846 - rubix - INFO - 
   ___  __  _____  _____  __
  / _ \/ / / / _ )/  _/ |/_/
 / , _/ /_/ / _  |/ /_>  <
/_/|_|\____/____/___/_/|_|


2025-11-10 17:15:54,847 - rubix - INFO - Rubix version: 0.0.post626+g42b4b7505.d20251110
2025-11-10 17:15:54,848 - rubix - INFO - JAX version: 0.7.2
2025-11-10 17:15:54,848 - rubix - INFO - Running on [CpuDevice(id=0)] devices
2025-11-10 17:15:54,849 - rubix - INFO - Rubix galaxy file already exists, skipping conversion
/home/annalena/.conda/envs/rubix/lib/python3.12/site-packages/jax/_src/numpy/scalar_types.py:50: UserWarning: Explicitly requested dtype float64 requested in asarray is not available, and will be truncated to dtype float32. To enable more dtypes, set the jax_enable_x64 configuration option or the JAX_ENABLE_X64 shell environment variable. See https://github.com/jax-ml/jax#current-gotchas for more.
  return asarray(x, dtype=self.dtype)
/home/annalena/.conda/envs/rubix/lib/python3.12/site-packages/rubix/core/data.py:491: UserWarning: Explicitly requested dtype <class 'jax.numpy.float64'> requested in array is not available, and will be truncated to dtype float32. To enable more dtypes, set the jax_enable_x64 configuration option or the JAX_ENABLE_X64 shell environment variable. See https://github.com/jax-ml/jax#current-gotchas for more.
  rubixdata.galaxy.center = jnp.array(data["subhalo_center"], dtype=jnp.float64)
2025-11-10 17:15:54,935 - rubix - INFO - Centering stars particles
2025-11-10 17:15:55,775 - rubix - WARNING - The Subset value is set in config. Using only subset of size 1000 for stars

You can simply access the data of the galaxy, e.g. the stellar coordinates by rubixdata.stars.coords.

# NBVAL_SKIP
import matplotlib.pyplot as plt
# Make a scatter plot of the stars coordinates
plt.scatter(rubixdata.stars.coords[:,0], rubixdata.stars.coords[:,1], s=1)

<matplotlib.collections.PathCollection at 0x7114241dcf80>

Step 3: Rotation

In the config we specify, how the galaxy should be orientated. In this example we want to orientate the galaxy edge-on. We plot the coordinates again and see that they are now rotated.

# NBVAL_SKIP
from rubix.core.rotation import get_galaxy_rotation
rotate = get_galaxy_rotation(config)

rubixdata = rotate(rubixdata)

2025-11-10 17:15:56,445 - rubix - DEBUG - Rotation Type found: edge-on
2025-11-10 17:15:56,446 - rubix - INFO - Rotating galaxy with alpha=90.0, beta=0.0, gamma=0.0
2025-11-10 17:15:56,447 - rubix - INFO - Rotating galaxy for simulation: IllustrisTNG
2025-11-10 17:15:56,447 - rubix - WARNING - Gas not found in particle_type, only rotating stellar component.

#NBVAL_SKIP
# Make a scatter plot of the stars coordinates after rotation
plt.scatter(rubixdata.stars.coords[:,0], rubixdata.stars.coords[:,1], s=1)

<matplotlib.collections.PathCollection at 0x7114241ee6f0>

Step 4: Filter particles

All particles outside field of view of the telescope are filtered. This has to be done, because we later bin the particles to the IFU grid and particles outside the arperture would make strange artefacts.

# NBVAL_SKIP
from rubix.core.telescope import get_filter_particles
filter_particles = get_filter_particles(config)

rubixdata = filter_particles(rubixdata)

2025-11-10 17:15:57,935 - rubix - INFO - Calculating spatial bin edges...

/home/annalena/.conda/envs/rubix/lib/python3.12/site-packages/rubix/telescope/factory.py:26: UserWarning: No telescope config provided, using default stored in /home/annalena/.conda/envs/rubix/lib/python3.12/site-packages/rubix/telescope/telescopes.yaml
  warnings.warn(
2025-11-10 17:15:58,094 - rubix - INFO - Getting cosmology...
2025-11-10 17:15:58,270 - rubix - INFO - Filtering particles outside the aperture...

Step 5: Spaxel assignment

We have an telescope aperture and a spatial resolution, which results in a spatial grid for the IFU cube. We can now assign the stars particles to the different spaxels in the IFU cube, i.e. define to which spaxel the stellar light of each stars particle contribute.

# NBVAL_SKIP
from rubix.core.telescope import get_spaxel_assignment
bin_particles = get_spaxel_assignment(config)

rubixdata = bin_particles(rubixdata)

2025-11-10 17:15:58,449 - rubix - INFO - Calculating spatial bin edges...
2025-11-10 17:15:58,460 - rubix - INFO - Getting cosmology...
2025-11-10 17:15:58,461 - rubix - INFO - Assigning particles to spaxels...

Step 6: Data cube calculation

This is the heart of the pipeline. Now we do the lookup for the spectrum for each stellar particle. For the simple stellar population model by BruzualCharlot2003, each stellar particle gets a spectrum assigned based on its age and metallicity.

We scale the stellar particle spectra by its mass. The stellar spectra have to be scaled by the stellar mass. Later heavier stellar particles should contribute more to the spectrum in a spaxel than lighter stellar particles.

The stellar particles are not at rest and therefore the emitted light is doppler shifted with respect to the observer. Before adding all stellar spectra in each spaxel, we dopplershift the spectra according to their particle velocity and we resample the spectra to the wavelength grid of the observing instrument.

Each stellar particle falls into one spaxel in the datacube. We ad the stellar particles spectra contribution to the according spaxel in the datacube. We do these steps for all atellar particles.

The first plot shows the spectra for two different spaxels. The second plot shows the spatial dimension of the datacube, where we summed over the wavelength dimension.

# NBVAL_SKIP
from rubix.core.ifu import get_calculate_datacube_particlewise
calculate_datacube_particlewise = get_calculate_datacube_particlewise(config)

rubixdata = calculate_datacube_particlewise(rubixdata)

2025-11-10 17:15:58,856 - rubix - WARNING - python-fsps is not installed. Please install it to use this function. Install using pip install fsps and check the installation page: https://dfm.io/python-fsps/current/installation/ for more details. Especially, make sure to set all necessary environment variables.

/home/annalena/.conda/envs/rubix/lib/python3.12/site-packages/rubix/telescope/factory.py:26: UserWarning: No telescope config provided, using default stored in /home/annalena/.conda/envs/rubix/lib/python3.12/site-packages/rubix/telescope/telescopes.yaml
  warnings.warn(
2025-11-10 17:15:58,915 - rubix - DEBUG - Method not defined, using default method: cubic
2025-11-10 17:15:58,954 - rubix - INFO - Calculating Data Cube (combined per‐particle)…
2025-11-10 17:15:59,865 - rubix - DEBUG - Datacube shape: (25, 25, 3721)

# NBVAL_SKIP
from rubix.core.pipeline import RubixPipeline 

pipe = RubixPipeline(config)

wave = pipe.telescope.wave_seq
print(wave)
print(rubixdata.stars.datacube[0][0][:])

plt.plot(wave, rubixdata.stars.datacube[12][12][:])
plt.plot(wave, rubixdata.stars.datacube[10][5][:])

/home/annalena/.conda/envs/rubix/lib/python3.12/site-packages/rubix/telescope/factory.py:26: UserWarning: No telescope config provided, using default stored in /home/annalena/.conda/envs/rubix/lib/python3.12/site-packages/rubix/telescope/telescopes.yaml
  warnings.warn(

[4700.15 4701.4  4702.65 ... 9347.65 9348.9  9350.15]
[0. 0. 0. ... 0. 0. 0.]

[<matplotlib.lines.Line2D at 0x7113c01f0710>]

# NBVAL_SKIP
datacube = rubixdata.stars.datacube
img = datacube.sum(axis=2)
plt.imshow(img, origin="lower")

<matplotlib.image.AxesImage at 0x7113e01f8560>

Step 7: PSF

The instrument and the earth athmosphere affect the spatial resolution of the observation data and smooth in spatial dimention. To take this effect into account we convolve our datacube with a point spread function (PSF).

# NBVAL_SKIP
from rubix.core.psf import get_convolve_psf
convolve_psf = get_convolve_psf(config)

rubixdata = convolve_psf(rubixdata)

2025-11-10 17:16:00,475 - rubix - INFO - Convolving with PSF...

# NBVAL_SKIP
datacube = rubixdata.stars.datacube
img = datacube.sum(axis=2)
plt.imshow(img, origin="lower")

<matplotlib.image.AxesImage at 0x7113c02554f0>

# NBVAL_SKIP
plt.plot(wave, datacube[12,12,:])
plt.plot(wave, datacube[10,5,:])

[<matplotlib.lines.Line2D at 0x7113800ff320>]

Step 8: LSF

The instrument affects the spectral resolution of the observation data and smooth in spectral dimention. To take this effect into account we convolve our datacube with a line spread function (LSF).

# NBVAL_SKIP
from rubix.core.lsf import get_convolve_lsf
convolve_lsf = get_convolve_lsf(config)

rubixdata = convolve_lsf(rubixdata)

plt.plot(wave, rubixdata.stars.datacube[12,12,:])
plt.plot(wave, rubixdata.stars.datacube[10,5,:])

2025-11-10 17:16:01,014 - rubix - INFO - Convolving with LSF...

[<matplotlib.lines.Line2D at 0x7113603bdd30>]

Step 9: Noise

Observational data are never noise-free. We apply noise to our mock-datacube to mimic real measurements.

# NBVAL_SKIP
from rubix.core.noise import get_apply_noise
apply_noise = get_apply_noise(config)

rubixdata = apply_noise(rubixdata)

datacube = rubixdata.stars.datacube
img = datacube.sum(axis=2)
plt.imshow(img, origin="lower")

2025-11-10 17:16:01,508 - rubix - INFO - Applying noise to datacube with signal to noise ratio: 1 and noise distribution: normal

<matplotlib.image.AxesImage at 0x7113601e6f60>

# NBVAL_SKIP
plt.plot(wave, rubixdata.stars.datacube[12,12,:])
plt.plot(wave, rubixdata.stars.datacube[10,5,:])

[<matplotlib.lines.Line2D at 0x71134034cd70>]

DONE!

Congratulations, you have now created step by step your own mock-observed IFU datacube! Now enjoy playing around with the RUBIX pipeline and enjoy doing amazing science with RUBIX :)

Store datacube in a fits file with header

Keep in mind that this it the luminosity cube. If you want to have a flux cube, you have to convert it. You can do this with the rubix.spectra.ifu.convert_luminoisty_to_flux function.

# NBVAL_SKIP
from rubix.core.fits import store_fits

store_fits(config, rubixdata.stars.datacube, "./output/")

/home/annalena/.conda/envs/rubix/lib/python3.12/site-packages/rubix/telescope/factory.py:26: UserWarning: No telescope config provided, using default stored in /home/annalena/.conda/envs/rubix/lib/python3.12/site-packages/rubix/telescope/telescopes.yaml
  warnings.warn(
2025-11-10 17:16:03,252 - rubix - INFO - Datacube saved to ./output/IllustrisTNG_id12_snap99_MUSE_calc_ifu.fits

Visualisation of the datacube

We show, how you can visualize the datacube and see the image and spectra and explore the datacube. We show our own build tool to visualize the datacube and second we provide the opportunity to load the datacube with the Cubeviz module from jdaviz.

visualize_rubix uses mpdaf to load the datacube. This is a package specialized to load MUSE datacubes. The function will thisplay you on the left an image collapsed along the wavelength and on the right a spectrum for a certain pixel or aperture.

Explanation of the sliders:

• Waveindex: Waveindex, which wavelength slice is plotted in the image.

• Wavelengthrange: Range in wavelength that is collapsed to the image.

• X Pixel: X coordinate of the displayed spectrum and x coordinate of the center of the aperture.

• Y Pixel: Y coordinate of the displayed spectrum and y coordinate of the center of the aperture.

• Radius: size of the circular aperture in pixels. If this value is set to zerro, only the spaxel specified in the x and y pixel is considered for the spectrum plot.

Now you can explore your datacube with the sliders!

# NBVAL_SKIP
from rubix.core.visualisation import visualize_rubix

#visualize_rubix("./output/IllustrisTNG_id11_snap99_stars_subsetTrue.fits")