3.5. Himawari

This tutorial/demo provides a quick and dirty example of how THUNER can be applied to Himawari observations.

3.5.1. Setup

 1%load_ext autoreload
 2%autoreload 2
 3
 4%matplotlib inline
 5
 6import shutil
 7import numpy as np
 8import thuner.data as data
 9import thuner.option as option
10import thuner.track.track as track
11import thuner.visualize as visualize
12import thuner.analyze as analyze
13import thuner.default as default
14import thuner.parallel as parallel
15import thuner.utils as utils
16import thuner.config as config

The autoreload extension is already loaded. To reload it, use:
  %reload_ext autoreload

 1# Set a flag for whether or not to remove existing output directories
 2remove_existing_outputs = False
 3
 4# Specify the local base directory for saving outputs
 5base_local = config.get_outputs_directory()
 6
 7output_parent = base_local / "runs/himawari/"
 8options_directory = output_parent / "options"
 9visualize_directory = output_parent / "visualize"
10
11# Remove the output parent directory if it already exists
12if output_parent.exists() and remove_existing_outputs:
13    shutil.rmtree(output_parent)

Run the cell below to get the demo data for this tutorial, if you haven’t already.

1# Download the demo data
2remote_directory = "s3://thuner-storage/THUNER_output/input_data/raw/satellite-products"
3data.get_demo_data(base_local, remote_directory)

2025-07-09 19:18:47,562 - thuner.data._utils - INFO - Syncing directory /home/ewan/THUNER_output/input_data/raw/satellite-products. Please wait.

3.5.2. Options

 1# Create the dataset options
 2start = "2023-01-01T00:00:00"
 3# Note the CPOL times are usually a few seconds off the 10 m interval, so add 30 seconds
 4# to ensure we capture 19:00:00
 5end = "2023-01-02T00:00:00"
 6times_dict = {"start": start, "end": end}
 7himawari_options = data.himawari.HimawariOptions(**times_dict)
 8data_options = option.data.DataOptions(datasets=[himawari_options])
 9data_options.to_yaml(options_directory / "data.yml")
10
11# Setup a grid over New Guinea.
12# Note the demo data contains the full disk, so vary the lat/lon as you like!
13spacing = [0.025, 0.025]
14latitude = np.arange(-10, 0 + spacing[0], spacing[0])
15longitude = np.arange(130, 150 + spacing[1], spacing[1])
16altitude = None
17grid_options = option.grid.GridOptions(
18    name="geographic", latitude=latitude, longitude=longitude, altitude=altitude
19)
20grid_options.to_yaml(options_directory / "grid.yml")
21
22# Create the track_options
23track_options = default.satellite_track(dataset_name="himawari")
24track_options.to_yaml(options_directory / "track.yml")

2025-07-09 17:20:14,108 - thuner.data.himawari - INFO - Generating Himawari filepaths.
2025-07-09 17:20:14,119 - thuner.data.himawari - INFO - Generating coordinates filepath.
2025-07-09 17:20:14,157 - thuner.option.grid - WARNING - altitude not specified. Using default altitudes.
2025-07-09 17:20:14,158 - thuner.option.grid - WARNING - shape not specified. Will attempt to infer from input.

3.5.3. Track

1times = utils.generate_times(data_options.dataset_by_name("himawari").filepaths)
2args = [times, data_options, grid_options, track_options]
3parallel.track(*args, output_directory=output_parent, dataset_name="himawari", num_processes=2)
4# track.track(*args, output_directory=output_parent)

2025-07-09 17:20:24,849 - thuner.parallel - INFO - Beginning parallel tracking with 2 processes.
2025-07-09 17:20:31,071 - thuner.track.track - INFO - Beginning thuner tracking. Saving output to /home/ewan/THUNER_output/runs/himawari/interval_0.
2025-07-09 17:20:31,072 - thuner.track.track - INFO - Beginning thuner tracking. Saving output to /home/ewan/THUNER_output/runs/himawari/interval_1.
2025-07-09 17:20:31,679 - thuner.track.track - INFO - Processing 2023-01-01T12:00:00.
2025-07-09 17:20:31,680 - thuner.utils - INFO - Updating himawari input record for 2023-01-01T12:00:00.
2025-07-09 17:20:31,680 - thuner.data.himawari - INFO - Converting himawari dataset for time 2023-01-01T12:00:00.
2025-07-09 17:20:31,685 - thuner.track.track - INFO - Processing 2023-01-01T00:00:00.
2025-07-09 17:20:31,686 - thuner.utils - INFO - Updating himawari input record for 2023-01-01T00:00:00.
2025-07-09 17:20:31,686 - thuner.data.himawari - INFO - Converting himawari dataset for time 2023-01-01T00:00:00.
2025-07-09 17:20:36,465 - thuner.data.himawari - INFO - Regridding Himawari data.
2025-07-09 17:20:36,467 - thuner.data._utils - INFO - Building regridder; this can take a while for large grids.
2025-07-09 17:20:36,521 - thuner.data.himawari - INFO - Regridding Himawari data.
2025-07-09 17:20:36,522 - thuner.data._utils - INFO - Building regridder; this can take a while for large grids.
2025-07-09 17:21:00,395 - thuner.track.track - INFO - Processing hierarchy level 0.
2025-07-09 17:21:00,395 - thuner.track.track - INFO - Processing hierarchy level 0.
...

3.5.4. Analyze/Visualize

1analysis_options = analyze.mcs.AnalysisOptions()
2analysis_options.to_yaml(options_directory / "analysis.yml")
3core_filepath = output_parent / "attributes/anvil/core.csv"
4analyze.utils.smooth_flow_velocities(core_filepath, output_parent)
5analyze.utils.quality_control("anvil", output_parent, analysis_options)

1style = "presentation"
2attribute_handlers = default.detected_attribute_handlers(output_parent, style)
3kwargs = {"name": "himawari_anvil", "object_name": "anvil", "style": style}
4kwargs.update({"attribute_handlers": attribute_handlers})
5figure_options = option.visualize.HorizontalAttributeOptions(**kwargs)
6args = [output_parent, start, end, figure_options, "himawari"]
7args_dict = {"parallel_figure": True, "by_date": False, "num_processes": 4}
8visualize.attribute.series(*args, **args_dict)

2025-07-09 17:28:04,167 - thuner.option.grid - WARNING - shape not specified. Will attempt to infer from input.
2025-07-09 17:28:04,289 - thuner.visualize.attribute - INFO - Visualizing attributes at time 2023-01-01T00:00:00.000000000.
2025-07-09 17:28:06,194 - thuner.data.himawari - INFO - Converting himawari dataset for time 2023-01-01T00:00:00.
2025-07-09 17:28:09,755 - thuner.data.himawari - INFO - Regridding Himawari data.
2025-07-09 17:28:09,758 - thuner.data._utils - INFO - Loading regridder from file.
2025-07-09 17:28:14,670 - thuner.visualize.attribute - INFO - Saving himawari_anvil figure for 2023-01-01T00:00:00.000000000.
2025-07-09 17:28:26,831 - thuner.visualize.attribute - INFO - Visualizing attributes at time 2023-01-01T00:30:00.000000000.
2025-07-09 17:28:26,837 - thuner.visualize.attribute - INFO - Visualizing attributes at time 2023-01-01T00:20:00.000000000.
2025-07-09 17:28:26,837 - thuner.visualize.attribute - INFO - Visualizing attributes at time 2023-01-01T00:10:00.000000000.
2025-07-09 17:28:28,502 - thuner.data.himawari - INFO - Converting himawari dataset for time 2023-01-01T00:30:00.
2025-07-09 17:28:28,705 - thuner.visualize.attribute - INFO - Visualizing attributes at time 2023-01-01T00:40:00.000000000.
2025-07-09 17:28:28,725 - thuner.data.himawari - INFO - Converting himawari dataset for time 2023-01-01T00:10:00.
2025-07-09 17:28:28,749 - thuner.data.himawari - INFO - Converting himawari dataset for time 2023-01-01T00:20:00.
2025-07-09 17:28:32,860 - thuner.data.himawari - INFO - Converting himawari dataset for time 2023-01-01T00:40:00.
2025-07-09 17:28:36,140 - thuner.data.himawari - INFO - Regridding Himawari data.
...