Overhall CLI

.github/workflows/process.yml

@@ -190,10 +190,12 @@ jobs:
           name: tracked-mfiles
           path: tracking/
       - run: mv tracking/large_tokamak_nof.SIG_TF.json tracking/large_tokamak_nof_SIG_TF.json
-      - name: Create an example plot proc
-        run: hatch run python process/core/io/plot_proc.py -f tracking/large_tokamak_nof_MFILE.DAT
-      - name: Move plot proc file to docs images
-        run: mv tracking/large_tokamak_nof_MFILE.DATSUMMARY.pdf documentation/source/images/plot_proc.pdf
+      - name: Create an example plot summary
+        run: |
+          hatch shell
+          process plot summary -f tracking/large_tokamak_nof_MFILE.DAT
+      - name: Move plot summary file to docs images
+        run: mv tracking/large_tokamak_nof_MFILE.DATSUMMARY.pdf documentation/source/images/plot_summary.pdf
       - run: hatch run docs:build
       - name: Download tracking html
         uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c

documentation/source/eng-models/machine-build.md

@@ -2,7 +2,7 @@
 
 
 Simplified scale diagrams of the vertical and horizontal cross-sections of the machine can be
-output in the summary created by using the utility `plot_proc.py` (currently stored in `process/process/core/io`).  
+output in the summary created by using the utility `summary.py` (currently stored in `process/process/core/io/plot`).
 
 The coordinate system is $(R,Z)$ system, where $R$ is the radial distance from the vertical
 centreline (axis) of the torus, and $Z$ is the vertical distance from the equatorial midplane.

documentation/source/io/output-guide.md

@@ -12,9 +12,6 @@ most of the same data as `OUT.DAT` but in a different format and has been design
 "machine-readable" by some of the [utility programs](utilities.md) for 
 post-processing and graphical output.
 
-An additional diagnostic output text file `VFILE.DAT` is generated if the `verbose` flag is set 
-to 1 in the input file.   
-
 (If there is already a file with any of the above filenames it will be overwritten.)
 
 Utilities generate additional output files.

documentation/source/io/python-lib-guide.md

@@ -1,6 +1,6 @@
 ## Library for `IN.DAT` Files
 
-> `process/core/io/in_dat.py`
+> `process/core/io/in_dat`
 
 A set of Python classes to read, modify and write an `IN.DAT` file. 
 
@@ -64,7 +64,7 @@ i.write_in_dat(output_filename="new_IN.DAT")
 
 ## Library for `MFILE.DAT` Files
 
-> `process/core/io/mfile.py`
+> `process/core/io/mfile`
 
 
 A set of Python classes to read and extract data from the `MFILE.DAT`.

documentation/source/usage/plotting.md

@@ -7,12 +7,12 @@
     The majority of the output plots can be automatically generated at the end of the `PROCESS` run by using the `--full-output` flag.
     See the [running process](./running-process.md) section for more info.
 
-### Summary document | `plot_proc.py`
+### Summary document
 
-`plot_proc` is used for plotting an overview of the results from an MFILE. It can be run using its own CLI (see [here](https://ukaea.github.io/process/core/io/utilities/) for full details):
+`plot summary` is used for plotting an overview of the results from an MFILE. It can be run using its own CLI (see [here](https://ukaea.github.io/process/core/io/utilities/) for full details):
 
 ```bash
-python process/core/io/plot_proc.py -f path/to/MFILE.DAT
+process plot summary -f path/to/MFILE.DAT
 ```
 
 or through PROCESS's main CLI:
@@ -21,8 +21,8 @@ or through PROCESS's main CLI:
 process -i path/to/IN.DAT --full-output
 ``` 
 
-An example of a plot proc output PDF for the large tokamak regression test is shown below:
-<embed src="../../images/plot_proc.pdf" type="application/pdf" width="100%" height="600">
+An example of a plot summary output PDF for the large tokamak regression test is shown below:
+<embed src="../../images/plot_summary.pdf" type="application/pdf" width="100%" height="600">
 
 ----------------
 
@@ -32,7 +32,7 @@ An example of a plot proc output PDF for the large tokamak regression test is sh
 Scans can be done in one or two dimensions.
 
 ```bash
-python process/core/io/plot_scans.py -f path/to/MFILE.DAT
+process plot scans -f path/to/MFILE.DAT
 ```
 <figure markdown>
 ![2D_contour_plot](../images/2D_contour_plot_example.png){figures-side, width="100%"}  
@@ -51,7 +51,7 @@ python process/core/io/plot_scans.py -f path/to/MFILE.DAT
 `plot_plotly_sankey` is to plot an interactive `.html` Sankey diagram for looking at the plants power balance. It can be run as follows:
 
 ```bash
-python process/core/io/plot_plotly_sankey.py -m path/to/MFILE.DAT
+process plot sankey -m path/to/MFILE.DAT --format html
 ```
 
 <figure markdown>

examples/introduction.ex.py

@@ -63,17 +63,15 @@
 
 # %% [markdown]
 # ## Plot summary
-# Create a summary of the generated `MFILE.DAT` using `plot_proc`.
+# Create a summary of the generated `MFILE.DAT` using `plot_summary`.
 #
-# You can also call `plot_proc` from the cli with `python -m process.core.io.plot_proc`
+# You can also call `plot_summary` from the cli with `process plot summary`
 
 # %%
-from process.core.io import plot_proc
+from process.core.io.plot import plot_summary
 
 # Pdf and png output are also available
-plot_proc.main(
-    args=["-f", single_run.mfile_path.as_posix(), "--output-format", "none", "--show"]
-)
+plot_summary(single_run.mfile_path, output_format="none", show=True)
 
 # %% [markdown]
 # ## View key output variables

examples/optimum_solutions_comparison.ex.py

@@ -28,7 +28,7 @@
 # %% [markdown]
 # This notebook demonstrates how the optimum solution found by `PROCESS` changes as we vary input parameters. We will use the large tokamak example input file to do this. The figure of merit for this example is to minimise the major radius, `rmajor`.
 #
-# We use the functionality from `plot_solutions.py` and from `plot_proc.py` to demonstrate this.
+# We use the functionality from `plot.solutions` and from `plot.summary` to demonstrate this.
 #
 # These tools plot the solution vectors (i.e. final values of optimisation parameters) for different runs of `PROCESS`. This allows visual comparisons of different solution points.
 #
@@ -54,7 +54,7 @@
 import tempfile
 from pathlib import Path
 
-from process.core.io.plot_solutions import (
+from process.core.io.plot.solutions import (
     RunMetadata,
     plot_mfile_solutions,
 )
@@ -140,7 +140,7 @@
 import matplotlib.pyplot as plt
 
 import process.core.io.mfile as mf
-from process.core.io.plot_proc import plot_inequality_constraint_equations
+from process.core.io.plot.summary import plot_inequality_constraint_equations
 
 original_mfile = mf.MFile((large_tokamak_mfile).as_posix())
 new_mfile = mf.MFile((large_tokamak_varied_min_net_electric_mfile).as_posix())

examples/scan.ex.py

@@ -48,19 +48,20 @@
 
 # %% slideshow={"slide_type": "subslide"}
 # %matplotlib inline
-from process.core.io import plot_scans
+from process.core.io.plot.scans import plot_scan
 
 # Define working directory relative to project dir and input file name
 mfile_name = data_dir / "scan_example_file_MFILE.DAT"
 output_dir = data_dir
 
-plot_scans.main(
-    args=[
-        "-f",
-        str(mfile_name),
-        "-yv",
-        "b_plasma_toroidal_on_axis rmajor p_plant_electric_net_mw p_fusion_total_mw capcost",
-        "--outputdir",
-        str(output_dir),
-    ]
+plot_scan(
+    mfile_name,
+    outputdir=output_dir,
+    output_names=[
+        "b_plasma_toroidal_on_axis",
+        "rmajor",
+        "p_plant_electric_net_mw",
+        "p_fusion_total_mw",
+        "capcost",
+    ],
 )

examples/single_model_evaluation.ex.py

@@ -22,7 +22,7 @@
 import matplotlib.pyplot as plt
 import numpy as np
 
-import process
+from process import data_structure
 from process.main import SingleRun
 
 # %% [markdown]
@@ -31,7 +31,7 @@
 
 # %%
 print(
-    f"p_plasma_separatrix_mw = {process.data_structure.physics_variables.p_plasma_separatrix_mw}"
+    f"p_plasma_separatrix_mw = {data_structure.physics_variables.p_plasma_separatrix_mw}"
 )
 
 # %% [markdown]
@@ -51,13 +51,11 @@
 # Print initial values of interest
 def print_values():
     print(
-        f"W frac = {process.data_structure.impurity_radiation_module.f_nd_impurity_electron_array[13]:.3e}"
+        f"W frac = {data_structure.impurity_radiation_module.f_nd_impurity_electron_array[13]:.3e}"
     )
+    print(f"p_plasma_rad_mw = {data_structure.physics_variables.p_plasma_rad_mw:.3e}")
     print(
-        f"p_plasma_rad_mw = {process.data_structure.physics_variables.p_plasma_rad_mw:.3e}"
-    )
-    print(
-        f"p_plasma_separatrix_mw = {process.data_structure.physics_variables.p_plasma_separatrix_mw:.3e}"
+        f"p_plasma_separatrix_mw = {data_structure.physics_variables.p_plasma_separatrix_mw:.3e}"
     )
 
 
@@ -67,9 +65,7 @@ def print_values():
 # Now try increasing the tungsten impurity fraction to see if there's a change in the divertor power.
 
 # %%
-process.data_structure.impurity_radiation_module.f_nd_impurity_electron_array[13] = (
-    5.0e-5
-)
+data_structure.impurity_radiation_module.f_nd_impurity_electron_array[13] = 5.0e-5
 single_run.models.physics.run()
 print_values()
 
@@ -94,23 +90,21 @@ def run_impurities(w_imp_fracs):
     # Loop over W impurity values, evaluate model and store responses at each point
     for i, imp_frac in enumerate(w_imp_fracs):
         # Set W impurity fraction, then run physics model
-        process.data_structure.impurity_radiation_module.f_nd_impurity_electron_array[
-            13
-        ] = imp_frac
+        data_structure.impurity_radiation_module.f_nd_impurity_electron_array[13] = (
+            imp_frac
+        )
         single_run.models.physics.run()
 
         # Evaluate constraint equation 15 (L-H threshold constraint)
         con15_value = ConstraintManager.evaluate_constraint(15).normalised_residual
 
         # Need to copy values
-        p_plasma_rad_mw[i] = (
-            process.data_structure.physics_variables.p_plasma_rad_mw.item()
-        )
+        p_plasma_rad_mw[i] = data_structure.physics_variables.p_plasma_rad_mw.item()
         p_plasma_separatrix_mw[i] = (
-            process.data_structure.physics_variables.p_plasma_separatrix_mw.item()
+            data_structure.physics_variables.p_plasma_separatrix_mw.item()
         )
         p_l_h_threshold_mw[i] = (
-            process.data_structure.physics_variables.p_l_h_threshold_mw.item()
+            data_structure.physics_variables.p_l_h_threshold_mw.item()
         )
         # Need to flip sign of constraint so negative means violated
         con15[i] = -con15_value

examples/vary_run_example.ex.py

@@ -82,7 +82,7 @@
 import tempfile
 from pathlib import Path
 
-from process.core.io.mfile_utils import get_mfile_initial_ixc_values
+from process.core.io.mfile import get_mfile_initial_ixc_values
 from process.core.repository import get_process_root
 from process.main import VaryRun
 

process/core/caller.py

@@ -7,12 +7,12 @@
 import numpy as np
 from tabulate import tabulate
 
-import process.core.solver.constraints as constraints
 from process import data_structure
 from process.core import constants
 from process.core.final import finalise
 from process.core.io.mfile import MFile
 from process.core.process_output import OutputFileManager, ovarre
+from process.core.solver import constraints
 from process.core.solver.iteration_variables import set_scaled_iteration_variable
 from process.core.solver.objectives import objective_function
 from process.models.tfcoil.base import TFConductorModel

process/core/final.py

@@ -2,10 +2,10 @@
 
 from tabulate import tabulate
 
-import process.core.solver.constraints as constraints
 from process.core import constants
 from process.core import output as op
 from process.core import process_output as po
+from process.core.solver import constraints
 from process.core.solver.objectives import objective_function
 from process.data_structure import numerics
 

process/core/init.py

@@ -9,13 +9,12 @@
 from warnings import warn
 
 import process
-import process.core.process_output as process_output
-import process.core.solver.iteration_variables as iteration_variables
 from process import data_structure
-from process.core import constants
+from process.core import constants, process_output
 from process.core.exceptions import ProcessValidationError
 from process.core.input import parse_input_file
 from process.core.log import logging_model_handler
+from process.core.solver import iteration_variables
 from process.core.solver.constraints import ConstraintManager
 from process.data_structure.blanket_library import init_blanket_library
 from process.data_structure.build_variables import init_build_variables

process/core/input.py

@@ -11,7 +11,7 @@
 from warnings import warn
 
 import process
-import process.data_structure as data_structure
+from process import data_structure
 from process.core.exceptions import ProcessValidationError, ProcessValueError
 from process.core.solver.constraints import ConstraintManager
 

process/core/io/cli_tools.py

@@ -0,0 +1,79 @@
+import importlib
+from pathlib import Path
+
+import click
+
+help_opt = click.help_option("-h", "--help")
+scan_opt = click.option("--scan", type=int, help="Scan to select")
+mfile_arg = click.argument(
+    "mfiles", nargs=-1, type=click.Path(exists=True, path_type=Path)
+)
+
+
+def mfile_opt(exists: bool = False):
+    return click.option(
+        "-f",
+        "--mfile",
+        "mfile",
+        default="MFILE.DAT",
+        type=click.Path(exists=exists, path_type=Path),
+        help="The mfile to read",
+    )
+
+
+def indat_opt(default="IN.DAT", exists=True):
+    return click.option(
+        "-i",
+        "--input",
+        "indat",
+        type=click.Path(exists, path_type=Path),
+        help="The path to the input file",
+        default=default,
+    )
+
+
+def save(help_):
+    return click.option("-s", "--save", "save", default=False, is_flag=True, help=help_)
+
+
+def split_callback(ctx: click.Context, param, value: str | None) -> list[str] | None:  # noqa: ARG001
+    return value.replace(" ", ":").split(":") if isinstance(value, str) else value
+
+
+### Taken from click documentation
+class LazyGroup(click.Group):
+    def __init__(self, *args, lazy_subcommands=None, **kwargs):
+        super().__init__(*args, **kwargs)
+        # lazy_subcommands is a map of the form:
+        #
+        #   {command-name} -> {module-name}.{command-object-name}
+        #
+        self.lazy_subcommands = lazy_subcommands or {}
+
+    def list_commands(self, ctx):
+        base = super().list_commands(ctx)
+        lazy = sorted(self.lazy_subcommands.keys())
+        return sorted(base + lazy)
+
+    def get_command(self, ctx, cmd_name):
+        if cmd_name in self.lazy_subcommands:
+            return self._lazy_load(cmd_name)
+        return super().get_command(ctx, cmd_name)
+
+    def _lazy_load(self, cmd_name):
+        # lazily loading a command, first get the module name and attribute name
+        import_path = self.lazy_subcommands[cmd_name]
+        modname, cmd_object_name = import_path.rsplit(".", 1)
+        # do the import
+        mod = importlib.import_module(modname)
+        # get the Command object from that module
+        cmd_object = getattr(mod, cmd_object_name)
+        # check the result to make debugging easier
+        if not isinstance(cmd_object, click.Command):
+            raise ValueError(
+                f"Lazy loading of {import_path} failed by returning a non-command object"
+            )
+        return cmd_object
+
+
+###