Skip to content

Segger fails on compute environment #45

@rcannood

Description

@rcannood

On a gpu enabled node:

/etc/bash.bashrc: line 9: PS1: unbound variable

A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.2.6 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.

If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.

Traceback (most recent call last):  File "/scratch/3p2VK70UaoOHsN/c0/a5d4209de158d717c4507b3d14daa6/.viash_script.py", line 13, in <module>
    import torch
  File "/usr/local/lib/python3.12/dist-packages/torch/__init__.py", line 2248, in <module>
    from torch import quantization as quantization  # usort: skip
  File "/usr/local/lib/python3.12/dist-packages/torch/quantization/__init__.py", line 2, in <module>
    from .fake_quantize import *  # noqa: F403
  File "/usr/local/lib/python3.12/dist-packages/torch/quantization/fake_quantize.py", line 10, in <module>
    from torch.ao.quantization.fake_quantize import (
  File "/usr/local/lib/python3.12/dist-packages/torch/ao/quantization/__init__.py", line 12, in <module>
    from .pt2e._numeric_debugger import (  # noqa: F401
  File "/usr/local/lib/python3.12/dist-packages/torch/ao/quantization/pt2e/_numeric_debugger.py", line 9, in <module>
    from torch.ao.quantization.pt2e.graph_utils import bfs_trace_with_node_process
  File "/usr/local/lib/python3.12/dist-packages/torch/ao/quantization/pt2e/graph_utils.py", line 9, in <module>
    from torch.export import ExportedProgram
  File "/usr/local/lib/python3.12/dist-packages/torch/export/__init__.py", line 60, in <module>
    from .decomp_utils import CustomDecompTable
  File "/usr/local/lib/python3.12/dist-packages/torch/export/decomp_utils.py", line 5, in <module>
    from torch._export.utils import (
  File "/usr/local/lib/python3.12/dist-packages/torch/_export/__init__.py", line 48, in <module>
    from .wrappers import _wrap_submodules
  File "/usr/local/lib/python3.12/dist-packages/torch/_export/wrappers.py", line 7, in <module>
    from torch._higher_order_ops.flat_apply import (
  File "/usr/local/lib/python3.12/dist-packages/torch/_higher_order_ops/__init__.py", line 1, in <module>
    from torch._higher_order_ops._invoke_quant import (
  File "/usr/local/lib/python3.12/dist-packages/torch/_higher_order_ops/_invoke_quant.py", line 8, in <module>
    from torch._higher_order_ops.base_hop import BaseHOP, FunctionWithNoFreeVars
  File "/usr/local/lib/python3.12/dist-packages/torch/_higher_order_ops/base_hop.py", line 12, in <module>
    from torch._subclasses.functional_tensor import disable_functional_mode
  File "/usr/local/lib/python3.12/dist-packages/torch/_subclasses/functional_tensor.py", line 46, in <module>
    class FunctionalTensor(torch.Tensor):
  File "/usr/local/lib/python3.12/dist-packages/torch/_subclasses/functional_tensor.py", line 279, in FunctionalTensor
    cpu = _conversion_method_template(device=torch.device("cpu"))
/usr/local/lib/python3.12/dist-packages/torch/_subclasses/functional_tensor.py:279: UserWarning: Failed to initialize NumPy: 
A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.2.6 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.

If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.

 (Triggered internally at /opt/pytorch/pytorch/torch/csrc/utils/tensor_numpy.cpp:81.)
  cpu = _conversion_method_template(device=torch.device("cpu"))
Using device: cuda
Reading input: _viash_par/input_1/spatial_unlabelled.zarr
Input:  SpatialData object, with associated Zarr store: /scratch/3p2VK70UaoOHsN/stage-14b82b41-b3e7-4a30-9b6d-1b4df3c97782/c7/d378169abc01fdf607d59f34830514/spatial_unlabelled.zarr
├── Images
│     └── 'image': DataTree[cyx] (1, 1000, 1000), (1, 500, 500), (1, 250, 250), (1, 125, 125), (1, 62, 62)
├── Labels
│     ├── 'cell_labels': DataTree[yx] (1000, 1000), (500, 500), (250, 250), (125, 125), (62, 62)
│     └── 'nucleus_labels': DataTree[yx] (1000, 1000), (500, 500), (250, 250), (125, 125), (62, 62)
├── Points
│     └── 'transcripts': DataFrame with shape: (<Delayed>, 8) (3D points)
└── Tables
      └── 'table': AnnData (0, 424)
with coordinate systems:
    ▸ 'global', with elements:
        image (Images), cell_labels (Labels), nucleus_labels (Labels), transcripts (Points)
Init segmentation mode: None
Traceback (most recent call last):
  File "/scratch/3p2VK70UaoOHsN/c0/a5d4209de158d717c4507b3d14daa6/.viash_script.py", line 339, in <module>
    raise ValueError(f"Unknown init_segmentation mode: {mode}")
ValueError: Unknown init_segmentation mode: None

task-10.command.out.txt
task-10.command.err.txt
task-10.command.log.txt

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions