Skip to content

[riscv64]:inductor/test_compile_worker.py::TestCompileWorker::test_shutdown_terminates_sidecar_worker_pool" #24

Closed as not planned

Description

@yuzibo

🐛 Describe the bug

Here is one case failed on riscv64 now:

"inductor/test_compile_worker.py::TestCompileWorker::test_shutdown_terminates_sidecar_worker_pool",

More log is here:

===== shard_8_ci.log =====

=================================== FAILURES ===================================
________ TestCompileWorker.test_shutdown_terminates_sidecar_worker_pool ________
Traceback (most recent call last):
  File "/home/jenkins/pytorch/test/inductor/test_compile_worker.py", line 144, in test_shutdown_terminates_sidecar_worker_pool
    result = subprocess.run(
        [sys.executable, "-c", code],
    ...<3 lines>...
        timeout=20,
    )
  File "/usr/lib/python3.13/subprocess.py", line 556, in run
    stdout, stderr = process.communicate(input, timeout=timeout)
                     ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/subprocess.py", line 1222, in communicate
    stdout, stderr = self._communicate(input, endtime, timeout)
                     ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/subprocess.py", line 2129, in _communicate
    self._check_timeout(endtime, orig_timeout, stdout, stderr)
    ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/subprocess.py", line 1269, in _check_timeout
    raise TimeoutExpired(
    ...<2 lines>...
            stderr=b''.join(stderr_seq) if stderr_seq else None)
subprocess.TimeoutExpired: Command '['/home/jenkins/pip/bin/python', '-c', '\nimport operator\nimport subprocess\nimport time\n\nfrom torch._inductor.compile_worker.subproc_pool import SubprocPool\n\npool = SubprocPool(2)\nassert pool.submit(operator.add, 1, 2).result() == 3\npool.submit(time.sleep, 5)\ntime.sleep(0.5)\n\nwait = pool.process.wait\n\ndef short_wait(timeout=None):\n    return wait(timeout=2)\n\npool.process.wait = short_wait\n\ntry:\n    pool.shutdown()\nexcept subprocess.TimeoutExpired:\n    pool.process.kill()\n    pool.process.wait()\n    raise\n\nprint("shutdown returned")\n']' timed out after 20 seconds
=========================== short test summary info ============================
FAILED [20.1359s] inductor/test_compile_worker.py::TestCompileWorker::test_shutdown_terminates_sidecar_worker_pool
!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!

===== shard_8_ci.log =====

=================================== FAILURES ===================================
________ TestCompileWorker.test_shutdown_terminates_sidecar_worker_pool ________
Traceback (most recent call last):
  File "/home/jenkins/pytorch/test/inductor/test_compile_worker.py", line 144, in test_shutdown_terminates_sidecar_worker_pool
    result = subprocess.run(
        [sys.executable, "-c", code],
    ...<3 lines>...
        timeout=20,
    )
  File "/usr/lib/python3.13/subprocess.py", line 556, in run
    stdout, stderr = process.communicate(input, timeout=timeout)
                     ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/subprocess.py", line 1222, in communicate
    stdout, stderr = self._communicate(input, endtime, timeout)
                     ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/subprocess.py", line 2129, in _communicate
    self._check_timeout(endtime, orig_timeout, stdout, stderr)
    ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/subprocess.py", line 1269, in _check_timeout
    raise TimeoutExpired(
    ...<2 lines>...
            stderr=b''.join(stderr_seq) if stderr_seq else None)
subprocess.TimeoutExpired: Command '['/home/jenkins/pip/bin/python', '-c', '\nimport operator\nimport subprocess\nimport time\n\nfrom torch._inductor.compile_worker.subproc_pool import SubprocPool\n\npool = SubprocPool(2)\nassert pool.submit(operator.add, 1, 2).result() == 3\npool.submit(time.sleep, 5)\ntime.sleep(0.5)\n\nwait = pool.process.wait\n\ndef short_wait(timeout=None):\n    return wait(timeout=2)\n\npool.process.wait = short_wait\n\ntry:\n    pool.shutdown()\nexcept subprocess.TimeoutExpired:\n    pool.process.kill()\n    pool.process.wait()\n    raise\n\nprint("shutdown returned")\n']' timed out after 20 seconds
=========================== short test summary info ============================
FAILED [20.1337s] inductor/test_compile_worker.py::TestCompileWorker::test_shutdown_terminates_sidecar_worker_pool
!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!

===== shard_8_ci.log =====

=================================== FAILURES ===================================
________ TestCompileWorker.test_shutdown_terminates_sidecar_worker_pool ________
Traceback (most recent call last):
  File "/home/jenkins/pytorch/test/inductor/test_compile_worker.py", line 144, in test_shutdown_terminates_sidecar_worker_pool
    result = subprocess.run(
        [sys.executable, "-c", code],
    ...<3 lines>...
        timeout=20,
    )
  File "/usr/lib/python3.13/subprocess.py", line 556, in run
    stdout, stderr = process.communicate(input, timeout=timeout)
                     ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/subprocess.py", line 1222, in communicate
    stdout, stderr = self._communicate(input, endtime, timeout)
                     ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/subprocess.py", line 2129, in _communicate
    self._check_timeout(endtime, orig_timeout, stdout, stderr)
    ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/subprocess.py", line 1269, in _check_timeout
    raise TimeoutExpired(
    ...<2 lines>...
            stderr=b''.join(stderr_seq) if stderr_seq else None)
subprocess.TimeoutExpired: Command '['/home/jenkins/pip/bin/python', '-c', '\nimport operator\nimport subprocess\nimport time\n\nfrom torch._inductor.compile_worker.subproc_pool import SubprocPool\n\npool = SubprocPool(2)\nassert pool.submit(operator.add, 1, 2).result() == 3\npool.submit(time.sleep, 5)\ntime.sleep(0.5)\n\nwait = pool.process.wait\n\ndef short_wait(timeout=None):\n    return wait(timeout=2)\n\npool.process.wait = short_wait\n\ntry:\n    pool.shutdown()\nexcept subprocess.TimeoutExpired:\n    pool.process.kill()\n    pool.process.wait()\n    raise\n\nprint("shutdown returned")\n']' timed out after 20 seconds
=========================== short test summary info ============================
FAILED [20.2431s] inductor/test_compile_worker.py::TestCompileWorker::test_shutdown_terminates_sidecar_worker_pool
!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!

It seems that due to timeout issue on riscv64

Versions

Collecting environment information...
PyTorch version: 2.13.0a0+git1449fa4
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: Debian GNU/Linux 13 (trixie) (riscv64)
GCC version: (Debian 14.2.0-19) 14.2.0
Clang version: 19.1.7 (3+b1)
CMake version: version 3.31.6
Libc version: glibc-2.41

Python version: 3.13.5 (main, Jun 25 2025, 18:55:22) [GCC 14.2.0] (64-bit runtime)
Python platform: Linux-6.6.0-138.0.0.121.oe2403sp3.riscv64-riscv64-with-glibc2.41
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
Is XPU available: False
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: False
Caching allocator config: N/A

CPU:
Architecture:        riscv64
Byte Order:          Little Endian
CPU(s):              64
On-line CPU(s) list: 0-63
Vendor ID:           0x5b7
Model name:          thead,c920
CPU family:          0x0
Model:               0x0
Thread(s) per core:  1
Core(s) per socket:  64
Socket(s):           1
L1d cache:           4 MiB (64 instances)
L1i cache:           4 MiB (64 instances)
L2 cache:            16 MiB (16 instances)
L3 cache:            64 MiB (1 instance)
NUMA node(s):        4
NUMA node0 CPU(s):   0-7,16-23
NUMA node1 CPU(s):   8-15,24-31
NUMA node2 CPU(s):   32-39,48-55
NUMA node3 CPU(s):   40-47,56-63

Versions of relevant libraries:
[pip3] mypy==1.16.0
[pip3] mypy_extensions==1.1.0
[pip3] numpy==2.1.2
[pip3] onnx==1.21.0
[pip3] onnx-ir==0.1.16
[pip3] onnxscript==0.6.2
[pip3] optree==0.13.0
[pip3] torch==2.13.0a0+git1449fa4
[conda] Could not collect

Activity

  1. changed the title [-][riscv64]:nductor/test_compile_worker.py::TestCompileWorker::test_shutdown_terminates_sidecar_worker_pool"[/-] [+][riscv64]:inductor/test_compile_worker.py::TestCompileWorker::test_shutdown_terminates_sidecar_worker_pool"[/+] on Jun 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions