Skip to content

fix: import celery issues#703

Merged
Prajna1999 merged 2 commits intomainfrom
hotfix/celery-import-issues
Mar 20, 2026
Merged

fix: import celery issues#703
Prajna1999 merged 2 commits intomainfrom
hotfix/celery-import-issues

Conversation

@Prajna1999
Copy link
Collaborator

@Prajna1999 Prajna1999 commented Mar 20, 2026

Summary

Target issue is #704
Explain the motivation for making this change. What existing problem does the pull request solve?

Checklist

Before submitting a pull request, please ensure that you mark these task.

  • Ran fastapi run --reload app/main.py or docker compose up in the repository root and test.
  • If you've fixed a bug or added code that is tested and has test cases.

Notes

Please add here if any other information is required for the reviewer.

Summary by CodeRabbit

  • Chores
    • Improved Celery worker startup: workers now pre-load configured job modules during initialization and include clearer startup logging and warnings for any module load issues, enhancing startup reliability and observability.

@Prajna1999 Prajna1999 requested a review from kartpop March 20, 2026 12:57
@coderabbitai
Copy link

coderabbitai bot commented Mar 20, 2026

📝 Walkthrough

Walkthrough

Adds a Celery worker startup handler that iterates a configured list of job module paths and imports each module during worker process initialization, logging debug on success and warnings on import failures (function connected to worker_process_init).

Changes

Cohort / File(s) Summary
Celery Worker Module Pre-warming
backend/app/celery/celery_app.py
Introduces warmup_job_modules(sender, **kwargs: object) -> None and _JOB_MODULES list; connects handler to worker_process_init; uses importlib to import job modules and logging to report successes and exceptions. All existing Celery app setup and autodiscovery are unchanged.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

🐰 I nibble at code as dawn draws near,
I fetch each module, bring them here,
I log a hop for every load,
I catch the crumbs when errors showed,
Warmed and ready — off they cheer! 🥕

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 inconclusive)

Check name Status Explanation Resolution
Title check ❓ Inconclusive The title 'fix: import celery issues' is vague and generic. It uses non-specific language ('import celery issues') that doesn't convey what specific problem is being solved or what the actual change accomplishes. Clarify the title to describe the specific fix, such as 'fix: add worker startup warmup for celery job module imports' or 'fix: prevent celery module import failures on worker initialization'.
✅ Passed checks (2 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch hotfix/celery-import-issues
📝 Coding Plan
  • Generate coding plan for human review comments

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Tip

CodeRabbit can use TruffleHog to scan for secrets in your code with verification capabilities.

Add a TruffleHog config file (e.g. trufflehog-config.yml, trufflehog.yml) to your project to customize detectors and scanning behavior. The tool runs only when a config file is present.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
backend/app/celery/celery_app.py (1)

28-38: Add type hint for sender parameter and use underscore prefix for unused arguments.

Per coding guidelines, all function parameters require type hints. Additionally, using _ prefix for sender and kwargs signals intentional non-use and silences the static analysis warnings.

Proposed fix
 `@worker_process_init.connect`
-def warmup_job_modules(sender, **kwargs: object) -> None:
+def warmup_job_modules(_sender: object, **_kwargs: object) -> None:
     """Pre-import all job modules so the first task execution is not delayed by cold imports."""
     for module_path in _JOB_MODULES:
         try:
             importlib.import_module(module_path)
             logger.info(f"[warmup_job_modules] Pre-imported {module_path}")
         except Exception as exc:
             logger.warning(
                 f"[warmup_job_modules] Failed to pre-import {module_path}: {exc}"
             )

As per coding guidelines: "Always add type hints to all function parameters and return values in Python code".

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@backend/app/celery/celery_app.py` around lines 28 - 38, Update the
warmup_job_modules function signature to add type hints for all parameters and
mark unused ones with a leading underscore: change the parameters to _sender:
object and _kwargs: object while keeping the return type None (function name
warmup_job_modules). No other logic changes are needed; this silences
static-analysis warnings about missing type hints and unused parameters.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@backend/app/celery/celery_app.py`:
- Around line 28-38: Update the warmup_job_modules function signature to add
type hints for all parameters and mark unused ones with a leading underscore:
change the parameters to _sender: object and _kwargs: object while keeping the
return type None (function name warmup_job_modules). No other logic changes are
needed; this silences static-analysis warnings about missing type hints and
unused parameters.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: e4ee8b82-c1af-4ee4-9061-c5f3a47b923c

📥 Commits

Reviewing files that changed from the base of the PR and between d92b4b6 and f75df33.

📒 Files selected for processing (1)
  • backend/app/celery/celery_app.py

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
backend/app/celery/celery_app.py (1)

35-38: Include traceback when module pre-import fails

Lines [35]-[38] only log the exception message; stack traces are dropped. Use exc_info=True (or logger.exception) so failures are diagnosable in worker logs.

Proposed fix
         except Exception as exc:
             logger.warning(
-                f"[warmup_job_modules] Failed to pre-import {module_path}: {exc}"
+                f"[warmup_job_modules] Failed to pre-import {module_path}: {exc}",
+                exc_info=True,
             )
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@backend/app/celery/celery_app.py` around lines 35 - 38, The current except
block that catches Exception while pre-importing modules (referenced by
module_path and using logger) only logs the exception message; modify the
handler in celery_app.py so it logs the full traceback — either replace
logger.warning(...) with logger.exception(...) or call logger.warning(...,
exc_info=True) — so the stack trace is recorded for failures during the
pre-import step.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@backend/app/celery/celery_app.py`:
- Line 29: The function warmup_job_modules should type and mark unused signal
parameters as intentionally unused: change the signature to accept a typed
sender parameter (e.g., _sender: Any) and a folded kwargs parameter named with a
leading underscore (e.g., **_kwargs: Any) while keeping the return type None;
update any imports to include Any from typing if necessary and leave the
function body unchanged.

---

Nitpick comments:
In `@backend/app/celery/celery_app.py`:
- Around line 35-38: The current except block that catches Exception while
pre-importing modules (referenced by module_path and using logger) only logs the
exception message; modify the handler in celery_app.py so it logs the full
traceback — either replace logger.warning(...) with logger.exception(...) or
call logger.warning(..., exc_info=True) — so the stack trace is recorded for
failures during the pre-import step.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 2436ee54-73ff-4c07-8478-232deabbc411

📥 Commits

Reviewing files that changed from the base of the PR and between f75df33 and 3c8dd98.

📒 Files selected for processing (1)
  • backend/app/celery/celery_app.py



@worker_process_init.connect
def warmup_job_modules(sender, **kwargs: object) -> None:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion | 🟠 Major

Type-annotate and underscore unused signal params

Line [29] leaves sender untyped and keeps two intentionally-unused args as regular names. Please type sender and prefix both unused params with _ to satisfy lint and readability.

Proposed fix
-@worker_process_init.connect
-def warmup_job_modules(sender, **kwargs: object) -> None:
+@worker_process_init.connect
+def warmup_job_modules(_sender: object, **_kwargs: object) -> None:

As per coding guidelines, "**/*.py: Always add type hints to all function parameters and return values in Python code" and "Use Python 3.11+ with type hints throughout the codebase".

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
def warmup_job_modules(sender, **kwargs: object) -> None:
`@worker_process_init.connect`
def warmup_job_modules(_sender: object, **_kwargs: object) -> None:
🧰 Tools
🪛 Ruff (0.15.6)

[warning] 29-29: Unused function argument: sender

(ARG001)


[warning] 29-29: Unused function argument: kwargs

(ARG001)

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@backend/app/celery/celery_app.py` at line 29, The function warmup_job_modules
should type and mark unused signal parameters as intentionally unused: change
the signature to accept a typed sender parameter (e.g., _sender: Any) and a
folded kwargs parameter named with a leading underscore (e.g., **_kwargs: Any)
while keeping the return type None; update any imports to include Any from
typing if necessary and leave the function body unchanged.

@Prajna1999 Prajna1999 self-assigned this Mar 20, 2026
@Prajna1999 Prajna1999 merged commit 161afb4 into main Mar 20, 2026
1 of 3 checks passed
@Prajna1999 Prajna1999 deleted the hotfix/celery-import-issues branch March 20, 2026 13:20
@coderabbitai coderabbitai bot mentioned this pull request Mar 20, 2026
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants