Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# cv-evaluator
CV Evaluator sends a CV to an LLM for review. A CodeYourFuture project.
CV Evaluator sends a CV to an LLM for review, with optional job-description matching. A CodeYourFuture project.

## Local Development Instructions

Expand Down Expand Up @@ -70,6 +70,12 @@ The application creates two FastAPI instances:
### Notes
- `slowapi` is used for rate limiting, since there's LLM cost involved with each evaluation. The default limit is set to 5 requests per minute per IP address.
- `markitdown` is used to convert uploaded CV files (PDF, DOCX) into markdown format for easier processing by the LLM.
- The UI accepts two inputs:
- Required CV input (text or PDF/DOCX upload)
- Optional job description input (text or PDF/DOCX upload)
- When a job description is provided, the API response may include:
- `jd_match_for_computers`
- `jd_match_for_people`

## GitHub App Setup

Expand Down
14 changes: 12 additions & 2 deletions app/llm_evaluator.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,8 @@ class CvEvaluation(BaseModel):
project: RuleResult
experience: RuleResult
education: RuleResult
jd_match_for_computers: Optional[RuleResult] = None
jd_match_for_people: Optional[RuleResult] = None
debug_info: Optional[str] = None


Expand Down Expand Up @@ -79,18 +81,26 @@ def __init__(self, config_path: Optional[str] = None):
self.system_message = config.get("system_message", "")
self.user_message_template = config.get("user_message", "{cv_text}")

async def eval(self, cv_text: str) -> CvEvaluation:
async def eval(self, cv_text: str, jd_text: Optional[str] = None) -> CvEvaluation:
"""
Evaluate a CV using OpenAI.

Args:
cv_text: The text content of the CV to evaluate.
jd_text: Optional job description text.

Returns:
CvEvaluation: Structured evaluation result.
"""
# Construct the user message with the CV text
user_message = self.user_message_template.format(cv_text=cv_text)
job_description_section = "[NO_JOB_DESCRIPTION_PROVIDED]"
if jd_text and jd_text.strip():
job_description_section = jd_text.strip()

user_message = self.user_message_template.format(
cv_text=cv_text,
job_description_section=job_description_section,
)

# Build the messages for the API call
messages = [
Expand Down
17 changes: 17 additions & 0 deletions app/llm_evaluator.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ system_message: |

Goal
- Given the text of a CV document, evaluate its compliance.
- If a job description is provided, also evaluate how well the CV matches it for ATS systems and human reviewers.

CV Verification Rules:
passed:
Expand Down Expand Up @@ -49,8 +50,24 @@ system_message: |
- References to CodeYourFuture should be under Education, not Experience.
- CodeYourFuture should not be referred to as a "bootcamp", and the CV should explain what CodeYourFuture is.
- Your CV should make clear what you learnt at CodeYourFuture, and why it’s impressive.
jd_match_for_computers:
- If no job description is provided, include this field with a null value.
- Evaluate whether the CV would likely pass ATS keyword-based screening against the job description.
- Focus on direct alignment with required skills, tools, role titles, and key domain terms.
jd_match_for_people:
- If no job description is provided, include this field with a null value.
- Evaluate whether the CV would likely pass human screening against the job description.
- Focus on practical fit across responsibilities, outcomes, and breadth of requirements rather than exact keyword overlap.

user_message: |
Evaluate the following CV per the system instructions and return plain JSON only.

Job Description (optional):
<<<JOB_DESCRIPTION_START>>>
{job_description_section}
<<<JOB_DESCRIPTION_END>>>

CV:
<<<CV_START>>>
{cv_text}
<<<CV_END>>>
93 changes: 57 additions & 36 deletions app/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -71,57 +71,78 @@
MAX_FILE_SIZE_MB = 30
MAX_FILE_SIZE_BYTES = MAX_FILE_SIZE_MB * 1024 * 1024


async def _extract_content_from_upload(upload: UploadFile, input_name: str) -> str:
"""Read and convert an uploaded PDF/DOCX file to markdown text."""
allowed_types = [
"application/pdf",
"application/vnd.openxmlformats-officedocument.wordprocessingml.document"
]
if upload.content_type not in allowed_types:
raise HTTPException(
status_code=400,
detail=f"Unsupported file type for {input_name}. Please upload PDF or DOCX files only."
)

try:
content = await upload.read()
if len(content) > MAX_FILE_SIZE_BYTES:
raise HTTPException(
status_code=413,
detail=f"{input_name} file too large. Maximum size is {MAX_FILE_SIZE_MB}MB."
)

result = markitdown.convert(io.BytesIO(content))
return result.text_content
except Exception as e:
raise HTTPException(status_code=400, detail=f"Error reading {input_name} file: {str(e)}")

# CV evaluation endpoint - handles both JSON and file uploads
# Protected by authentication - only org members can access
@api_app.post("/cv/evaluate", response_model=CvEvaluation)
@limiter.limit("5/minute")
async def evaluate_cv(
request: Request,
cv_text: Optional[str] = Form(None),
file: Optional[UploadFile] = File(None),
cv_file: Optional[UploadFile] = File(None),
jd_text: Optional[str] = Form(None),
jd_file: Optional[UploadFile] = File(None),
user: User = Depends(require_auth),
):
"""
Evaluate a CV either from text input or file upload.
Accepts either form data with cv_text field or a file upload.
Evaluate a CV from text input or file upload, with an optional
job description provided as text input or file upload.
"""

if cv_text is None and file is None:
raise HTTPException(status_code=400, detail="Either cv_text or file upload is required")

if cv_text is not None and file is not None:
raise HTTPException(status_code=400, detail="Provide either cv_text OR file upload, not both")

cv_content = ""

if cv_text:
# Handle text input
cv_content = cv_text
elif file:
# Handle file upload
allowed_types = [
"application/pdf",
"application/vnd.openxmlformats-officedocument.wordprocessingml.document"
]
if file.content_type not in allowed_types:
raise HTTPException(status_code=400, detail="Unsupported file type. Please upload PDF or DOCX files only.")

try:
# Convert the file to markdown for sending to the LLM
content = await file.read()
if len(content) > MAX_FILE_SIZE_BYTES:
raise HTTPException(status_code=413, detail=f"File too large. Maximum size is {MAX_FILE_SIZE_MB}MB.")

result = markitdown.convert(io.BytesIO(content))
cv_content = result.text_content
except Exception as e:
raise HTTPException(status_code=400, detail=f"Error reading file: {str(e)}")


if cv_text is None and cv_file is None:
raise HTTPException(status_code=400, detail="Either cv_text or cv_file is required")

if cv_text is not None and cv_file is not None:
raise HTTPException(status_code=400, detail="Provide either cv_text OR cv_file, not both")

if jd_text is not None and jd_file is not None:
raise HTTPException(status_code=400, detail="Provide either jd_text OR jd_file, not both")

cv_content = cv_text.strip() if cv_text else ""
jd_content: Optional[str] = jd_text.strip() if jd_text else None

if cv_file is not None:
cv_content = await _extract_content_from_upload(cv_file, "CV")

if jd_file is not None:
jd_content = await _extract_content_from_upload(jd_file, "job description")

if not cv_content:
raise HTTPException(status_code=400, detail="CV content is required")

if jd_content is not None and not jd_content:
jd_content = None

# Evaluate the CV using the LLM evaluator
last_exc: Exception | None = None
for attempt in range(1 + settings.llm_retry_count):
try:
result = await evaluator.eval(cv_content)
result = await evaluator.eval(cv_content, jd_content)
return result
except Exception as e:
last_exc = e
Expand Down
79 changes: 56 additions & 23 deletions app/static/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@

<div class="container">
<h1>CodeYourFuture: CV Evaluation Tool</h1>
<p>Evaluate CVs by entering text or uploading files.</p>
<p>Evaluate your CV and optionally provide a job description.</p>

<!-- Auth Error Message -->
<div id="authError" class="auth-error" style="display: none;">
Expand All @@ -48,33 +48,66 @@ <h3>Authentication Required</h3>

<div id="mainContent" style="display: none;">
<div class="privacy-warning">
<p><strong>Privacy warning:</strong> This tool sends CV data to an external API. If you have privacy concerns, please replace your name, phone number, email address, and so forth with fake data. You may also wish to redact any other personally identifying information. If you're still not sure, skip this step and reach out in <span style="white-space: nowrap">#cyf-profile-review</span> to request a manual review.</p>
<p><strong>Privacy warning:</strong> This tool sends CV and job description data to an external API. If you have privacy concerns, please replace your name, phone number, email address, and so forth with fake data. You may also wish to redact any other personally identifying information. If you're still not sure, skip this step and reach out in <span style="white-space: nowrap">#cyf-profile-review</span> to request a manual review.</p>
</div>

<div class="demo-section">

<!-- Mode Switcher -->
<div class="mode-switcher">
<button id="textModeBtn" class="mode-btn active">Text Input</button>
<button id="fileModeBtn" class="mode-btn">File Upload</button>
</div>

<!-- Text Input Section -->
<div id="textSection" class="input-section">
<div class="form-group">
<label for="cvText">CV Content:</label>
<textarea id="cvText" placeholder="Paste your CV content here..."></textarea>

<section class="input-card">
<h2>Your CV</h2>
<p class="input-description">Provide your CV as text or upload a PDF/DOCX file.</p>

<div class="mode-switcher">
<button id="cvTextModeBtn" class="mode-btn active">Text Input</button>
<button id="cvFileModeBtn" class="mode-btn">File Upload</button>
</div>
</div>

<!-- File Upload Section -->
<div id="fileSection" class="input-section" style="display: none;">
<div class="form-group">
<label for="cvFile">Upload CV (PDF or DOCX):</label>
<input type="file" id="cvFile" accept=".pdf,.docx">
<div id="fileInfo" class="file-info"></div>

<div id="cvTextSection" class="input-section">
<div class="form-group">
<label for="cvText">CV Text:</label>
<textarea id="cvText" placeholder="Paste your CV text here..."></textarea>
</div>
</div>
</div>

<div id="cvFileSection" class="input-section" style="display: none;">
<div class="form-group">
<label for="cvFile">Upload CV (PDF or DOCX):</label>
<input type="file" id="cvFile" accept=".pdf,.docx">
<div class="file-meta">
<div id="cvFileInfo" class="file-info"></div>
<button type="button" id="cvFileRemoveBtn" class="btn-file-remove" style="display: none;">Remove file</button>
</div>
</div>
</div>
</section>

<section class="input-card optional-card">
<h2>Job Description<span class="optional-badge">Optional</span></h2>
<p class="input-description">Add a job description to get ATS and human match checks.</p>

<div class="mode-switcher">
<button id="jdTextModeBtn" class="mode-btn active">Text Input</button>
<button id="jdFileModeBtn" class="mode-btn">File Upload</button>
</div>

<div id="jdTextSection" class="input-section">
<div class="form-group">
<label for="jdText">Job Description Text (Optional):</label>
<textarea id="jdText" placeholder="Paste the job description here..."></textarea>
</div>
</div>

<div id="jdFileSection" class="input-section" style="display: none;">
<div class="form-group">
<label for="jdFile">Upload Job Description (PDF or DOCX):</label>
<input type="file" id="jdFile" accept=".pdf,.docx">
<div class="file-meta">
<div id="jdFileInfo" class="file-info"></div>
<button type="button" id="jdFileRemoveBtn" class="btn-file-remove" style="display: none;">Remove file</button>
</div>
</div>
</div>
</section>

<!-- Form Actions -->
<div class="form-actions">
Expand Down
Loading