Skip to content
View giosullutrone's full-sized avatar
🏠
Working from home
🏠
Working from home

Block or report giosullutrone

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
giosullutrone/README.md

Giovanni Sullutrone

LLM safety researcher. I build evaluations of when language models should refuse, ask for clarification or abstain.

I am completing a PhD in Information and Communication Technologies at the University of Modena and Reggio Emilia. My research examines how LLM systems behave when a request involves safety boundaries, incomplete information, sensitive data, or external evidence. I am particularly interested in evaluations that expose behaviour that are commonly given less attention: whether a refusal is justified, whether retrieval changes the model's decision, and whether an agent can recognize what it needs from a user before acting.

My PhD is supervised by Prof. Sonia Bergamaschi. The PhD cycle ends in October 2026, and the defence is expected in early 2027. I have also worked with Dr. Georgia Koutrika at the DARE Lab on the evaluation of Text-to-SQL systems.

Selected research

  • COVER: Context-Driven Over-Refusal Verification in LLMs (Findings of ACL 2025) — Introduces context-driven over-refusal: cases in which a model rejects a benign task because of the documents supplied with it. The study shows that refusals depend on both the requested task and the evidence, with translation and summarization affected more strongly than question answering. Findings of ACL 2025.

  • Text-to-Refused-SQL: A Comprehensive Evaluation of LLMs Refusal in Text-to-SQL (SEBD 2025) — Studies refusal when a database schema contains personal or sensitive fields. Across models and prompting conditions, legitimate requests can be misread as attempts to bypass safeguards, revealing a tension between privacy protection and usable database access. SEBD 2025.

  • Sensitive Topics Retrieval in Digital Libraries: A Case Study of ḥadīṯ Collections (TPDL 2024) — Evaluates retrieval over historically sensitive religious documents through questions generated from the collection itself. The results show that modern embedding models can rank sensitive passages lower than comparable non-sensitive ones, making parts of a digital library less visible to researchers. TPDL 2024.

  • ABISS: Evaluating Text-to-SQL Systems Through Agent Interaction (Findings of EACL 2027) — Unifies the reasons why a database question may be ambiguous or impossible to answer, then evaluates complete interactions with simulated users. The benchmark shows that agents often recognize that a request is problematic without identifying why, and still struggle to translate useful clarification into the correct SQL query. Code and benchmark: coming soon.

Research methods and tools

My work combines LLM evaluation, benchmarking, multi-agent data generation and validation, retrieval-augmented generation, Text-to-SQL, and synthetic data.

I mainly work with Python, PyTorch, Hugging Face, vLLM, TRL, Docker, and SQL.

Contact

Website · CV (academic) · CV (industry) · Email · ORCID · ACL Anthology · LinkedIn · DARE Lab profile

Pinned Loading

  1. COVER COVER Public

    COVER: Context-Driven Over-Refusal Verification in LLMs (Findings of ACL 2025)

    Python

  2. ABISS-Evaluating-Text-to-SQL-Systems-Through-Agent-Interaction ABISS-Evaluating-Text-to-SQL-Systems-Through-Agent-Interaction Public

    ABISS: Evaluating Text-to-SQL Systems Through Agent Interaction