Can small language models handle context-summarized multi-turn customer-service QA? A synthetic data-driven comparative evaluation.

Customer-service question answering (QA) systems increasingly rely on conversational language understanding. While Large Language Models (LLMs) achieve strong performance, their high computational cost and deployment constraints limit practical use in resource-constrained environments. Small Language Models (SLMs) provide a more efficient alternative, yet their effectiveness for multi-turn customer-service QA remains underexplored, particularly in scenarios requiring dialogue continuity and contextual understanding. In this study, we evaluate whether instruction-tuned SLMs, fine-tuned using parameter-efficient finetuning, can effectively handle context-summarized multi-turn customer-service QA while preserving contextual consistency, response quality and task relevance under computational constraints. We further investigate instruction-tuned SLMs for context-summarized multi-turn customer-service QA using a history summarization strategy to preserve essential conversational state and introduce a conversation stage-based qualitative analysis to evaluate model behavior across different phases of customer-service interactions. The main contributions of this work include the application of parameter-efficient fine-tuning to adapt SLMs for context-summarized multi-turn customer-service QA, a synthetic data construction pipeline for generating a context-summarized multi-turn QA dataset, and a structured evaluation framework combining quantitative metrics with human and LLM-as-a-judge assessments for customer-service QA evaluation. Nine instruction-tuned SLMs are evaluated against three commercial LLMs using lexical and semantic similarity metrics alongside qualitative assessments, including human evaluation and LLM-as-a-judge methods. Results show notable variation across SLMs, with some models demonstrating near-LLM performance, while others struggle to maintain dialogue continuity and contextual alignment. These findings highlight both the potential and current limitations of low-parameter language models for real-world customer-service QA systems.