• Computer Science > Computation and Language [Submitted on 1 Feb 2026] Title:Task-Aware LoRA Adapter Composition via Similarity Retrieval in Vector Databases View PDF HTML (experimental)Abstract:Parameter efficient fine tuning methods like LoRA have enabled task specific adaptation of large language models, but efficiently composing multiple specialized adapters for unseen tasks remains challenging • We present a novel framework for dynamic LoRA adapter composition that leverages similarity retrieval in vector databases to enable zero-shot generalization across diverse NLP tasks • Our approach constructs a task-aware vector database by embedding training examples from 22 datasets spanning commonsense reasoning, question answering, natural language inference, and sentiment analysis • At inference time, we retrieve the most similar training examples, compute task similarity distributions via nucleus sampling, and dynamically merge relevant LoRA adapters using retrieval weighted fusion strategies • We evaluated four merging methods Linear, Concatenation, TIES, and Magnitude Prune demonstrating that our dataset centric retrieval approach often matches or exceeds the performance of individua

Article Summaries:

  • Computer Science > Computation and Language [Submitted on 1 Feb 2026] Title:Task-Aware LoRA Adapter Composition via Similarity Retrieval in Vector Databases View PDF HTML (experimental)Abstract:Parameter efficient fine tuning methods like LoRA have enabled task specific adaptation of large language models, but efficiently composing multiple specialized adapters for unseen tasks remains challenging. We present a novel framework for dynamic LoRA adapter composition that leverages similarity retrieval in vector databases to enable zero-shot generalization across diverse NLP tasks. Our approach co

Sources: