When Does Bigger Help? A Controlled Study of LLM Scale for Ontology Learning
A controlled study evaluated 13 large language models from Qwen3.5, Qwen3.6, and proprietary GPT release variants on ontology learning tasks using the OntoLearner retrieval-augmented generation pipeline. All models used the same embedding model, retrieval configuration, prompt templates, decoding settings, datasets, and metrics across term typing, taxonomy discovery, and non-taxonomic relationship extraction on four biomedical and materials science and engineering ontologies. Within the dense Qwen3.5 lineage, increasing parameter count primarily improved precision rather than recall, with the largest gains between 9B and 27B parameters. Dense 27B models outperformed larger sparse models on term typing, while larger Mixture-of-Experts models achieved the strongest open-weight results on taxonomy discovery. Non-taxonomic relationship extraction remained difficult across model scales.
The study provides a controlled comparison of LLM scale effects on ontology learning, finding that scale benefits are task-dependent and not monotonic. Dense models up to 27B parameters show precision gains in term typing, while larger sparse models excel in taxonomy discovery. Non-taxonomic relationship extraction remains challenging regardless of scale.
The finding that dense 27B models outperform larger sparse models on term typing suggests that for certain ontology learning subtasks, dense parameter efficiency may be more important than total parameter count. The non-monotonic scaling behavior indicates that architecture (dense vs. MoE) interacts with task requirements, and that retrieval-augmented generation may mitigate some benefits of scale. Future work could investigate whether task-specific fine-tuning or different retrieval strategies alter these scaling trends.
For organizations building ontology learning systems, the results imply that simply using the largest available model may not yield the best performance for all subtasks. A portfolio approach—using dense mid-size models for term typing and larger MoE models for taxonomy discovery—could optimize cost and accuracy. The difficulty in non-taxonomic relationship extraction highlights a persistent gap that may require specialized training or hybrid symbolic methods.
The study offers actionable guidance for AI teams in biomedical and materials science domains: selecting model size and architecture based on the specific ontology learning task can improve precision without unnecessary compute costs. It also identifies a clear opportunity for vendors to develop specialized solutions for non-taxonomic relationship extraction.
Next signals to watch include whether newer model releases (e.g., beyond Qwen3.6) change the scaling patterns, and whether improvements in retrieval or prompting reduce the gap in non-taxonomic relationship extraction. Replication on additional ontologies and domains would strengthen generalizability.