Tech Specification Extractor - Fine-tuned DistilBERT

[Link to paper](https://www.twine.net/signin
Extracting technical specifications from engineering documentation is challenging due to specialized terminology and complex textual structures. Traditional Natural Language Processing (NLP) techniques struggle with this specialized content, especially when limited annotated data is available for model training. This thesis explores the adaptation of DistilBERT, a lightweight pre-trained language model, to effectively extract technical specifications from engineering documents with minimal manual annotation requirements. Through implementation of both conventional fine-tuning and a novel pattern-enhanced training approach, the research evaluates strategies for reducing labeled data dependency while maintaining extraction accuracy. The experimental methodology employs masked language modeling to adapt DistilBERT to engineering text, followed by targeted fine-tuning with pattern-based data augmentation. Cross-domain evaluations between electrical and mechanical engineering documentation reveal the transferability of learned patterns across technical domains. Results demonstrate that domain adaptation improves model performance across multiple technical entity types, with a 2.69% reduction in perplexity observed through adaptation. Notable performance differences emerge between entity types, with standardized specifications showing stronger cross-domain transfer potential than domain-specific attributes.