Papers by Francesco Fusco
pNLP-Mixer: an Efficient all-MLP Architecture for Language (2023.acl-industry)
Copied to clipboard
| Challenge: | large pre-trained language models are impractical for on-device applications due to their size and inference cost. |
| Approach: | They propose an embedding-free MLP-Mixer model for on-device NLP that achieves high weight-efficiency thanks to a novel projection layer. |
| Outcome: | The proposed model beats state-of-the-art of tiny models by 97.8% on two datasets . it beats mBERT on MTOP and multiATIS, while using 170x less parameters . |
Unsupervised Term Extraction for Highly Technical Domains (2022.emnlp-industry)
Copied to clipboard
| Challenge: | Term extraction is an important task for knowledge discovery platforms because domain specific terms are the linguistic representation of domainspecific concepts. |
| Approach: | They propose a term extraction subsystem that uses an unsupervised annotator to generate training data to fine-tune transformer models. |
| Outcome: | The proposed system can generalize across domains while reducing latency and inference time while preserving the high performance of the existing system. |
Extracting Text Representations for Terms and Phrases in Technical Domains (2023.acl-industry)
Copied to clipboard
| Challenge: | Large pre-trained language models are extensively used in modern NLP systems. |
| Approach: | They propose an unsupervised approach to encoding using character-based models and pre-trained sentence encoders to reconstruct large pre-trained embedding matrices. |
| Outcome: | The proposed approach matches the quality of sentence encoders in technical domains and is 5 times smaller and up to 10 times faster on high-end GPUs. |