Papers by Lifu Tu
Efficiently Aligned Cross-Lingual Transfer Learning for Conversational Tasks using Prompt-Tuning (2024.findings-eacl)
Copied to clipboard
| Challenge: | Cross-lingual transfer of language models trained on high-resource languages such as English has been limited due to the high cost of obtaining non-English conversational data. |
| Approach: | They introduce a parallel and large-scale multilingual conversation dataset that is used for cross-lingual alignment pretraining by translating the English-only Schema-Guided Dialogue dataset into 105 other languages. |
| Outcome: | The proposed model performs well on slot-filling and intent classification tasks, and is able to perform well in other languages. |
ENGINE: Energy-Based Inference Networks for Non-Autoregressive Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | a non-autoregressive machine translation model can minimize the autoregressive teacher's energy . engINE is an inference network trained to minimize the teacher' energy based on distilled corpora . |
| Approach: | They propose to train a non-autoregressive machine translation model to minimize autoregressive teacher energy by using an inference network instead of distilled corpora. |
| Outcome: | The proposed model achieves state-of-the-art non-autoregressive results on two datasets . the proposed model is trained to minimize the autoregressive teacher energy . |
Traffic Light or Light Traffic? Investigating Phrasal Semantics in Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Phrases are fundamental linguistic units through which humans convey semantics. |
| Approach: | They assess the capacity of API-based large language models to comprehend phrase semantics . they use three human-annotated datasets to analyze their results . |
| Outcome: | The proposed model outperforms embedding-based methods in phrase semantic reasoning tasks . the proposed model does not show significant advantage over fine-tuned methods . |
An Exploration of Arbitrary-Order Sequence Labeling via Energy-Based Inference Networks (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows that conditional random fields (CRFs) perform well in sequence labeling tasks. |
| Approach: | They propose several high-order energy terms to capture dependencies among labels in sequence labeling . they use convolutional, recurrent, and self-attention networks to construct these energy terms . |
| Outcome: | The proposed approach improves on four sequence labeling tasks while having the same decoding speed as simple classifiers. |
An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Models (2020.tacl-1)
Copied to clipboard
| Challenge: | Recent work shows that pre-trained language models perform poorly on challenging datasets where spurious correlations do not hold. |
| Approach: | They propose to use multi-task learning to improve generalization from minority examples . they propose to combine MTL with auxiliary tasks to improve performance . |
| Outcome: | The proposed model generalizes from minority examples without hurting in-distribution performance. |
Prompt-Tuning Can Be Much Better Than Fine-Tuning on Cross-lingual Understanding With Multilingual Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained multilingual language models show significant performance gains for zero-shot cross-lingual model transfer on a wide range of natural language understanding (NLU) tasks. |
| Approach: | They do cross-lingual evaluation using prompt tuning and compare it with fine-tuning . prompt tuning achieves much better cross-linguistic transfer than fine- tuning . |
| Outcome: | The results show that prompt tuning achieves better cross-lingual transfer than fine-tuning across datasets, with only 0.1% to 0.3% tuned parameters. |
Benchmarking Approximate Inference Methods for Neural Structured Prediction (N19-1)
Copied to clipboard
| Challenge: | Structured prediction models often involve complex inference problems for which finding exact solutions is intractable. |
| Approach: | They propose to perform gradient descent with respect to the output structure directly and train a neural network to perform inference. |
| Outcome: | The proposed methods achieve better speed/accuracy/search error trade-off than gradient descent while being faster than exact inference at similar accuracy levels. |
Unlocking Anticipatory Text Generation: A Constrained Approach for Large Language Models Decoding (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models have shown a powerful ability for text generation, but undesired behaviors such as toxicity and hallucinations can manifest. |
| Approach: | They propose to formalize text generation as a future-constrained generation problem to minimize undesirable behaviors and enforce faithfulness to instructions. |
| Outcome: | The proposed approach is effective across three tasks, including keyword-constrained generation, toxicity reduction, and factual correctness in question-answering. |
Generating Diverse Story Continuations with Controllable Semantics (D19-56)
Copied to clipboard
| Challenge: | a new framework for controllable story continuation generation is proposed . we use frames to generate story continuations based on sentence attributes . |
| Approach: | They propose a framework for controlled generation of multiple, diverse outputs . they use sentiment, length, predicates, frames, and automatically-induced clusters as controllable dimensions . |
| Outcome: | The proposed model produces outputs that match target attributes, the authors show . it also yields higher metric scores than previous models, they show ." |