Papers by Baohao Liao
Parameter-Efficient Fine-Tuning without Introducing New Latency (2023.acl-long)
Copied to clipboard
| Challenge: | Parameter-efficient fine-tuning of pre-trained language models has been demonstrated to be effective, but its inherent characteristics limit its performance. |
| Approach: | They propose to generate a sparse mask in a task-agnostic manner by modifying only a small subset of existing parameters and adding new parameters. |
| Outcome: | The proposed method surpasses existing methods on the GLUE benchmark by a significant margin. |
Unifying Input and Output Smoothing in Neural Machine Translation (2020.coling-main)
Copied to clipboard
| Challenge: | Recent methods that smooth input and output of neural machine translation systems bring significant improvements in performance. |
| Approach: | They propose a method that replaces one-hot representations with soft posterior distributions of an external language model, smoothing the input of machine translation systems. |
| Outcome: | The proposed method improves translation performance on small datasets and larger datasets. |
Multi-Agent Mutual Learning at Sentence-Level and Token-Level for Neural Machine Translation (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) has achieved significant progress over recent years. |
| Approach: | They extend mutual learning to the machine translation task and operate at both the sentence-level and the token-level. |
| Outcome: | The proposed method improves on the IWSLT’14 German-English task and also on the WMT’14 English-German task. |
Unilogit: Robust Machine Unlearning for LLMs Using Uniform-Target Self-Distillation (2025.findings-acl)
Copied to clipboard
| Challenge: | Extensive experiments on public benchmarks and an in-house e-commerce dataset demonstrate Unilogit’s superior performance in balancing forget and retain objectives, outperforming state-of-the-art methods such as NPO and UnDIAL. |
| Approach: | They propose a self-distillation method that dynamically adjusts target logits to achieve a uniform probability for the target token. |
| Outcome: | Extensive experiments on public benchmarks and an in-house e-commerce dataset demonstrate Unilogit’s superior performance in balancing forget and retain objectives. |
ApiQ: Finetuning of 2-Bit Quantized Large Language Model (2024.emnlp-main)
Copied to clipboard
| Challenge: | Memory-efficient finetuning of large language models (LLMs) has attracted huge attention with the increasing size of LLMs due to the constraints posed by GPU memory limitations and the effectiveness of these methods compared to full finetune. |
| Approach: | They propose a memory-efficient finetuning framework called ApiQ to restore lost information from quantization by initializing LoRA components and quantizing weights of LLMs. |
| Outcome: | The proposed framework maintains the original LLM’s activation precision while mitigating error propagation from shallower into deeper layers. |
Mask More and Mask Later: Efficient Pre-training of Masked Language Models by Disentangling the [MASK] Token (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Large-scale pre-trained MLMs can be used to generalize well to a wide range of tasks. |
| Approach: | They propose to append [MASK]s at a later layer to reduce sequence length for earlier layers. |
| Outcome: | The proposed method outperforms RoBERTa for 6 out of 8 GLUE tasks on average by 0.4%. |
Ask Language Model to Clean Your Noisy Translation Data (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Neural machine translation models exhibit a noticeable decline in translation quality when exposed to noisy input. |
| Approach: | They use a dataset to evaluate the robustness of NMT models against noisy inputs. |
| Outcome: | The proposed dataset cleaners the noise from the target sentences while preserving the semantic integrity of the original sentences. |
ClusComp: A Simple Paradigm for Model Compression and Efficient Finetuning (2025.findings-acl)
Copied to clipboard
| Challenge: | Weight-only quantization reduces model size but suffers from performance degradation at lower bit widths. |
| Approach: | They propose a weight-only quantization paradigm that clusters weight matrices into codebooks and finetunes them block-by-block. |
| Outcome: | The proposed paradigm outperforms quantization methods and fine tunes LLMs to 1-bit compression and fine tuning. |
The SIFo Benchmark: Investigating the Sequential Instruction Following Ability of Large Language Models (2024.findings-emnlp)
Copied to clipboard
Xinyi Chen, Baohao Liao, Jirui Qi, Panagiotis Eustratiadis, Christof Monz, Arianna Bisazza, Maarten Rijke
| Challenge: | Current evaluation resources for instruction following focus on single task instructions, but the instruction sequences in these benchmarks often lack coherence. |
| Approach: | They propose to evaluate models’ abilities to follow multiple instructions through sequential instruction following tasks using four tasks to assess different aspects of sequential instruction followed. |
| Outcome: | The proposed benchmark outperforms open-source and closed-source models on four tasks assessing different aspects of sequential instruction following. |