Papers by Jianwei Niu
Temporal Scaling Law for Large Language Models (2025.emnlp-main)
Copied to clipboard
Yizhe Xiong, Xiansheng Chen, Xin Ye, Hui Chen, Zijia Lin, Haoran Lian, Zhenpeng Su, Wei Huang, Jianwei Niu, Jungong Han, Guiguang Ding
| Challenge: | Existing studies have found that the test loss of LLMs scales as power-laws with model size, computational budget, and dataset size. |
| Approach: | They propose a concept of Temporal Scaling Law to study test loss of LLMs . they break down test loss into fine-grained token positions and develop a dynamic hyperbolic-law . |
| Outcome: | The proposed model predicts the test loss of LLMs as the training steps scale up. |
Fine-grained Factual Consistency Assessment for Abstractive Summarization Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have shown that around 30% of the summaries generated by abstractive summarization models contain factual errors. |
| Approach: | They propose a fine-grained two-stage Fact Consistency assessment framework for summarization models that uses fine-grain consistency reasoning to find subtle clues to identify whether a model-generated summary is consistent with the original document. |
| Outcome: | The proposed framework improves on the state-of-the-art models and distinguishes detailed differences better. |
Can Federated Learning Safeguard Private Data in LLM Training? Vulnerabilities, Attacks, and Defense Evaluation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | federated learning (FL) fine-tunes large language models with local data, but organizations are reluctant to share local data. |
| Approach: | They propose a framework for fine-tuning large language models with local data . they propose centralized fine- tuning with local datasets is a good idea . |
| Outcome: | The proposed framework allows clients to retain local data while sharing only model parameters for training. |
LogicST: A Logical Self-Training Framework for Document-Level Relation Extraction with Incomplete Annotations (2024.emnlp-main)
Copied to clipboard
| Challenge: | Document-level relation extraction (DocRE) is difficult due to the vast number of entity pairs. |
| Approach: | They propose a neural-logic self-training framework that iteratively resolves conflicts and constructs the minimal diagnostic set for updating models. |
| Outcome: | The proposed framework outperforms existing methods on the document-level relation extraction (docRE) benchmark. |
CETA: A Consensus Enhanced Training Approach for Denoising in Distantly Supervised Relation Extraction (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods for relation extraction use noisy instances and poor quality training data. |
| Approach: | They propose a sentence-level DSRE method that denies noisy samples from the wrong classification space on the feature space by enhancing the classification consensus between two discrepant classifiers. |
| Outcome: | The proposed method outperforms existing methods on widely-used benchmarks and significantly outperformed existing methods. |
EdgeFormer: Latency-Aware Collaborative Multi-Head Attention of Transformer Inference in Edge Networks (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for inference in centralized cloud pose privacy risks due to sensitive data. |
| Approach: | They propose a latency-aware framework for distributed Transformer inference in resource-constrained edge networks. |
| Outcome: | The proposed framework achieves 2.01 times inference acceleration over state-of-the-art baselines with leq1.06% accuracy loss, maintaining robustness under varying edge conditions. |
Breaking the Stage Barrier: A Novel Single-Stage Approach to Long Context Extension for Large Language Models (2025.coling-main)
Copied to clipboard
Haoran Lian, Junmin Chen, Wei Huang, Yizhe Xiong, Wenping Hu, Guiguang Ding, Hui Chen, Jianwei Niu, Zijia Lin, Fuzheng Zhang, Di Zhang
| Challenge: | Recent studies show that Large language models struggle with handling long token sequences due to limited training context size. |
| Approach: | They propose a single-stage continual pretraining method to equip LLMs with long context modeling capabilities. |
| Outcome: | The proposed method outperforms existing methods on 4 language modeling benchmarks. |
Explore Better Relative Position Embeddings from Encoding Perspective for Transformer Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results on nine authoritative datasets demonstrate the effectiveness of our methods empirically. |
| Approach: | They propose to use a method to explicitly encode position information into Transformer models by using absolute position embedding. |
| Outcome: | The proposed methods improve Shaw-RPE and XL-R PE and achieve the best overall performance among five different RPEs. |
Key Mention Pairs Guided Document-Level Relation Extraction (2022.coling-1)
Copied to clipboard
| Challenge: | Document-level Relation Extraction (DocRE) aims to identify the relations between entities in a given document. |
| Approach: | They propose a document-level relation extraction model with two modules to model mention-level relations. |
| Outcome: | The proposed model outperforms existing state-of-the-art models on two public DocRE datasets and outperformed existing models. |
Bridging the Gap between Synthetic and Natural Questions via Sentence Decomposition for Semantic Parsing (2023.tacl-1)
Copied to clipboard
| Challenge: | Existing methods to train a parser to perform zero-shot learning are limited by the lack of training data. |
| Approach: | They propose a decomposition-based method to unify the sentence structures of questions . their method can generalize to natural questions with novel text expressions . |
| Outcome: | The proposed method improves on synthetic data and on complex web questions with novel expressions. |
Boosting Document-Level Relation Extraction by Mining and Injecting Logical Rules (2022.emnlp-main)
Copied to clipboard
| Challenge: | Document-level relation extraction (DocRE) aims to extract relations of all entity pairs in document. |
| Approach: | They propose a logic enhanced framework that boosts DocRE by mining and injecting logic rules. |
| Outcome: | The proposed framework outperforms LogiRE on two benchmarks. |