Papers by Myeongjun Jang
Improving Language Models’ Meaning Understanding and Consistency by Learning Conceptual Roles from Dictionary (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have exploited data augmentation or implemented specialised loss functions to alleviate the inconsistent behaviour issue, but they consume expensive training resources and can only handle a certain consistency type. |
| Approach: | They propose a method that allows PLMs to capture accurate meaning by learning precise interrelationships between concepts from word-definition pairs in a dictionary. |
| Outcome: | The proposed method can improve multiple types of consistency and integrate pre-trained knowledge with PLMs’ pre-training knowledge. |
KoBEST: Korean Balanced Evaluation of Significant Tasks (2022.coling-1)
Copied to clipboard
| Challenge: | a well-formulated benchmark allows objective and precise evaluation of diverse models. |
| Approach: | They propose a benchmark for Korean balanced evaluation of significant tasks that requires advanced Korean linguistic knowledge. |
| Outcome: | The proposed benchmarks are based on five Korean-language downstream tasks . the data is annotated by humans and thoroughly reviewed to guarantee high data quality. |
BECEL: Benchmark for Consistency Evaluation of Language Models (2022.coling-1)
Copied to clipboard
| Challenge: | Existing definitions of behavioural consistency are inconsistent across many studies. |
| Approach: | They propose a behavioural consistency model and propose behavioural taxonomy that classifies consistencies into several sub-categories. |
| Outcome: | The proposed model performs poorly on 19 test cases while exhibiting high inconsistency in many cases. |
KNOW How to Make Up Your Mind! Adversarially Detecting and Alleviating Inconsistencies in Natural Language Explanations (2023.acl-short)
Copied to clipboard
| Challenge: | eIA is an adversarial attack that generates inconsistent natural language explanations (NLEs) a model that generate In-NLE is undesirable, as it has a faulty decision-making process or is prone to inconsistencies. |
| Approach: | They propose an off-the-shelf mitigation method to alleviate inconsistencies by grounding the model into external background knowledge. |
| Outcome: | The proposed method reduces inconsistencies detected by previous models . it is based on external knowledge bases and a novel approach to mitigate inconsistent models based upon the proposed method . |
Consistency Analysis of ChatGPT (2023.emnlp-main)
Copied to clipboard
| Challenge: | ChatGPT and GPT-4 have been reported to be more reliable and trustworthy, provided they behave similarly to humans. |
| Approach: | They propose to compare ChatGPT and GPT-4 in terms of logically consistent behaviour and the properties of negation, symmetric, and transitive consistency. |
| Outcome: | The proposed models show that they can be more reliable and trustworthy provided they behave similarly to humans. |
Beyond Distributional Hypothesis: Let Language Models Learn Meaning-Text Correspondence (2022.findings-naacl)
Copied to clipboard
| Challenge: | Recent evidence shows that large-size pre-trained language models do not satisfy the logical negation property (LNP) However, their reliability is being challenged due to faulty behaviours and incomprehension on number-related representations. |
| Approach: | They propose a new intermediate training task to directly learn meaning text correspondence instead of relying on the distributional hypothesis. |
| Outcome: | The proposed approach outperforms previous models on 7 GLUE tasks and outperformed previous models. |