Papers with BERT-wwm
Molweni: A Challenge Multiparty Dialogues-based Machine Reading Comprehension Dataset with Discourse Structure (2020.coling-main)
Copied to clipboard
| Challenge: | Multiparty dialog applications such as discourse parsing and meeting summarization are now mainstream research. |
| Approach: | They propose to annotate a machine reading comprehension dataset with discourse structure built over multiparty dialog using a modified Segmented Discourse Representation Theory (SDRT) style. |
| Outcome: | The proposed dataset contributes large-scale discourse dependency annotations in a modified Segmented Discourse Representation Theory (SDRT) style for all of its multiparty dialogs, and achieves only 67.7% F1 on Molweni’s questions, a 20+% significant drop as compared against its SQuAD 2.0 performance. |
Enhancing Pre-trained Chinese Character Representation with Word-aligned Attention (2020.acl-main)
Copied to clipboard
| Challenge: | Pre-trained Chinese language models take character as the basic unit and learn representation according to character’s external contexts, ignoring the semantics expressed in the word, which is the smallest meaningful utterance in Chinese. |
| Approach: | They propose to pool character-level attention to the word level and propose to alleviate the potential issue of segmentation error propagation by multi-source information fusion. |
| Outcome: | The proposed approach improves on five Chinese NLP benchmark tasks against BERT, ERNIE and BERT-wwm. |