To evaluate the linguistic and aesthetic quality of the artificial poems, we collected behavioral ratings through online crowdsourcing evaluation, which enabled us to sample a wide range of population and acquire a large amount of data. The 150 artificial poems were evaluated. We designed five questions to probe five aspects of poem appreciation:
- Aesthetics:
这首诗美吗？(translation: is this poem beautiful?) - Fluency:
这首诗顺口吗？(translation: does the poem read fluently?) - Emotion:
这首诗触动了你吗？(translation: are you moved by this poem?) - Meaning:
你觉得这首诗有意义吗？(translation: is this poem meaningful?) - Value:
你愿意给这首诗打多少赏？(translation: how much would you tip the poet?)
Rating results
The sheet of the rating result were organized as the following format:
- poem:
the poem ID - subject:
the subject ID - score_1:
the rating score on 'aesthetics' question - score_2:
the rating score on 'fluency' question - score_3:
the rating score on 'emotion' question - score_4:
the rating score on 'meaning' question - score_5:
the rating score on 'value' question - rt_1:
the reaction time on 'aesthetics' question - rt_2:
the reaction time on 'fluency' question - rt_3:
the reaction time on 'emotion' question - rt_4:
the reaction time on 'meaning' question - rt_5:
the reaction time on 'value' question
Data samples (Download the full data)
| poem | subject | score_1 | score_2 | score_3 | score_4 | score_5 | rt_1 | rt_2 | rt_3 | rt_4 | rt_5 |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 147 | 12 | 3 | 3 | 4 | 4 | 4 | 1732 | 1916 | 15132 | 1585 | 2594 |
| 16 | 12 | 5 | 3 | 5 | 4 | 5 | 1316 | 7062 | 6214 | 1464 | 1220 |
| 65 | 12 | 3 | 3 | 5 | 4 | 4 | 2163 | 1233 | 1835 | 10507 | 1996 |
| 92 | 12 | 3 | 3 | 3 | 2 | 2 | 1362 | 1154 | 7783 | 1466 | 1718 |
| 56 | 29 | 8 | 9 | 8 | 8 | 9 | 19760 | 13651 | 13578 | 5705 | 2309 |