Correlations Between Positive or Negative Utterances and Basic Acoustic Features of Voice: a Preliminary Analysis

Łukasz Stolarski

doi:10.18778/1731-7533.20.2.03

Authors

Łukasz Stolarski Jan Kochanowski University in Kielce, Poland

DOI:

https://doi.org/10.18778/1731-7533.20.2.03

Keywords:

sentiment analysis, acoustic features, feature selection

Abstract

The major aim of this paper is to establish possible correlations between continuous sentiment scores and four basic acoustic characteristics of voice. In order to achieve this objective, the text of “A Christmas Carol” by Charles Dickens was tokenized at the sentence level. Next, each of the resulting text units was assessed in terms of sentiment polarity and aligned with the corresponding fragment in an audiobook. The results indicate weak but statistically significant correlations between sentiment scores and three acoustic features: the mean F0, the standard deviation of F0 and the mean intensity. These findings may be useful in selecting optimal acoustic features for model training in multimodal sentiment analysis. Also, they are essential from a linguistic point of view and could be applied in studies on such language phenomena as irony.

References

Abbasi, Ahmed, Hassan, Ammar and Dhar, Milan.2014. Benchmarking Twitter Sentiment Analysis Tools. In LREC, Vol. 14, 26–31.
Google Scholar

Aldeneh, Zakaria, Khorram, Soheil, Dimitriadis, Dimitrios and Provost, Emily Mower. 2017. Pooling acoustic and lexical features for the prediction of valence. In Proceedings of the 19th ACM International Conference on Multimodal Interaction, 68–72. ACM. https://doi.org/10.1145/3136755.3136760
Google Scholar DOI: https://doi.org/10.1145/3136755.3136760

Audacity Team. 2014. Audacity(R): Free audio editor and recorder (version 2.0.5) [computer software].
Google Scholar

Aue, Anthony and Gamon, Michael. 2005. Customizing sentiment classifiers to new domains: A case study. In Proceedings of recent advances in natural language processing (RANLP), Vol. 1, 2–10. Citeseer.
Google Scholar

Austin, John Langshaw. 1962. How to do things with words. Oxford: Clarendon Press.
Google Scholar

Bezooijen, Renée. 1984. Characteristics and recognizability of vocal expressions of emotion. Dordrecht, Netherlands: Foris Publications. https://doi.org/10.1515/9783110850390
Google Scholar DOI: https://doi.org/10.1515/9783110850390

Boersma, Paul and Weenink, David. 2014. Praat, a system for doing phonetics by computer (version 5.4.01) [computer software]. Amsterdam: University of Amsterdam.
Google Scholar

Borth, Damian, Ji, Rongrong, Chen, Tao, Breuel, Thomas and Chang, Shih-Fu. 2013. Large-scale visual sentiment ontology and detectors using adjective noun pairs. In Proceedings of the 21st ACM international conference on Multimedia, 223–232. ACM. https://doi.org/10.1145/2502081.2502282
Google Scholar DOI: https://doi.org/10.1145/2502081.2502282

Breitenstein, Caterina, Van Lancker, Diana and Daum, Irene. 2001. The contribution of speech rate and pitch variation to the perception of vocal emotions in a German and an American sample. Cognition and Emotion, 15(1), 57–79. https://doi.org/10.1080/02699930126095
Google Scholar DOI: https://doi.org/10.1080/0269993004200114

Cambria, Erik, Poria, Soujanya, Bajpai, Rajiv and Schuller, Björn W. 2016. SenticNet 4: A Semantic Resource for Sentiment Analysis Based on Conceptual Primitives. In COLING, 2666–2677.
Google Scholar

Chen, Minghai, Wang, Sen, Liang, Paul Pu, Baltrušaitis, Tadas, Zadeh, Amir and Morency, Louis-Philippe. 2017. Multimodal sentiment analysis with word-level fusion and reinforcement learning. In Proceedings of the 19th ACM International Conference on Multimodal Interaction, 63–171. ACM. https://doi.org/10.1145/3136755.3136801
Google Scholar DOI: https://doi.org/10.1145/3136755.3136801

Choi, Yejin and Cardie, Claire. 2008. Learning with compositional semantics as structural inference for subsentential sentiment analysis. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (pp. 793–801). Association for Computational Linguistics. https://doi.org/10.3115/1613715.1613816
Google Scholar DOI: https://doi.org/10.3115/1613715.1613816

Coleman, Robert F. and Williams, Robert. 1979. Identification of emotional states using perceptual and acoustic analyses. In Transcript of the 8th Symposium: Care of the Professional Voice, Part I. The Voice Foundation, New York.
Google Scholar

Collier, William G. and Hubbard, Timothy L. 1998. Judgments of happiness, brightness, speed, and tempo change of auditory stimuli varying in pitch and tempo. Psychomusicology, 17(1/2), 36–55. https://doi.org/10.1037/h0094060
Google Scholar DOI: https://doi.org/10.1037/h0094060

Collier, William G. and Hubbard, Timothy L. 2001. Musical scales and evaluations of happiness and awkwardness: Effects of pitch, direction, and scale mode. American Journal of Psychology, 114(3), 355–375. https://doi.org/10.2307/1423686
Google Scholar DOI: https://doi.org/10.2307/1423686

Davitz, Joel R. 1964. Auditory correlates of vocal expressions of emotional meaning. The Communication of Emotional Meaning, 101–112.
Google Scholar

Diniz, Joao P., Bastos, Lucas, Soares, Elias, Ferreira, Miller, Ribeiro, Filipe and Benevenuto, Fabrıcio. 2016. ifeel 2.0: A multilingual benchmarking system for sentence-level sentiment analysis.
Google Scholar

Eldred, Stanley H. and Price, Douglas B. 1958. A linguistic evaluation of feeling states in psychotherapy. Psychiatry, 21(2), 115–121. https://doi.org/10.1080/00332747.1958.11023120
Google Scholar DOI: https://doi.org/10.1080/00332747.1958.11023120

Eyben, Florian, Weninger, Felix, Gross, Florian and Schuller, Björn. 2013. Recent developments in opensmile, the munich open-source multimedia feature extractor. In Proceedings of the 21st ACM international conference on Multimedia, 835–838. ACM. https://doi.org/10.1145/2502081.2502224
Google Scholar DOI: https://doi.org/10.1145/2502081.2502224

Fairbanks, Grant and Pronovost, Wilbert. 1939. An experimental study of the pitch characteristics of the voice during the expression of emotion. Speech Monographs, 6(1), 87–104. https://doi.org/10.1080/03637753909374863
Google Scholar DOI: https://doi.org/10.1080/03637753909374863

Fonagy, Ivan.1978. A new method of investigating the perception of prosodic features. Language and Speech, 21(1), 34–49. https://doi.org/10.1177/002383097802100102
Google Scholar DOI: https://doi.org/10.1177/002383097802100102

Frick, Robert W. 1985. Communicating emotion: The role of prosodic features. Psychological Bulletin, 97(3), 412–429. https://doi.org/10.1037/0033-2909.97.3.412
Google Scholar DOI: https://doi.org/10.1037/0033-2909.97.3.412

Goldman, Jean-Philippe. 2011. EasyAlign: an automatic phonetic alignment tool under Praat. In Proceedings of Interspeech, 3233–3236. https://doi.org/10.21437/Interspeech.2011-815
Google Scholar DOI: https://doi.org/10.21437/Interspeech.2011-815

Gonçalves, Pollyanna, Araújo, Matheus, Benevenuto, Fabrício and Cha, Meeyoung. 2013. Comparing and combining sentiment analysis methods. In Proceedings of the first ACM conference on Online social networks (pp. 27–38). ACM. https://doi.org/10.1145/2512938.2512951
Google Scholar DOI: https://doi.org/10.1145/2512938.2512951

Govindaraj, Sureshkumar and Gopalakrishnan, Kumaravelan. 2016. Intensified sentiment analysis of customer product reviews using acoustic and textual features. ETRI Journal, 38(3), 494–501. https://doi.org/10.4218/etrij.16.0115.0684
Google Scholar DOI: https://doi.org/10.4218/etrij.16.0115.0684

Hargreaves, William A., Starkweather, John A. and Blacker, K. H. 1965. Voice quality in depression. Journal of Abnormal Psychology, 70(3), 218–220. https://doi.org/10.1037/h0022151
Google Scholar DOI: https://doi.org/10.1037/h0022151

Höffe, Wilhelm L. 1960. Über Beziehungen von Sprachmelodie und Lautstärke. Phonetica, 5(3–4), 129–159. https://doi.org/10.1159/000258054
Google Scholar DOI: https://doi.org/10.1159/000258054

Huddleston, Rodney. 1988. English grammar: An outline. Cambridge University Press. https://doi.org/10.1017/CBO9781139166003
Google Scholar DOI: https://doi.org/10.1017/CBO9781139166003

Huron, David. 2008. A comparison of average pitch height and interval size in major-and minor-key themes: Evidence consistent with affect-related pitch prosody. Empirical Musicology Review, 3(2), 59–63. https://doi.org/10.18061/1811/31940
Google Scholar DOI: https://doi.org/10.18061/1811/31940

Huron, David, Yim, Gary, and Chordia, Parag. 2010. The effect of pitch exposure on sadness judgments: An association between sadness and lower than normal pitch. In Proceedings of the 11th International Conference on Music Perception and Cognition, 63–66.
Google Scholar

Huttar, George L. 1968. Relations between prosodic variables and emotions in normal American English utterances. Journal of Speech, Language, and Hearing Research, 11(3), 481–487. https://doi.org/10.1044/jshr.1103.481
Google Scholar DOI: https://doi.org/10.1044/jshr.1103.481

Hutto, C. J. and Gilbert, Eric. 2014. Vader: A Parsimonious Rule-Based Model for Sentiment Analysis of Social Media Text. In Eighth International AAAI Conference on Weblogs and Social Media (ICWSM-14), 216–255. Ann Arbor, MI. https://doi.org/10.1609/icwsm.v8i1.14550
Google Scholar DOI: https://doi.org/10.1609/icwsm.v8i1.14550

Kaiser, L. 1962. Communication of affects by single vowels. Synthese, 14(4), 300–319. https://doi.org/10.1007/BF00869311
Google Scholar DOI: https://doi.org/10.1007/BF00869311

Kappas, Arvid, Hess, Ursula and Scherer, Klaus R. 1991. Voice and emotion. In R. S. Feldman and B. Rim (eds.), Fundamentals of nonverbal behavior (pp. 200–238). Paris, France: Editions de la Maison des Sciences de l’Homme.
Google Scholar

Kennedy, Alistair and Inkpen, Diana. 2005. Sentiment classification of movie and product reviews using contextual valence shifters. In Proceedings of the Workshop on the Analysis of Informal and Formal Information Exchange during Negotiations. Ottawa, Ontario, Canada.
Google Scholar

Kennedy, Alistair and Inkpen, Diana. 2006. Sentiment classification of movie reviews using contextual valence shifters. Computational Intelligence, 22(2), 110–125. https://doi.org/10.1111/j.1467-8640.2006.00277.x
Google Scholar DOI: https://doi.org/10.1111/j.1467-8640.2006.00277.x

Kiritchenko, Svetlana and Mohammad, Saif M. 2016a. Happy Accident: A Sentiment Composition Lexicon for Opposing Polarity Phrases. In Proceedings of the 10th edition of the Language Resources and Evaluation Conference (LREC). Portoro, Slovenia. https://doi.org/10.18653/v1/N16-1128
Google Scholar

Kiritchenko, Svetlana and Mohammad, Saif M. 2016b. Sentiment composition of words with opposing polarities. In Proceedings of the 15th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL), pp. 1102–1108. San Diego, California. https://doi.org/10.18653/v1/N16-1128
Google Scholar DOI: https://doi.org/10.18653/v1/N16-1128

Kiritchenko, Svetlana, Zhu, Xiaodan and Mohammad, Saif M. 2014. Sentiment analysis of short informal texts. Journal of Artificial Intelligence Research, 50, 723–762. https://doi.org/10.1613/jair.4272
Google Scholar DOI: https://doi.org/10.1613/jair.4272

Ladd, D. Robert, Silverman, Kim E.A., Tolkmitt, Frank, Bergmann, Günther and Scherer, Klaus R. 1985. Evidence for the independent function of intonation contour type, voice quality, and F0 range in signaling speaker affect. The Journal of the Acoustical Society of America, 78(2), 435–444. https://doi.org/10.1121/1.392466
Google Scholar DOI: https://doi.org/10.1121/1.392466

Lee, Akinobu and Kawahara, Tatsuya. 2009. Recent development of open-source speech recognition engine julius. In Proceedings: APSIPA ASC 2009: Asia-Pacific Signal and Information Processing Association, 2009 Annual Summit and Conference, 131–137. Asia-Pacific Signal and Information Processing Association, 2009 Annual ….
Google Scholar

Leinonen, Lea, Hiltunen, Tapio, Linnankoski, Ilkka and Laakso, Maija-Liisa. 1997. Expression of emotional–motivational connotations with a one-word utterance. The Journal of the Acoustical Society of America, 102(3), 1853–1863. https://doi.org/10.1121/1.420109
Google Scholar DOI: https://doi.org/10.1121/1.420109

Li, Bryan, Dimitriadis, Dimitrios and Stolcke, Andreas. 2019. Acoustic and Lexical Sentiment Analysis for Customer Service Calls. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 5876–5880. IEEE. https://doi.org/10.1109/ICASSP.2019.8683679
Google Scholar DOI: https://doi.org/10.1109/ICASSP.2019.8683679

Liu, Bing. 2012. Sentiment analysis and opinion mining. Synthesis Lectures on Human Language Technologies, 5(1), 1–167. https://doi.org/10.1007/978-3-031-02145-9
Google Scholar DOI: https://doi.org/10.2200/S00416ED1V01Y201204HLT016

Liu, Bing and Zhang, Lei. 2012. A survey of opinion mining and sentiment analysis. In Mining text data, 415–463. Springer. https://doi.org/10.1007/978-1-4614-3223-4_13
Google Scholar DOI: https://doi.org/10.1007/978-1-4614-3223-4_13

Mairesse, François, Polifroni, Joseph and Di Fabbrizio, Giuseppe. 2012. Can prosody inform sentiment analysis? experiments on short spoken reviews. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2012 5093–5096. IEEE. https://doi.org/10.1109/ICASSP.2012.6289066
Google Scholar DOI: https://doi.org/10.1109/ICASSP.2012.6289066

McAuliffe, Michael, Socolof, Michaela, Mihuc, Sarah, Wagner, Michael and Sonderegger, Morgan. 2017. Montreal Forced Aligner: Trainable Text-Speech Alignment Using Kaldi. In Interspeech, 498–502. https://doi.org/10.21437/Interspeech.2017-1386
Google Scholar DOI: https://doi.org/10.21437/Interspeech.2017-1386

Mohammad, Saif M., Kiritchenko, Svetlana and Zhu, Xiaodan. 2013. NRC-Canada: Building the state-of-the-art in sentiment analysis of tweets. ArXiv Preprint ArXiv:1308.6242.
Google Scholar

Mohammad, Saif M. and Turney, Peter D. 2010. Emotions evoked by common words and phrases: Using Mechanical Turk to create an emotion lexicon. In Proceedings of the NAACL HLT 2010 workshop on computational approaches to analysis and generation of emotion in text, 26–34. LA, California: Association for Computational Linguistics.
Google Scholar

Mohammad, Saif M. and Turney, Peter D. 2013. Crowdsourcing a word–emotion association lexicon. Computational Intelligence, 29(3), 436–465. https://doi.org/10.1111/j.1467-8640.2012.00460.x
Google Scholar DOI: https://doi.org/10.1111/j.1467-8640.2012.00460.x

Pang, Bo and Lee, Lillian. 2008. Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval, 2(1–2), 1–135. https://doi.org/10.1561/1500000011
Google Scholar DOI: https://doi.org/10.1561/1500000011

Peng, Zeshan. 2017. Acoustic feature-based sentiment analysis of call center data (PhD Thesis). University of Missouri–Columbia.
Google Scholar

Pereira, Moisés Henrique Ramos, Pádua, Flávio Luis Cardeal, Pereira, Adriano César Machado, Benevenuto, Fabrício and Dalip, Daniel Hasan. 2016. Fusing audio, textual, and visual features for sentiment analysis of news videos. In Tenth International AAAI Conference on Web and Social Media.
Google Scholar

Pérez-Rosas, Verónica, Mihalcea, Rada and Morency, Louis-Philippe. 2013. Utterance-level multimodal sentiment analysis. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 973–982.
Google Scholar

Plutchik, Robert. 1980. Emotion: A psychoevolutionary synthesis. Harper and Row.
Google Scholar

Plutchik, Robert. 1997. The circumplex as a general model of the structure of emotions and personality. In R. Plutchik and H. R. Conte (eds.), Circumplex models of personality and emotions, 7–45. Washington, DC, US: American Psychological Association. https://doi.org/10.1037/10261-001
Google Scholar DOI: https://doi.org/10.1037/10261-000

Plutchik, Robert. 2000. Emotions in the practice of psychotherapy: Clinical implications of affect theories, Vol. 13. Washington, DC, US: American Psychological Association. https://doi.org/10.1037/10366-000
Google Scholar DOI: https://doi.org/10.1037/10366-000

Plutchik, Robert. 2001a. Integration, differentiation, and derivatives of emotion. Evolution and Cognition, 7(2), 114–125.
Google Scholar

Plutchik, Robert. 2001b. The nature of emotions. American Scientist, 89(4), 344–350.
Google Scholar

R Development Core Team. 2018. R: A language and environment for statistical computing (version 3.4.4) [computer software]. Vienna, Austria. https://doi.org/10.1511/2001.4.344
Google Scholar DOI: https://doi.org/10.1511/2001.4.344

Razak, Aishah Abd, Abidin, Mohd Izani Zainal and Komiya, Ryoichi. 2003. Emotion pitch variation analysis in Malay and English voice samples. In The 9th Asia-Pacific Conference on Communications 2003, Vol. 1, 108–112.
Google Scholar

Ribeiro, Filipe N., Araújo, Matheus, Gonçalves, Pollyanna, Gonçalves, Marcos André and Benevenuto, Fabrício. 2016. Sentibench-a benchmark comparison of state-of-the-practice sentiment analysis methods. EPJ Data Science, 5(1), 1–29. https://doi.org/10.1140/epjds/s13688-016-0085-1
Google Scholar DOI: https://doi.org/10.1140/epjds/s13688-016-0085-1

Rosas, Verónica Pérez, Mihalcea, Rada and Morency, Louis-Philippe. 2013. Multimodal sentiment analysis of spanish online videos. IEEE Intelligent Systems, 28(3), 38–45. https://doi.org/10.1109/MIS.2013.9
Google Scholar DOI: https://doi.org/10.1109/MIS.2013.9

Scherer, Klaus R. 1986. Vocal affect expression: A review and a model for future research. Psychological Bulletin, 99(2), 143–165. https://doi.org/10.1037/0033-2909.99.2.143
Google Scholar DOI: https://doi.org/10.1037/0033-2909.99.2.143

Schuller, Björn, Batliner, Anton, Seppi, Dino, Steidl, Stefan, Vogt, Thurid, Wagner, Johannes, … Kessous, Loic. 2007. The relevance of feature type for the automatic classification of emotional user states: low level descriptors and functionals. In Eighth Annual Conference of the International Speech Communication Association. https://doi.org/10.21437/Interspeech.2007-612
Google Scholar DOI: https://doi.org/10.21437/Interspeech.2007-612

Schuller, Björn, Steidl, Stefan and Batliner, Anton. 2009. The interspeech 2009 emotion challenge. In Tenth Annual Conference of the International Speech Communication Association. https://doi.org/10.21437/Interspeech.2009-103
Google Scholar DOI: https://doi.org/10.21437/Interspeech.2009-103

Sheikh, Imran, Dumpala, Sri Harsha, Chakraborty, Rupayan and Kopparapu, Sunil Kumar. 2018. Sentiment analysis using imperfect views from spoken language and acoustic modalities. In Proceedings of Grand Challenge and Workshop on Human Multimodal Language (Challenge-HML), 35–39. https://doi.org/10.18653/v1/W18-3305
Google Scholar DOI: https://doi.org/10.18653/v1/W18-3305

Skinner, E. Ray. 1935. A calibrated recording and analysis of the pitch, force and quality of vocal tones expressing happiness and sadness. Communications Monographs, 2(1), 81–137. https://doi.org/10.1080/03637753509374833
Google Scholar DOI: https://doi.org/10.1080/03637753509374833

Sobin, Christina and Alpert, Murray. 1999. Emotion in speech: The acoustic attributes of fear, anger, sadness, and joy. Journal of Psycholinguistic Research, 28(4): 347–365. https://doi.org/10.1023/A:1023237014909
Google Scholar DOI: https://doi.org/10.1023/A:1023237014909

Sperber, Dan and Wilson, Deirdre. 1986. Relevance: Communication and Cognition. Oxford: Basil Blackwell.
Google Scholar

Stolarski, Łukasz. 2018. Lack of effects of gender on the reading rate of long texts. Sociolinguistic Studies, 12(3–4), 461–479. https://doi.org/10.1558/sols.32924
Google Scholar DOI: https://doi.org/10.1558/sols.32924

Stolarski, Łukasz. 2020. The influence of character’s gender and the basic emotions of ‘happiness’ and ‘sadness’ on voice pitch in the reading of fiction. Brno Studies in English, 46(1), 49–89. https://doi.org/10.5817/BSE2020-1-3
Google Scholar DOI: https://doi.org/10.5817/BSE2020-1-3

Stolarski, Łukasz. 2021. Comparison of key statistical instruments used in lexicon-based tools for sentiment analysis in the English language. Token: A Journal of English Linguistics, 13: 219–248.
Google Scholar

Taboada, Maite, Brooke, Julian, Tofiloski, Milan, Voll, Kimberly and Stede, Manfred. 2011. Lexicon-based methods for sentiment analysis. Computational Linguistics, 37(2): 267–307. https://doi.org/10.1162/COLI_a_00049
Google Scholar DOI: https://doi.org/10.1162/COLI_a_00049

Thelwall, Mike. 2017. Heart and soul: Sentiment strength detection in the social web with SentiStrength (summary book chapter). In J. Holyst (ed.), Cyberemotions: Collective emotions in cyberspace, 119–134. Berlin, Germany: Springer. https://doi.org/10.1007/978-3-319-43639-5_7
Google Scholar DOI: https://doi.org/10.1007/978-3-319-43639-5_7

Thelwall, Mike and Buckley, Kevan. 2013. Topic-based sentiment analysis for the social web: The role of mood and issue-related words. Journal of the Association for Information Science and Technology, 64(8), 1608–1617. https://doi.org/10.1002/asi.22872
Google Scholar DOI: https://doi.org/10.1002/asi.22872

Thelwall, Mike, Buckley, Kevan and Paltoglou, Georgios. 2012. Sentiment strength detection for the social web. Journal of the Association for Information Science and Technology, 63(1), 163–173. https://doi.org/10.1002/asi.21662
Google Scholar DOI: https://doi.org/10.1002/asi.21662

Thelwall, Mike, Buckley, Kevan, Paltoglou, Georgios, Cai, Di, and Kappas, Arvid. 2010. Sentiment strength detection in short informal text. Journal of the American Society for Information Science and Technology, 61(12): 2544–2558. https://doi.org/10.1002/asi.21416
Google Scholar DOI: https://doi.org/10.1002/asi.21416

Thelwall, Mike, Buckley, Kevan, Paltoglou, George, Skowron, Marcin, Garcia, David, Gobron, Stephane, … Holyst, Janusz A. 2013. Damping sentiment analysis in online communication: discussions, monologs and dialogs. In International Conference on Intelligent Text Processing and Computational Linguistics (pp. 1–12). Berlin, Germany: Springer. https://doi.org/10.1007/978-3-642-37256-8_1
Google Scholar DOI: https://doi.org/10.1007/978-3-642-37256-8_1

Traunmüller, Hartmut and Eriksson, Anders. 1995. The frequency range of the voice fundamental in the speech of male and female adults. Unpublished Manuscript.
Google Scholar

Wallbott, Harald G. and Scherer, Klaus R. 1986. Cues and channels in emotion recognition. Journal of Personality and Social Psychology, 51(4), 690–699. https://doi.org/10.1037/0022-3514.51.4.690
Google Scholar DOI: https://doi.org/10.1037/0022-3514.51.4.690

Wöllmer, Martin, Weninger, Felix, Knaup, Tobias, Schuller, Björn, Sun, Congkai, Sagae, Kenji and Morency, Louis-Philippe. 2013. Youtube movie reviews: Sentiment analysis in an audio-visual context. IEEE Intelligent Systems, 28(3), 46–53. https://doi.org/10.1109/MIS.2013.34
Google Scholar DOI: https://doi.org/10.1109/MIS.2013.34

Wu, Wei, Zheng, Thomas Fang, Xu, Ming-Xing, and Bao, Huanjun. 2006. Study on speaker verification on emotional speech. In Proceedings of Ninth International Conference on Spoken Language Processing, INTERSPEECH, 2102–2105. Pittsburgh, Pennsylvania. https://doi.org/10.21437/Interspeech.2006-191
Google Scholar DOI: https://doi.org/10.21437/Interspeech.2006-191

Zadeh, Amir, Chen, Minghai, Poria, Soujanya, Cambria, Erik, and Morency, Louis-Philippe. 2017. Tensor fusion network for multimodal sentiment analysis. ArXiv Preprint ArXiv:1707.07250. https://doi.org/10.18653/v1/D17-1115
Google Scholar DOI: https://doi.org/10.18653/v1/D17-1115

Zhu, Xiaodan, Kiritchenko, Svetlana and Mohammad, Saif M. 2014. NRC-Canada-2014: Recent Improvements in the Sentiment Analysis of Tweets. In SemEval@ COLING, 443–447. https://doi.org/10.3115/v1/S14-2077
Google Scholar DOI: https://doi.org/10.3115/v1/S14-2077

Zuberbier, Erika. 1957. Zur Schreib-und Sprechmotorik der Depressiven. Zeitschrift Für Psychotherapie Und Medizinische Psychologie, 7, 239–249.
Google Scholar