{"id":384,"date":"2024-06-17T09:23:54","date_gmt":"2024-06-17T09:23:54","guid":{"rendered":"https:\/\/wp.lancs.ac.uk\/public-discourses-of-dementia\/?p=384"},"modified":"2024-06-17T09:29:20","modified_gmt":"2024-06-17T09:29:20","slug":"eyes-gaze-and-vision-do-humans-and-ai-see-eye-to-eye-when-it-comes-to-representing-people-living-with-dementia","status":"publish","type":"post","link":"https:\/\/wp.lancs.ac.uk\/public-discourses-of-dementia\/2024\/06\/17\/eyes-gaze-and-vision-do-humans-and-ai-see-eye-to-eye-when-it-comes-to-representing-people-living-with-dementia\/","title":{"rendered":"Eyes, gaze and vision: Do humans and AI see \u201ceye to eye\u201d when it comes to representing people living with dementia?"},"content":{"rendered":"\n

Why look at eyes, gaze and vision?<\/em><\/strong><\/p>\n\n\n\n

Eyes and vision have great cultural significance: eyes are seen as \u201cwindows to the soul\u201d and in (Western, neurotypical) societies, eye contact has been associated with trust, confidence, attention, social recognition, inclusion and rapport building \u2013 although of course this is context dependent (Jarrett, 2019; Jongerius et al., 2020; Kreysa et al., 2016; MacDonald, 2009; Tessler and Sushelsky, 1978). Interestingly, research indicates that images, such as portraits and photos that mimic eye contact through a direct gaze, can actually evoke similar<\/em> brain responses to real-life eye contact (Kesner et al., 2018), making them important artefacts to examine. In real life interactions, eye contact can be very important for people living with dementia, for instance, Allan Davis (2015)<\/a> says that: \u2018My attention span and ability to pay attention are not as good as they once were. Please make eye contact with me before you start talking. A nice smile always gets my attention.\u2019<\/p>\n\n\n\n

Relatedly, humans draw on our physical experiences with the world to help us to communicate less tangible aspects. Within this, vision <\/strong>is metaphorically linked to cognition<\/strong>, at least in a Western context (Ibarretxe-Antu\u00f1ano, 2008). For instance, we talk about seeing<\/em> one another\u2019s viewpoints<\/em> when we communicate what we\u2019re thinking and feeling \u2013 and you can hopefully see<\/em> the point I\u2019m making here, rather than it being about as clear as mud<\/em>\u2026 As Schweda (2019: p. 4) has argued: when truth is conceptualised in terms of light (e.g., not telling someone the truth is \u201ckeeping them in the dark\u201d) and cognition is conceptualised as visual perception, \u2018declining cognition for conditions such as dementia can be symbolized as the impairment <\/strong>of this visual perception\u2019, often by meteorological phenomena such as fog, rain and darkness. Such a metaphor can be useful; for instance, Keith Oliver (2015)<\/a> says that: \u2018whilst it\u2019s easy [to] exaggerate using metaphors, I think it can be useful to give people insight into my world of living with dementia. One metaphor I use is about the fog. Which on bad days it\u2019s like living in a fog, on good days \u2013 and there are more good days than bad \u2013 the sun shines and life is much clearer.\u2019<\/p>\n\n\n\n

Amongst humans, then, eyes and vision are important tools both for relationship building and communicating about our subjective experiences (or inner worlds). In a society where artificial intelligence (AI) is rapidly evolving and increasingly being integrated into images and texts, we were interested in how AI might in turn represent people living with dementia \u2013 and because that\u2019s such a big question for a little blog post, today, for the reasons mentioned above, I\u2019m just focusing on eyes, gaze and vision.<\/p>\n\n\n\n

<\/div>\n\n\n\n

 Generating images and character descriptions with AI<\/em><\/strong><\/p>\n\n\n\n

In our research, we used Stable Diffusion (version 1.4) to generate images. Stable Diffusion is a deep learning model that has been trained on a huge dataset of captioned images so that it can generate images in a range of styles in response to the user\u2019s textual prompts. We used the text prompt \u201cdementia\u201d to generate a total of 171 images by using all of the available image generation models and the three main formats: landscape, portrait and square.<\/p>\n\n\n\n

To generate character descriptions of people with dementia, we used Sudowrite, which advertises itself as \u2018the non-judgemental, unexpectedly creative AI writing tool that sounds like you, not a robot\u2019 (Sudowrite, 2024<\/a>). Sudowrite is an AI writing programme based on GPT-3 and GPT-4 that is aimed at fiction writers and \u2018generates text by guessing what’s most likely to come next, one word at a time\u2019 \u2013 these guesses are based on its training data (Sudowrite 2024<\/a>). In this study, we used Sudowrite\u2019s \u201cdescribe\u201d function, which categorises its output according to the five senses (sight, sound, touch, taste, smell) and metaphors. We used four distinct textual prompts that considered two genders and the perspectives of a character with (\u2018I am<\/strong> a (wo)man with dementia\u2019) and without dementia (\u2018I saw<\/strong> a (wo)man with dementia\u2019) to generate a total of 52 separate character descriptions, totalling 22,638 words.<\/p>\n\n\n\n

<\/div>\n\n\n\n

What did we find?<\/em><\/strong><\/p>\n\n\n\n

1. Fogginess: Dementia is associated with impaired vision.<\/strong><\/p>\n\n\n\n

The metaphorical representation of dementia as impaired vision was surprisingly prevalent in our character descriptions. The experience of characters with dementia is one of a world of fog, blurriness, dimness and greyscale, for example:<\/p>\n\n\n\n

\u2018My eyes are dimmed with fog<\/em>. [\u2026] My surroundings are grey<\/em>, colors have long since left me. A world of grey faces and black and white is all I know.\u2019<\/p>\n\n\n\n

\u2018I see the world as blurry<\/em>, like an old film shot inside a rain cloud<\/em>. Colors become muted<\/em> and indistinct<\/em>, objects lose their sharpness. It is a drab<\/em> world with little contrast<\/em>.\u2019<\/p>\n\n\n\n

Interestingly, this lack of colour can also be seen visually. Indeed, 56 of the 171 images are in greyscale, and many more are predominately <\/em>greyscale, or otherwise use a limited colour palette of blues, beiges and shadows when representing the person with dementia (see Figure 1).<\/p>\n\n\n

\n
\"Three
Figure 1.<\/strong> Three AI-generated image examples. The first image is in full greyscale (see also the featured image), the second image exemplifies the use of shadows\/darkness and the third uses a partial (blue oriented) colour palette when representing the individual.<\/figcaption><\/figure>\n<\/div>\n\n\n
<\/div>\n\n\n\n

2.<\/strong> Both AI outputs establish social distance from people with dementia.<\/strong><\/p>\n\n\n\n

    <\/ol>\n\n\n\n

    Importantly, 107 of the 130 images that we coded as showing people showed individuals<\/em>, usually not looking at viewers and instead either looking off camera (most frequently downwards) or seeming to have their eyes closed. This encourages viewers to observe <\/em>these individuals rather than seek to connect, which helps to create a sense of social distance between the represented individuals and viewers.<\/p>\n\n\n

    \n
    \"Three
    Figure 2. <\/strong>Examples of characters\u2019 gaze in the AI-generated images.<\/figcaption><\/figure>\n<\/div>\n\n\n
    <\/div>\n\n\n\n

    3. Both AI outputs associate people with dementia with living death.<\/strong><\/p>\n\n\n\n

    While eyes can be positioned as a window to inner emotional states (usually of fear or suffering, as with \u2018His face looked right, but his eyes did not. They showed fear and pain within\u2019), oftentimes, eyes are instead used to attribute an emptiness<\/em> to people with dementia:<\/p>\n\n\n\n

    \u2018Her eyes are empty<\/em> windows, a blank<\/em> gaze, like a doll<\/em>\u2019s\u2019<\/p>\n\n\n\n

    \u2018The woman is a ghost<\/em> of her former self. [\u2026] Most unsettling are her eyes. They look at something that isn\u2019t there. One sees only blankness<\/em>\u2019<\/p>\n\n\n\n

    Associated with this sense of emptiness, or blankness, is an inhumanity<\/em>, as exemplified by the comparisons to inanimate objects (a doll) and spirits (ghost) above. In the images, this sense of deathliness is instead conveyed through the visual metaphor of bare trees, which I\u2019ve unpacked further in a previous blog post<\/a>. Arguably, as well as being related to the impaired vision metaphor, the trend of using a drab colour palette (Figure 1) adds to this sense of lifelessness.<\/p>\n\n\n\n

    <\/div>\n\n\n\n

    So what?<\/em><\/strong><\/p>\n\n\n\n

    Although there some examples of social connection (e.g., \u2018\u2018When she saw me, her eyes lit up<\/em> with surprise and warmth\u2019) and colour in the AI-generated representations, these were few and far between. The tropes therefore indicate an overall imbalance in the representation of people living with dementia that foregrounds social distance, blankness, deathliness and a grey, foggy and dark world. Yet, to return to Keith Oliver\u2019s use of the fog metaphor, although \u2018on bad days it\u2019s like living in a fog\u2019, there can also be good (or at least better) days, where \u2018the sun shines and life is much clearer\u2019. Our dataset would certainly suggest that more sunshine and colour is needed when training AI to represent dementia. However, thinking of the recent, highly controversial Alzheimer\u2019s Society advert (which claims that \u2018With dementia, you don\u2019t just die once. You die again, and again, and again<\/a>\u2019), a similarly critical eye needs to also be turned to human-produced representations, too. \u00a0Dementia is a diverse and complex condition, and people\u2019s experiences are similarly diverse and multifaceted. Conversations need to be had (again, and again, and again) about how to better reflect this breadth in public discourses moving forward. \u00a0<\/p>\n\n\n\n

    <\/div>\n\n\n\n

    Interested in reading more?<\/strong><\/p>\n\n\n\n

    Putland, E., Chikodzore-Paterson, C. and Brookes, G. (2023). \u2018Artificial intelligence and visual discourse: a multimodal critical discourse analysis of AI-generated images of \u201cDementia\u201d\u2019, Social Semiotics<\/em>, 0(0): https:\/\/doi.org\/10.1080\/10350330.2023.2290555<\/a><\/p>\n\n\n\n

    Putland, E., Chikodzore-Paterson, C. and Brookes, G. (Forthcoming). ‘\u201dWe no longer recognized her as a human being\u201d: A Critical Discourse Analysis of AI-generated character descriptions of men and women with dementia\u2019.<\/p>\n\n\n\n

    <\/div>\n\n\n\n

    <\/p>

    <\/p>

    References<\/b><\/p><\/p>\n\n\n\n

    Davis, A. (2015). ‘Thought for the week No. 6: \u201cPlease make eye contact with me before you start talking\u201d’, Dementia Diaries<\/em>. https:\/\/dementiadiaries.org\/entry\/2164\/thought-for-the-week-no-6-my-attention-span-and-ability-to-pay-attention-are-not-as-good-as-they-once-were-please-make-eye-contact-with-me-before-you-start-talking\/?highlight=eye<\/a><\/p>\n\n\n\n

    Ibarretxe-Antu\u00f1ano, I. (2008). \u2018Vision Metaphors for the Intellect: Are they Really Cross-Linguistic?\u2019, <\/span>Atlantis<\/span><\/i>, 30(1): pp. 15-33. <\/span><\/p><\/p>\n\n\n\n

    Jarrett, C. (2019). \u2018<\/span>Why meeting another\u2019s gaze is so powerful\u2019, BBC Future. Available at: <\/span>https:\/\/www.bbc.com\/future\/article\/20190108-why-meeting-anothers-gaze-is-so-powerful<\/span><\/a> <\/span><\/p><\/p>\n\n\n\n

    Jongerius, C., Hessels, R. Romijn, J.,\u00a0Smets, E. and\u00a0Hillen, M. (2020). \u2018<\/span>The Measurement of\u00a0Eye Contact
    in\u00a0Human Interactions: A\u00a0Scoping Review\u2019, <\/span>Journal of Nonverbal Behavior<\/i>, 44: pp. 363\u2013389. <\/p><\/p>\n\n\n\n

    Kesner, L., Grygarov\u00e1a, D., Fajnerov\u00e1, I., Lukavsk\u00fda, J., Nekov\u00e1\u0159ov\u00e1, T., Tint\u011bra, J., Zaytseva,
    Y., Hor\u00e1\u010dek, J. (2019). \u2018<\/span>Perception of direct vs. averted gaze in portrait paintings: An fMRI and eyetracking study\u2019, Brain and Cognition<\/i>, 125: pp. 88-99. <\/span><\/p><\/p>\n\n\n\n

    Kreysa, H., Kessler, L. and Schweinberger, S. (2016). \u2018<\/span>Direct Speaker Gaze Promotes Trust in Truth-Ambiguous Statements\u2019, PLOS One 11(9): pp. 1-15. <\/span><\/p><\/p>\n\n\n\n

    MacDonald, K. (2009). \u2018Patient-Clinician Eye Contact: Social Neuroscience and Art of Clinical Engagement\u2019, Postgraduate Medicine<\/i>, 121(4): pp. 136-144.<\/p><\/p>\n\n\n\n

    Oliver, K. (2015). ‘\u201cOn bad days it\u2019s like living in a fog\u201d’, Dementia Diaries<\/em>. https:\/\/dementiadiaries.org\/entry\/1208\/on-bad-days-its-like-living-in-a-fog\/<\/a><\/p>\n\n\n\n

    Sudowrite (2024) Sudowrite homepage<\/em>. https:\/\/www.sudowrite.com\/<\/a>.<\/p>\n\n\n\n

    Tessler, R. and Sushelsky, L. (1978). \u2018Effects of eye contact and social status on the perception of a job applicant in an employment interviewing situation\u2019, Journal of Vocational Behavior<\/i>, 13(3): pp.338-347.<\/p>

    \u00a0<\/p>

    <\/p>\n","protected":false},"excerpt":{"rendered":"

    Inspired by a project presentation given at the Dementia in Film, Media and Culture symposium at Queen Margaret University (Edinburgh, 31st May 2024), here, Emma Putland reflects on representations of people living with dementia in relation to eyes, gaze and vision.<\/p>\n","protected":false},"author":1550,"featured_media":388,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"neve_meta_sidebar":"","neve_meta_container":"","neve_meta_enable_content_width":"off","neve_meta_content_width":70,"neve_meta_title_alignment":"","neve_meta_author_avatar":"","neve_post_elements_order":"","neve_meta_disable_header":"","neve_meta_disable_footer":"","neve_meta_disable_title":"","_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[39],"tags":[37,13,28,27],"class_list":["post-384","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-project-blog","tag-ai","tag-communication","tag-discourses","tag-social-narratives"],"jetpack_featured_media_url":"https:\/\/wp.lancs.ac.uk\/public-discourses-of-dementia\/files\/2024\/06\/Featured-image.png","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/wp.lancs.ac.uk\/public-discourses-of-dementia\/wp-json\/wp\/v2\/posts\/384","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/wp.lancs.ac.uk\/public-discourses-of-dementia\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/wp.lancs.ac.uk\/public-discourses-of-dementia\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/wp.lancs.ac.uk\/public-discourses-of-dementia\/wp-json\/wp\/v2\/users\/1550"}],"replies":[{"embeddable":true,"href":"https:\/\/wp.lancs.ac.uk\/public-discourses-of-dementia\/wp-json\/wp\/v2\/comments?post=384"}],"version-history":[{"count":4,"href":"https:\/\/wp.lancs.ac.uk\/public-discourses-of-dementia\/wp-json\/wp\/v2\/posts\/384\/revisions"}],"predecessor-version":[{"id":393,"href":"https:\/\/wp.lancs.ac.uk\/public-discourses-of-dementia\/wp-json\/wp\/v2\/posts\/384\/revisions\/393"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/wp.lancs.ac.uk\/public-discourses-of-dementia\/wp-json\/wp\/v2\/media\/388"}],"wp:attachment":[{"href":"https:\/\/wp.lancs.ac.uk\/public-discourses-of-dementia\/wp-json\/wp\/v2\/media?parent=384"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/wp.lancs.ac.uk\/public-discourses-of-dementia\/wp-json\/wp\/v2\/categories?post=384"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/wp.lancs.ac.uk\/public-discourses-of-dementia\/wp-json\/wp\/v2\/tags?post=384"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}