Preprint
Article

The Impact of Digital Media on Event-Related Perception

Altmetrics

Downloads

119

Views

51

Comments

0

This version is not peer-reviewed

Submitted:

28 June 2023

Posted:

30 June 2023

You are already at the latest version

Alerts
Abstract
The digital breakthrough of the nineties has profoundly changed the way events are recognized. Starting from EST (Event Segmentation Theory) elaborated by J.M. Zacks and his collaborators, this contribution shows how the way we segment the experiential flow has been changed by the introduction of digital: while throughout modernity the perception of events was based on immediate segmentation processes, founded on five elements (agent, space, time, intention, purpose) and determined by an evident con-clusion, we currently see in the perceptive models of individuals and consequently also in aesthetic texts (films, television series, novels, graphic novels etc.) a weakening of the end. This determines not only a greater difficulty in perceiving an event, but also the need to introduce micro-caesuras between one event and another: it is space that con-stitutes the most salient category in differentiating one event from another, as is evident in the transmedia storytelling that characterizes contemporary aesthetics.
Keywords: 
Subject: Arts and Humanities  -   Literature and Literary Theory
The digital turning point at the end of the twentieth century has undoubtedly changed both the act of writing and reading, and the most widespread hypothesis today is that writing and reading have become synonyms of look. The hypothesis, which naturally involves a historic overtaking of the verbal code by the iconic code, was put forward in time by the French neuroscientist Stanislas Dehaene in the belief that the plasticity of the brain makes it possible to recycle and readapt pre-existing neuronal networks to new cognitive and perceptive needs [1] (pp. 169-170). In the specific case, in an extreme adaptive effort to the new forms of communication induced by the digital, the areas of Broca and Wernicke - traditionally dedicated to the syntactic articulation and semantic decoding of verbal language, domiciled in the left temporal lobe - would be systematically linking with the much larger areas of vision, occupying a large part of the occipital lobe of the left hemisphere and a large portion of the right hemisphere. A very unequal competition.
Until twenty years ago, neuro-imaging tests detected an activation of the vision areas during the reading of verbal texts only in the case of oriental ideogrammatic languages, especially Chinese and Japanese, while today reading texts on digital media reveal a surprising activation of the visual areas. To read, we turn to the gaze, first of all because the unavoidable presupposition of reading is the eye's willingness to concentrate on a single thing to the exclusion of all the rest, and this focusing exercise takes place, as it were, in the basements of the brain, precisely in the thalamus. The times are short, but during reading in about 400 milliseconds both hemispheres light up in the areas dedicated to vision. Maryanne Wolf, an attentive scholar of the changes in reading processes in the era of transference from the verbal to the iconic, has demonstrated how during the process of reading a verbal text today subjects activate both the areas used for verbal language and those which preside to visual perception. Wolf uses the metaphorical field of photography (literally, the transcription of light) to be more suggestive in her arguments, but in reality this is what can be seen in neuro-imaging experiments [2]. Now, digital has not only given a pre-eminent value to the iconic code compared to the verbal one, but it has also changed the way of conceiving events, and it is on this problem that I will now focus. Segmentation into events is a fundamental process of human cognition that organizes the continuous flow of activities into discrete and hierarchical units: it is an automatic cognitive process of fragmenting information about experience into significant events. Event segmentation theory [3,4,5] believes that segmentation plays a fundamental role in normal daily perceptual processing and attentional control. In everyday life, to understand reality, we cognitively segment the continuous flow of experience into discrete parts and dynamically recompose these segments into a meaningful and dynamic whole: our lives unfold fluidly, yet when we reflect on the past we remember experiences as distinct events. Numerous studies have also highlighted the link between the ability to segment events and the development of many areas of cognition, such as memory, social competence and language.
The definition of an event within discrete units is related to the activity of the hippocampus which encodes information about the event just concluded in episodic memory, so that memories of such events can be reintegrated into cortical regions and into long-term memory, with a stronger impact for more strongly codified events and with stronger emotional involvement [6]. There is a deep connection between temporal inference processes and the way in which the brain segments events: the segmentation of events produces the mental representations that allow temporal reasoning. Recent research focuses in particular on how the brain is able to elaborate continuous experiences until constructing discrete temporal representations. Behavioral and neuroimaging data suggest that to construct representations of events, individuals rapidly integrate multiple conceptual and perceptual cues, such as a movement to a new spatial location or the introduction of a new character or object into the perceiver's environment [7] (p. 160). Neural evidence supports three hypotheses about event segmentation: (a) event segmentation is a continuous and automatic process; (b) events are segmented into discrete representations relative to a time sequence, where events are incorporated into other events; (c) segmentation into events is activated thanks to the detection of perceptual changes in stimuli and conceptual changes in mental representations of speech [8] (p. 490).
Not only do event models have an internal structure, but they also form rich connections between the events themselves and hierarchy can be considered both as an aspect of the internal structure of the event and in terms of the relationships between events; hierarchical organization is ubiquitous in event segmentation and description data [9] (p. 653) and appears to increase with familiarity with a particular sequence of events [10] (p. 28), as the perception of the event is strongly influenced by knowledge. How an observer understands a particular event depends on how well he applies knowledge about the immediate stimulus, and a key-thing that knowledge does for the perception of events is to enable predictions about how events will unfold: perception of the event therefore has a predictive nature, whereby the more unpredictable the activity becomes, the more likely it is that explicit predictions about the future will become difficult, and that the attempt to segment is associated with a greater presence of forecast errors [7] (p. 158).
Developments in event segmentation studies justify the use of the broader term "event cognition" to describe the new state of the art; these empirical developments have been accompanied by a new generation of theories, which have sought to explain the interplay between perception, narrative understanding, action, memory, segmentation of ongoing activity, and temporal organization of events. Therefore, according to the theory of segmentation into events, as it has been elaborated mainly by J. M. Zacks [9] (p. 753), narratives are understood thanks to the innate ability in every human being to segment into events, an ability that allows readers to identify narrative units and to segment the story based on the presence of salient events. During the encoding of a narrative, changes in causes, characters, interactions between characters, interactions with objects, objectives and space can be identified. These are the elements that viewers of a film or readers of a text are likely to monitor and use to identify boundaries of an event in such a way as to realize that another event is being outlined and therefore the previous one is concluding: at this point the event model is updated and the understanding of the narrative takes place [11] (pp. 1821- 1826).
According to EST (Event Segmentation Theory), the perception of event boundaries involves multiple cognitive and neural mechanisms that interact in a specific way, and perceptual processing leads to predictions about the near future, being influenced by event patterns held in working memory. Event models in turn are updated when predictions are wrong; during updating, event patterns are influenced by episodic long-term memory, general semantic knowledge, and event-specific semantic knowledge [9] (p. 753). Events, central elements of human experience, can be formally defined in terms of the entities that compose them, together with the characteristics and relationships between these entities; on a cognitive level, the representations of events are made up of their space-time position, the people (and the objects involved and the relationships between these elements), the objectives of the protagonists and the causes that manifest themselves during the action. Representations of events are like images isomorphic to the situations they represent. However, they are also partial models or representations that leave out some abstract elements: the representations of the individual events are formed by the schematic knowledge of the general classes of events and are built thanks to a process that transforms continuous reality into discrete events. The construction of a series of relevant representations forms a basis for predicting and planning the future and imagining alternatives. Objects are central to human experience, and much research has been devoted to how objects themselves are represented by the mind and brain. Arrays of objects form scenes, and this notion too has played a major role in theories of perception; one can easily imagine life on a desert island without other people, but not life without objects or scenes. Žižek [12] (p. 80) argued that events are the raw material of human experience: they seem to be the elements that make up the flow of experience, of things that are remembered or forgotten in autobiographical memory, and the components of plans for future action. Žižek also introduce the concept of “event model” as a representation of a current event and explain how it relates to similar concepts, describing how such representations are structured and how, in their creation, continuous reality is segmented into events and this includes events that take place in a space-time context, involving entities that have properties and functional relationships with each other.
Therefore, segmentation is a powerful perceptual operation: by reducing a continuous stream of activities to a modest number of discrete events, a perceiver can achieve a formidable economy of representation for perception and subsequent memory. Segmentation is not only economical, but it also allows us to think of events as discrete parts that can be combined in new ways. This type of cognition is notoriously difficult with continuous, unsegmented representations. For this reason, people generally perceive space as consisting not of continuous gradations of color and texture, but of spatially coherent objects. The same is true for time: just as our everyday perceptual world is made up of discrete objects, it is made up of discrete events [13]. Perceptual processing transforms sensory input into elaborate representations that include predictions [14] (p. 614): this processing is affected by event patterns; such models influence the perceptual processing flow, allowing to fill in missing information and to complete ambiguous information.
For event models to be effective, they must be mostly protected from the variety of sensory inputs, maintaining stability in the face of ambiguous or biased data. To be effective, event templates also need to be updated from time to time. But how to update event templates at the right time? EST argues that as prediction error increases, participants update their event models based on available sensory and perceptual information. In most cases, the new event model will be more effective than the old one and the error will be reduced as the system stabilizes [14]. Updating will tend to be hierarchical, such that upgrading to higher levels rarely occurs without concurrent upgrading to lower levels, and there is specific evidence for these mechanisms. Behavioral and neurophysiological data demonstrate that people construct a series of event patterns during perception and reading, and that in perception, passive viewing of events produces transient increases in brain activity at event boundaries. In summary, there are several ways in which an event model can be updated [14] (p. 619): at one extreme, event models can be rebuilt whenever an event boundary is encountered and this is one way economically viable, but it may be inefficient. At the other extreme, event models can be incrementally updated, revised, and combined: this mode is flexible and powerful, but perhaps too flexible and powerful, robbing event models of their explanatory force in cognitive theories.
In the literature after the 1990s we do not find the characteristics of nineteenth-century naturalism but rather the will to take hold of a reality aimed at narrating events which, by appealing to the theory of possible worlds, can be read not so much as realistic, but plausible and endowed with a certain cognitive density. The event is no longer a functional element for the effectiveness of the plot, but is identified with the narration itself, which precisely conveys a message-event that runs through the entire novel. In the contemporary novel the way of acting of the characters is proposed by skilfully mixing events (what happens to me) and actions (what I make happen) to the point of eliding the very status of actions and events: in Everyman (2006) Philip Roth, for example, shows a character who dies right in the first pages of the novel – a character who has acted and has undergone events, marriages and divorces, adultery and quarrels with his children, but nothing is now distinguishable: there are no more actions, and there are no more events.
Baudrillard argued that “we live in an integral reality, everything is represented online, in real time, there is no longer the distance of representation between what happens and me who looks, or between me who acts and the fruit of my action. There is no longer this distance, there is no longer a sign, there is nothing anymore”. Well, within this nothingness what disappears for Baudrillard is also the event, the historical cataclysm, the one that came suddenly and unrepeatable, like the French Revolution or the world conflicts, but that all in all it was thought that it really had to happen, since everything conspired to the fact that it happened. And it is here that Baudrillard spoke of "nostalgia for the event":
"There is in us an immense desire for an event and an immense disappointment, because the contents of information are desperately inferior to the power of the means of diffusion. This disproportion creates a need ready to launch itself upon any accident, to crystallize upon any catastrophe. It is not a matter of voyeurism, but a spontaneous reaction to an immoral situation. Information overload creates an unethical situation in that it has no equivalent in the real event. Automatically we want a maximum event, a fatal event, which repairs the immense trivialization of life brought about by information. We dream of senseless events capable of freeing us from this tyranny of meaning and the bond of causes. At the same time, we live in terror of the excess of signification and in that of total insignificance” [15] (p.58).
Truly an extraordinary aphorism, which encompasses the meaning and new meaning of an event, which becomes a founding element, engaging, almost rampant: it is no longer just part of the structure of the narrative plot, but at the same time it loses itself in the excess of information.
Now, the digital has had a great responsibility in instilling the segmental logic that presided over the novels of the twentieth century, erasing the boundaries that enclosed the stories. Today the phenomenon of serialization has effectively eliminated the concept of "end", and in fact even the identity of the characters is no longer perimeter: gender slips, individual memory changes with the passage of time, past and future often come together in a fascinating liquid future anterior [16] (p. 26). Everything changes because the perception of the correlated event changes and this is demonstrated by the so-called transmedia storytelling, i.e. In his book Convergence Culture, Jenkins defines a transmedia story as a story that "develops across multiple media platforms, with each new text contributing distinctive and valuable as a whole" [17]. More precisely, Ivan Askwith and Jonathan Gray [18] (pp. 519-520) stated that:
“taken by itself, the term 'transmedia' simply describes the process of moving or expanding content from one medium to another…the transmedia storytelling is more specific and is used to describe the process of further developing a coherent narrative (or elaborating a narrative universe) through the distribution of related narrative components across multiple media platforms” [18] (pp. 519-520).
Let's think of the Marvel universe: Marvel is owned by the Walt Disney Company, a large company that operates on several levels in the communication sector, with strategies that touch on various areas, from production to distribution. The Marvel Universe is expanded and shared by the many superheroes that are part of the Marvel Studios media properties. Starting from the film Iron Man and (2008) up to Black Widow (2020) passing through the Avengers films, Marvel has given life to a constellation of media products that have not only to do with a narrative that unfolds on multiple media, but with a multifaceted project, which draws on the world of Marvel comics, from which it draws the characters and narrative lines to recombine them according to production needs which translate into a design that can be modified and expanded over time [19].
In the fiction dedicated to individual characters, some narrative elements are taken up in the collective story that brings together all the characters (The Avengers) but they are not essential for understanding the entire plot. Moments of interconnectivity constitute but a few scenes in otherwise self-contained films: for example in The Incredible Hulk (2008) a WWII-era serum is partly responsible for turning the villain Emil Blonsky into an abominable being. While readers of Marvel comics might have recognized this serum as the very one that gave Captain America his heightened abilities, such knowledge is unnecessary for an unsuspecting millennial audience. Indeed, most of the continuity moments in Marvel films are wrapped up in mid-film teasers (for example, Nick Fury's appearance in Iron Man), which do not disturb the flow of the film. In this way, authors have enough autonomy to engage a mass audience, while still including the "flow tags" that direct consumers to the wider transmedia story. This careful balance is essential for the success of the transmedia, and here it is of interest in order to understand what has been said previously: while not being recognizable by everyone, the event concerning the serum is distributed across multiple channels. The discriminating element is the action, represented by the act of ingesting the serum, which has fundamental consequences for the entire narrative plot. With reference to the hierarchical nature of the events, it can be seen how precisely the construction and reticular structure of transmedia storytelling gradually allows an increasingly vast audience to recognize events in the narrative flow on the basis of the previous knowledge of the users.
For this reason, it can be said that the difference in the recognizability of events in transmedia storytelling is entrusted, more than in linear narration, to the skills and knowledge of the users with respect to the entire narrative universe. Narrative ecosystems are in fact the result of a dialectic between project design and use made by users; the individual products can be personalized and diversified, and there is a high degree of unpredictability even in the most carefully planned project, because misalignments can occur between the events narrated and the use of the media objects by the audience. The event is therefore the basis of the various modifiable worlds, capable of creating not only universes, but entire ecosystems that take their cue from a single event that becomes more central and flexible than ever: in conclusion we could say that the event in transmedia storytelling can be identified exactly as in the linear narrative structure, but it is the reader who makes the event what it is, while before digital the event presented itself as such to the gaze of the reader, who only had to acknowledge it, without making a segmentation work today become much more complex.

References

  1. Dehaene, S. I neuroni della lettura. Italian translation by Corrado Sinigaglia; Raffaello Cortina: Milano, 2009. [Google Scholar]
  2. Wolf, M. Lettore vieni a casa. Il cervello che legge in un mondo digitale. Italian translation by Patrizia Villani; Vita e Pensiero: Milano, 2020. [Google Scholar]
  3. Zacks, J.M; Tversky, B. Event structure in perception and conception. Psychological Bulletin 2001, 127, 3–21. [Google Scholar] [CrossRef]
  4. Zacks, J.M. Ten Lectures on the Representation of Events in Language, Perception, Memory, and Action Control; Brill: New York, 2020. [Google Scholar]
  5. Radvansky, G.A.; Zacks, J.M. Event cognition; Oxford University Press: Oxford, 2014. [Google Scholar]
  6. Feller, D.P.; Eerland, A.; Ferretti, T.R.; Magliano, J.P. Aspect and narrative event segmentation. Collabra: Psychology 2019, 5, 1–12. [Google Scholar] [CrossRef]
  7. Zacks, J.M.; Swallow, K.M.; Vettel, J.M.; McAvoy, M.P. Visual movement and the neural correlates of event perception. Brain Res 2006, 1076, 150–162. [Google Scholar] [CrossRef] [PubMed]
  8. Schapiro, A.C.; Rogers, T.T.; Cordova, N.I.; Turk-Browne, N.B.; Botvinick, M.M. Neural representations of events arise from temporal community structure. Natural Neurosciences 2013, 16, 486–492. [Google Scholar] [CrossRef]
  9. Zacks, J.M.; Braver, T.S.; Sheridan, M.A.; Donaldson, D.I.; Snyder, A.Z.; Ollinger, J.M.; Buckner, R.L.; Raichle, M.E. Human brain activity time-locked to perceptual event boundaries. Nature Neuroscience 2001, 4, 651–655. [Google Scholar] [CrossRef]
  10. Hard, B.M.; Meyer, M.; Baldwin, D. Attention reorganizes as structure is detected in dynamic action. Memory & cognition 2019, 47, 17–32. [Google Scholar]
  11. Stahl, A.E. Infants segment continuous events using transitional probabilities. Child development 2014, 85, 1821–1826. [Google Scholar]
  12. Žižek, S. Event: A Philosophical Journey Through a Concept; Penguin Book: London, 2014. [Google Scholar]
  13. Zacks, J.M.; Swallow, K.M. Event segmentation. Current directions in psychological science 2007, 16, 80–84. [Google Scholar] [CrossRef]
  14. Radvansky, G.A.; Zacks, J.M. Event perception. Wiley Interdisciplinary Reviews: Cognitive Science 2011, 2, 608–620. [Google Scholar] [CrossRef]
  15. Baudrillard, J. Power inferno. Requiem per le Twin towers. Ipotesi sul terrorismo. La violenza globale; Raffaello Cortina: Milano, 2003. [Google Scholar]
  16. Baricco, A. La via della narrazione; Feltrinelli: Milano, 2023. [Google Scholar]
  17. Jenkins, H. Convergence Culture; New York University Press: New York, 2006. [Google Scholar]
  18. Askwith, I.; Gray, J. Transmedia Storytelling and Media Franchises. In Battleground: the Media; Anderson, R., Gray, J., Eds.; Greenwood: WestPort, 2008; pp. 519–620. [Google Scholar]
  19. Calabrese, S. Narrare al tempo della globalizzazione; Carocci: Roma, 2016. [Google Scholar]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
Prerpints.org logo

Preprints.org is a free preprint server supported by MDPI in Basel, Switzerland.

Subscribe

© 2024 MDPI (Basel, Switzerland) unless otherwise stated