Prerpints.org logo
Preprint
Article

The Impact of Types of Recommendation System on User Satisfaction: A Moderated Mediation Analysis

Altmetrics

Downloads

180

Views

65

Comments

0

A peer-reviewed article of this preprint also exists.

This version is not peer-reviewed

Submitted:

21 January 2024

Posted:

23 January 2024

You are already at the latest version

Alerts
Abstract
A significant body of research has examined the accuracy and diversity of several types of recommendation systems. However, little is known about the psychological mechanisms underlying the effects of the different types of recommendation systems on user satisfaction. The findings of our experiment showed that user satisfaction is influenced by shopping goal. Specifically, when a user’s shopping goal aligns with the type of recommendation system, the user satisfaction is enhanced. Our study evaluated the mediating role of feeling right and psychological reactance for a better understanding of this relationship. We further tested the moderated mediation effect of feeling right and psychological reactance moderated by the user shopping goal. For goal-directed users, accurate recommendations trigger the activation of feeling right, consequently increasing the user satisfaction. Conversely, when exploratory users face accurate recommendations, they activate psychological reactance, which leads to a reduction in user satisfaction.
Keywords: 
Subject: Business, Economics and Management  -   Marketing

1. Introduction

As Internet technology advances swiftly, the ways in which enterprises meet user needs are undergoing significant change. Enterprises can leverage recommendation systems to provide personalized recommendations that align with the user preferences or requirements. By collecting extensive transaction data from users, enterprises gain insights into their needs and preferences, and apply this knowledge to inform new product designs and marketing plans (Liang, Lai, & Ku, 2006). For instance, comprehensive online shopping websites can discern a user's shopping preferences by analyzing their transaction data or browsing history, thereby enabling targeted product recommendations. Furthermore, users benefit from recommendation systems not only by accessing a wealth of preferred information, but also by reducing information overload and minimizing the costs associated with data collection and decision-making (Ricci, Rokach, & Shapira, 2015).
Recommendation systems have become indispensable in shopping websites. However, it is essential to recognize that recommendation systems may not universally cater to all users, and in some cases, could evoke resistance, creating a sense of aversion among users (Fitzsimons & Lehmann, 2004); (Lee, Lee, Lee, & Lee, 2009). Initially, research on recommendation systems was focused on enhancing accuracy to improve user satisfaction. Yet, scholars later argued that accuracy alone is insufficient (McNee, Riedl, & Konstan, 2006), leading to studies exploring other characteristics such as diversity and novelty (Hu & Pu, 2011); (Kunaver & Požrl, 2017). However, most of these studies have focused on the recommendation system itself, ignoring the ever-changing needs of users.
The purpose of the present study is to evaluate whether the fit between the type of recommendation system and user shopping goal can increase user satisfaction. This study sheds light on the moderated mediation effect of feeling right and psychological reactance in explaining the impact of the fit between the type of recommendation system and the user shopping goal on satisfaction. The remainder of this paper is organized as follows. The conceptual framework and hypotheses are briefly described. The research methodology is presented, with a description of our measures and data collection. Data and statistical analyses were performed using a regression-based path analysis. Finally, we discuss the study’s research and managerial implications.

2. Theoretical Background

2.1. Type of Online shopping recommendation system

A recommendation system is an information filtering system designed to address the issue of information overload by filtering significant information fragments from a vast pool of dynamically generated content, based on user preferences, interests, or browsing history. In other words, a recommendation system can predict whether a user is likely to favor a particular item based on their individual profile (Isinkaye, Folajimi, & Ojokoh, 2015). Companies and users can benefit from these recommendation systems. In the context of online shopping, these systems assist users in reducing the costs related to information search, product selection, and final decision-making (Isinkaye et al., 2015). As a result, recommendation systems have found widespread application on online shopping websites. Through personalized recommendations, users are aided in decision making, ultimately enhancing user satisfaction (Lee et al., 2009).
The methods used to analyze user preferences in recommendation systems can be broadly categorized into two main types (Figure 1). One is content-based filtering, which is based on the attributes of products, such as the keywords associated with the products. The content-based (CB) recommendation approach involves recommending products similar to those previously liked by the user. The fundamental principles of content-based recommendation systems are as follows: 1) analyze the product that a specific user prefers, identify the common attributes of these products, and store these preferences in the user's profile; and 2) compare the attributes of each product with the user's profile, recommending products that exhibit a high degree of similarity to the user's profile (Lu, Wu, Mao, Wang, & Zhang, 2015). However, this type of recommendation approach carries the risk of excessive personalization, meaning that the recommendations received by users are limited to products that are highly similar to their user profiles (Ricci et al., 2015).
The other is collaborative filtering, which relies on user behavior, such as historical purchases (Liang et al., 2006). Collaborative filtering (CF) can assist users in decision making by taking the choices of other users with similar preferences. Collaborative filtering can primarily be categorized into user- and item-based CF. User-based CF involves recommending products preferred by other similar users, whereas item-based CF involves recommending products similar to the user's past preferences (Lu et al., 2015). CF is considered one of the most effective recommendation types in recommendation systems and is widely employed (Goldberg, Roeder, Gupta, & Perkins, 2001). However, owing to its reliance on recommendations based on other users or products with high similarities, CF can potentially lead to repetitive recommendations and a reduced variety of recommended products (Lee & Hosanagar, 2019).

2.2. Recommendation system and user satisfaction

Previous research on recommendation systems has primarily focused on accuracy. However, user satisfaction does not necessarily depend solely on accuracy. In other words, the accuracy alone is insufficient (Ziegler, McNee, Konstan, & Lausen, 2005). For example, if a user accidentally clicks on a product on a shopping website, the recommendation system may continuously suggest similar products based on the recommendation principle, even though the user's initial interaction may have been accidental. Such recommendation approaches can lead users into a similarity hole, where they continuously receive high-accuracy recommendations and the suggested products share high similarity or are already well known to the user (Matt, Benlian, Hess, & Weiß, 2014).
The concept of filter bubbles was first introduced by Pariser (2011). Websites can provide personalized services to users based on their preferences, actions, and algorithms (Pariser, 2011). Pariser argued that such algorithms create bubbles around users, confining them to a single perspective. This implies that the algorithm excludes diverse perspectives and information outside the bubble. Moreover, because this bubble is crafted using a user's personal information, the filter bubble for each user is unique. This bubble also possesses the characteristics of being invisible as users become trapped within it (Pariser, 2011). In other words, as users become confined within their preference cycle, it becomes increasingly difficult for them to discover alternative viewpoints or domains (Zhang, Séaghdha, Quercia, & Jambor, 2012). However, since user preferences are not permanently fixed, relying solely on high-accuracy recommendations, as mentioned above, is insufficient (McNee et al., 2006).
Furthermore, from the user's perspective, when shopping, if consumers are uncertain about their preferences, they tend to seek variety before making decisions (Zhang, Yuan, 2022). Therefore, when evaluating a recommendation system, it is essential to consider not only its accuracy but also its diversity. Additionally, an increase in the diversity of recommendations implies a decrease in the similarity among recommended products, that is, a loss in recommendation accuracy (Adomavicius & Kwon, 2011). In other words, there is a trade-off between the accuracy and diversity.
User satisfaction is typically used to evaluate the success of information systems (Al-Fraihat, Joy, & Sinclair, 2020). According to Zipf's principle of least effort, a fundamental principle of human action is to exert the least effort to do things (Zipf, 2016). The contrasting concept of least effort is information overload, which involves providing users with information beyond their processing capabilities within a specified timeframe. Personalized recommendation systems can alleviate information overload by offering users information that aligns with their preferences. In other words, following the principle of least effort, personalized recommendation systems can mitigate users' information overload, thereby enhancing user satisfaction.
However, recommendation systems cannot satisfy all users equally. As mentioned earlier, both accuracy and diversity are criteria for evaluating recommendation systems. Therefore, there are users who prefer systems that prioritize high accuracy and others who favor systems that emphasize diversity. The present study posits that user satisfaction depends not only on the type of recommendation system but is also influenced by users themselves, specifically their shopping goals. Currently, no study has considered the impact of alignment between user shopping goals and the type of recommendation system on user satisfaction.

2.3. User shopping goal

The users have diverse online shopping purposes. Some users have a general or ambiguous concept of the product they are looking for (e.g., "I want to buy the latest newly released smartphone"), while others have very specific purchase goals (e.g., "I want to buy the iPhone 14 Pro 256GB space black"). Based on prior research, shopping behavior can generally be categorized into two types: goal-directed and exploratory. Hoffman and Novak (1996) reported that in the online world, users' information processing can be divided into goal-directed and exploratory processes. In other words, goal-directed users have a specific shopping goal before online shopping, whereas exploratory users do not have a goal and must search for information to determine their purchasing objective.
During online shopping, users' behavior varies depending on whether they have a specific shopping goal (Smith, Newman, & Parks, 1997). Specifically, if there is a specific shopping goal, the search for a goal can be considered a goal-directed behavior. If there is no specific shopping goal, the process of making vague goals more concrete can be seen as exploratory behavior. When searching for information on shopping websites, users’ shopping goals can change instantly, based on the information they find (Smith et al., 1997). In summary, the present study categorizes users into goal-directed and exploratory users, based on whether they have a specific shopping goal.

2.4. Feeling right

Previous research suggests that if these two factors mutually fit, users will experience a subjective experience of engagement. This subjective experience of engagement, which increases based on matching, is referred to as ‘feeling right’ (Wadhwa & Zhang, 2015). Feeling is a form of information that can be utilized when making judgments or decisions (Cesario, Grant, & Higgins, 2004). Such a feeling of experience can influence users' evaluations of a product and enhance their assessment of the product in which they are initially interested (Wadhwa & Zhang, 2015). Therefore, during online shopping, if a user's shopping goal fits the type of recommendation system, he or she would experience a ‘feeling right’, which may influence his or her level of satisfaction.

2.5. Psychological reactance

Psychological reactance theory (Brehm, 1966) suggests that individuals have a certain degree of freedom in their behavior. If freedom is diminished or threatened, individuals are motivated to regain it. Psychological reactance consists of three stages: first, perceiving a threat to freedom; second, leading to psychological resistance; ultimately, attempting to restore the threatened freedom (Song, McComas, & Schuler, 2018).
Users need to make judgments and choices from thousands of pieces of information on the internet. Personalized recommendations play an invaluable role in reducing the costs associated with this process. Personalized recommendations can provide users with services or products that match their preferences, helping them reduce the time and effort required for the decision-making process. This convenience enhances the quality of decision-making. Although personalized recommendations can simplify and streamline users' decision-making processes, they also have the potential to undermine their freedom of choice (André et al., 2018). If personalized recommendations interfere with user autonomy in the decision-making process, they may have a counterproductive effect, leading to psychological reactance.
Users' negative responses to information technology are mainly divided into two types: psychological and behavioral responses (Ma, Sun, Guo, Lai, & Vogel, 2022). Research has primarily focused on resistance from the perspective of psychological responses. This refers to the actions that individuals take in opposition to situations in which they perceive themselves to be in a compelled condition (Tucker, 2014). Moreover, psychological reactance has been studied in various fields such as advertising (Youn & Kim, 2019), artificial intelligence (Pizzi, Scarpi, & Pantano, 2021), and personalized recommendations (Fitzsimons & Lehmann, 2004; Ma, Sun, Guo, Lai, & Vogel, 2022). Particularly in the field of personalized recommendations, the research results indicate that personalized recommendations can lead to psychological reactance.
More specifically, Youn and Kim (2019) conducted research on users' avoidance behaviors toward Facebook ads. They suggests that when using Facebook, if users feel that they have the freedom to avoid ads, their perception of ad intrusion decreases. That is, when ad intrusion or the freedom to avoid ads is threatened, it leads to reactance and users engage in actions to avoid ads. Pizzi et al. (2021) indicated that non-anthropomorphic digital assistants could trigger psychological reactance and lead to negative evaluations of artificial intelligence.
Fitzsimons and Lehmann (2004) showed that when users receive recommendations that do not align with their expectations, a resistance state is activated. In such situations, users not only ignore recommendations from the recommendation system, but may also exhibit resistance. Ma et al. (2022) suggested that the greedy recommendation of algorithms, by narrowing the scope of information or recommending repetitive content, can lead to information overload. This, in turn, causes users to feel fatigued, increasing psychological reactance and their intention to interrupt.

3. Conceptual Framework and Hypotheses Development

Recommendation systems have become essential for shopping on websites. According to previous research, when evaluating the quality of recommendation, the focus is often on accuracy, Later, due to the sound of being accuracy is not enough (McNee et al., 2006), some studies relative to accuracy began to appear, such as diversity, novelty, etc. (Hu & Pu, 2011; Kunaver & Požrl, 2017). As there is a trade-off between diversity and accuracy (Adomavicius & Kwon, 2011), these two cannot coexist. However, to evaluate the applicability of the two recommendation approaches in diverse situations, this study believes that it is more meaningful to evaluate their accuracy and diversity.
The present study suggests that recommendation approaches for recommendation systems can be divided into two types based on the recommendation results: recommendations with high accuracy (similarity) and those with high diversity. According to previous research, both types of recommendation approach have positive and negative effects on user satisfaction. Therefore, this study believes that user satisfaction depends not only on the type of recommendation system but also on the user shopping goal. When the type of recommendation system fits the user shopping goal, the user may feel right, which in turn leads to a higher user satisfaction. By contrast, under the condition of unfitness, user satisfaction decreases owing to the activation of psychological reactance. Figure 2 illustrates the research model.
Task-Technology Fit (TTF) refers to the Fit between the characteristics of technology and task requirements of tasks (Goodhue & Thompson, 1995). In the present study, TTF refers to the fit between the type of recommendation system (technology) and user shopping goal (task). Prior research shows that TTF has a positive impact on user satisfaction (Isaac, Abdullah, Ramayah, & Mutahar, 2017). When the user is goal-directed, the user already has the desired product and a clear shopping goal before online shopping. When a user searches on a shopping website, he or she usually searches directly for the target product. In a situation where a shopping website recommends products that are highly aligned with its search content (high accuracy), we consider such recommendations to be in line with the user shopping goal. On the other hand, when the user is an exploratory user, the user does not have a clear shopping goal before online shopping or simply wants to browse the website. The user usually first explores the goal in a variety of ranges, and then gradually refines it during the search process. When a shopping website recommends a variety of products to users (with a high diversity), we believe that such recommendations are consistent with the user’s shopping goal.
Therefore, this study believes that when the type of recommendation system fits the user shopping goal, the user satisfaction improves. Conversely, in the case of unfitness, the user satisfaction decreases. Specifically, when a user has a clear shopping goal and is goal-directed, highly accurate recommendations improve user satisfaction compared with diverse recommendations. When a user’s shopping goal is mainly exploratory with vague purchase objectives, diverse recommendations will improve user satisfaction compared to accurate recommendations.
H1: The fit between recommendation type and user shopping goal increases user satisfaction. However, unfitness decreases user satisfaction.
According to Westbrook (1987), consumers who experience emotions during consumption can be categorized as positive or negative. Positive emotions were associated with satisfaction, whereas negative emotions were associated with dissatisfaction. Emotional responses are often used as mediating variables in research studies. When the type of recommendation system fits the user shopping goal, the user undergoes a subjective experience of engagement, termed ‘feeling right’ (Avnet et al., 2013). We predicted that this positive emotional response would positively affect the user satisfaction.
However, our study posits that ‘being fit’ does not necessarily lead to ‘feeling right. According to expectancy disconfirmation theory (EDT), user satisfaction can be assessed by measuring the disparity between users' expectations and their experiences in perceiving a product or service (Oliver, 1980). Expectancy refers to user' expectations regarding the performance of a product or service. Perceived performance relates to users' experiences of using a product or service, which may be better or worse than their expectations. Disconfirmation refers to the difference between users' initial expectations and their actual performance of the product or service they perceive. If the performance of a specific product or service exceeds customer expectations, positive disconfirmation can lead to increased user satisfaction (Al-Nuaimi, Mahmood, Khalid, & Alazzawi, 2020). In the context of the present study, compared to exploratory users, goal-directed users already have specific expectations for the products they want before entering an online shopping site. Therefore, when a shopping website recommends products with high accuracy (i.e., high similarity to user expectations), users may experience positive disconfirmation. Consequently, users are more likely to feel right, thereby experiencing increased user satisfaction.
H2a: Feeling right mediates the effect of fitness of recommendation type and user shopping goal on user satisfaction.
H2b: User shopping goal moderates the mediation effect of the fitness of recommendation type and user shopping goal on user satisfaction. Specifically, when the user shopping goal is specific, feeling right will mediate the effect of the fitness of recommendation type and user shopping goal on user satisfaction (however, such a mediation effect will not occur when the user’s shopping goal is ambiguous).
However, it has often been reported that this type of recommendation system does not meet a user’s shopping goal. When users receive recommendations that do not align with their expectations, resistance is activated. In such cases, users not only ignore recommendations from the recommendation system but may also exhibit resistance (Fitzsimons & Lehmann, 2004). In the present study, we argue that ‘being unfit’ may not necessarily lead to psychological reactance. Goal-directed users who have a clear shopping goal, even if they receive diverse recommendations, may not experience psychological reactance because of their tendency to purchase products with various attributes or brands while shopping (Zhang, Yuan, 2022). Conversely, exploratory users who lack specific goals may feel that their freedom of choice is threatened and experience psychological reactance when they receive highly accurate recommendations. This leads to a decrease in the user satisfaction.
H3a: Psychological reactance mediates the effect of unfitness of the recommendation system and user shopping goal on user satisfaction.
H3b: User shopping goals moderate the mediation effect of unfitness of the recommendation system and user shopping goals on user satisfaction. Specifically, when user shopping goals are ambiguous, psychological reactance will mediate the effect of the unfitness of the recommendation system and the user shopping goal on user satisfaction (but such a mediation effect will not occur when the user’s shopping goal is specific).

4. Research Design and Methodology

4.1. Sample

A total of 206 university students in South Korea participated in our experiment, and they earned three participation points at the end of the semester. After excluding participants with dishonest responses, the final sample consisted of 184 individuals (101 males and 83 females, Mage= 22.36, SDage = 2.14). The participants were randomly assigned to four experimental conditions. The specific distribution of the participants is presented in Table 1. The demographic characteristics of the participants are as follows. The participants included 101 males (54.9%) and 84 females (45.1%). Most participants were between 20 and 29 years old (97.8%), whereas only three were less than 19 years old (1.6%), and only one was more than 30 years old (0.6%).

4.2. Experimental factors

At the beginning of the experiment, the participants were presented with a scenario in which they needed to buy a camera online. Participants assigned to the Goal-directed User condition were given the scenario: "Due to recent interest in VLOGs, combining recommendations from friends and online reviews, you decided to buy a SONY ZV series Vlog camera." Participants assigned to the Exploratory User condition were given the following scenario: "You are planning a trip recently, and you want to buy a camera." Subsequently, two pre-made recommendation pages on online shopping websites were displayed, each containing eight products.
The high-accuracy (high similarity) recommendation page includes seven SONY cameras, six of which belong to the SONY ZV series and one Panasonic Vlog camera. This design reflects the characteristics of a high accuracy (similarity). In addition, the diverse recommendation page, which emphasizes product diversity and reduces recommendation accuracy (similarity), includes six cameras from different brands, featuring Vlog Cameras, disposable film cameras, camera lenses, camera rentals, and various other types. Specific recommendation pages are shown in Figure 3.
After the participants were randomly assigned to four different conditions to verify the success of our manipulation (accuracy/diversity) of the recommendation system, they were required to answer two manipulation check items after browsing the recommended page on the shopping website. A single-item, seven-point scale was used to assess the participants' perceived differences in the various recommendation types. Participants were asked, ‘How do you perceive the products recommended by the shopping site?’ (1 = very accurate; 7 = very diverse).
The results indicated that our manipulation was successful: Participants assigned to different recommendation types (accuracy/diversity) showed significant differences in their perception of the accuracy and diversity of the recommended products (M_accuracy = 3.23, t(87) = -7.58, p < .001; M_diversity = 4.84, t(97) = -7.52, p < .001). Another single-item, seven-point scale was used to assess the participants' perceived differences in the scenarios. Participants were asked, ‘What do you think of your shopping goal?’ (1 = not specific; 7 = very specific). The results indicate that our manipulation was also successful: Participants assigned to different scenarios showed significant differences in their perception of the specificity of their shopping goals (M_goal-directed = 5.13, t(98) = 12.31, p < .001; M_exploratory = 2.88, t(86) = 12.32, p < .001). The details are presented in Table 1.

4.3. Measures

After answering the two manipulation check questions, participants assigned to the four different scenarios responded to the same set of questions to measure "Feeling right," "Psychological reactance," and "User satisfaction." After presenting the participants with the scenario conditions and pre-made shopping site recommendation pages, they were asked to evaluate their feelings toward the presented products on the recommendation pages.
A single-item, seven-point scale was used to check ‘feeling right’ toward the fit of recommendation type and user shopping goal by asking participants to indicate ‘‘how right’’ or ‘‘how wrong’’ they felt about the recommendation (1 – feeling wrong to 7 – feeling right), based on Cesario & Higgins(Cesario & Higgins, 2008) (Mfit = 5.10, t(101) = 3.43, p = 0.01; Munfit = 4.42, t(83) = 3.40, p = 0.01). To measure the extent to which users felt that their freedom of choice was threatened after seeing the shopping site's recommendation page, we used the scale developed by Bleier and Eisenbeiss (2015) with appropriate modifications for this experiment. To measure user satisfaction with the recommendation page after viewing the shopping site, we employed the scale developed by Liang et al. (2006) with appropriate modifications for this experiment. The final scale comprised eight items. (e.g., “whether the recommendation finds the item that the user wants to view,” α = .928; M = 4.96; SD = 1.00). Table 2 shows the reliability of measurement instruments used in this study.

4.4. Procedure

The purpose of this experiment was to explore the psychological reactions of users with different shopping goals when exposed to different types of recommendation systems during online shopping as well as their impact on their satisfaction. After presenting participants with scenarios containing a shopping goal, pre-made recommendation pages with either high accuracy or diversity were shown. Subsequently, the participants answered two manipulation check questions and provided subjective feedback on the recommendation system, including “Feeling Right”, “Psychological Reactance”, and “User Satisfaction.” Finally, the participants answered demographic questions, including gender, age, and occupation.
To test the proposed hypotheses on moderation, mediation, and conditional indirect effects, we employed the SPSS PROCESS macro (Hayes, 2017) using a regression-based approach. To test the mediation effect (Baron & Kenny, 1986), we first examined the interaction effect of the type of recommendation system (1 = accurate recommendation, -1 = diverse recommendation) and users’ shopping goals (1 = goal-directed user, -1 = exploratory user) on their satisfaction. The indirect effect of the interaction term on user satisfaction through feeling right or psychological reactance was tested using bootstrapping (Preacher & Hayes, 2008).

5. Results

5.1. Moderation effect

To test H1, we examined whether the interaction effect between recommendation type and shopping goal on user satisfaction is statistically significant. The results (Table 3) support this hypothesis. More specifically, there was a significant interaction between recommendation type and user-shopping goals (b = 0.31, t = 4.47, p < .001). The main effect of the recommendation type was not significant (b = 0.46, t = 0.66, p = .51). The main effect of the user-shopping goal was significant (b = 0.22, t = 3.19, p < .01). To decompose the significant interactions, we plotted the predicted values at two different levels of shopping goals (Aiken, West, and Reno 1991). Figure 4 shows the interaction patterns.
Simple effects analyses (Dawson, 2014) further supported our hypotheses: for goal-directed users, recommendation system (diversity = -1, accuracy = 1) had a significant and positive influence on user satisfaction (b = 0.36, t = 3.78, p < .001). In contrast, for exploratory users, recommendation system (diversity = -1, accuracy = 1) had a significant negative influence on user satisfaction (b = -0.27, t = -2.60, p < .05).
The results were consistent with H1: higher satisfaction is achieved when accurate recommendations are provided to goal-directed users than to exploratory users. Likewise, exploratory users were more satisfied with diverse recommendations than goal-directed users were.

5.2. The mediating role of feeling right and psychological reactance

To test H2a, we analyzed the impact of the fit of the recommendation type and user shopping goal on the user’s feeling of being right. The results (see Table 4) are consistent with the hypothesis that when the recommendation type fits the user shopping goal, the user’s feeling of being right is stronger (b = 0.34, t = 3.63, p < .001). Next, we examined whether a user’s feeling of right affected their satisfaction. The results showed that feeling right was positively associated with user satisfaction (b = 0.39, t = 11.14, p < .001). The indirect effect was positive and significant (b = 0.13, 95% CI [0.06, 0.22]). Thus, the mediating effect was significant and Hypothesis 2a was supported.
To test H3a, we analyzed the impact of the unfit recommendation type and the user shopping goal on the user’s psychological reactance. The results (see Table 4) were consistent with the hypothesis that when the recommendation type did not fit the user shopping goal, the user’s psychological reactance is stronger (b = -0.28, t = -2.83, p = .0052). Next, we examined whether the users’ psychological reactance affected their satisfaction. The results showed that psychological reactance was negatively associated with user satisfaction (b = -0.34, t = -9.97, p < .001). The indirect effect was positive and significant (b = 0.09, 95% CI [0.03, 0.17]). Thus, the mediating effect was significant, and Hypotheses3a was supported.

5.3. Testing conditional indirect effects

To gain a deeper understanding of the impact of recommendation system type and user satisfaction, we further investigated the conditional indirect effect of the former on the latter through feeling the right and psychological reactance across levels of shopping goals. Table 5 presents the results for hypothesis 2b. With regard to H2b, we predicted that the positive relationship between recommendation type and user feeling right would be stronger for users with specific shopping goals. The results indicate that the cross-product term between recommendation type and user shopping goal on the user feeling right was significant (b = 0.34, t = 3.63, p < .001). To fully support H2b, we applied conventional procedures to plot the simple slopes (Figure 5). The results are consistent with our expectations (and supporting H2b), where specific shopping goals had a significant and positive influence on users’ feeling right (simple slope = 0.58, t = 4.51, p < .001). In contrast, exploratory shopping goals did not have a significant influence on users’ feeling right (simple slope = -0.11, t = -0.77, p = .44).
Although the results showed that user shopping goals interacted with recommendation type to influence user feeling right, they did not directly assess the conditional indirect effects. Therefore, we analyzed the conditional indirect effects (Table 5). The results supported H2b by showing that the indirect and positive effect of accurate recommendations on user satisfaction was significant (b = 0.25, 95%, CI [0.14, 0.36]) when the user was goal-directed; however, the indirect and positive effect of diverse recommendations on user satisfaction was not significant (b = -0.05, 95%, CI [-0.18, 0.07]) when the user was an exploratory user.
Table 6 presents the results for hypothesis 3b. Regarding H3b, we predicted that there is a positive relationship between recommendation type and user psychological reactance for users with exploratory shopping goals. The results indicate that the cross-product term between recommendation type and user shopping goal on user psychological reactance is significant (b = -0.28, t = -2.83, p < .01). To fully support H3b, we applied conventional procedures to plot the simple slopes (Figure 6). These results are consistent with our expectations (and supporting H3b). For ambiguous shopping goals, recommendation system (diversity = -1, accuracy = 1) had a significant and positive influence on users’ psychological reactance (simple slope = 0.37, t = 2.56, p < .05). On the contrary, for specific shopping goals, recommendation system did not have a significant impact on users’ reactance (simple slope = -0.19, t = -1.40, p = .16).
Although the results showed that user shopping goals interacted with recommendation type to influence users’ psychological reactance, they did not directly assess the conditional indirect effects. Therefore, we analyzed the conditional indirect effects (see Table 6). The results supported H3b by showing that the indirect and negative effects of accurate recommendations on user satisfaction through psychological reactance were significant (b = -0.12, 95%, CI [-0.24, -0.03]) when the user has an exploratory shopping goal, but not when the user was goal-directed (b = 0.06, 95%, CI [-0.02, 0.16]).
In sum, user shopping goals would moderate the indirect effect of recommendation type on user satisfaction through feeling right or psychological reactance. Specifically, for users with specific shopping goals, an accurate recommendation would make them feel right and increase their satisfaction. For users who do not have a specific shopping goal, an accurate recommendation activates their psychological reactance and decreases their satisfaction.

6. Discussions

This study investigated user satisfaction from the perspective of matching user shopping goals with a recommendation system. We also aimed to understand the psychological mechanisms underlying the relationship between recommendation system and user satisfaction. This study found that the user shopping goal moderates the relationship between the type of recommendation system and user satisfaction. When the user-shopping goal fits the type of recommendation system, user satisfaction is expected to increase (H1). Second, we investigated the mediating role of feeling right and psychological reactance on the interaction effect between shopping goals and recommendation systems on user satisfaction. When the user shopping goal matches the type of recommendation system (fit), it leads to a feeling of right, thereby enhancing the user satisfaction. Conversely, when the user shopping goal and type of recommendation system do not match (are unfit), it may lead to psychological reactance, thus reducing user satisfaction (H2a, H3a). Third, we examined the conditional indirect effect of the type of recommendation system on user satisfaction through feeling right and psychological reactance at different levels of user shopping goals. Because of the alignment of recommendations with initial expectations, goal-directed users who experience accurate recommendations are expected to feel right, thereby increasing user satisfaction. By contrast, as exploratory users inherently seek diversity, accurate recommendations may activate psychological reactance, leading to a decrease in user satisfaction (H2b, H3b). These results offer new insights for e-commerce websites regarding how to enhance user satisfaction when designing recommendation systems.

6.1. Theoretical implications

This study makes several important theoretical contributions to the literature on recommendation system. First, this study uniquely combines the user shopping goal with the type of recommendation system, and investigates the interactive effects of both factors on user satisfaction. Building on previous research, which primarily examined the accuracy and diversity of recommendation systems (McNee et al., 2006; Hu & Pu, 2011; Matt et al., 2014), we took a step further by incorporating user shopping goals. Earlier studies initially emphasized the accuracy of recommendation systems and later recognized the need for diversity; however, they were confined to recommendation systems, overlooking the diverse shopping needs of users. When users shop online, they have different needs based on various factors (products, purchase timing, etc.). Recommendation systems should dynamically adapt to meet the changing needs of users in order to enhance their satisfaction. Based on the results of our study, depending on whether users have specific shopping goals, a recommendation system should provide different approaches to satisfy their shopping needs. Second, it is also found that the the mediating process between the recommendation system type and user satisfaction differs depending on the user shopping goals. Users with specific shopping goal are associated with the process of feeling right, whereas those with exploratory goal are related to the process of psychological reactance.

6.2. Managerial implications

The practical implications of this study are as follows: First, when designing a recommendation system, e-commerce websites should consider a user’s shopping goal. Websites can classify users based on their search routine. Goal-directed users typically search for a specific product using precise keywords, whereas exploratory users are more likely to explore the product using product categories or general keywords. An e-commerce website may provide recommendations that best fit a user’s shopping goal, potentially enhancing user satisfaction. Second, because feeling right is activated only when goal-directed users encounter accurate recommendations, it should strive to provide highly accurate information to users with specific shopping goals, thereby enhancing their satisfaction. Conversely, psychological reactance is activated only when exploratory users encounter accurate recommendations, and websites should avoid recommending highly accurate information to users without specific shopping goals.

6.3. Limitations

This study has several limitations. First, in our experiment, the recommendation system was divided into two basic categories: accurate and diverse recommendations. However, recommendation approaches can also take other forms such as novelty recommendations (Vargas & Castells, 2011). The two pre-made recommendation pages in our study do not represent mutually exclusive recommendation approaches. In addition, because the recommendation system itself is built on a large amount of user data and algorithms, our pre-made recommendation pages can only reflect the surface features presented by the recommendation system (accuracy/diversity) and cannot fully represent the recommendation approaches that users encounter during online shopping.
Second, the participants only provided evaluations of the recommendation pages as soon as they browsed them, and this process did not involve comparison, selection, decision making, or other aspects. Therefore, the participants' evaluations of the recommended pages may differ from their actual feelings during real-life shopping activities. Thus, we suggest that future research should attempt to make the recommendation pages more realistic by involving participants in the entire online shopping process (including comparison, selection, and decision-making) before assessing the recommendation pages.

Author Contributions

Conceptualization, X. H. and S. J.; Methodology, S. J.; Formal analysis, Q.L.; Investigation, X. H.; Data curation, Q. L.; Writing – original draft, X. H.; Writing – review & editing, Q. L.; Supervision, S. J.; Funding acquisition, S. J.

Funding

This research was funded by Kyung Hee University (20191246).

Institutional Review Board Statement

Not applicable

Informed Consent Statement

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Adomavicius, G., & Kwon, Y. (2011, October). Maximizing aggregate recommendation diversity: A graph-theoretic approach. In Proc. of the 1st International Workshop on Novelty and Diversity in Recommender Systems (DiveRS 2011) (pp. 3-10).
  2. Al-Fraihat, D., Joy, M., & Sinclair, J. (2020). Evaluating E-learning systems success: An empirical study. Computers in Human Behavior, 102, 67-86. [CrossRef]
  3. Al-Nuaimi, I. T. I., Mahmood, T. M., Khalid, O. W., & Alazzawi, A. K. (2020). An integrated model of the relation between E-service quality and user satisfaction in IHL. Paper presented at the Journal of Physics: Conference Series, , 1529(4) 042072. [CrossRef]
  4. André, Q., Carmon, Z., Wertenbroch, K., Crum, A., Frank, D., Goldstein, W., ... & Yang, H. (2018). Consumer choice and autonomy in the age of artificial intelligence and big data. Customer needs and solutions, 5, 28-37. [CrossRef]
  5. Baron, R. M., & Kenny, D. A. (1986). The moderator–mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of personality and social psychology, 51(6), 1173-1182.
  6. Bleier, A., & Eisenbeiss, M. (2015). The importance of trust for personalized online advertising. Journal of Retailing, 91(3), 390-409. [CrossRef]
  7. Brehm, J. W. (1966). A theory of psychological reactance. [CrossRef]
  8. Cesario, J., Grant, H., & Higgins, E. T. (2004). Regulatory fit and persuasion: Transfer from" feeling right.". Journal of personality and social psychology, 86(3), 388-404. [CrossRef]
  9. Cesario, J., & Higgins, E. T. (2008). Making message recipients “feel right” how nonverbal cues can increase persuasion. Psychological Science, 19(5), 415-420.
  10. Dawson, J. F. (2014). Moderation in management research: What, why, when, and how. Journal of business and psychology, 29(1), 1-19. [CrossRef]
  11. Fitzsimons, G. J., & Lehmann, D. R. (2004). Reactance to recommendations: When unsolicited advice yields contrary responses. Marketing Science, 23(1), 82-94. [CrossRef]
  12. Goldberg, K., Roeder, T., Gupta, D., & Perkins, C. (2001). Eigentaste: A constant time collaborative filtering algorithm. information retrieval, 4, 133-151. [CrossRef]
  13. Goodhue, D. L., & Thompson, R. L. (1995). Task-technology fit and individual performance. MIS quarterly, 213-236. [CrossRef]
  14. Hayes, A.F. (2017). Introduction to Mediation, Moderation, and Conditional Process Analysis: A Regression-Based Approach. Guilford Publications.
  15. Hoffman, D. L., & Novak, T. P. (1996). Marketing in hypermedia computer-mediated environments: Conceptual foundations. Journal of Marketing, 60(3), 50-68.
  16. Hu, R., & Pu, P. (2011, October). Helping users perceive recommendation diversity. In DiveRS@ RecSys (pp. 43-50).
  17. Isaac, O., Abdullah, Z., Ramayah, T., & Mutahar, A. M. (2017). Internet usage, user satisfaction, task-technology fit, and performance impact among public sector employees in yemen. The International Journal of Information and Learning Technology, 34(3), 210-241.
  18. Isinkaye, F. O., Folajimi, Y. O., & Ojokoh, B. A. (2015). Recommendation systems: Principles, methods and evaluation. Egyptian informatics journal, 16(3), 261-273. [CrossRef]
  19. Kunaver, M., & Požrl, T. (2017). Diversity in recommender systems–A survey. Knowledge-based systems, 123, 154-162.
  20. Lee, D., & Hosanagar, K. (2019). How do recommender systems affect sales diversity? A cross-category investigation via randomized field experiment. Information Systems Research, 30(1), 239-259. [CrossRef]
  21. Lee, G., & Lee, W. J. (2009). Psychological reactance to online recommendation services. Information & Management, 46(8), 448-452. [CrossRef]
  22. Liang, T. P., Lai, H. J., & Ku, Y. C. (2006). Personalized content recommendation and user satisfaction: Theoretical synthesis and empirical findings. Journal of Management Information Systems, 45-70. [CrossRef]
  23. Lu, J., Wu, D., Mao, M., Wang, W., & Zhang, G. (2015). Recommender system application developments: a survey. Decision support systems, 74, 12-32. [CrossRef]
  24. Matt, C., Benlian, A., Hess, T., & Weiß, C. (2014). Escaping from the filter bubble? The effects of novelty and serendipity on users’ evaluations of online recommendations.
  25. Ma, X., Sun, Y., Guo, X., Lai, K. H., & Vogel, D. (2022). Understanding users’ negative responses to recommendation algorithms in short-video platforms: a perspective based on the Stressor-Strain-Outcome (SSO) framework. Electronic Markets, 1-18. [CrossRef]
  26. McNee, S. M., Riedl, J., & Konstan, J. A. (2006, April). Being accurate is not enough: how accuracy metrics have hurt recommender systems. In CHI'06 extended abstracts on Human factors in computing systems (pp. 1097-1101).
  27. Muller, D., Judd, C. M., & Yzerbyt, V. Y. (2005). When moderation is mediated and mediation is moderated. Journal of personality and social psychology, 89(6), 852-863.
  28. Oliver, R. L. (1980). A cognitive model of the antecedents and consequences of satisfaction decisions. Journal of Marketing Research, 17(4), 460-469.
  29. Pariser, E. (2011). The filter bubble: What the Internet is hiding from you. penguin UK.
  30. Pizzi, G., Scarpi, D., & Pantano, E. (2021). Artificial intelligence and the new forms of interaction: Who has the control when interacting with a chatbot? Journal of Business Research, 129, 878-890.
  31. Preacher, K. J., & Hayes, A. F. (2008). Asymptotic and resampling strategies for assessing and comparing indirect effects in multiple mediator models. Behavior research methods, 40(3), 879-891.
  32. Ricci, F., Rokach, L., & Shapira, B. (2015). Recommender systems: introduction and challenges. Recommender systems handbook, 1-34.
  33. Smith, P. A., Newman, I. A., & Parks, L. M. (1997). Virtual hierarchies and virtual networks: some lessons from hypermedia usability research applied to the World Wide Web. International Journal of Human-Computer Studies, 47(1), 67-95. [CrossRef]
  34. Song, H., McComas, K. A., & Schuler, K. L. (2018). Source effects on psychological reactance to regulatory policies: The role of trust and similarity. Science Communication, 40(5), 591-620. [CrossRef]
  35. Tucker, C. E. (2014). Social networks, personalized advertising, and privacy controls. Journal of Marketing Research, 51(5), 546-562.
  36. Vargas, S., & Castells, P. (2011, October). Rank and relevance in novelty and diversity metrics for recommender systems. In Proceedings of the fifth ACM conference on Recommender systems (pp. 109-116).
  37. Wadhwa, M., & Zhang, K. (2015). This number just feels right: The impact of roundedness of price numbers on product evaluations. Journal of Consumer Research, 41(5), 1172-1185. [CrossRef]
  38. Westbrook, R. A. (1987). Product/consumption-based affective responses and postpurchase processes. Journal of Marketing Research, 24(3), 258-270.
  39. Youn, S., & Kim, S. (2019). Understanding ad avoidance on Facebook: Antecedents and outcomes of psychological reactance. Computers in human behavior, 98, 232-244. [CrossRef]
  40. Zhang, Y. (2022). Variety-seeking behavior in consumption: A literature review and future research directions. Frontiers in Psychology, 13, 874444. [CrossRef]
  41. Zhang, Y. C., Séaghdha, D. Ó., Quercia, D., & Jambor, T. (2012, February). Auralist: introducing serendipity into music recommendation. In Proceedings of the fifth ACM international conference on Web search and data mining (pp. 13-22).
  42. Ziegler, C. N., McNee, S. M., Konstan, J. A., & Lausen, G. (2005, May). Improving recommendation lists through topic diversification. In Proceedings of the 14th international conference on World Wide Web (pp. 22-32).
  43. Zipf, G. K. (2016). Human behavior and the principle of least effort: An introduction to human ecology. Ravenio Books.
Preprints 96934 g001
Figure 1. Content-based filtering and collaborative filtering.
Figure 1. Content-based filtering and collaborative filtering.
Preprints 96934 g001
Preprints 96934 g002
Figure 2. Research model.
Figure 2. Research model.
Preprints 96934 g002
Preprints 96934 g003
Figure 3. Accurate recommendation type (left); Diverse recommendation type (right).
Figure 3. Accurate recommendation type (left); Diverse recommendation type (right).
Preprints 96934 g003
Preprints 96934 g004
Figure 4. Moderating role of shopping goal.
Figure 4. Moderating role of shopping goal.
Preprints 96934 g004
Preprints 96934 g005
Figure 5. The moderating role of shopping goal on the relationship between types of recommendation system and feeling right.
Figure 5. The moderating role of shopping goal on the relationship between types of recommendation system and feeling right.
Preprints 96934 g005
Preprints 96934 g006
Figure 6. The moderating role of shopping goal on the relationship between types of recommendation system and psychological reactance.
Figure 6. The moderating role of shopping goal on the relationship between types of recommendation system and psychological reactance.
Preprints 96934 g006
Table 1. Goal-directed User: ‘SONY ZV series Camera’; Exploratory User: ‘Camera’.
Table 1. Goal-directed User: ‘SONY ZV series Camera’; Exploratory User: ‘Camera’.
Preprints 96934 t001
Table 2. Reliability.
Table 2. Reliability.
Preprints 96934 t002
Table 3. Moderation analyses.
Table 3. Moderation analyses.
Preprints 96934 t003
Table 4. Mediation analyses. Independent variable: fit (recommendation type * shopping goal); mediator variable: feeling right / psychological reactance.
Table 4. Mediation analyses. Independent variable: fit (recommendation type * shopping goal); mediator variable: feeling right / psychological reactance.
Preprints 96934 t004
Table 5. Moderated mediation: Feeling right.
Table 5. Moderated mediation: Feeling right.
Preprints 96934 t005
Table 6. Moderated mediation: Psychological reactance.
Table 6. Moderated mediation: Psychological reactance.
Preprints 96934 t006
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
Prerpints.org logo

Preprints.org is a free preprint server supported by MDPI in Basel, Switzerland.

facebook logotwitter logolinkedin logo
MDPI Initiatives
Important Links
Subscribe

Disclaimer

Terms of Use

Privacy Policy

© 2024 MDPI (Basel, Switzerland) unless otherwise stated