Prerpints.org logo
Preprint
Article

Reducing the Inequality of Citations based on Local Indexers: The Brazilian Spell (Scientific Periodicals Electronic Library) Case

Altmetrics

Downloads

110

Views

40

Comments

0

A peer-reviewed article of this preprint also exists.

This version is not peer-reviewed

Submitted:

11 November 2023

Posted:

13 November 2023

You are already at the latest version

Alerts
Abstract
The inclusion of scientific journals in prestigious indexers is often associated with higher citation rates; journals included in such indexers are significantly more acknowledged than those that are not included in them. This phenomenon refers to the Matthew effect on journal citations, according to which, journals in exclusive rankings tend to be increasingly cited. This paper shows the opposite: that the inclusion of journals in local indexers ruled by inclusive logic reduces the Matthew effect on journal citations, since it enables them to be equally exposed. Thus, we based our arguments on the comparison of 68 Brazilian journals before and after they were indexed in the Scientific Periodicals Electronic Library (Spell), which ranks journals in the Brazilian management field based on local citations. Citation impact indicators and iGini (a new individual inequality analysis measure) were used to show that the inclusion of journals in Spell has increased their impact factor and decreased their citation inequality rates. In addition, the results indicated that the effect between ranking and inequality has declined after journals were included in Spell; this outcome indicates that the database has reduced the Matthew effect on journal citations.
Keywords: 
Subject: Social Sciences  -   Library and Information Sciences

Introduction

It is known that scientific journals aim at differentiating from one another based on their quality. Thus, one of the most adopted strategies, if not the main one, lies on being inserted in databases and indexers that rank the impact of journals based on their citations (Harzing & Alakangas, 2016; Kulkarni et al., 2009; Martín-Martín et al., 2018, 2020). Consequently, most journals focus their efforts on the most prestigious indexers, mainly on Web of Science and Scopus (Mongeon & Paul-Hus, 2016) to increase their visibility and reputation in the global scientific debate (Lariviere & Gingras, 2010; Vessuri, 1995). It makes any sense, since many researchers define the journals they want to publish in based on their ranking position in such indexers (Drivas & Kremmydas, 2020).
Such a strategy of differentiating quality through prestigious indexes is also valid for journals from countries that are on the margins of global scientific production (Collazo-Reyes, 2014; Leta & Chaimovich, 2002; Vélez-Cuartas et al., 2016; Rodrigues & Abadal, 2014). However, since some scientific fields have a more regional or local focus (Cetto et al. 2010; Hicks et al., 2015), not all journals would be eligible based on criteria adopted by such indexers (Alcadipani, 2017), regardless of their quality. In fact, this fact ends up dividing journals into two classes, namely: indexed and non-indexed in international indicators. Since it is extremely hard to assess the quality of a given journal, researchers tend to trust any reliable status indication (Azoulay et al., 2014), which is the case of indexers such as Web of Science and Scopus. Thus, these differences in status attribution - which can be subtle when journals join a given indexer - can increase over time, generate cumulative advantages and influence one’s perception about a given journal’s quality (Azoulay et al., 2014), as well as the volume of citations (Bornmann et al., 2020; Drivas & Kremmydas, 2020; Lariviere & Gingras, 2010). This process relates to the Matthew effect in Science (1968; 1988), which also applies to journals, since status differentiation signs lead to increasing gains in audience and citations.
If there is a journal stratification process inherent to their participation in prestigious indexers, what would be the alternatives for journals that do not participate in them in order to have their quality acknowledged? Being indexed in alternative databases (Guédon, 2008; Marchitelli et al., 2017; Silva et al., 2018), such as DOAJ (Directory of Open Access Journals) is likely the most common path, mainly for Open Access (OA) journals. The Ibero-American context presents some successful experiences (Fischman et al., 2010; López et al., 2008; Meneghini et al., 2006; Mesquita et al., 2019), such as Redalyc (Red de Revistas Científicas de América Latina y El Caribe, Spain and Portugal) and Scielo (Scientific Electronic Library Online).
Although the influence of Open Access databases and indexers has been extensively explored at researcher, journal and country level, little is known about the effects of creating such indexers on the set of journals in these databases, mainly on the dynamics of citations and impact factors of indexed journals. Overall, studies focusing on investigating such dimensions have put greater emphasis on prestigious indexers (Drivas & Kremmydas, 2020; Lariviere & Gingras, 2010; Starbuck, 2005), which often show higher Matthew effect intensity overtime. However, the ranking logic adopted in such indexers is based on the exclusion principle, according to which, the best journals are selected, whereas the others are excluded. Would the Matthew effect of journal citations also happen in databases and indexers adopting an open and more inclusive logic?
In order to answer this question, this paper has analyzed a successful alternative indexing experiment carried out in Brazil, namely: the Scientific Periodicals Electronic Library (Spell), which is a scientific database that indexes Brazilian open access journals in the Business Administration, Accounting and Tourism fields and that currently accounts for more than 55 thousand documents, 41 million accesses and 14 million downloads. Spell was created by the local scientific community in the field in order to fill the gap observed in databases and indexers focused on Brazilian journals. Based on the inclusive logic, Spell was launched in mid-2012 to incorporate most journals in the field and to provide free access to all articles published on a single platform, and it enabled researchers to search for articles based on topics and terms. Since all journals - regardless of their ranking - had the same visibility in the database, there was the expectation that they would be evaluated based on the perceived quality of their articles, rather than on the previous ranking of journals.
In methodological terms, we selected 68 journals that had been present in Spell since its beginning of operation. Through fixed panel data models, we compared the impact indicators of citations before (2010-2012) and after (2013-2015) Spell was created, which allowed us to verify if the rise of Spell increased the impact of the journals. Additionally, we used a new measure of individual inequality analysis called iGini (Liao, 2019), which allowed us to analyze whether Spell reduced the Matheus effect of citations of journals, or not.
The analysis of whether the rise of Spell has simultaneously increased the mean citation of journals and reduced the inequality between them has contributions to the scientometric field. First, we explored an alternative database of local contents rather than focused on large databases and indexers (Martín-Martín et al., 2018; 2021). The impact of alternative databases on journal citations is hardly analyzed, since many of them do not often provide such an information, and when they do, there are often doubts about its validity and reliability (Gutierrez et al., 2015; Xia, 2019). Secondly, data about journal citations before and after Spell was created have enabled comparing the effect of the emergence of the investigated database in terms of impact and inequality, which is extremely rare in this type of research (Drivas & Kremmydas, 2020). Finally, the current study is pioneer in introducing a new inequality assessment measure called iGini (Liao, 2019) in the scientometry field. This assessment measure enables analyzing individual cases in comparison to all others, which allows a direct measurement of the Matheus effect.
The current article was structured in six parts. First, it contextualizes the journal evaluation system available in Brazil. Next, it addresses the herein analyzed database and indexer called Spell. Then, it analyzes the incidence of Matthew effect in journal evaluation processes. Subsequently, it provides detailed information about the analysis method and, finally, it presents the results, which are discussed in the final section.

Science, indexers and the evaluation of journals in Brazil

Before highlighting the relevance of Spell to the local academic community, it is necessary contextualizing some aspects about the way science is organized in Brazil, since the emergence of this indexer is associated with peculiarities of the Brazilian science, as well as with the production logic adopted in the Business Administration, Accounting and Tourism field the assessed indexer was consolidated in.
Unlike what happens in other countries, science in Brazil is strongly, directed and regulated by the government, both in terms of financing and in directing policies focused on contributing to scientific progress in the country (Leite et al., 2011; Strehl et al., 2016). However, the scientific environment in the country is featured by low interaction between universities and companies, by great financial and structural dependence on the federal government (Freitas et al., 2013) and by internationalization barriers due to low proficiency in the English language (Meneghini & Packer, 2007). This profile also leads to many journals and articles written in the local language in certain science fields such as applied human social sciences (Mugnaini et al., 2019; Rocha et al., 2020).
However, despite having a large number of studies published in local journals, Brazil is currently the 13th country in the world that mostly produces articles indexed in the Web of Science, 95% of them are produced by public universities (Clarivate, 2017). According to the Science-Metrix report (2018), Brazil presents the highest rates of open access articles in the world (74%). Most of them are published in Brazilian journals that are not indexed in databases such as Scopus and the Web of Science (Mugnaini et al., 2019). Even when articles are published in indexed databases such as the Web of Science, 80.4% of journals are Brazilian (Strehl et al., 2016).
The successful internationalization of scientific production in international journals, as well as its outspread in the country through open access systems, can be significantly attributed to the university evaluation system implemented by the Coordination for the Improvement of Higher Education Personnel (Capes), which is linked to the Brazilian Ministry of Education (Marenco, 2015). Capes is one of the main foundations accounting for the expansion of university post-graduate programs; it plays key role in assessing and monitoring the performance of Brazilian universities in the scope of academic research, since it guides the scientific production of public and private universities.
Nowadays, Brazil has more than 400 universities and colleges that have, in total, 4,642 master's and/or doctorate programs distributed among 49 assessment fields and organized into 9 major fields: Agrarian Sciences, Biological Sciences, Health Sciences, Human Sciences, Applied Social Sciences, Linguistics, Letters and Arts, Exact and Earth Sciences, Engineering and Multidisciplinary fields1. Such information highlights the wide dimension of Brazilian science and its heterogeneous nature with respect to its evaluation fields. Therefore, specific commissions in each of these fields were created; they account for evaluating all Brazilian postgraduate programs every 4 years, in order to outline strategies focused on expanding, internationalizing and improving the Brazilian science.
Publications of post-graduate programs linked to Brazilian universities and colleges are evaluated by a scoring system called Qualis. Articles published by researchers associated with these programs are evaluated based on criteria set by Capes, which are used along with other indicators to evaluate the quality of post-graduate programs nationwide. Qualis Capes comprises 8 assessment levels, from A1 to C (A1, A2, B1, B2, B3, B4, B5 and C); each level corresponds to certain criteria and scores (from 100 to 0) established for each of the 49 assessment fields. For example, level A1 corresponds to high impact journals; since high impacts of JCR and/or H-Scopus are overall established to define the journals to be part of this level (Carvalho Neto et al., 2016). This procedure shows certain alignment between Qualis and international assessments (Rocha et al., 2020). As previously mentioned, depending on the field, not all criteria are dimensioned based on international impact measurements (e.g., JCR and SCOPUS) Overall, they can only include indexing in specific databases and directories (e.g., Scielo, Redalyc, Latindex, DOAJ) or in other mostly subjective criteria.
Given these dimensions, what could explain the internationalization and, at the same time, the endogenous nature of Brazilian science? One of the likely answers to this question lies on different journal assessment paths established by Capes for each science field. The so-called hard science fields have favored – since the beginning - international impact metrics used to classify the quality of journals at the time to assess their researchers and programs. On the other hand, the so-called soft sciences did not used impact metrics as criterion to evaluate their publications for approximately 10 years (Trzesniak 2016). Consequently, the process of classifying journals, over a period-of-time, was artisanal and poorly systematized (Rossoni & Guarido Filho, 2012; Souza & Paula, 2002). Although journal evaluation processes have parameterized criteria based on normalization, regularity, circulation, visibility, editorial management, in addition to the number, proportion and perceived quality of published articles, the essence of evaluation in social sciences and humanities remained based on the opinions of experts (Machado-da-Silva et al., 2008)
Lack of criteria based on international metrics has also enabled the emergence and proliferation of national journals, which mainly contributed for scientific fields such as Business Administration, Accounting and Tourism in Brazil to have approximately 304 open access journals (Rosa & Romani-Dias, 2019). This feature of the field has highlighted a quite different publication dynamic in comparison to the hard science field. Thus, there is a community featured by high concentration of studies published in national journals - such studies belonging to the Business Administration, Accounting and Tourism fields - that are not indexed in the mainstream indexers.
Despite the current local endogeny in the Business Administration, Accounting and Tourism field, from 2010 onwards, the improvement observed in evaluations carried out by Capes ended up putting pressure on the field itself to use metrics such as JCR and H-Scopus (Elsevier) at the time to evaluate journals published by researchers (Trzesniak, 2016). However, these metrics worked as basis to qualify part of the international journals indexed in these databases. In the absence of such metrics, national journals ended up being evaluated again based on poorly consistent criteria; they were subjected to the scrutiny of specialists at the time to determine the most or least prestigious journals, since most of them did not appear in any indexer (Rossoni & Guarido Filho, 2012).
Qualis Capes has recently made radical changes to make evaluations more objective based on international metrics. However, there was rupture in the institutional evaluation process because most Brazilian journals were not indexed in such databases. On the one hand, a small number of journals were already indexed, or in the process of being indexed, to adapt to institutional requirements. On the other hand, most national journals did not have indexing alternatives in such databases due to several factors like the large number of requirements for foreign journals to be indexed, a long evaluation time on the part of the mainstream databases (Web of Science, Scopus) and lack of managerial and technical resources to enable the internationalization process.
Part of this endogeny can also be explained by the valorization of Brazilian journals indexed in alternative national or Ibero-American open access databases (Rocha et al., 2020), such as Scielo, Redalyc and Spell itself, which will be analyzed in the present article. The Brazilian experience with national and Ibero-American databases - not only in the Business Administration, Accounting and Tourism fields - has shown that databases such as Scielo and Redalyc have contributed to improve the quality level of national journals, as well as their visibility and outspread at national and international level (Meneghini et al., 2006; López et al., 2008). According to Mesquita et al. (2019), the success of such initiatives goes beyond the open access and free publication in Brazilian journals and databases, since they are also inducers of the knowledge democratization principle called Open Access (AO). They aim at increasing the access to, and speed of, information; knowledge sharing; data visibility; as well as transparency and collaboration of published research by using the San Francisco Declaration on Research Assessment (DORA) as reference.
The research carried out by Carvalho Neto et al. (2016) has emphasized the important role played by Open Access databases and directories in the Brazilian context. The authors have investigated a sample comprising 350 journals classified by Qualis Capes evaluation and found strong association between open access journals presenting high Qualis classification level: the OA rate at the highest evaluation levels (from A1 to B2) ranged from 90% to 80%, whereas the most poorly evaluated journals recorded much lower rates (36% for B4 with and 28% for C). Rosa and Romani-Dias (2019) observed similar results for the Business Administration, Accounting and Tourism fields: journals presenting higher Qualis levels presented high incidence in databases and indexers such as Google Scholar, Latindex, Spell, Redalyc, Scielo, among others. In addition, evidence has also shown that these journals have higher mean indexing than journals presenting the worst Qualis evaluation.
It is worth emphasizing that Brazilian publishing and indexing initiatives keep journals away from predatory and commercial models of scientific publication, which is not often seen in the international OA context. According to Xia (2019), who investigated 27 alternative and open access indexers, many of them are associated with predatory journals, which, in their turn, do not show evidence of quality at the time to be indexed. This is not the case of the Brazilian experience (Perlin et al., 2018), since publications in predatory journals that charge fees is discouraged and morally repressed; besides, these journals are excluded from official evaluation mechanisms such as Qualis.
In addition to the local scope, the relevance of Open Access goes beyond the borders of peripheral countries like Brazil and the interest in it has gained room in several countries worldwide. According to Piwowar et al. (2018), at least 28% of the world literature comprises OA. If one analyzes the difference in impact between OA and non-OA articles, OA articles receive 18% more citations than non-OA articles, on average. Thus, it is worth analyzing how open access databases and indexers based on the inclusive logic - such as the Scientific Periodicals Electronic Library (Spell) - contribute to rise the number of citations and the impact of scientific articles.

The Brazilian experience with the Scientific Periodicals Electronic Library (Spell)

The Brazilian experience with open access databases has been successful in terms of their institutional assessment quality, increased visibility, and understanding about their scientific impacts (Carvalho Neto et al., 2016; López et al., 2008; Meneghini et al., 2006; Mesquita et al., 2019). One of the reasons for such a success lies on the widespread use of the Open Journal System (OJS) of the Public Knowledge Project developed at the University of British Columbia (Canada), which was translated and customized in Brazil by the Brazilian Institute of Information in Science and Technology (IBICT - Instituto Brasileiro de Informação em Ciência e Tecnologia) and made available through the Electronic Journal Editing System (SEER - Sistema Eletrônico de Editoração de Revistas). According to IBICT, this initiative enabled more than a thousand Brazilian scientific journals to be created or to migrate to online versions in the first decade of the 2000s.
However, the access to such journals took place in a fragmented way, since researchers rarely use databases and search engines to access articles produced in Brazil. Despite international search initiatives in free access journals such as DOAJ (Directory of Open Access Journals), they were unable to draw enough attention from Brazilian journals and researchers because they aim the general public. Thus, Scielo has conquered prominent position at national level in the attempt to prepare, store, spread and evaluate national scientific production. Its project was originally supported by official Brazilian development agencies such as FAPESP and CNPq. However, Scielo was not capable of presenting fast response to the demand for indexing national journals - so far, it has only incorporated 5.92% of journals in the Business Administration, Accounting and Tourism field (Rosa & Romani-Dias, 2019). It mainly happened because technologies adopted for Scielo's indexing process were not accessible to most national journals, which also ended up being excluded from the indexing process due to several editorial adequacy requirements.
Although Scielo has emerged as a platform aimed at fostering open access to national journals, it was not capable of meeting the need of different research fields in Brazil to be included in databases and indexers. Thus, instead of being a project focusing on including non-indexed national journals in international databases, it was structured as a journal selection mechanism that excluded most national initiatives. Scielo's response to the Business Administration, Accounting and Tourism field was not different: what started as the initial bet on the indexing of most journals in the field ended up leading to great frustration, since Scielo’s rejection of these journals got more and more frequent.
Given the frustration generated by Scielo, in association with Capes' interest in increasingly valuing impact indicators in the Brazilian journal evaluation system, the Business Administration, Accounting and Tourism fields started discussing about the likelihood of building their own database and indexing system. The initial inspiration to create a national portal for journals has emerged in 2006; however, it was only in 2010 that a real development proposal was articulated with the National Association of Postgraduation and Research in Administration (ANPAD). Given ANPAD’s prominence in the field - since it incorporates the main scientific meetings and the most prominent journals in the field, as well as represents postgraduate programs before official research and development agencies - the design of a database and indexing system started to take shape.
The database and indexer had no name yet, but their idealization was based on four principles, namely: inclusion, free access, visibility, and information. With respect to inclusion, despite the increasing number of publications in the field, as well as the significant increase in the number of professors and postgraduate programs in Brazil, national and international databases, and indexers, such as Web of Science, Scopus and Scielo, used exclusion criteria, since they adopted deliberately strict access criteria that were inaccessible to most national journals in our field. According to Rosa and Romani-Dias (2019), Brazilian journals in the Business Administration, Accounting and Tourism field present total indexation rate of 0.99%, 4.38% and 5.92%, respectively. The new database aimed at breaking up with the exclusion logic and at including any national journal in the field that was classified in Qualis and met the criteria of good publishing practices, such as peer review.
If inclusive logic was a principle, it would make no sense to have a closed database with restricted access. This is the reason why Spell should necessarily be a free access portal for any user. The free access would enable increasing the visibility of national journals - regardless of their prestige and Qualis stratum -, as well as organizing the entire content in a single database to enable integrated searches based on topic and subject. Such an organization process was essential to increase the visibility of the national production. At that time, it was much more practical to search for articles in English than in Portuguese, because there was no database capable of integrating a relatively large volume of national journals in the Business Administration, Accounting and Tourism field.
With respect to the higher quality information expected to be acquired through the database and indexer, Capes has implemented a process to review the journal evaluation approach in the Qualis System, which started to take into consideration the impact factor and the indexing in databases with academic-scientific reputation. This change, which meets the worldwide trend to use the impact factor of journals as instrument to evaluate scientific publications, would lead to changes in the stratification of national journals. Hence, the idea of producing impact indicators based on citations of national journals by the journals themselves, which would highlight the relevance of what is produced for Brazilians in terms of use. In strategic terms, the concern in the field was to have an alternative local impact in order to avoid the massive downgrade of national journals in Qualis, since most of them are not part of international indexers.
Bearing these four principles in mind, the Scientific Periodicals Electronic Library (Spell) was officially launched in June 2012, after two years of discussions and development; it is freely accessible at www.spell.org.br. Seventy (70) journals composed the database in its first operation year - 8,784 articles, in total (Figure 1). At that time, the database was limited to articles published from 2000 onwards and it only presented mechanisms focused on searching and sharing articles and journals. From 2013 to 2015, the procedure adopted to include all collection articles, which comprised documents published since 1961, was implemented. Thus, in 2015, the database presented 33,253 documents deriving from 100 journals. Spell impact indicators - which included citations per document, immediacy, self-citation rate, among others2 - started to be generated in that very same year. By the end of 2020, Spell had 53,149 documents distributed in 130 journals; most of them had their impact assessed on a yearly basis.
Although Spell database/indexer is restricted to the Business Administration, Accounting and Tourism fields in Brazil, it has some advantages over alternative indexers. The first advantage refers to the reliability of its sources. Most of the 21 alternative indexers evaluated by Gutierrez et al. (2015) incorporate predatory journals with questionable quality, not to mention that their journal impact measurement method is not properly described. In addition, according to Xia (2019), such indexers are not very judicious since they accept many journals that do not require peer review. In addition to not incorporating any predatory journals, Spell also exposes its data sources.
The second advantage of Spell lies on its scope in indexing journals researchers in the Administration, Accounting and Tourism field publish in. As shown in Figure 2, Spell indexes 11% of journals that have received publications from researchers in the field; however, these journals account for 49% of the entire volume of articles produced in the field. Although Scopus and the JCR of Web of Science present higher rates of journals (35% and 16%, respectively), their volume of published articles is much lower (17% and 7%, respectively). Google Scholar, which indexes 58% of all publications, only exceeds Spell's publication volume by two percentage points. Although Google Scholar has greater coverage than Spell, its use as impact indicator is discouraged in many cases due to limitations observed in its indicators (Martín-Martín et al 2021).
Given these advantages, CAPES’ evaluation department accounting for defining Qualis classification criteria to rank the journals has incorporated Spell as indexer for intermediate extracts, alongside h-index and Citescore of Scopus and the JCR of Web of Science, which were used as ranking criteria for superior extracts. However, what we want to highlight with Spell’s experience is how the incorporation of both a database and indexer based on inclusive logic helped reducing asymmetries in, and partly reverting the Matthew effect on, journal citations.

Citation inequalities and the Matthew effect on journal evaluations

As previously mentioned, the system used to evaluate journals published by Brazilian researchers is based on CAPES’ Qualis system. One of the advantages of such a system lies on the fact that it allows journals that are not listed in impact indicators to have their quality acknowledged. This evaluation was based on the assessment of experts, who ranked the journals based on subjective criteria used to assess the quality of the publications. However, although the ranking by experts is useful and often carried out, it tends to perpetuate asymmetries and biases of judgment, as well as to favor the most prestigious journals (Lariviere & Gingras 2010; Wang, 2014) and the ones presenting the best status (Drivas & Kremmydas, 2020) - either due to their history or to the status of the publisher or institution of origin (Medoff, 2006) - regardless of effective changes in the quality of the least prestigious journals (Starbuck, 2005). Consequently, journals that were best evaluated in the past always tend to be better evaluated (Bornmann et al., 2020). Likewise, journals that were poorly evaluated in the past are likely to be poorly evaluated in the present. Such an unequal merit, and quality acknowledgment allocation phenomenon, is called Matthew effect in Science (Merton, 1968, 1988).
The Matthew effect in Science, in the sense treated by Merton (1988), refers to social processes inherent to the academic activity, which lead some social actors in Science, such as researchers, institutions and - in the herein investigated case – journals, to accumulate material or symbolic rewards in a disproportional way, regardless of merit. Such an accumulation of rewards takes place through initial comparative advantages in status (Drivas and Kremmydas, 2020), prestige (Wang, 2014) or access to resources (Laudel, 2006), which produce incremental benefits overtime for journals presenting privileged position or evaluation (Merton, 1988). Such a cumulative advantage generation mechanism can be identified in several social stratification systems (DiPrete & Eirich, 2006); it is not only observed in Science. In fact, it always produces the same result, namely: the rich get richer, whereas the poor become relatively poorer (Merton, 1968).
The Matthew effect was named after the Bible verse of Matthew 25:29: “for to everyone who has will more be given, and he will have abundance; but from him who has not, even what he has will be taken away”. More than metaphorical, the consolidation of this effect took place through the concept of cumulative advantage, which is often used in the literature about social mobility and poverty (DiPrete & Eirich, 2006). However, there is a striking cumulative advantage typification in bibliometrics. According to Price (1976), success tends to generate success, mainly in the case of journals - those that were mostly consulted in the past are likely to be most cited in the future.
Price’s (1976) finding about cumulative advantages generated due to unequal attention and merit attribution to international journals is not an isolated case (Bonitz et al., 1997). There is evidence that the initial status differentiation can generate long-lasting asymmetries in assessment processes carried out in Brazil, mainly in expert-based evaluation systems such as Qualis. Rossoni and Guarido Filho (2012) have investigated how the prestige of the editorial boards of different journals affected both the Qualis score (subjective assessment) and the impact of the assessed journals (objective assessment). Despite the ceremonial nature surrounding the activity performed by editorial boards, boards comprising both more prestigious and international members tended to have higher Qualis score, but not greater impact factor. It only happened due to the effect such boards had on experts’ evaluation, which did not show practical association with greater article reading and citations in the journals. However, besides enabling journals to have higher scores in Qualis, this effect has also increased the number of times they were cited in subsequent periods (Rossoni & Guarido Filho, 2012), which has self-fulfilled experts’ prophecy about the accuracy of their evaluation. Thus, the academic distinction element was given by the symbolic appeal of the evaluated journals rather than by the effective contribution of their publications.
Very little or nothing can be done about the accumulation of advantages by the most visible journals in expert-based evaluation systems. According to Machado-da-Silva et al. (2008), the citation of Brazilian journals in the Business Administration, Accounting and Tourism field has increased by 22% from 2005 to 2007 (9 out of 10 journals), whereas the citation of the three most cited journals has increased by 44% - i.e., twice as much the increase observed in the previous ones, within a 3-year interval. Besides lack of search engines to enable all journals in the field to have the same chances of being identified through searches based on topic, the inequality in the acknowledgement of these journals would only increase.
According to Starbuck (2005), the emphasis placed on prestigious journals results much more from administrative choices than from the natural adaptive result of scientific practice in Social Sciences; this factor leads to perception biases towards the value of published articles and hinders knowledge improvement processes. The emphasis on indicators and rankings further increases the biases of such a perception. Bornmann et al. (2020) have used agent-based simulations to show that the introduction of new indicators, such as Hirsch's hα-index (2019), reinforced the Matthew effect at researcher level and led to the disproportionate acknowledgement of journals that had initially presented high h-index. Lariviere and Gingras (2010) have found similar effect between journals. They performed a natural experiment to compare 4,532 identical studies published in different journals and found that journals’ impact factor has strongly influenced citation rates. Based on the comparison of each pair of duplicated articles, those published in the highest-impact journals had, on average, twice as many citations as the version published in the lowest-impact journals. Drivas and Kremmydas (2020) have also found similar results. They assessed the ranking effect on the Academic Journal Guide published by the Association of Business Schools and found causal evidence that increasing journals’ ranking on the list would increase their number of citations.
The Scientific Periodicals Electronic Library (Spell) was generated due to quality assessment biases that led to Matthew effect on journal citations. It was done in order to increase the visibility of Brazilian academic productions in the Business Administration, Accounting and Tourism field, as well as to reduce asymmetries and inequalities in the journal quality acknowledgement system so that any journal would have the same change to be identified in the database, regardless of ranking. The next section presents, in detail, the analysis method adopted in the current study to find evidence that Spell has increased the number of citations in articles available in the database, reduced the inequality between national journals in the field and mitigated the Mathew effect on journal citations.

Matherials and Methods

Data

Impact indicators of journals composing Spell were collected in its homepage (http://www.spell.org.br/impacto). Spell makes available impact indicators of indexed journals - whose first publication dates back to 2010 - on a yearly basis. Until the end of the current research, 9 listings had been published between 2010 and 2018. Thus, although Spell had 130 journals registered in 2020, there was variation in the number of journals presenting impact indicators between 2010 and 2018. It happened because not all journals were in the listing, since the service started operating in late 2012, as well as because other journals had been terminated. In addition, journals must be up to date in order to be part of the list of impact indicators released every year. Consequently, the number of indexed journals has ranged from the minimum of 78 journals in 2010 to the maximum of 114 journals in 2018.
However, the aim of the current study was to investigate the impact of Spell implementation in 2012 on journal’s impact and inequality. We herein compared the same interval before and after its launching, namely: 2010-2012 and 2013-2015. Only journals listed in all editions released within that time interval were taken into consideration in the analysis. This procedure was adopted to avoid the effect of including or excluding journals listed in different periods, as well as significant increase in the volume of citations due to incoming journals. Thus, our database was formed by a balanced data panel comprising 68 journals observed for 6 consecutive years providing 408 cases, in total.

Variables

Spell freely makes available several impact indicators for journals organized by year; it is also possible exporting them to electronic spreadsheets. Our job was to compile these indicators in a single matrix, whose definition - given by the database - can be seen at http://www.spell.org.br/impacto/ajuda. Of the available indicators, we stick to two:
Two-year impact without self-citations - remarkably similar to impact indicators used in databases such as Web of Science and SCOPUS, which take into consideration the number of times a given journal is cited – except for self-citations - in a given year (for example, 2015), in articles published in the two immediately preceding years (2014 and 2013), divided by the number of articles published in the same period. Thus, the impact of the 68 analyzed journals was annuallyobserved 6 times, between 2010 and 2015.
Five-year impact without self-citations – it was computed in the same way as the previous index; however, it took into consideration a window of citations to articles published in the previous five years and divided by the total number of articles published in those same five years. For example, the impact of each journal in the sample was assessed by taking into consideration the number of citations recorded in 2015 for articles published between 2010 and 2014, divided by the total number of articles produced in that same period. The same process was applied to the other years, until 2010.
It should be taken into consideration that, in addition to disregarding self-citations that can inflate impact indicators, the indicators described above have the advantage of being a relative measurement, since they take into consideration the number of received citations in comparison to the total number of published articles. This factor helps avoiding the effect of journal longevity and size on impact indicator composition, as observed in absolute measurements, such as the H Index, which end up reinforcing the Matthew effect (Bornmann et al., 2020; Leydesdorff et al., 2019).
In addition to indicators generated by Spell, it was necessary creating two other variables: the first one was created to show that the emergence of Spell in 2012 has increased the impact of journals and reduced the inequality of citations. The second variable was created to capture the relative impact of a given journal in comparison to the impact of all other journals - this variable enabled exposing the inequality in citations:
Period when one variable with two categories was created: 2010-2012 - aggregates all the observations recorded in the years immediately before Spell creation; 2013-2015 - also aggregates observations in our sample in the three years following the database emergence. In terms of effect capturing, this variable was treated as dummy - value 1 corresponds to the period between 2013 and 2015.
Our intention was to meet the criteria of event studies focused on evaluating the effect of external events on journals (Binder, 1998). In order to do so, it was necessary specifying an equal time window before and after Spell creation in 2012. As Spell citation data refer to the year of 2010, the time window was limited to six years: three years before and three years after its creation. Although this cut has reduced the sample to 204 observations for each period, it did not compromise the power of the herein performed tests, since sample size remained large.
iGini (individual Gini) is an individual inequality measurement tool developed by Liao (2019). It derives from one of the aggregate inequality indicators mostly used in studies about wealth distribution (Oancea and Pirjol 2019), namely: the Gini index. Ceriane and Verme (2015) have highlighted that the Gini index captures inequality based on the aggregation of individual contributions. According to Liao (2019), the aim of iGini is to analyze inequality at individual level to help improving the understanding about the ways inequality takes place. Thus, iGini can be seen as the difference in scale between a given observation and all other observations in the same sample (Liao 2019). In the current case, iGini captures the difference between the impact of one journal and that of all other journals in the same year. Thus, based on the formalization by Liao (2019, p. 7, equation 2), for each journal i, the iGini or g i can be defined as:
g i = ∑ j = 1 n x i − x j 2 n 2 x ̿
Wherein xi represents the Spell impact of journal i and xj refers to the Spell impact of all other journals j on the sample with size n. The denominator is used to normalize the indicator between 0 and 1 - the higher the value, the greater the distance in terms of citations from one journal i to the other journals j.
It is worth emphasizing that the use of Liao's iGini (2019) as measurement tool enabled analyzing the effect of Spell on reducing the inequality of citations, since any other aggregate inequality assessment tool would significantly reduce the degrees of freedom of the tests.

Analysis Models

Since annual observations referring to three years before and three years after Spell creation were recorded for all 68 journals, we used panel data models with fixed effects. It is worth emphasizing that these models are mainly useful when heterogeneity is not observed in balanced panels (Bell & Jones, 2015). In our study, these cases refer to journals, whose other effects in the period would not be easily identified. Thus, 6 models were generated in the current study: the first 4 were used to evaluate the effect of Spell implementation on the 2- and 5-year impacts and on inequality in the 2- and 5-year impacts based on the dummy variable “Period”, which captured the effect of the 2013-2015 period on dependent variables. As for the last two models, variable “Impact” was added to Spell as moderator in inequality models to capture whether Spell implementation affected journals with different impact levels in different ways. We ran the same models as robustness check by taking into consideration the likelihood of heteroscedasticity issues (based on the White test). In addition, we used random-effect models to run the panels. There was no significant change in coefficients, regardless of the adopted models and error types.
In addition to the fixed and random models of panel data, we used the Analysis of Variance between Subjects (ANOVA) to calculate the size of the Spell effect; the explained variance rate was calculated by dividing the value of the sum of Type III squares by the total corrected variance, as suggested by Snijders and Bosker (2011). Thus, 2- and 5-year impacts, as well as 2- and 5-year iGini, were incorporated to these models as variables to be tested. The effect of including journals in Spell was also tested - based on variable “Period” - in order to check whether the means recorded for impact and inequality differed from each other between 2013-2015 and 2010-2012. Similar to fixed panel data models, we controlled part of the non-observed effects by incorporating factor Journal to increase the concurrent validity degree. With this, we hope that fluctuations that occurred at the journal level caused by variables or events not included in the model, will not severely contaminate the analysis of the Spell effect. ANOVA models were included in Appendix 1.

Results

Spell-Indexed Journal Citations

The citation rate recorded for articles belonging to the Business Administration, Accounting and Tourism field, which were published in journals listed in the Spell database has been constantly increasing in Brazil, as shown in Figure 3. A little more than 37% of citations observed in 2010 referred to articles in journals, whereas the total citation rate recorded in 2015 has increased to 52%. This outcome has indicated trend to increasingly cite peer-reviewed documents to the detriment of books or other document types.
Regardless of the reasons why this increase has been taking place - from the larger number of databases providing access to articles to the constant valuing of journal citations, mainly of the international ones - it was pertinent comparing the increased number of citations of Spell-indexed journals (all Brazilian journals) to the total number of journal citations. Figure 3 shows this proportion, according to which the rate of article citations in Spell has increased from 12.8% in 2012 (when Spell was launched) to 15.2% in 2015.
Therefore, despite the emphasis given to the fact that Brazilian research in the Business Administration, Accounting and Tourism field has been increasingly using foreign literature, primarily in English, the data points out exactly the opposite: that even though it is not the majority of citations, there is a constant increase in citations of Brazilian papers indexed in Spell. This outcome meets the aim of the database, which was created to increase the access to, and visibility of, Brazilian scientific production.

Impact of, and inequality in, Spell citations

The percentile growth of citations in Spell must also be taken into consideration based on the impact index, which compares the total number of citations recorded for a given journal to the total number of published articles. Figure 4 shows the mean impact of journals indexed in Spell, in a 5-year window - self-citations were always excluded. Up to Spell implementation in 2012, the impact of database journals ranged from 0.09 citations per document in 2010 to 0.17 in 2012. After Spell implementation, this rate increased to 0.20 in 2013 and reached its peak (0.28) in 2015.
Although the citation increase trend was observed before Spell was implemented in 2012, we understand that this trend was consolidated by the emergence of the aforementioned indexer, since it was widely adopted by the research community in the Business Administration, Accounting and Tourism field, which needed topic-based search engines comprising all database journals. This outcome has encouraged us to make more robust comparisons.
However, it is worth emphasizing that increase in mean citations of Spell articles does not happen in a concentrated manner. On the contrary, since the platform enables equal access to articles, data shown in Figure 4 indicate that the distribution of citations is getting lesser unequal.
This outcome was evidenced through the Gini coefficient of Spell impact recorded for a 5-year window. Such a measurement tool is used to measure inequality in distributions, mainly to measure income inequality between countries. As previously mentioned, this index ranges from zero to one - higher values correspond to greater inequality. Gini coefficient tended to remain stable until 2012. After this year, when Spell had started its activities, the coefficient has systematically declined, and it indicates that the inequality of citations among journals in the database has decreased overtime.
For the purpose of analogy with income inequality between countries, it is as if we have left an inequality condition equal to that of Brazil to the inequality of countries such as Israel and Bulgaria. However, without analogy, the decreased inequality of Spell citation impact indicates that, despite the differences in resources and power asymmetries observed between journals listed in the database, Brazilian researchers in the Business Administration, Accounting and Tourism field started to acknowledge the usefulness of journals in a more egalitarian and lesser centralized way. As we have pointed out, Spell's prominence in promoting fairer conditions for Brazilian journals has likely led to lesser inequality, since all journals have the same visibility space and can be mainly accessed based on content rather than on journal status.
Apparently, the "richest" journals in terms of citations are not proportionally getting "richer", which is the reason why we must carefully analyze whether Spell has reduced the Mathew effect on journal citations. Since such assessments cannot be conducted with aggregate indices such as Gini, we used the measurement tool recently formulated by Liao (2019), iGini, whose trend is also shown in Figure 4. It is worth emphasizing that Gini corresponds to the exact sum of iGini; however, the Gini excludes cases presenting value 0, which is not the case of iGini calculation. This is the reason why iGini recorded higher values than Gini between 2010 and 2012. These journals without citations were listed in the database; therefore, they could not be featured as omitted cases; thus, they were included in the analysis applied to Spell effect on reducing the inequality of citations.

Comparing the impact of, and inequality in, citations before and after Spell implementation

Table 1 presents the mean impact of Spell, without self-citations, recorded for 2- and 5-year windows in the periods right before (2010-2012) and after (2013-2015) Spell implementation. It also presents the mean iGini recorded for the same periods, whose inequality was calculated based on the impact of journals. As expected, the mean 2-year (0.097 to 0.181) and 5-year impacts (0.124 to 0.236) have increased between the two periods. Impact inequality recorded for the 2-year (0.086 to 0.0067) and 5-year (0.0085 to 0.0065) windows between the two periods has decreased.
The lowest inequality of citations observed between the evaluated periods is highlighted in Figure 5, which depicts the distribution of the 2-year (violin chart on the left) and 5-year (on the right) impacts. Greater volume of journals recorded lower impact values in 2010-2012; however, this volume has decreased in 2013-2015, due to higher density of cases in intermediate portions to journals listed in the database. Thus, Spell's mean impact has shown increase in the number of citations recorded for documents between the analyzed periods, whereas iGini has evidenced whether such an increase was better distributed.
Although the herein recorded means have indicated association of Spell implementation with increased impact and reduced inequality, it was necessary testing whether the period-change effect was significant, or not. It was done based on the application of regression tests to fixed panel models; results are shown in Table 2. After the effects of journals were controlled in models 1 and 2, there was significant increase in the impact of journals after Spell implementation (variable 2013-2015), both for the 2- (b = 0.084) and 5-year (b = 0.112) windows. In other words, based on the comparison between three years before Spell implementation and three years after it, there was significant increase in mean impact recorded for both time windows, whose explanatory power ranged from 7.6% to 7.7% (Appendix 1).
Models 3 and 4 presented significant effect on reducing impact inequality between journals in the 2- (b = 0.002) and 5-years (b = 0.002) windows. Thus, based on the comparison between 2010-2012 and 2013-2015, there was decreased difference in the impact of journals (explanatory power of 3.3%). This outcome enables saying that journals’ inclusion in Spell is associated with increased equitable distribution of citations per document, which, consequently, reduced the Matthew effect at journal level.
Finally, the dummy variable representing journals’ inclusion in Spell (2013-2015) has interacted with the absolute impact of journals (2- and 5-Year Impacts) in models 5 and 6. In was done to investigate whether inequality reduction was associated with journals’ impact factor. Results have shown that the greater the impact of the journal, the lesser unequal it became in relation to the others, both for the 2- (b = -0.015) and 5-year impacts (b = -0.011).
Figure 6 depicts the association between impact and inequality before and after Spell implementation. Both impact measurements - 2 and 5 years – have shown decreased association between impact and iGini in 2013-2015 in comparison to 2010-2012. Thus, the Matthew effect of journals refers to the effect of rankings on citations, whereas the current data show that journals’ inclusion in Spell has reduced this effect and made them more egalitarian.

Discussion and Conclusion

This paper has shown how the inclusion of journals in open databases and indexers based on inclusive logic has the potential to increase the impact of citations and to reduce inequality between journals. Based on the Brazilian experience in the Business Administration, Accounting and Tourism research field, which created a database and indexer to cover most national scientific productions in the field in 2012, the current study has: 1) compared the impact of journals before and after Spell implementation; and 2) investigated whether these journals got lesser unequal in terms of impact after Spell implementation.
With respect to the first point, after the non-observed heterogeneity at journal level was controlled, results have shown that the mean 2- and 5-year impact of journals recorded in the three years right after Spell implementation was significantly higher than that recorded in the three years before it. It is worth emphasizing that the comparison of citations before and after journals’ inclusion in Spell was only possible because the database and indexer has calculated the impact indicators for the three years prior to its launch.
As previously mentioned, the most likely interpretation for the fact that Spell has increased the mean number of citations lies on the fact that the platform made available articles openly published in the management research field on a single database that enabled equal search possibilities for all journals. Before Spell, there was no efficient search engine encompassing the scientific production of the management field in a single environment; consequently, researchers in the field had to search journals’ websites in separate, a fact that favored the best-known ones. The visibility Spell has given to journals results from the prominence of its owner, National Association of Postgraduation and Research in Administration (ANPAD), which is the association of graduate programs and researchers in the field that has outspread its use countrywide. Thus, without ANPAD's prominence in the field, the database adoption process would have been much slower and would not allow the improvement in information and visibility to be used an impact increase mechanism.
As for the second point, the inequality measurement newly created by Liao (2019) – iGini – enabled showing that journals’ inclusion in Spell has made the impact of citations lesser unequal within the three years after such an inclusion than within the three years before it, both for the 2- and 5-year impact measurements. In addition, we have also shown that the influence of the ranking of journals in Spell on the inequality between journals has decreased after journals were indexed in it. This outcome enabled concluding that Spell has likely reduced the Matthew effect on journal citations, since the highest-impact journals were not so far from the other journals.
Our understanding is that the reduced Matthew effect on Spell journals’ citations is linked to two mechanisms. The first mechanism refers to reduced information asymmetry. As most journals in the management field were not ranked on databases such as Scopus and Web of Science, the evaluation of their likely quality was based on Qualis CAPES, which is the basis used to classify Brazilian journals (Souza & Paula, 2002) Until 2012 Qualis CAPESa guided the evaluation of Brazilian journals almost exclusively based on subjective criteria (Rosa & Romani-Dias, 2019b; Rossoni & Guarido Filho, 2012). Moreover, such an assessment was based on the status of the institution editing the journal (Medoff, 2006). Thus, after the creation of Spell, which provided the same possibilities for articles to be found through thematic searches, researchers had a better opportunity to assess the quality of articles, regardless of the journal. In fact, if articles with better perceived quality tend to be more cited (Wang, 2014), it has likely happened at Spell.
The second mechanism refers to decrease in the perception about the differentiated status attributed to some journals that are better classified in rankings such as Qualis or due to the prestige of institutions accounting for editing them. This status leads researchers to cite articles deriving from better-evaluated journals, rather than basing the citation on articles’ quality (Drivas & Kremmydas, 2020; Larivière & Gingras, 2010). On the contrary, limitations in process of investigating all the relevant literature often lead researchers to only search for studies in the most prestigious journals (Bornmann et al., 2020; Wang, 2014). This phenomenon reinforces the Matthew effect, since the most prestigious journals always have disproportionate acknowledgement gains. However, the rise of Spell - which has incorporated equitable search engine to impact measurements based on local production – has decreased the incentives to only cite the most prestigious journals. Individual inequality measurements point towards decreased differences in citations between journals and show that such a decrease is inversely proportional to the impact of journals.
Our findings have important implications for studies about the impact of journals, as well as about scientific research acknowledgement mechanisms. First, based on the analytical perspective, results have indicated that the unwanted trend of researchers to keep on citing scientific articles from better ranked journals mainly takes place because indexers (e.g., Scopus and Web of Science) and listings of journals with differentiated status (e.g., Academic Journal Guide of the Association of Business Schools) operate based on a segregation logic and exclude most relevant journals. This process is evident in significant part of the literature, which shows string association between journal status and citations (Bornmann et al. 2020; Drivas & Kremmydas, 2020; Larivière & Gingras, 2010; Leydesdorff et al., 2019; Starbuck, 2005; Wang, 2014). Without this segregation mechanism, the cumulative advantages of such rankings would most likely be mitigated. According to Merton (1968), Price (1976) and DiPrete and Eirich (2006), the Matthew effect only happens in systems based on different status that are not always associated with quality. The effects of citing the best evaluated journals would be mitigated in systems based on the inclusive logic, such as Spell and Redalyc (López et al. 2008).
Secondly, in methodological terms, the analysis of inequality between actors in a given social system has always been a methodological challenge (Ceriani & Verme, 2015), mainly if one investigates the individual contribution of each actor. Thus, only recently, Liao (2019) has developed an individual indicator – iGini -, which has several advantages such as aggregated or segmented measurements between and within groups. Thus, we advocate that incorporating iGini as measurement to assess differences in impact and citations can open room for relevant research to be developed by scholars focused on investigating the Matthew effect and science acknowledgement, mainly because iGini can capture a given actor's distance from everyone else, as well as enables analyzing different forms of inequality.
Thirdly, it is necessary highlighting the practical implications of this study, mainly to research assessment policies. By incorporating an open, inclusive, free database and indexer without the intermediation of commercial publishers, as in the case of Spell, such a system directly attacks one of the biggest issues observed in the use of rankings, listings and impact factors: acknowledgement based on citations that do not necessarily refer to the quality of articles. In other words, since such initiatives reduce the Matthew effect, they bring the practice of citations closer to an idealistic ethos of scientific acknowledgement based on merit (Merton 1996), which is only possible through inclusive, non-exclusive databases and indexers. At individual level, if there is clear evidence that authors cite articles based on journals’ status and impact factor (Drivas & Kremmydas, 2020; Larivière & Gingras, 2010), the academic community must explicitly acknowledge these citations for the quality of the documents, regardless of the journals’ prestige. One of the recent alternatives lies on the likelihood of journals, indexers, institutions and research funds to be signatories to the San Francisco Declaration on Research Evaluation (DORA), which makes recommendations about the separation of journal metrics from the research evaluation itself. What DORA signatories want - and there are indications that they are achieving it (Schmid, 2017) - is that science assessment systems that currently attribute impact through citations do not distort the system to the point of making such metrics useless. In addition, it is worth emphasizing that Spell is the initiative of a large, although peripheral, academy that faces great issues in legitimizing itself at local and international levels, namely: the Brazilian academy. Since the Matthew effect on citations also happens at country level (Bonitz et al., 1997), initiatives focused on promoting the dissemination of local science, in an open and free way, help fighting the hegemonic practices of commercial publishers (Meriläinen et al., 2008) that limit knowledge dissemination and distort the acknowledgement of academic ventures.
Finally, it is necessary highlighting that this research has some limitations that should be overcome in future research. The main limitation lies on the fact that we conducted an event study in which the exogenous shock - i.e., the creation of Spell - affected all journals at the same time. Although there is no suspicion of another relevant event for the journals in the sample, our research strategy may have contemporary correlation issues. In addition, as data about citations of Brazilian journals in the management field that are not included in Spell are not available in virtually any other index, it was not possible performing a quasi-experimental treatment to compare the sample of journals included in Spell to those that were not included in it. Future studies could investigate mechanisms used to estimate citations of journals that are not included in indexers, such as Spell, in order to check the causality of including a given journal in them, both in terms of impact and inequality. Since we were not interested in explaining the individual impact of journals, but rather the change caused by Spell implementation, our study has ignored effects at author and article level, as well as some proxies at journal level. Future studies focused on better understanding the inequality of citations could decompose the effect at these levels. Finally, since iGini is a measurement of individual inequality, we strongly recommend that this indicator should be used by other researchers to investigate the effect of other databases on the stratification of science, mainly by comparing the effect of international indexers.

Author Contributions

Conceptualization, LR and RAR; methodology, LR ; validation, LR and RAR; investigation, LR and RAR; formal analysis, LR; writing—original draft preparation, LR and RAR; writing—review and editing, LR and RAR; visualization, LR; funding acquisition, LR. The authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by CNPq (Grant number 435946/2018-8) and FAPERJ (Grant number E-26/203.238/2017).

Institutional Review Board Statement

Not applicable.

Data Availability Statement

Data used in the study are freely available.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix 1: Tests of Between-Subjects Effects (ANOVA)

(1) (2) (3) (4) (5) (6)
2-Year Impact 5-Year Impact 2-Year iGini 5-Year iGini 2-Year iGini 5-Year iGini
2013-2015 x Spell Impact 0.001*** 0.001***
(0.026) (0.029)
2013-2015 0.724*** 1.285*** 0.001*** 0.001*** 0.001*** 0.001**
(0.076) (0.077) (0.033) (0.033) (0.008) (0.002)
Journals 6.621*** 13.196*** 0.008*** 0.011*** 0.002*** 0.003***
(0.697) (0.789) (0.723) (0.863) (0.201) (0.224)
Corrected model (R²) 7.345*** 14.481*** 0.008*** 0.011*** 0.009*** 0.011***
(0.773) (0.866) (0.753) (0.884) (0.847) (0.965)
Error 2.152 2.236 0.003 0.001 0.002 0.001
(0.227) (0.134) (0.243) (0.104) (0.174) (0.073)
Corrected Total 9.497 16.717 0.011 0.012 0.009 0.012
(1.000) (1.000) (1.000) (1.000) (1.000) (1.000)
Intercept 7.891*** 13.168*** 0.024*** 0.023*** 0.003*** 0.002***
Total 17.388 29.885 0.035 0.036 0.035 0.036
Note: Effect size in parenthesis. N = 408. *** p<0.01, ** p<0.05, * p<0.1.

Notes

1
Source: https://sucupira.capes.gov.br/sucupira/public/consultas/coleta/programa/quantitativos/quantitativoAreaAvaliacao.xhtml
2
Annual Spell impact indicators available at http://www.spell.org.br/impacto

References

  1. Alcadipani, R. (2017). Periódicos brasileiros em inglês: A mímica do publish or perish" global". Revista de Administração de Empresas, 57(4), 405-411. [CrossRef]
  2. Azoulay, P., Stuart, T., & Wang, Y. (2014). Matthew: Effect or fable?. Management Science, 60(1), 92-109. [CrossRef]
  3. Bell, A., & Jones, K. (2015). Explaining fixed effects: Random effects modeling of time-series cross-sectional and panel data. Political Science Research and Methods, 3(1), 133-153. [CrossRef]
  4. Binder, J. (1998). The event study methodology since 1969. Review of Quantitative Finance and Accounting, 11(2), 111-137. [CrossRef]
  5. Bonitz, M., Bruckner, E., & Scharnhorst, A. (1997). Characteristics and impact of the Matthew effect for countries. Scientometrics, 40(3), 407-422. [CrossRef]
  6. Bornmann, L., Ganser, C., Tekles, A., & Leydesdorff, L. (2020). Does the hα-index reinforce the Matthew effect in science? The introduction of agent-based simulations into scientometrics. Quantitative Science Studies, 1(1), 331-346. [CrossRef]
  7. Carvalho Neto, S. C., Willinsky, J., & Alperin, J. P. (2016). Measuring, rating, supporting, and strengthening open access scholarly publishing in brazil. Education policy analysis archives, 24(54), 1-26. [CrossRef]
  8. Ceriani, L., & Verme, P. (2015). Individual diversity and the Gini decomposition. Social Indicators Research, 121(3), 637-646. [CrossRef]
  9. Cetto, A. M., Alonso-Gamboa, J. O., & González, S. C. (2010). Ibero-American systems for the dissemination of scholarly journals: a contribution to public knowledge worldwide. Scholarly and Research Communication, 1(1), 1-16. [CrossRef]
  10. Clarivate, (2017). Research in Brazil A report for CAPES by Clarivate Analytics. Retrieved, November 10, 2020, from http://www.sibi.usp.br/wp-content/uploads/2018/01/Relat%C3%B3rio-Clarivate-Capes-InCites-Brasil-2018.pdf.
  11. Collazo-Reyes, F. (2014). Growth of the number of indexed journals of Latin America and the Caribbean: the effect on the impact of each country. Scientometrics, 98(1), 197-209. [CrossRef]
  12. Costa, M. P. D., & Leite, F. C. L. (2016). Open access in the world and Latin America: A review since the Budapest Open Access Initiative. TransInformação, 28(1), 33-46. [CrossRef]
  13. DiPrete, T. A., & Eirich, G. M. (2006). Cumulative advantage as a mechanism for inequality: A review of theoretical and empirical developments. Annual Review of Sociology, 32, 271-297. [CrossRef]
  14. Drivas, K., & Kremmydas, D. (2020). The Matthew effect of a journal's ranking. Research Policy, 49(4), 103951. [CrossRef]
  15. Fischman, G. E., Alperin, J. P., & Willinsky, J. (2010). Visibility and quality in Spanish-language Latin American scholarly publishing. Information technologies & international development, 6(4), 1-21. Retrieved, March 9, 2021, from https://itidjournal.org/index.php/itid/article/download/639/639-1694-2-PB.pdf.
  16. Freitas, I. M. B., Marques, R. A., & Silva, E. M. D. P. (2013). University–industry collaboration and innovation in emergent and mature industries in new industrialized countries. Research Policy, 42(2), 443-453. [CrossRef]
  17. Guédon, J. C. (2008). Open Access and the divide between “mainstream” and “peripheral” science. In: Ferreira, S.M.S, & Targino, M (Eds.). Como gerir e qualificar revistas científicas (pp. 1-21, forthcoming in 2007, in Portuguese). Napoli: e-LIS. Retrieved, November 29, 2020, from http://eprints.rclis.org/10778/1/Brazil-final.pdf.
  18. Gutierrez, F. R., Beall, J., & Forero, D. A. (2015). Spurious alternative impact factors: The scale of the problem from an academic perspective. Bioessays, 37(5), 474-476. [CrossRef]
  19. Harzing, A. W., & Alakangas, S. (2016). Google Scholar, Scopus and the Web of Science: a longitudinal and cross-disciplinary comparison. Scientometrics, 106(2), 787-804. [CrossRef]
  20. Hicks, D., Wouters, P., Waltman, L., De Rijcke, S., & Rafols, I. (2015). Bibliometrics: the Leiden Manifesto for research metrics. Nature, 520(7548), 429-431. [CrossRef]
  21. Hirsch, J. E. (2019). hα: An index to quantify an individual’s scientific leadership. Scientometrics, 118(2), 673–686. [CrossRef]
  22. Kulkarni, A. V., Aziz, B., Shams, I., & Busse, J. W. (2009). Comparisons of citations in Web of Science, Scopus, and Google Scholar for articles published in general medical journals. Jama, 302(10), 1092-1096. [CrossRef]
  23. Larivière, V., & Gingras, Y. (2010). The impact factor's Matthew Effect: A natural experiment in bibliometrics. Journal of the American Society for Information Science and Technology, 61(2), 424-427. [CrossRef]
  24. Larivière, V., Haustein, S., & Mongeon, P. (2015). The oligopoly of academic publishers in the digital era. PloS one, 10(6), e0127502. [CrossRef]
  25. Laudel, G. (2006). The ‘quality myth’: Promoting and hindering conditions for acquiring research funds. Higher Education, 52(3), 375–403. [CrossRef]
  26. Leite, P., Mugnaini, R., & Leta, J. (2011). A new indicator for international visibility: exploring Brazilian scientific community. Scientometrics, 88(1), 311-319. [CrossRef]
  27. Leta, J., & Chaimovich, H. (2002). Recognition and international collaboration: the Brazilian case. Scientometrics, 53(3), 325-335. [CrossRef]
  28. Leydesdorff, L., Bornmann, L., & Opthof, T. (2019). hα: The scientist as chimpanzee or bonobo. Scientometrics, 118(3), 1163–1166. [CrossRef]
  29. Liao, T. F. (2019). Individual Components of Three Inequality Measures for Analyzing Shapes of Inequality. Sociological Methods & Research. [CrossRef]
  30. López, E. A., Salazar, R. R., Oropeza, G. G., & Zúñiga, M. F. (2008). Redalyc: una alternativa a las asimetrías en la distribución del conocimiento científico. Ciencia, docencia y tecnologia, 19(37), 11-30. Retrieved, March 9, 2021, from https://www.redalyc.org/pdf/145/14511370002.pdf.
  31. Machado-da-Silva, C. L., Guarido Filho, E. R., Rossoni, L., & Graeff, J. F. (2008). Periódicos Brasileiros de Administração: Análise Bibliométrica de Impacto no Triênio 2005-2007. RAC-Eletrônica, 2(3), 351-373. Retrieved, March 9, 2021, from http://www.spell.org.br/documentos/download/31030.
  32. Marchitelli, A., Galimberti, P., Bollini, A., & Mitchell, D. (2017). Improvement of editorial quality of journals indexed in DOAJ: a data analysis. JLIS. it: Italian Journal of Library, Archives and Information Science. Rivista italiana di biblioteconomia, archivistica e scienza dell'informazione, 8(1), 1-21. [CrossRef]
  33. Marenco, A. (2015). When institutions matter: Capes and political science in Brazil. Revista de Ciência Política, 35(1), 33-46. Retrieved, March 9, 2021, from https://www.redalyc.org/pdf/324/32439319003.pdf.
  34. Martín-Martín, A., Orduna-Malea, E., Thelwall, M., & López-Cózar, E. D. (2018). Google Scholar, Web of Science, and Scopus: A systematic comparison of citations in 252 subject categories. Journal of Informetrics, 12(4), 1160-1177. [CrossRef]
  35. Martín-Martín, A., Thelwall, M., Orduna-Malea, E., & López-Cózar, E. D. (2021). Google Scholar, Microsoft Academic, Scopus, Dimensions, Web of Science, and OpenCitations’ COCI: a multidisciplinary comparison of coverage via citations. Scientometrics, 126(1), 871-906. [CrossRef]
  36. Medoff, M.H. (2006). Evidence of a Harvard and Chicago Matthew Effect. Journal of Economic Methodology, 13(4), 485–506. [CrossRef]
  37. Meneghini, R., Mugnaini, R., & Packer, A. L. (2006). International versus national oriented Brazilian scientific journals. A scientometric analysis based on SciELO and JCR-ISI databases. Scientometrics, 69(3), 529-538. [CrossRef]
  38. Meneghini, R., & Packer, A. L. (2007). Is there science beyond English? Initiatives to increase the quality and visibility of non-English publications might help to break down language barriers in scientific communication. EMBO reports, 8(2), 112-116. [CrossRef]
  39. Meriläinen, S., Tienari, J., Thomas, R., & Davies, A. (2008). Hegemonic academic practices: experiences of publishing from the periphery. Organization, 15(4), 584-597. [CrossRef]
  40. Merton, R. K. (1968). The Matthew effect in science. Science, 159(3810), 56-63. [CrossRef]
  41. Merton, R. K. (1988). The Matthew effect in science, II: cumulative advantage and the symbolism of intellectual property. Isis, 79(4), 606-623. Retrieved, March 9, 2021, from https://www.jstor.org/stable/234750.
  42. Merton, R. K. (1996). The ethos of science. In R. K. Merton. (1996). On social structure and science (pp. 267-276). Chicago: The University of Chicago Press.
  43. Mesquita, C. T., Borim, D., & Rochitte, C. E. (2019). Open Science, Cardiology and 20 years of SciELO (Scientific Electronic Library Online). International Journal of Cardiovascular Sciences, 32(3), 203-204. [CrossRef]
  44. Mongeon, P., & Paul-Hus, A. (2016). The journal coverage of Web of Science and Scopus: a comparative analysis. Scientometrics, 106(1), 213-228. [CrossRef]
  45. Mugnaini, R., Damaceno, R. J. P., Digiampietri, L. A., & Mena-Chalco, J. P. (2019). Panorama da produção científica do Brasil além da indexação: uma análise exploratória da comunicação em periódicos. Transinformação, 31, 1-15. [CrossRef]
  46. Oancea, B., & Pirjol, D. (2019). Extremal properties of the Theil and Gini measures of inequality. Quality & Quantity, 53(2), 859-869. [CrossRef]
  47. Perlin, M. S., Imasato, T., & Borenstein, D. (2018). Is predatory publishing a real threat? Evidence from a large database study. Scientometrics, 116(1), 255–273. [CrossRef]
  48. Piwowar, H., Priem, J., Larivière, V., Alperin, J. P., Matthias, L., Norlander, B. et al. (2018). The state of OA: a large-scale analysis of the prevalence and impact of Open Access articles. PeerJ, 6, e4375. [CrossRef]
  49. Price, D. D. S. (1976). A general theory of bibliometric and other cumulative advantage processes. Journal of the Association for Information Science and Technology, 27(5), 292-306. [CrossRef]
  50. Rocha, F. G., Sabino, R. F., & Frery, A. C. (2020). Analysis of the international impact of the Brazilian base “Qualis”-Education. Scientometrics, 125(3), 1949-1963. [CrossRef]
  51. Rodrigues, R. S., & Abadal, E. (2014). Ibero-American journals in Scopus and Web of Science. Learned Publishing, 27(1), 56-62. [CrossRef]
  52. Rosa, R. A., & Romani-Dias, M. R. (2019). Indexação de periódicos e a política de avaliação científica: uma análise do campo de administração, contabilidade e turismo no Brasil. International Journal of Professional Business Review, 4(2), 1-24. [CrossRef]
  53. Rosa, R. A., & Romani-dias, M. (2019b). A Presença e o Impacto de Periódicos Brasileiros da Área de Administração, Contabilidade e Turismo em Bases Científicas. Revista Eletrônica de Ciência Administrativa, 18(3), 327-348. [CrossRef]
  54. Rossoni, L., & Guarido Filho, E. R. (2012). Onipresença nos conselhos editoriais: prestígio e cerimonialismo na atividade científica. Redes. Revista hispana para el análisis de redes sociales, 22(1), 189-218. [CrossRef]
  55. Science-Metrix. (2018). Analytical Support for Bibliometrics Indicators Open access availability of scientific publications. Montréal: Science-Metrix Inc. Retrieved, November 10, 2021, from https://www.science-metrix.com/sites/default/files/science-metrix/publications/science-metrix_open_access_availability_scientific_publications_report.pdf.
  56. Silva, J. A. T., Dobránszki, J., Al-Khatib, A., & Tsigaris, P. (2018). Challenges facing the DOAJ (Directory of Open Access Journals) as a reliable source of open access publishing venues. Journal of Educational Media & Library Sciences, 55(3), 349-358. [CrossRef]
  57. Schmid, S. L. (2017). Five years post-DORA: promoting best practices for research assessment. Molecular Biology of The Cell, 28(22), 2941-2944. [CrossRef]
  58. Snijders, T. A., & Bosker, R. J. (2011). Multilevel analysis: An introduction to basic and advanced multilevel modeling. Thousand Oaks, Sage.
  59. Souza, E. P., & Paula, M. C. S. (2002). Qualis: a base de qualificação dos periódicos científicos utilizada na avaliação CAPES. InfoCAPES, 10(2), 6-24. Retrieved, November 10, 2021, https://www.scielo.br/scielo.php?script=sci_nlinks&ref=000109&pid=S0104-1169200900030001900006&lng=en.
  60. Starbuck, W. H. (2005). How much better are the most-prestigious journals? The statistics of academic publication. Organization Science, 16(2), 180-200. [CrossRef]
  61. Strehl, L., Calabró, L., Souza, D. O., & Amaral, L. (2016). Brazilian science between national and foreign journals: Methodology for analyzing the production and impact in emerging scientific communities. PloS One, 11(5), e0155148. [CrossRef]
  62. Swan, A. (2008). Why Open Access for Brazil? Liinc em Revista, 4(2), 158-171. [CrossRef]
  63. Trzesniak, P. (2016). Qualis in four quarters: history and suggestions for the Administration, Accounting and Tourism area. Revista Contabilidade & Finanças, 27(72), 279-290. [CrossRef]
  64. Vélez-Cuartas, G.; Lucio-Arias, D.; Leydesdorff, L. (2016). Regional and global science: Publications from Latin America and the Caribbean in the SciELO Citation Index and the Web of Science. El profesional de la información, 25(1), 35-46. [CrossRef]
  65. Vessuri, H. (1995). Recent strategies for adding value to scientific journals in Latin America. Scientometrics, 34(1), 139-161. [CrossRef]
  66. Xia, J. (2019). A preliminary study of alternative open access journal indexes. Publishing Research Quarterly, 35(2), 274-284. [CrossRef]
  67. Wang, J. (2014). Unpacking the Matthew effect in citations. Journal of Informetrics, 8(2), 329-339. [CrossRef]
Preprints 90283 g001
Figure 1. Journals and papers indexed in Spell. Source: Scientific Periodicals Electronic Library.
Figure 1. Journals and papers indexed in Spell. Source: Scientific Periodicals Electronic Library.
Preprints 90283 g001
Preprints 90283 g002
Figure 2. Presence of Spell and other indexers in Capes’ evaluation of the Business Administration, Accounting and Tourism fields. Source: CAPES’ Mid-Term Seminar.
Figure 2. Presence of Spell and other indexers in Capes’ evaluation of the Business Administration, Accounting and Tourism fields. Source: CAPES’ Mid-Term Seminar.
Preprints 90283 g002
Preprints 90283 g003
Figure 3. Article citations in Spell. The blue line corresponds to the citation rate recorded for articles published in journals, whose rate is shown on the left axis. The orange line represents article citations in Spell in comparison to the total number of article citations. Source: Spell citation data.
Figure 3. Article citations in Spell. The blue line corresponds to the citation rate recorded for articles published in journals, whose rate is shown on the left axis. The orange line represents article citations in Spell in comparison to the total number of article citations. Source: Spell citation data.
Preprints 90283 g003
Preprints 90283 g004
Figure 4. Impact of, and inequality in, Spell citations (5 years). The herein adopted Spell impact indicators excludes self-citations. The impact indicator takes into consideration the mean recorded for each year, whereas iGini is an absolute added value formed by the sum of the individual contribution of each journal, which does not exclude cases with value 0 from the sample. Thus, Gini differed from the sum of iGini in the years when journals presented zero impact. Source: Spell impact indicators of 68 journals listed in all evaluated years.
Figure 4. Impact of, and inequality in, Spell citations (5 years). The herein adopted Spell impact indicators excludes self-citations. The impact indicator takes into consideration the mean recorded for each year, whereas iGini is an absolute added value formed by the sum of the individual contribution of each journal, which does not exclude cases with value 0 from the sample. Thus, Gini differed from the sum of iGini in the years when journals presented zero impact. Source: Spell impact indicators of 68 journals listed in all evaluated years.
Preprints 90283 g004
Preprints 90283 g005
Figure 5. Frequency distribution of impact on Spell. The higher the concentration of journals in a given stratum, the thicker the graph area.
Figure 5. Frequency distribution of impact on Spell. The higher the concentration of journals in a given stratum, the thicker the graph area.
Preprints 90283 g005
Preprints 90283 g006
Figure 6. Inequality in Mean Citations in Spell.
Figure 6. Inequality in Mean Citations in Spell.
Preprints 90283 g006
Table 1. Impact and mean inequality of Journals in Spell.
Table 1. Impact and mean inequality of Journals in Spell.
2-Year Impact 5-Year Impact
2010-2012 2013-2015 2010-2012 2013-2015
Spell Impact Factor 0.097 0.181 0.124 0.236
iGini 0.0086 0.0067 0.0085 0.0065
Note: N = 408 (68 journals x 6 years). Spell Impact Factor without self-citations. IGini was calculated based on the impact of Spell. The higher the mean value recorded for iGini, the greater the impact inequality.
Table 2. Regression results recorded panel fixed effect models.
Table 2. Regression results recorded panel fixed effect models.
(1) (2) (3) (4) (5) (6)
2-Year Impact 5-Year Impact iGini2 iGini5 iGini2 iGini5
2013-2015 0.084*** 0.112*** -0.002*** -0.002*** -0.001*** -0.001***
(0.008) (0.008) (0.000) (0.000) (0.000) (0.000)
2-Year Impact 0.028***
(0.002)
2013-2015 x 2-Year Impact -0.015***
(0.002)
5-Year Impact 0.013***
(0.001)
2013-2015 x 5-Year Impact -0.011***
(0.001)
Constant 0.097*** 0.124*** 0.009*** 0.009*** 0.006*** 0.007***
(0.006) (0.006) (0.000) (0.000) (0.000) (0.000)
R2 0.252 0.365 0.121 0.241 0.414 0.460
Adjusted R2 0.101 0.237 0.056 0.089 0.293 0.348
F 113.97*** 194.70*** 46.45*** 107.67*** 79.47*** 95.74***
Note: Standard errors are presented in parenthesis. N = 408. *** p<0.01, ** p<0.05, * p<0.1.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
Prerpints.org logo

Preprints.org is a free preprint server supported by MDPI in Basel, Switzerland.

facebook logotwitter logolinkedin logo
MDPI Initiatives
Important Links
Subscribe

Disclaimer

Terms of Use

Privacy Policy

© 2024 MDPI (Basel, Switzerland) unless otherwise stated