Prerpints.org logo
Preprint
Article

An Assessment Model for Sustainability City Using Crowdsourced Data Based on General System Theory: A Design Science Methodology Approach

Altmetrics

Downloads

229

Views

60

Comments

0

A peer-reviewed article of this preprint also exists.

This version is not peer-reviewed

Submitted:

04 September 2023

Posted:

06 September 2023

You are already at the latest version

Alerts
Abstract
Historically, the lack of comprehensive and real-time data is the main challenge in city sustainability assessments. However, with the increase in the availability of crowdsourced data over the last decade, a rich and real-time data source provides detailed images of urban systems. This has opened up new opportunities for practitioners, including geographers, social researchers, and data scientists, to gather and utilize social media data as a core source for studying cities and assessing their sustainability. This paper proposes an assessment model for the sustainability of cities based on crowdsourced data and the principles of General System Theory. The model aims to utilize crowdsourced data for the analysis and decision-making processes to enhance the sustainability and resilience of smart future cities.
Keywords: 
Subject: Social Sciences  -   Urban Studies and Planning

1. Introduction

In recent years, the concept of smart cities has gained significant attention in urban development, aiming to enhance the quality of life for city residents. The increasing urban population, which now accounts for over half of the global populace and consumes 75% of the world's energy [1], has driven the interest in creating efficient, productive, and sustainable urban environments using innovative technologies and data-driven strategies. These smart cities strive to address diverse objectives, acting as catalysts for urban progress by embracing physical, social, and digital dimensions, anticipating challenges, and fostering integrated services and innovation within urban institutions [2].
The foundation of the smart city concept is rooted in general systems theory, which envisions cities as intricate systems composed of various subsystems, each reflecting different facets of smart city attributes [3]. Smart cities are identified by their embodiment of characteristics such as sustainability, urbanization, quality of life, and intelligence. These traits collectively form the basis for assessing a city's level of intelligence and sustainability using smart city evaluation methodologies [4]. Integral to the advancement of smart cities is evaluating their potential, a task reliant on essential smart city evaluation models that play a pivotal role in achieving the aforementioned goals [5]. Successful smart city assessments provide guidance for decision-making while gauging the alignment of implementation with desired trajectories [6]. To achieve this, assessment models and data sources serve as indispensable tools.
Numerous smart city assessment models have emerged, encompassing paradigms like Sustainable Development Indicators [7,8], Lisbon ranking for smart sustainable cities [9], Smart City Performance [10], IESE Cities in Motion Index [11], ITU-T Y.4903/L.1603 Indicators [12], Sustainability Perspectives Indicators [13], Smart City Index Master Indicators Survey [14,8], Dimensions of the Smart City Vienna UT [15], Characteristics of Smart City Indicators [16], Criteria set for evaluating smart cities [17], China smart city performance [18], Juniper analysis of smart city frameworks [19], Assessing the Effectiveness of Smart Transport [20], Smart City Dimension [21], and City Sustainability Assessment [22]. These models draw from a range of resources, including primary and secondary data [23].
In the realm of smart city assessments, data collection methodologies primarily involve gathering primary data through means like questionnaires, interviews, surveys, on-site analyses, field observations, and photographic documentation. Complementing these efforts, secondary data sources encompass census data, city audits, annual reports, and historical records. However, both primary and secondary data have limitations. Collecting primary data can be resource-intensive and time-consuming, potentially leading to prolonged assessment processes. Conversely, secondary data may suffer from obsolescence and data quality issues [24]. Governmental statistical data forms a foundation for smart city assessments in current models. While it provides a broad overview of cities, it possesses limitations like its static nature and inability to capture real-time shifts in urban environments. Establishing unified evaluation standards across diverse cities also presents a significant challenge [25]. Traditional evaluation models based on statistical data struggle to monitor real-time developments within the dynamic landscape of smart cities [25].
A notable limitation in existing smart city assessment models is their incapacity to effectively understand and evaluate interactions among distinct subsystems within urban systems [23]. As the smart city landscape evolves, addressing this limitation becomes pivotal for refining the accuracy and efficacy of smart city assessments. In summary, the trajectory of smart cities in the past decade reflects efforts to leverage technology and data for urban improvement. With a focus on multifaceted attributes and holistic development, evaluating smart city potential through diverse models and data sources holds the key to steering urban centers towards a more intelligent and sustainable future.
Leveraging big data through strategic crowdsourcing presents a compelling solution for overcoming challenges in smart city assessments [23,26]. Crowdsourced data offers advantages in creating precise evaluation models, enhancing accuracy, and uncovering intricate urban dynamics. This approach seamlessly integrates into real-time assessments, reducing data collection costs while maintaining integrity [27,22]. Utilizing crowdsourced data, driven by the widespread use of information and communication technologies (ICT), provides an exceptional avenue for assessing smart cities. With a substantial increase in home internet adoption (64.6 percent of the global population) and approximately 5.18 billion internet users as of April 2023 (statista.com), a wealth of social media data is accessible from platforms like Facebook, Instagram, Twitter, and more.
Building on an understanding of the challenges, limitations, and untapped potential within crowdsourced data, this research focuses on developing an advanced smart city assessment paradigm. This model places emphasis on the strategic integration of crowdsourced data, addressing existing limitations and gaps in conventional smart city evaluations. The envisioned model represents an innovative approach, introducing more comprehensive and sustainable indicators to underpin smart city assessments. Concurrently, it establishes crucial connections between inter-indicator dynamics, a crucial step towards comprehensively evaluating the sustainability of urban landscapes.

2. Literature Review

2.1. Significance of Study

This study lies in its imperative to bridge a critical gap that exists within contemporary methodologies for assessing smart cities [5]. The unprecedented rate at which the global population is urbanizing has ushered in a complex era of challenges, encompassing swift urban expansion, heightened resource consumption, and escalating environmental ramifications. While conventional models for evaluating smart cities hold value, they grapple with encapsulating the intricate interplay among the diverse subsystems that constitute urban settings. This limitation impairs our capacity to accurately gauge the genuine intelligence, sustainability, and resilience exhibited by modern cities [25].
In the wake of advanced technologies and the proliferation of data, the prospect of a more comprehensive and dynamic assessment framework is palpably within reach. The integration of strategic crowdsourced data emerges as a promising panacea [27]. However, this potential remains largely untapped owing to the absence of an innovative evaluation model adept at adeptly harnessing and scrutinizing this profusion of information [23,24].
The urgency of addressing these constraints and formulating an advanced paradigm for smart city evaluation cannot be overstated. Primarily, this endeavor holds the potential to furnish city planners, policymakers, and invested parties with a profound comprehension of the intricate interdependencies that shape and sway distinct subsystems within urban landscapes. This cognizance forms the bedrock for making judicious decisions that propel sustainable growth, optimize resource utilization, and heighten the quality of life for urban denizens. Secondarily, this study is poised to expedite the identification of nascent trends, challenges, and prospects that unfold within smart cities. The ability to dissect real-time data sourced from the crowd, garnered through social media platforms, bestows the agility needed to react adeptly to urban evolutions, empowering cities to adapt and flourish within an ever-fluctuating milieu. Tertiary to its significance, by forging an all-encompassing and sustainable set of benchmarks for the assessment of smart cities, we can forge a holistic grasp of the long-term viability inherent within urban systems. This, in turn, can serve as the compass guiding strategic investments, policy shifts, and urban planning ventures that align harmoniously with the evolving prerequisites of both inhabitants and the ecosystem.
Finally, the crux of this research lies in its urgency to surmount the shortcomings pervading current smart city assessment paradigms while fully embracing the latent capabilities inherent in crowdsourced data and cutting-edge technology. By embarking on this trajectory, we have the capacity to pave the pathway for urban centers that epitomize heightened intelligence, adaptability, and sustainability, addressing the needs of current and forthcoming generations. The time to act is undeniably now, as the complexities confronting cities become ever more labyrinthine, and the solutions we proffer are destined to mold the contours of our urban horizons in the days to come.

2.2. Smart City Assessment

In accordance with [28], a smart city can be defined as a dynamic ecosystem comprising interconnected and automated components. Drawing from the principles of the General System Theory (GST), the urban landscape can be likened to a multi-tiered system, housing diverse subsystems or functions. These functions can be classified into two overarching domains: the tangible "hard" domain encompassing energy, water, waste management, transportation, environmental systems, buildings, and healthcare infrastructure, and the intangible "soft" domain comprising education, social welfare, community cohesion, public administration, employment opportunities, civic engagement, and economic activities. Hence, the crux of a smart city lies in the seamless interlinking of these subsystems within the city's framework, as emphasized by [29].
The assessment of a smart city hinges on the application of the General System Theory (GST) to the evolution of the smart city paradigm, as expounded by [30]. The GST provides a universal language to articulate and exchange challenges across diverse disciplines, rendering it adaptable to a multitude of contexts. As a result, the problem-solving approach within smart cities aligns with the GST system theory, where real-world complexities are tackled within the purview of the hard and soft subsystem realms [31]. Rooted in this approach, the evaluation of a smart city is gauged through its input-output (IO) model.
The smart city's IO model embodies four key facets: inputs (resources), processes (transformative mechanisms), outputs (practical applications), and outcomes (external impacts). Inputs encompass human capital, knowledge and innovation, information dissemination technology (ICT) infrastructure, and financial reservoirs. Processes encompass dynamic conduits, as well as the governance and leadership proficiencies that amplify the value of resources, converting them into tangible outputs. Outputs epitomize practical applications, while externalities mirror the results stemming from the smart city's IO processes, contributing to the city's holistic sustainability. Consequently, the evaluation of a smart city assumes pivotal importance, as it determines its sustainability encompassing environmental, economic, and societal dimensions.
Fundamentally, the core objective of assessing a smart city is to furnish insights and counsel for decision-making, facilitating a comprehensive appraisal of whether the implementation aligns with the envisioned trajectory [6]. The assessment of a smart city represents an emergent realm, teeming with potential for future advancements [23]. Moreover, it caters to performance monitoring, serving as a yardstick to evaluate diverse benefits for varied stakeholders, including city authorities, investors, funding institutions, scholars, and residents [5].
The evaluation of smart cities has ushered in a multitude of advantages for a diverse array of stakeholders, spanning city authorities, investors, funding institutions, researchers, and society at large. City authorities, first and foremost, glean significant value from the recognition of both strengths and vulnerabilities within their smart city development blueprints [32]. This insightful perspective empowers them to forge judicious decisions regarding the trajectory of growth and enhancement, while also bolstering the transparency quotient of the urban landscape [33]. Moreover, by honing their comprehension of the intricate technical requisites integral to smart cities, authorities are better poised to chart a course for the future with enhanced foresight [34].
Investors and funding institutions, equally integral players in the smart city paradigm, reap substantial rewards through an augmented capacity to strategically map out investment avenues within smart cities. By adeptly identifying and capitalizing on novel business prospects, they hold the potential to garner tangible dividends from the unfolding smart city landscape [35]. The utilization of empirical evidence in dictating resource allocations further equips them to make well-grounded determinations about the optimal allocation of their financial reservoirs [36]. This confluence of benefits serves as a catalyst for fostering increased investments and funding into the evolution of smart cities [37].
For researchers, the realm of smart city assessment engenders a fertile ground for cultivating innovative strategies aimed at enhancing performance and distilling the intricate complexities inherent to smart cities [36]. This intellectual pursuit yields fresh perspectives on the contours of smart city development, concurrently elevating the efficacy of their research endeavors [38]. As the smart city landscape burgeons, the concomitant demand for research that galvanizes heightened performance and addresses the multifaceted challenges enmeshed within their growth trajectory escalates proportionally.
The broader society stands to reap the rewards of elevated awareness vis-à-vis smart cities, an insight-rich vantage point that enables shrewd decisions regarding investment participation and active engagement in the evolution of smart city realms [35]. This informed populace, by articulating their aspirations to city authorities, becomes instrumental in shaping the contours of smart city progression to more harmoniously align with their needs [36]. The active involvement of society in the developmental arc of smart cities simultaneously serves as a catalyst for instilling transparency and accountability within the developmental process [34]. In summation, the dividends borne out of smart city development encompass a diverse tapestry of dimensions, spanning societal, economic, and governance spheres.

2.3. Crowdsourced Data

Crowdsourced data is an effort to contribute bottom-up information from society to a particular media, Crowdsourced data represents a grassroots effort to harness information from the public and funnel it into specific media channels, transcending mere geographic parameters [39]. This type of data holds immense potential in deciphering urban dynamics and the underlying patterns that drive them. It has proven instrumental in tackling intricate challenges and bridging critical gaps in data analysis that conventional urban methodologies struggle to address [40]. With its inherent high-frequency attributes, crowdsourced data offers real-time insights, portraying the present state of affairs accurately. Moreover, it presents a cost-effective alternative to traditional data acquisition methods like governmental surveys or censuses. Central to the significance of crowdsourced data is its voluntary generation by community members, who contribute a wealth of information tied to urban activities [41].
As highlighted by [41], crowdsourced data originates from three primary sources: (1) the realm of social media encompassing platforms like Facebook, Twitter, Instagram, and Sina Weibo; (2) Point of Interest (POI) data derived from sources such as OpenStreetMap, prominent business mapping services like Google Places and Gaode Maps, and check-in records from social networks like Foursquare or Yelp; and (3) diverse websites offering web services and open street maps. An alternate classification offered by [42] expands the scope to four categories: (1) familiar social media platforms like Flickr, Foursquare, Instagram, Tencent QQ, Twitter, and Weibo; (2) outdoor activity-sharing platforms like Condoon, Geocaching, GPSies, MapMyFitness, Strava, and Wikiloc; (3) community knowledge portals exemplified by eBird and iNaturalist; and (4) cellular signal data furnished by telecommunications companies.
It's evident that social media constitutes a prominent wellspring of crowdsourced data. Illustrated in Figure 1 is the process through which such data is procured from social media platforms. The agility of crowdsourced data's updates, stemming from the frequent nature of social media interactions, not only reflects the latest developments but also captures the prevailing conditions accurately. The economic advantage of crowdsourced data over conventional sources like surveys or official censuses is a compelling factor. Enriched by the voluntary contributions of community members, crowdsourced data is a trove of insights pertaining to urban life [41].
In summation, crowdsourced data stands as a potent tool for decoding urban dynamics and patterns, capable of untangling intricate problems and bridging gaps left by traditional urban data analysis methods. Leveraging social media as a data source, city authorities, investors, research institutions, and society at large stand to gain manifold benefits. These include enhanced city transparency, data-driven decision-making for funding allocations, and a channel for expressing civic aspirations to local authorities. The ample reserve of crowdsourced data derived from social media serves as a valuable resource that can inform new strategies and streamline the complexities of smart city development.

3. Methods

This study delves into the intricate design of dimensions and indicators, which forms the bedrock for the creation of an insightful smart city assessment model within the realm of sustainable urban landscapes. This model is meticulously crafted by channeling the perspectives of citizens through the lens of crowdsourced data. To navigate this endeavor, the study draws upon the invaluable framework of the Design Science Method, exemplified in Figure 2 [43]. This method, a fusion of conceptual prowess stemming from behavioral science and the design science paradigm, is harnessed to not just apprehend and implement, but also rigorously evaluate the model. It converges with key pillars within the research journey, prominently including the environment and the knowledge base [44].
In the sphere of design science, the environment assumes the role of a problem space, a dynamic arena pulsating with captivating phenomena. These phenomena intertwine harmoniously with three pivotal dimensions: people, organizations, and technology [43]. Within this expansive realm, one unearths an array of tasks, challenges, and opportunities that directly resonate with the needs of the business ecosystem. It is these very needs that crystallize as the compass directing the formation of the theoretical and artefactual prerequisites. In this context, these prerequisites seamlessly align with the dimensions and indicators pivotal for the crafting of a sustainable city assessment model. The research endeavor systematically deciphers tasks by discerning the contours of the application domain. This entails people entwined with the fabric of citizens, actively participating in smart city initiatives and offering feedback that enriches the inputs and outputs of the smart city sphere. Simultaneously, the organizational facet is intertwined with the pressing need to amplify urban competitiveness and elevate the tapestry of public services. The technological facet, playing a harmonious tune with urban evolution, thrives at the intersection of technological impetus and the nuanced needs of urban existence.
Interwoven amidst these tasks, challenges loom large as the proverbial hurdles to surmount. These challenges are strategically interwoven with the tapestry of business requirements. In this exploration, they unfold as the driving forces compelling (1) the evolution of a more efficient and effective smart city assessment process, (2) the integration of budget-friendly data streams via Information and Communication Technology (ICT) channels such as Big Data and crowdsourcing, and (3) the imperative to fathom and dissect the intricate interactions knitting together the subsystems within urban ecosystems. Concomitantly, opportunities emerge as latent catalysts, poised to empower the resolution of these challenges. In the present study, these opportunities manifest as the means to (1) streamline temporal and financial investments within the assessment process, (2) unveil real-time panoramas of the sustainable urban landscape, and (3) unearth the reservoir of crowdsourced data that holds the potential to reshape urban insights.
At the very foundation of this research journey lies the knowledge base, an essential reservoir of data manifesting in two distinct realms: the foundational and the methodological. The foundational stratum stands as a repository of rudimentary theories, frameworks, instruments, structures, models, and methodologies that have paved the path for scientific inquiry. On the other hand, the methodological sphere stands as a compass guiding the research trajectory through a structured set of guidelines and protocols [45]. In the context of this study, the knowledge base is characterized by its rich amalgamation of (1) innovative strategies for devising assessments, encapsulated within the Taxonomy framework, (2) pre-existing artefacts including taxonomies, frameworks, and models tailor-made for smart city assessments, (3) theories unraveling the complex dimensions of smart and sustainable urban paradigms, and (4) established methodologies for validating the constructs of taxonomies and models. In the realm of information systems research, the components embedded within the framework of the design science methodology serve as the pivotal drivers in tackling intricate challenges aligned with research objectives. These components are orchestrated through a duo of primary processes: the creative forging of artifacts and the subsequent validation or substantiation of findings. Within the context of this research endeavor, the exploration embarks by delineating dimensions and indicators, followed by expert validation, the formulation of keywords to harness crowdsourced data, and culminating in the conceptualization of a sustainable city assessment model, elegantly depicted in Figure 3.
The identification of dimensions and indicators is underpinned by a taxonomy-driven approach. Facilitating this endeavor is the strategic employment of the multi-criteria analysis (MCA) method—a decision-making instrument meticulously crafted to surmount complex multi-dimensional challenges, encompassing both qualitative and quantitative facets in the decision-making continuum [46]. The fruits of this taxonomy endeavor assume the mantle of a bedrock upon which criteria and indicators are judiciously chosen to weave the fabric of the sustainable city assessment model. The meticulous development of this taxonomy, orchestrated through the prism of MCA, is shaped by a triad of pivotal facets—principles, criteria, and indicators—skillfully illustrated in Figure 4. Principles, as the cornerstone, are interlaced with fundamental truths and laws, laying the foundation for the contextual underpinning of the assessment model. Within the scope of this study, the contextual bedrock finds resonance in the smart city assessment domain, situated within the broader expanse of the sustainable city landscape. Criteria, akin to guiding stars, delineate the benchmarks against which the assessment voyage unfolds. They concurrently serve as guiding principles for the direct measurement of performance and as conduits for assimilating insights from the array of indicators. The indicators, akin to the vital organs within an ecosystem, breathe life into each assessment criterion. Singular in purpose, they collectively manifest as an aggregate portrayal of elements traversing the assessment journey [46].
To forge this dimension (criteria) and indicator taxonomy, a wealth of data is drawn from diverse sources: frameworks, instruments, models, and methods, all rooted in the bedrock of prior smart city assessments (knowledge base), as eloquently displayed in Table 2. However, the selection of these criteria and indicators finds its anchor in the outputs of the smart city input-output (I/O) dynamics [47]. The process of handpicking these essential components, as depicted in Figure 5, entails a meticulous compilation of insights gleaned from frameworks, instruments, models, and methods germane to smart city assessment. A discerning evaluation then ensues, scrutinizing whether these components align with the dimensions and criteria associated with the social, economic, and environmental outcomes emanating from the intricate web of smart city input-output activities. Should they stand the test, they find their rightful place within the taxonomy of dimensions and indicators—a testament to their robust alignment with the intricate fabric of holistic assessment [48].
Following the development of taxonomy, the art of formulating keywords takes center stage, entailing the careful curation of terms to facilitate effective data aggregation. This undertaking assumes even greater importance when the model is employed to assess a city's sustainability. Particularly noteworthy is the meticulous process of selecting these pivotal keywords, which occurs subsequent to defining dimensions and indicators. These critical keywords are drawn from an existing knowledge base, serving as the bedrock for their construction. The process of determining or selecting keywords is guided by a consideration of their relevance and contextual appropriateness for each indicator generated through the taxonomy, as well as the initial indicators prior to the taxonomy's formation.
Upon the establishment of the taxonomy for dimensions, indicators, and keywords, the subsequent phase involves a meticulous refinement process, conducted in collaboration with domain experts, to authenticate and validate the taxonomy dan keywords. The purification endeavor is directed towards appraising the criteria and indicators, evaluating their appropriateness in gauging urban sustainability. Embarking on this journey, a meticulously designed questionnaire takes the forefront. This questionnaire finds its roots in the comprehensive taxonomy that has been meticulously crafted, encompassing a spectrum of criteria and indicators. The questionnaire itself adheres to a semi-structured format, encompassing a blend of both closed-ended and open-ended inquiries [49].
In the realm of closed statements, experts are presented with a gamut of options, facilitating their assessment of dimensions or indicators on a discerning 5-point Likert scale. This scale encompasses degrees of relevance, spanning from 1) not at all relevant, 2) slightly relevant, 3) moderatly relevant, 4) very relevant, and (5) extramely relevant. In contrast, the open statements section empowers experts to contribute suggestions and insights tied to the indicators, fostering a comprehensive discourse [50]. The methodology employed for expert selection is underpinned by purposive sampling, with a deliberate consideration for expertise and proficiency in areas like smart city dynamics, sustainable urban development, assessment methodologies, or disciplines closely aligned with the scope of this study [51]. Subsequently, the process of data aggregation unfolds by initially engaging experts via email communication and succinct phone messages. Upon their concurrence, the questionnaire is put forth for their input. A meticulous validation process guarantees that the experts constituting the validator cohort epitomize a diverse cross-section, thereby underwriting the robustness of the results. Notably, Table 1 delineates the panel of experts substantively involved as validators, contributing their insights towards the shaping of an assessment model for sustainable city, leveraging crowdsourced data.
The responses tendered by the experts are subjected to a comprehensive synthesis and analysis, employing statistical measures of central tendency and variability. These analytical findings, in turn, foster nuanced dialogues with experts, should the need arise. The distilled insights, encapsulating the collective wisdom of the experts, are subsequently subjected to a ranking process. This discerning process culminates in the curation of the final array of dimensions and indicators [49]. The threshold for determining this selection is anchored in a mean value equal to or exceeding 3.7. The strategic deployment of this cut-off value serves to not only acknowledge but effectively address the potential influence of limited variability, as communicated by the esteemed panel of experts [52].
The final phase encompasses the presentation of the generated model, the creation of an evaluative framework for the model, and a comprehensive analysis of its pertinence within the context of a sustainable city. Furthermore, it offers a panoramic view of the operational dynamics of the generated model when applied to the assessment of smart cities, with a specific emphasis on sustainability considerations. To construct the assessment model, the Smart City Reference Architecture Framework serves as a guiding beacon. This framework's directives encompass three pivotal dimensions: the foundational backdrop of the assessment, the aspired objectives, and the metrics – all meticulously illustrated in Figure 6 [53].
Within the contextual discussion, the magnitude of smart city assessment within cities that have pioneered smart urban initiatives is explored. This stems from the inherent need for substantial resources when embarking on smart city development within a municipality. As a result, measurements become indispensable tools to assess achievements holistically, encompassing diverse perspectives, including the citizenry's standpoint, as gleaned through crowdsourced data. In the pursuit of the “goals”, the assessment of a smart city takes root within the city's ecosystem, with sustainability as its ultimate zenith. These objectives find their articulation through dimensional and indicative representations, notably the Key Goal Indicators (KGI) and Key Performance Indicators (KPI). Concurrently, the “measure” encapsulates the dataset instrumental in the assessment. This study's principal focus rests on crowdsourced data, meticulously cataloged according to keywords tailored for each unique indicator. Subsequently, the fruits of this phase find their elaboration within the realm of the results and discussion section, where outcomes are dissected and explored in depth.

4. Results

Building upon the research procedure detailed in the methodology section, the forthcoming discourse will intricately explore the results achieved in each distinct research phase. This journey begins with the inception of the dimensions and indicators taxonomy and culminates with the unveiling of the model. Subsequent elucidations will meticulously elaborate on each of these integral components.

4.1. Taxonomy of Dimensions and Indicators

The process of establishing dimensions for the creation of a taxonomy is intricately intertwined with the principles of General System Theory (GST) and the dynamic input-output model inherent to smart cities. The utilization of GST serves as a comprehensive framework to fathom the intricate workings of urban ecosystems. This necessity is rooted in the contrasting viewpoints that distinguish traditional cities from their smart counterparts [30,31]. In essence, a traditional city is characterized as an expansive and enduring human habitation that intricately comprises multifaceted subsystems encompassing vital domains such as sanitation, utilities, land allocation, housing, and transportation. The deliberate concentration of developmental efforts significantly expedites synergistic interactions among individuals and enterprises, which often culminate in mutual advantages throughout the progression. However, this accelerated development also begets challenges in effectively orchestrating urban expansion [54]. Conversely, a smart city, grounded in the tenets of GST, embodies an intricate amalgamation of diverse systems, intricately interlinked and divisible into interdependent subsystems. Within the confines of a city, these systems are emblematic of energy and transportation, exemplified by energy grids and power generation systems within the energy realm, and vehicles and infrastructure components within the transportation domain [3].
The urban ecosystem thriving within the context of the smart city paradigm, as illuminated by the GST perspective, is inherently guided by the imperative to address urban demands (urban needs). This drive is fortified by the dynamic interplay of technology-driven advancements (technology push), intricately woven into the input-output framework of the smart city model [47]. This synergistic interplay gives rise to a pivotal process through which a smart city systematically cultivates sustainability, encompassing three fundamental dimensions: the societal, economic, and environmental realms. Thus, the dimensions that take center stage in crafting a comprehensive taxonomy, forming the bedrock for the development of this assessment model for urban sustainability, closely align with the outcomes set forth by the smart city model's input-output structure, spanning social, economic, and environmental considerations. These three dimensions collectively serve as the benchmark (triple bottom lines) for evaluating the sustainability of a city [9,22,55].
In the realm of indicator taxonomy development, meticulous adherence to established dimensions is a guiding principle. The selection of indicators is orchestrated through a procedure intricately outlined in Figure 5. This procedure mandates that the chosen indicators seamlessly align with the social, economic, and environmental dimensions. The wellspring for the development of these indicators springs from a diverse array of artifacts, encompassing taxonomies, frameworks, and models intrinsically linked to the domain of smart city assessments, as evidenced in reference [56]. These sources collectively amount to a comprehensive knowledge base, numbering thirty-one (31) entries. It is pivotal to note that these sources undergo a meticulous curation process, wherein the prerequisite of possessing at least one dimension—be it social, economic, or environmental—acts as the litmus test for inclusion. In the pursuit of precision, any taxonomies, frameworks, or models that fail to meet this pivotal criterion are judiciously excluded from the pool of potential sources for shaping indicators. Table 2 contains the essential material pertinent to this study.
Table 2. Artefact for Developing Indicators.
Table 2. Artefact for Developing Indicators.
No Ref Number of Indicators
Social Economy Environemt
1 Smart Sustainable City Indicators [7,8] 6 3 5
2 Sustainable Development Indicators [8] 11 3 6
3 Smart City Index Master [8,14] 0 3 3
4 Lisbon ranking for smart sustainable cities [9] 6 6 6
5 Smart city performance indeks [10] 3 4 4
6 IESE Cities in Motion Index 2018 [11] 13 8 11
7 ITU-T Y.4903/L.1603 [12,57] 6 7 6
8 Sustainability Perspectives Indicators [13] 11 5 13
9 Dimensions of the smart city Vienna UT [15] 0 6 4
10 Characteristics Smart City [16] 0 3 3
11 Criteria set for evaluating smart cities [17] 0 5 7
12 China smart city performance [18] 0 3 3
13 Sustainable development of communities [19] 0 5 7
14 Assess effectiveness of the smart transport [20] 0 0 2
15 Smart City Dimension [21] 0 4 7
16 City Sustainability Assessment [22] 12 7 5
17 Smart Sustainable Cities [58] 0 4 5
18 Global Power City Index 2018 [59] 0 5 3
19 ITU-T Y.4901/L.1601 [5,60,57] 6 7 6
20 ITU-T Y.4902/L.1602 [5,57,61] 6 7 6
Total of Indicators 80 95 112
The culmination of this selection procedure crystallizes into a succinct cohort: twenty (20) taxonomies, frameworks, or models that seamlessly fulfill the stipulated criteria. These exemplary entities, collectively serving as the bedrock of knowledge, lay the foundation for the construction of the indicator taxonomy. Within this assemblage, it becomes evident that nine (9) knowledge bases, while comprehensive in many aspects, lack indicators germane to the social dimension. However, in the context of economic and environmental dimensions, each knowledge base proffers a robust repertoire of indicators, ensuring a comprehensive analytical framework.
Subsequently, the endeavor transitions to the intricate art of crafting indicator taxonomies for each individual dimension. This artistry is manifested through a meticulous process of categorization, wherein indicators that share semantic affinities or contextual correlations are elegantly merged into unified entries. This astute stratagem is deployed to circumvent redundancy, particularly given the substantial magnitude of indicators—numbering eighty (80) for the social dimension, ninety-five (95) for the economic dimension, and one hundred and twelve (112) for the environmental dimension. The integration of indicators, guided by contextual information, serves as the foundation for the delineation of indicator themes. These thematic clusters play a pivotal role in the meticulous curation of indicators for each dimension. This strategic selection is geared towards procuring tangible insights when intertwined with crowdsourced data. Notably, the social dimension emerges as a focal point of diversity, encompassing a comprehensive spectrum of five distinct indicator themes: equity, health, education, security, and culture & equality, elegantly depicted in Table 3.
In the realm of societal analysis, the concept of equity unfurls a rich tapestry interwoven with a myriad of thematic threads. These interconnected elements, comprising equity itself, housing, social inclusion, and property prices, collectively offer a multidimensional perspective on the subject. At its core, equity stands as the bedrock of impartiality and fairness for getting live equipment. Housing, closely intertwined with equity, accentuates the quality and accessibility of housing options. The notion of social inclusion underscores the imperative of fostering a diverse and integrated society. Moreover, delving into the social dimension, Property Prices introduce a critical lens, considering how property values influence equitable resource distribution. Together, these elements form an intricate tapestry that encapsulates the essence of Equity and its intricate connections to various facets of society.
In the domain of well-being, the exploration of Health takes on a holistic approach, considering an array of factors that collectively shape the wellness of individuals and communities. The central pillar, Health, shines a spotlight on vitality and well-being. Complementing this core element, Health Status delves into specific health conditions and their quality experienced by individuals. Hospitals, as pivotal institutions, play a critical role in providing healthcare services. Mortality statistics provide insights into life expectancy and the broader effects of health on longevity. Nutritional regimes shed light on the profound impact of dietary patterns on health outcomes. Sanitation conditions contribute to disease prevention by ensuring a hygienic environment. Additionally, the availability of Drinking Water stands as a fundamental requirement for maintaining robust health. These interconnected elements together constitute the thematic framework of Health, offering a comprehensive understanding of the multifaceted factors influencing individual and communal well-being.
Within the realm of knowledge and growth, Education's thematic construct draws from a trio of fundamental elements: education itself, educational level, and literacy. These three components collectively form a comprehensive framework for comprehending the multifaceted nature of education and its profound effects on individuals and society. At its core, Education encapsulates the broader concept of learning and knowledge acquisition. Educational level explores the stages and degrees of education attained by individuals, reflecting their educational journey. Literacy, an indispensable facet, emphasizes the ability to read, write, and comprehend information, playing a pivotal role in personal development and societal progress. Collectively, these elements weave the thematic tapestry of Education, revealing its intricate connections to personal growth, intellectual enrichment, and the overall advancement of communities and nations.
Transitioning into the sphere of safety and stability, the thematic framework of Security emerges as a complex interplay of interconnected factors. This intricate network, comprising Security itself, Population Dynamics, Safety Measures, Crime Rates, Unemployment Rates, the Global Peace Index, the Global Slavery Index, Government Responses to Slavery, Terrorism, and Violence, collectively shapes our understanding of security across its multifaceted dimensions. At its core, Security embodies protection and well-being. Population considerations offer insights into the demographic context that influences security dynamics. Safety emphasizes safeguarding individuals and communities from harm. Crime rates and Unemployment reflect the interplay of socio-economic stability with overall security. Metrics such as the Global Peace Index and the Global Slavery Index offer nuanced perspectives on global stability and human rights. Government responses to slavery underscore the role of governance in addressing violations. Terrorism and Violence shed light on threats to security and societal cohesion. This intricate tapestry of interconnected elements forms the thematic foundation of Security, unveiling its ties to societal well-being, governance, human rights, and global stability.
Lastly, the synthesis of culture and equality emerges as a harmonious fusion of diverse elements, including Culture itself, Female Workers' Participation, the Happiness Index, and Gender Equality. This amalgamation offers a nuanced lens through which to explore the intricate interplay between cultural dynamics and the pursuit of equality. Culture, as a foundational element, encompasses the mosaic of traditions, values, and beliefs shaping societies. Female workers serve as a prism to examine gender roles, labor participation, and women's empowerment within cultural contexts. The Happiness Index illuminates well-being and contentment within cultural and societal settings. Gender equality, a pivotal pursuit, underscores fairness and opportunities for all genders. Together, these elements form the thematic synthesis of culture and equality, unveiling the intricate connections between cultural heritage, societal inclusivity, individual fulfillment, and the ongoing journey toward greater equality.
Within the realm of the economic dimension, a quartet of overarching indicator themes emerges: innovation, income, infrastructure, and business opportunity. Delving deeper, the innovation theme takes shape through a compilation of eleven meticulous indicators, while the income theme finds its foundation upon a comprehensive set of eight indicators. Similarly, the infrastructure theme derives its essence from a cluster of six pivotal indicators, while the business opportunity theme coalesces impressively through the synergy of twenty-two indicators, elegantly showcased in the tabulated data presented in Table 4.
The thematic foundation of Innovation is intricately woven from a diverse spectrum of elements, drawing upon Entrepreneurship and innovation as its core, and encompassing the Ability to transform, Innovation Industries, Innovative spirit, Innovative output, and Entrepreneurial enterprises. This intricate interplay creates a comprehensive framework for understanding the multifaceted dynamics of innovation and its far-reaching impacts. Entrepreneurship and innovation serve as the bedrock, emphasizing the proactive and creative endeavors that drive progress. The Ability to transform underscores the capacity to adapt and evolve in response to changing environments. Innovation Industries delve into the sectors that fuel technological and creative advancement. Innovative spirit embodies the mindset of curiosity, experimentation, and pushing boundaries. Innovative output showcases the tangible results of novel ideas and approaches. Entrepreneurial enterprises showcase the role of business ventures in driving innovation-driven economies. Collectively, these elements form the thematic essence of Innovation, revealing the intricate interconnections between human ingenuity, economic growth, and the continuous pursuit of advancement.
The thematic underpinning of Income is synthesized from a multifaceted array of elements, with roots in Availability of employment finding services, Employment, GDP estimate, GDP, GDP per capita, Labour Force Participation, Talent Pool, and Human Capital. This intricate blend offers a comprehensive lens through which to understand the complex dynamics that shape individual and societal economic prosperity. Availability of employment finding services lays the groundwork for job opportunities and workforce support. Employment serves as the core element, highlighting the participation in productive endeavors. GDP estimate and GDP provide macroeconomic insights into the overall economic activity and production. GDP per capita offers a measure of economic output per person, reflecting the standard of living. Labour Force Participation captures the engagement of the population in the workforce. Talent Pool showcases the skills and expertise available within a society. Human Capital emphasizes the knowledge and capabilities that contribute to economic growth. Collectively, these elements constitute the thematic essence of Income, unveiling the intricate connections between employment, economic output, individual potential, and the broader socio-economic landscape.
The thematic construct of Infrastructure is intricately woven from a diverse tapestry of elements, rooted in Local and global connection. This fabric extends to include ICT Infrastructure, encompassing the digital frameworks that underpin modern communication and technology. Physical infrastructure forms a vital thread, encompassing the tangible systems that support societies, such as transportation and utilities. Headquarters serve as nodal points of administrative and operational significance. The Use of information and communication technologies showcases the role of digital tools in enhancing efficiency and connectivity. Global interconnectedness further amplifies the theme, highlighting the interdependence and collaboration across borders. Collectively, these elements form the thematic essence of Infrastructure, offering a multi-dimensional perspective on the foundational systems that enable societies to function, connect, and thrive in an increasingly interconnected world.
The thematic foundation of Business Opportunity is intricately woven from a diverse array of elements, drawing from Productivity, Trade, Economic image and trademarks, Flexibility of labor market international embeddedness, Economic performance, Trading, Financial status, material consumption, Energy consumption, Economic Vitality and Planning, Online services made it easy to start a new business, E-commerce companies, Time required to start a business, Ease of starting a business, Motivation for early-stage entrepreneurial activity, Competitiveness, Socially responsible use of resources, Market size, Market Attractiveness, Business Environment, Ease of Doing Business, and Public sector. This intricate interplay offers a comprehensive framework for understanding the multifaceted dynamics that define business opportunities and their impact on economic growth. The amalgamation of these elements underscores the complex interrelationships between productivity, trade, technological innovation, market conditions, regulatory environments, and economic values. Collectively, they compose the thematic essence of Business Opportunity, revealing the intricate web of factors that shape entrepreneurial endeavors, economic prosperity, and the broader business landscape.
The subsequent thematic indicator is closely related to the environmental dimension, which is characterized by having the fewest number of thematic indicators. Specifically, this dimension is organized into three distinct themes: air, energy, and public facilities. The foundation for the creation of these themes lies in the indicators established within the environmental dimension, as elaborated earlier. The theme focusing on air quality is constructed upon a set of fifteen indicators, while the energy theme is derived from six key indicators. Similarly, the theme pertaining to public facilities draws from a compilation of thirteen indicators. The culmination of these thematic indicators is presented in illustrative Table 5.
The thematic composition of "Air" is intricately interwoven from a diverse spectrum of components, with its genesis rooted in factors such as PM2.5 and PM10 particulate matter, Pollution, Air quality, Availability and quality of applications for air pollution monitoring, Air pollution index, Volume of CO2 emissions, Pollution control, Air and water quality, Emission monitoring, Industrial wastewater, Emissions of industrial waste gases, Discharge of industrial solid waste, Release of hazardous waste, Natural Environment, and Noise. This intricate tapestry constitutes a holistic framework that enables a comprehensive comprehension of the multifaceted dynamics dictating air quality and environmental well-being. These interrelated elements encompass the physical constitution of the atmosphere, the caliber and repercussions of pollution, technological strides in monitoring practices, and the regulatory endeavors to manage emissions and waste discharge. In unison, these elements encapsulate the thematic essence of "Air," unveiling the intricate nexus of factors that mold air quality, environmental sustainability, and the overall vitality of ecosystems and communities.
The thematic core of "Energy" is intricately derived from a constellation of interrelated elements, including Renewable energy production, Energy consumption, Energy management, Energy, Energy Efficiency, and Recycling. This intricate web forms a comprehensive framework for understanding the intricate dynamics that govern the production, consumption, and sustainability of energy resources. Renewable energy production encapsulates the utilization of environmentally friendly sources like solar, wind, and hydroelectric power. Energy consumption reflects the societal demand and utilization of energy for various purposes. Energy management delves into strategic planning to optimize energy use. Energy is the foundational unit that powers countless aspects of modern life. Energy Efficiency underscores the judicious use of energy resources to minimize waste. Recycling highlights the circular economy concept that repurposes and reuses materials to conserve energy. Together, these elements constitute the thematic essence of Energy, revealing the complex interplay between resource availability, environmental impact, technological innovation, and the pursuit of sustainable energy practices.
The thematic underpinning of "Public Facilities" arises from an intricate interweaving of interconnected components, encompassing factors such as the allure of natural surroundings, the stewardship of sustainable resources, environmental safeguarding, the caliber of fundamental sanitation, intelligent construction and refurbishment, urban and resource planning, investments in urban amenities, per capita green space availability, the degree of waste reuse and recycling, enhancements in waste disposal practices, management of residences and facilities, vehicles for urban environmental initiatives, and the continuum of environmental quality and sustainability. This intricate interplay establishes a comprehensive framework that elucidates the infrastructure and strategies pivotal in enhancing the quality of communal spaces and the overall environment. From the preservation of natural aesthetics to the pursuit of sustainable resource stewardship, from sanitation standards to pioneering technological advancements, these elements collectively shape the thematic core of "Public Facilities," thereby revealing the intricate interconnections that bind resource allocation, waste management, urban development, and the holistic thriving of societies.
The progression from the expounded thematic indicators leads to the subsequent stage – the delineation or arrangement of a taxonomy that encompasses dimensions and indicators, meticulously guided by the Multi-Criteria Analysis (MCA) methodology. Primarily, this entails the establishment of the fundamental taxonomy. A focal point of the principle taxonomy is the notion of a sustainable city, with a pivotal objective to evaluate the feasibility of city sustainability. A sustainable city, in essence, embodies the fulfillment of urban requisites while safeguarding prospective interests [62]. Subsequently, the identification of criteria within the purview of this study, a task intrinsically linked to the dimensions that forge the framework of a sustainable city. These dimensions encompass a triad of facets: social, economic, and environmental. The social dimension is centered upon an exhaustive exploration of both affirmative and adverse repercussions stemming from the urban ecosystem, the processes, establishments, and undertakings of citizens in their communal existence [63]. On the other hand, the economic dimension delves into the revelation of impacts tied to the efficient and conscientious utilization of urban resources (64), evaluated through the lens of economic performance [63]. Meanwhile, the environmental dimension undertakes the excavation of effects emanating from the city's endeavors towards environmental sustainability within the complex tapestry of the urban ecosystem [64,65].
Moving ahead, the third facet involves the identification of indicators as dynamic variables or integral constituents for each dimension. Within the framework of this taxonomy construction, these indicators encompass the full spectrum of thematic indicators, spanning the realms of social, economic, and environmental dimensions. In consequence, the outcomes yielded by the taxonomy of dimensions and indicators metamorphose into the foundational bedrock for the evolution of a comprehensive model for evaluating the sustainability of a city, as eloquently depicted in Figure 7. The social dimension encapsulates equity, health, education, security, and culture, interwoven seamlessly with equality. Simultaneously, the economic dimension encompasses innovation, income, infrastructure, and prospects for business pursuit. Concomitantly, the environmental dimension encompasses the vital elements of air, energy, and public amenities.

4.2. Keywords of Crowdsourced Data

The formulation of keywords for crowdsourced data involves a strategic alignment with indicator themes representing each dimension. These meticulously chosen keywords serve the purpose of procuring pertinent crowdsourced data that aligns with the indicators' thematic essence, especially when applied in the context of assessments. This process of keyword selection is driven by a comprehensive assessment framework that encompasses multiple criteria, including synonymous meanings, terminologies intricately tied to the indicators, and the fundamental indicators that lay the foundation for these dimensions.
This structured approach ensures that the vast realm of information dispersed across social media platforms is not only efficiently harnessed but also endowed with a sense of accuracy and authenticity. As a consequence of this approach, the quantity of keywords exhibits variability across different indicators. Within the social dimension, for instance, the equity indicator is linked with five distinct keywords, while health, education, security, and culture & equality indicators correspond to five, five, ten, and four keywords respectively. In the realm of economic dimensions, the innovation indicator is tethered to six specific keywords, while income, infrastructure, and business opportunity indicators are associated with seven, three, and seven keywords respectively. Transitioning towards the environmental dimension, the air indicator is harmonized with five keywords, energy with four, and public facilities with six. The foundation for deriving these keywords lies in a systematic taxonomy-based approach, exemplified in the presentation of results in Table 6. This meticulous process ensures the alignment of keywords with indicator nuances, reinforcing the integrity of the gathered data and its ultimate application.

4.3. Experts Validation

The process of expert validation involves conducting a thorough evaluation of the appropriateness and significance of the taxonomy and keywords linked with crowdsourced data. These components serve as the fundamental bedrock on which the assessment model is meticulously crafted. The panel tasked with the role of validation is composed of eight individuals, precisely delineated in Table 1. Within the framework of expert assessment, the verification procedure is meticulously carried out, incorporating a blend of verbal discourse and succinct elucidations. This rigorous process is streamlined through the utilization of a validation form instrument. The intricate outcomes of expert validation, as illustrated in Figure 8, represent the culmination of this meticulous endeavor.
The outcomes of expert validation, as illustrated in Figure 8, uncover a spectrum of scores for all indicators and keywords, reflecting inconsequential variations. Notably, the "public facilities" indicator attains the highest mean score at 4.88, whereas the "income" indicator registers the lowest score of 4.13 on the one-to-five scale. This pattern harmonizes with the standard deviation data, underscoring that the "public facilities" indicator garners the most modest score, while the "income" indicator secures the highest rating. However, the results of expert validation underscore that all indicators and keywords surpass the established threshold of 3.7. In essence, this implies the pertinence of all indicators and keywords across each dimension, validating their inclusion in shaping and advancing the model for evaluating sustainable cities.

4.4. Sustainable City Assessment Model

The culmination of the design science methodology's iterative process lies in the formulation of a sustainable city assessment model. Building upon the previously expounded concepts, the architectural structure of this model adheres to the framework provided by the smart city reference architecture. Its constituents are distilled from a fusion of established theories, taxonomies, and rigorously validated keywords, all of which have been curated by domain experts. This model's developmental trajectory is meticulously compartmentalized into three distinctive components: issues, goals, and measures. The symbiotic interplay of these components assumes a pivotal role in the holistic integration of the model. The crux of the issues component resides in its direct relevance to the conundrums and imperatives necessitating resolution within the purview of the assessment model. Conversely, the goals component is intricately linked with the intricate tapestry of measurements encompassed within the assessment model. Concurrently, the measure component is inextricably entwined with the systematic processes employed to derive quantifiable values for each domain nested within the goals component.
Within the framework of the assessment model, the issues component's prime function is to elucidate the precise information objectives during the assessment's implementation. These objectives converge around three cardinal facets: validation, knowledge base, and understanding. The issue associated with validation encapsulates a depiction of whether the implementation of a smart city aligns harmoniously with the city's ecological dynamics. Simultaneously, the issue associated with the knowledge base furnishes a repository of insights catering to city planners, policymakers, and vested stakeholders. This knowledge base constitutes a foundational compass for informed decision-making in the context of cultivating a sustainable urban landscape. Conversely, the issue concerning understanding furnishes insights conducive to unraveling and appraising the intricate interplay among extant subsystems within city ecosystems. These tripartite issues collectively serve as the cornerstone upon which the architecture of a sustainable city is raised.
The pivotal role of the goal component materializes as a discerning yardstick in categorizing a city's sustainability quotient. This component's hierarchy comprises three strata: major, mid, and sub-goals. The major goal delineates a panoramic vista of a city's sustainability and encapsulates the essence of all mid-level goals. The mid-goals, in turn, offer a comprehensive portrayal of a city's sustainable dimensions, spanning social, economic, and environmental realms. Moreover, the mid-goals stand as the embodiment of sub-goals' values. As for sub-goals, they intricately illustrate the intricate tapestry of a city's ecosystem or signify the nuanced values aligned with each indicator, all rooted in crowdsourced data.
The measures facet orchestrates the calculation of assessments for every sub-goal or indicator nestled within the sustainable city assessment model, enshrining the multidimensional nature of the endeavor. This evaluation tableau enlists multiple strata: crowdsourced data, keywords, and weighting. Crowdsourced data serves as the lifeblood of assessment and its subsequent quantification. The tapestry of data sources spans diverse reservoirs, including the social media platform (LinkedIn, Twitter, Facebook, Instagram, YouTube, and Telegram), the troves of smart city platforms (mobile and web applications), and the realm of third-party platforms—harboring official governmental announcements and non-governmental organization (NGO) websites tethered to the tapestry of urban sustainable development.
Keywords unfurl as filters, winnowing the bounty of data from the fount of crowdsourced resources. Within the crucible of these keywords, data coalesces—data that paints a vivid tableau of citizen perspectives concerning the implementation of smart city paradigms within their urban environs. The annals of processed crowdsourced data undergo intricate weighting and scrutiny, a symphony conducted through diverse conduits: deep or machine learning, semantic analysis, sentiment weighting, and the uncharted waters of exploratory data analysis. The fruits of this weighty labor bestow values upon each sub-goal and assessment indicator. For an all-encompassing grasp of the panorama rendered by the sustainable city assessment model, all meticulously illustrated in Figure 9.

5. Discussion

This study has successfully crafted an assessment model tailored to sustainable urban development, with a specific focus on harnessing the potential of crowdsourced data. The model's evolution adheres to a systematic protocol, commencing with the establishment of a robust framework for designing taxonomy dimensions and indicators. This foundational phase is followed by the meticulous arrangement of these dimensions and indicators, the curation of pertinent keywords for each indicator, and a rigorous expert validation process to gauge the pertinence of both the taxonomy and keywords. Through these carefully chosen methodologies, the resultant model attains a high level of validity and credibility, rendering it a robust tool for city assessments.
Central to this investigation is the strategic utilization of crowdsourced data to delve into the intricacies of a smart city ecosystem, viewed through the lens of its citizens. The viewpoints and recommendations of these citizens hold profound significance in the evaluation of a city's sustainability. These perspectives are intricately intertwined with a spectrum of issues within the smart city ecosystem, spanning social, economic, environmental, and technological aspects. Additionally, they encompass matters of trust in shaping urban policies [66], and beyond. This stance resonates harmoniously with the input-output model that characterizes a smart city, where citizens, city authorities, and technology coalesce as indispensable constituents [67,66]. Against the backdrop of decision-making processes, the comprehensive understanding of a city's ecosystem and the validation of smart city initiatives emerge as pivotal dimensions, wherein citizens assume an active role. Their invaluable insights, often channeled through crowdsourced data [68], contribute substantively to these undertakings.
In light of this context, the acknowledgment of citizens' role within smart cities emerges as a pivotal force propelling the sustainability of these urban environments [69]. By actively engaging citizens, not only does their participation contribute to the development of smarter urban systems, but it also fosters a deeper sense of community ownership and responsibility. Furthermore, their involvement acts as a catalyst, nurturing a profound sense of ownership, as they actively contribute feedback during the execution of smart city initiatives [70,71]. This reciprocal exchange of information and input becomes a cornerstone in refining the city's functionality and enhancing its livability. This feedback finds its expression through the channeling of insights using a range of crowdsourced mediums [71], encompassing social media platforms, smart city platforms, and third-party platforms. As citizens' perspectives flow through these diverse channels, they form a mosaic of valuable data that not only aids in informed decision-making but also establishes a collaborative and inclusive foundation for the sustainable development of the city.
In the realm of comprehending the citizen's viewpoint through crowdsourced data, the meticulously crafted assessment model has presented an all-encompassing array of dimensions and indicators. Within the sphere of social dynamics, five pivotal indicators—namely equity, health, education, security, and culture & equality—converge to epitomize the urban ecosystem's social tapestry. In this narrative, equity assumes a poignant relevance, interwoven with citizens' perceptions regarding asset ownership, encompassing domains like housing and other tangible possessions [72]. The tenets of health intertwine intricately with the domain of healthcare [73], spanning a spectrum from hospitals, health centers, and nutritional provisions to sanitation and potable water. The educational facet pertains to the multifaceted educational landscape within the urban expanse [74], encapsulating aspects such as literacy, formal schooling, academic campuses, and institutions of higher learning. The fabric of security finds its essence in citizens' ecological security conditions [75], encompassing spheres like unemployment, subjugation, criminality, peace, aggression, terrorism, terrorists, and terror. Conversely, the indicators of culture & equality find their mooring in citizens' perspectives concerning the role of culture and gender parity in shaping their lives [76].
From an economic dimension perspective, there are four indicators: innovation, income, infrastructure, and business opportunities. These indicators serve as representations of the city's ecosystem, depicting its economic condition based on the citizens' perspective from crowdsourced data. Innovation pertains to the city's efforts to sustain itself through innovation. It encompasses areas such as entrepreneurship, company advancements, technological innovations, and industrial progress [77]. Income relates to how the city generates and expends its financial resources, including factors like GDP [78], salaries, employment rates, poverty levels, financial stability, talent availability, and human capital. Infrastructure concerns both physical and non-physical connectivity. This encompasses aspects of cooperation, connections, and the overall structural foundation [79]. Lastly, business opportunities are linked to the city's capacity to provide avenues for economic growth [80]. This includes economic performance, consumption patterns, market dynamics, trade potential, competitiveness, productivity, and overall business environment.
The dimension of the environment is delineated by three key indicators that facilitate the comprehension of citizen perspectives through the utilization of crowdsourced data. These indicators encompass air quality, energy consumption, and public infrastructure. In particular, the notion of air quality pertains to the purity of the air and its implications within the urban milieu [81]. This encompasses a spectrum of factors, including pollution, emissions, defilement, and waste management [82]. The indicators of energy is intricately interwoven with the judicious consumption of energy and the active pursuit of employing sustainable sources [83]. This encompasses a diverse array of components such as renewable energy, electricity usage, green industrial practices, and solar power initiatives [84]. Lastly, the purview of public facilities is entwined with the city's proactive endeavors in provisioning infrastructure that aligns with the city's environmental priorities [85]. This encompasses facets such as the provisioning of green spaces, parks, city parks, efficient vehicle systems, comprehensive public transportation networks, and the availability of environmental facilities and equipment [86,87].
The study findings reveal a distinct and robust quality embedded within the sustainable city assessment model's dimensions and indicators. These dimensions and indicators are strategically aligned with the assessment objectives, which revolve around harnessing crowdsourced data generously provided by citizens. This active participation from citizens serves as a pivotal component in materializing the smart city paradigm. What's even more remarkable is the meticulous customization of the resulting assessment model. It serves as a comprehensive yardstick for measuring the outcomes of a smart city, offering a holistic portrayal of the urban ecosystem across social, economic, and environmental facets. By adopting this approach, the assessment model triumphs over longstanding challenges that have hindered a profound understanding and evaluation of interactions among diverse subsystems within the urban setup.
This groundbreaking model adeptly tackles an enduring issue that has long troubled conventional evaluation techniques. These time-honored approaches, which rely on fragmented data sources, have historically grappled to yield effective resolutions. In stark contrast, the freshly devised model is constructed upon a bedrock of authenticated taxonomies and keywords, which have received the endorsement of field experts. This pivotal step significantly diminishes the potential for subjective biases stemming from individual understandings and experiences. However, it's essential to underscore that despite the strides made, the resultant model's foundation in validated taxonomies and keywords does not entirely eliminate the prospect of subjective influence. The collective expertise and unique experiences of the experts involved could still exert an impact on the model's outcomes.

6. Conclusions

This study aims to craft an inventive assessment model tailor-fit for sustainable cities, underpinned by the fusion of crowdsourced data. Beyond resolving intricate challenges inherent in smart city evaluations, the model's evolution has adeptly addressed intricate issues surrounding data acquisition and the interconnected dynamics of urban subsystems. Guided by the beacon of the design science methodology, the model's construction was rooted in a methodical and well-informed process. The resultant model emerges as an intricate tapestry artfully illustrating the diverse urban ecosystems, encompassing the intricate dimensions of social, economic, and the environment. Its synchronization with the validation process, upheld by domain experts, reaffirms the relevance of the chosen dimension and indicator structure, thus cementing its stature as a robust evaluative tool. The architectural blueprint of this assessment model, meticulously fashioned around three pivotal elements—issues, goals, and measures—derives its essence from the scaffold of the smart city reference architecture. This thoughtful architecture guarantees not only ease of comprehension but also practical utility within the vast expanse of urban evaluations.
The contribution of this research to the discourse encircling smart cities is undeniably profound, particularly in its spotlight on the assessment of urban sustainability. As a natural progression, a call resonates for a pertinent case study, poised to actualize research outcomes within a tangible milieu, effectively demonstrating the model's real-world efficacy. Moreover, the trajectory of exploration should traverse beyond the confines of sustainable cities, embarking upon the exhilarating journey that harmonizes the realms of smartness and sustainability, culminating in a comprehensive paradigm for smart city assessments.

Author Contributions

Conceptualization, UE and AFR; methodology, UE and AW; validation, AFR. and AW; formal analysis, UE; investigation, UE and AW; resources, UE; data curation, UE; writing original draft preparation, UE; writing review and editing, UE, AFR, and AW; visualization, UE; supervision, AFR. and AW; project administration, UE.

Funding

Please add: This research received no external funding.

Acknowledgments

We would like to express our sincere gratitude to Universitas Diponegoro adn Universitas Bina Darma for generously providing the necessary facilities and resources that have greatly contributed to the successful completion of this study.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. K. Kourtzanidis, K. Angelakoglou, V. Apostolopoulos, P. Giourka, and N. Nikolopoulos, “Assessing impact, performance and sustainability potential of smart city projects: Towards a case agnostic evaluation framework,” Sustainability, vol. 13, no. 13, 2021. [CrossRef]
  2. D. Y. Bayar, H. Guven, H. Badem, and E. Soylu Sengor, “National Smart Cities Strategy and Action Plan: the Turkey’s Smart Cities Approach,” Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci., vol. XLIV-4/W3-, no. October, pp. 129–135, 2020. [CrossRef]
  3. M. Lom, “Smart city model based on systems theory,” Int. J. Inf. Manage., vol. 56, no. February 2019, p. 102092, 2021. [CrossRef]
  4. Khan, S. Aslam, K. Aurangzeb, M. Alhussein, and N. Javaid, “Multiscale modeling in smart cities: A survey on applications, current trends, and challenges,” Sustain. Cities Soc., vol. 78, no. November 2021, p. 103517, 2022. [CrossRef]
  5. C. Patrão, P. Moura, and A. T. de Almeida, “Review of Smart City Assessment Tools,” Smart Cities, vol. 3, no. 4, pp. 1117–1132, 2020. [CrossRef]
  6. H. Ahvenniemi, A. Huovila, I. Pinto-Seppä, and M. Airaksinen, “What are the differences between sustainable and smart cities?,” Cities, vol. 60, pp. 234–245, 2017. [CrossRef]
  7. J. L. Hass, F. Brunvoll, and H. Hoie, “Overview of Sustainable Development Indicators used by National and International Agencies,” OECD Stat. Work. Pap., 2002. [CrossRef]
  8. M. Pira, “A novel taxonomy of smart sustainable city indicators,” Humanit. Soc. Sci. Commun., vol. 8, no. 1, pp. 1–10, 2021. [CrossRef]
  9. A. Akande, P. Cabral, P. Gomes, and S. Casteleyn, “The Lisbon ranking for smart sustainable cities in Europe,” Sustain. Cities Soc., vol. 44, no. August, pp. 475–487, 2019. [CrossRef]
  10. T. Yigitcanlar, K. Degirmenci, L. Butler, and K. C. Desouza, “What are the key factors affecting smart city transformation readiness ? Evidence from Australian cities,” Cities, vol. 120, no. August, p. 103434, 2022. [CrossRef]
  11. P. Berrone and J. E. Ricart, IESE Cities in Motion Index 2018, 2018th ed. Barcelona: IESE Business School’s, 2018. [CrossRef]
  12. ITU-T, “Key performance indicators for smart sustainable cities to assess the achievement of sustainable development goals,” 2022. [Online]. Available: https://www.itu.int/rec/T-REC-Y.4903.
  13. A. J. Benites and A. F. Simões, “Assessing the urban sustainable development strategy: An application of a smart city services sustainability taxonomy,” Ecol. Indic., vol. 127, p. 107734, 2021. [CrossRef]
  14. B. Cohen, “Smart city index master indicators survey. Smart cities council.” 2022. [Online]. Available: http://smartcitiescouncil.com/resources/smart-city-index-master-indicators-survey.
  15. G. Koca, O. Egilmez, and O. Akcakaya, “Evaluation of the smart city: Applying the dematel technique,” Telemat. Informatics, vol. 62, no. June, p. 101625, 2021. [CrossRef]
  16. F. Purnomo, Meyliana, and H. Prabowo, “Smart city indicators: A systematic literature review,” J. Telecommun. Electron. Comput. Eng., vol. 8, no. 3, pp. 161–164, 2016.
  17. R. M. Kimiya and S. A. Torabi, “Ranking cities based on their smartness level using MADM methods,” Sustain. Cities Soc., vol. 72, no. May, p. 103030, 2021. [CrossRef]
  18. L. Shen, Z. Huang, S. Wai, S. Liao, and Y. Lou, “A holistic evaluation of smart city performance in the context of China,” J. Clean. Prod., vol. 200, no. November, pp. 667–679, 2018. [CrossRef]
  19. V. Fernandez-anez, G. Velazquez, and F. Perez-prada, “Smart City Projects Assessment Matrix: Connecting Challenges and Actions in the Mediterranean Region,” J. Urban Technol., vol. 27, no. 4, pp. 1–25, 2018. [CrossRef]
  20. S. Gutman and P. Vorontsova, “Issues of Development of Smart Transport Assessment Indicators,” 2020. [CrossRef]
  21. R. Al Sharif and S. Pokharel, “Smart City Dimensions and Associated Risks: Review of literature,” Sustain. Cities Soc., vol. 77, no. February, 2022. [CrossRef]
  22. Y. Zhou, P. Yi, W. Li, and C. Gong, “Assessment of city sustainability from the perspective of multi-source data-driven,” Sustain. Cities Soc., vol. 70, no. January, p. 102918, 2021. [CrossRef]
  23. A. Sharifi, “A critical review of selected smart city assessment tools and indicator sets,” J. Clean. Prod., vol. 233, no. 1 October 2019, pp. 1269–1283, 2019. [CrossRef]
  24. A. Sharifi, “A typology of smart city assessment tools and indicator sets,” Sustain. Cities Soc., vol. 53, no. May, pp. 1–15, 2020. [CrossRef]
  25. A. Sharifi, “A global dataset on tools, frameworks, and indicator sets for smart city assessment,” Data Br., vol. 29, 2020. [CrossRef]
  26. A. Dawodu, A. Cheshmehzangi, A. Sharifi, and J. Oladejo, “Neighborhood sustainability assessment tools : Research trends and forecast for the built environment,” Sustain. Futur., vol. 4, no. January, p. 100064, 2022. [CrossRef]
  27. Z. Wu, X. Li, X. Zhou, and T. Yang, “City Intelligence Quotient Evaluation System Using Crowdsourced Social Media Data : A Case Study of the Yangtze River Delta Region , China,” Int. J. Geo-Information, vol. 10, no. 702, 2021. [CrossRef]
  28. P. Neirotti, A. De Marco, A. C. Cagliano, G. Mangano, and F. Scorrano, “Current trends in smart city initiatives: Some stylised facts,” Cities, vol. 38, pp. 25–36, 2014. [CrossRef]
  29. C. Rochet and F. P. de Villechenon, “System Architecture As A Means To Build An Extended Administration : The Case of Smart Cities,” Puyricard, France, 2014.
  30. P. B. Checkland and M. G. Haynes, “Varieties of systems thinking: The case of soft systems methodology,” Manag. Control Theory, vol. 3, no. January, pp. 151–159, 1994.
  31. N. Noori, T. Hoppe, and M. de Jong, “Classifying pathways for smart city development: Comparing design, governance and implementation in Amsterdam, Barcelona, Dubai, and Abu Dhabi,” Sustain., vol. 12, no. 10, 2020. [CrossRef]
  32. C. Garau and V. M. Pavan, “Evaluating urban quality: Indicators and assessment tools for smart sustainable cities,” Sustainability, vol. 10, no. 3, p. 575, 2018.
  33. V. Fernandez-Anez, J. M. Fernández-Güell, and R. Giffinger, “Smart City implementation and discourses: An integrated conceptual model. The case of Vienna,” Cities, vol. 78, pp. 4–16, 2018.
  34. A. K. Debnath, H. C. Chin, M. M. Haque, and B. Yuen, “A methodological framework for benchmarking smart transport cities,” Cities, vol. 37, pp. 47–56, 2014.
  35. A. Mohan, G. Dubey, F. Ahmed, and A. Sidhu, “Smart Cities Index: A Tool for Evaluating Cities,” Indian School of Business: Hyderabad. Indian School of Business-Hyderabad, 2020.
  36. A. S. Caird, L. Hudson, and G. Kortuem, “A tale of evaluation and reporting in UK smart cities,” Open Univ. Milt. Keynes, 2016.
  37. R. Giffinger, G. Haindlmaier, and H. Kramar, “The role of rankings in growing city competition,” Urban Res. Pract., vol. 3, no. 3, pp. 299–312, 2010.
  38. A. Akande, P. Cabral, and S. Casteleyn, “Assessing the gap between technology and the environmental sustainability of European cities,” Inf. Syst. Front., vol. 21, no. 3, pp. 581–604, 2019. [CrossRef]
  39. A. Crooks et al., “Crowdsourcing urban form and function,” Int. J. Geogr. Inf. Sci., vol. 29, no. 5, pp. 720–741, 2015. 0.1080/13658816.2014.977905.
  40. Y. Long and L. Liu, “Transformations of urban studies and planning in the big/open data era: a review,” Int. J. Image Data Fusion, vol. 7, no. 4, pp. 295–308, 2016. [CrossRef]
  41. H. Niu and E. A. Silva, “Crowdsourced Data Mining for Urban Activity: Review of Data Sources, Applications, and Methods,” J. Urban Plan. Dev., vol. 146, no. 2, p. 04020007, 2020. [CrossRef]
  42. I. Havinga, P. W. Bogaart, L. Hein, and D. Tuia, “Defining and spatially modelling cultural ecosystem services using crowdsourced data,” Ecosyst. Serv., vol. 43, no. March, p. 101091, 2020. [CrossRef]
  43. J. vom Brocke, A. Hevner, and A. Maedche, “Introduction to Design Science Research,” Cases, no. September, pp. 1–13, 2020. [CrossRef]
  44. S. T. March and V. C. Storey, “Design science in the information systems discipline: An introduction to the special issue on design science research,” MIS Q. Manag. Inf. Syst., vol. 32, no. 4, pp. 725–730, 2008. [CrossRef]
  45. A. Hevner and S. Chatterjee, Design Research in Information Systems. London: Springer, 2010.
  46. N. Ahmad, D. Berg, and G. R. Simons, “The integration of analytical hierarchy process and data envelopment analysis in a multi-criteria decision-making problem,” Int. J. Inf. Technol. Decis. Mak., vol. 5, no. 2, pp. 263–276, 2006. [CrossRef]
  47. . N. Noori, M. de Jong, M. Janssen, D. Schraven, and T. Hoppe, “Input-Output Modeling for Smart City Development,” J. Urban Technol., vol. 28, no. 1–2, pp. 71–92, 2020. [CrossRef]
  48. . G. A. Mendoza, P. Macoun, R. Prabhu, D. Sukadri, H. Purnomo, and H. Hartanto, Guidelines for Applying Multi-Criteria Analysis to the Assessment of Criteria and Indicators. Center for International Forestry Research (CIFOR), 1999. [CrossRef]
  49. A. Saihi, M. Ben-Daya, and R. As’ad, “An Investigation of Sustainable Maintenance Performance Indicators: Identification, Expert Validation and Portfolio of Future Research,” IEEE Access, vol. 10, no. November, pp. 124259–124276, 2022. [CrossRef]
  50. . Day and M. Bobeva, “A generic toolkit for the successful management of delphi studies,” Electron. J. Bus. Res. Methods, vol. 3, no. 2, pp. 103–116, 2005.
  51. R. Helmy et al., “ESPACOMP Medication Adherence Reporting Guidelines (EMERGE): A reactive-Delphi study protocol,” BMJ Open, vol. 7, no. 2, pp. 1–7, 2017. [CrossRef]
  52. N. Nordin, B. M. Deros, D. A. Wahab, and M. N. A. Rahman, “Validation of lean manufacturing implementation framework using delphi technique,” J. Teknol. (Sciences Eng., vol. 59, no. SUPPL.2, pp. 1–6, 2012.
  53. Government Japan, Smart City Reference Architecture White Paper Cross-ministerial Strategic Innovation Promotion Program, vol. March. Government Japan, 2020.
  54. P. James, M. Holden, M. Lewin, and L. Neilson, “Managing metropolises by negotiating urban growth,” in Institutional and social innovation for sustainable urban development, Routledge, 2013, pp. 233–248.
  55. J. Liu et al., “Analysis of sustainability of Chinese cities based on network big data of city rankings,” Ecol. Indic., vol. 133, no. October, p. 108374, 2021. [CrossRef]
  56. U. Ependi, A. F. Rochim, and A. Wibowo, “Smart City Assessment for Sustainable City Development on Smart Governance: A Systematic Literature Review,” 2022 Int. Conf. Decis. Aid Sci. Appl. DASA 2022, pp. 1088–1097, 2022. [CrossRef]
  57. A. Huovila, P. Bosch, and M. Airaksinen, “Comparative analysis of standardized indicators for Smart sustainable cities: What indicators and standards to use and when?,” Cities, vol. 89, no. January, pp. 141–153, 2019. [CrossRef]
  58. X. Li, P. S. W. Fong, S. Dai, and Y. Li, “Towards sustainable smart cities: An empirical comparative assessment and development pattern optimization in China,” J. Clean. Prod., vol. 215, pp. 730–743, 2019. [CrossRef]
  59. H. Takenaka and H. Ichikawa, “Global Power City Index 2018,” London, New York, Toko, Paris, Singapore, 2018. [Online]. Available: https://mori-m-foundation.or.jp/pdf/GPCI2018_summary.pdf.
  60. ITU-T, “Key Performance Indicators Related to The Use of Information and Communication Technology In Smart Sustainable Cities,” 2016.
  61. ITU-T, “Key performance indicators related to the sustainability impacts of information and communication technology in smart sustainable cities,” 2016.
  62. C.-L. Goi, “The impact of technological innovation on building a sustainable city,” Int. J. Qual. Innov., vol. 3, no. 1, 2017. [CrossRef]
  63. Ş. Y. Balaman, “Sustainability Issues in Biomass-Based Production Chains,” Decis. Biomass-Based Prod. Chain., pp. 77–112, Jan. 2019. [CrossRef]
  64. G. Del Río Castro, M. C. González Fernández, and Á. Uruburu Colsa, “Unleashing the convergence amid digitalization and sustainability towards pursuing the Sustainable Development Goals (SDGs): A holistic review,” J. Clean. Prod., vol. 280, 2021. [CrossRef]
  65. K. P. Bithas and M. Christofakis, “Environmentally sustainable cities. Critical review and operational conditions,” Sustain. Dev., vol. 14, no. 3, pp. 177–189, 2006. [CrossRef]
  66. Z. Tomor, A. Meijer, A. Michels, and S. Geertman, “Smart Governance For Sustainable Cities: Findings from a Systematic Literature Review,” J. Urban Technol., vol. 26, no. 4, pp. 3–27, 2019. [CrossRef]
  67. W. Castelnovo, G. Misuraca, and A. Savoldelli, “Smart Cities Governance: The Need for a Holistic Approach to Assessing Urban Participatory Policy Making,” Soc. Sci. Comput. Rev., vol. 34, no. 6, pp. 724–739, 2016. [CrossRef]
  68. W. R. Webster and C. Leleux, “Smart governance: Opportunities for technologically-mediated citizen co-production,” Inf. Polity, vol. 23, no. 1, pp. 95–110, 2018. [CrossRef]
  69. S. Y. Tan and A. Taeihagh, “Smart city governance in developing countries: A systematic literature review,” sustainability, vol. 12, no. 3, p. 899, 2020.
  70. G. Viale Pereira, M. A. Cunha, T. J. Lampoltshammer, P. Parycek, and M. G. Testa, “Increasing collaboration and participation in smart city governance: a cross-case analysis of smart city initiatives,” Inf. Technol. Dev., vol. 23, no. 3, pp. 526–553, 2017. [CrossRef]
  71. A. Joia and A. Kuhl, “Smart city for development: A conceptual model for developing countries,” in International conference on social implications of computers in developing countries, 2019, pp. 203–214.
  72. M.J. Nikki Han and M. J. Kim, “A critical review of the smart city in relation to citizen adoption towards sustainable smart living,” Habitat Int., vol. 108, no. January, p. 102312, 2021. [CrossRef]
  73. S. Chang and M. K. Smith, “Residents’ Quality of Life in Smart Cities: A Systematic Literature Review,” Land, vol. 12, no. 4, 2023. [CrossRef]
  74. Indrawati, W. Widodo, and H. Amani, “Indicators to measure a smart education: An Indonesian perspective,” Proc. Int. Conf. Ind. Eng. Oper. Manag., vol. 2018-March, pp. 3486–3495, 2018.
  75. B. D. Tampubolon, A. B. Mulyono, F. Isharyadi, E. H. Purwanto, W. C. Anggundari, and Gensly, “Indicator Analysis Of Smart City Standard Sni ISO 37122 Plays A Role In The Covid-19 Pandemic,” Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. - ISPRS Arch., vol. 46, no. 4/W5-2021, pp. 523–528, 2021. [CrossRef]
  76. D. Šulyová and J. Vodák, “The impact of cultural aspects on building the smart city approach: Managing diversity in Europe (London), North America (New York) and Asia (Singapore),” Sustain., vol. 12, no. 22, pp. 1–11, 2020. 0.3390/su12229463.
  77. M. Duygan, M. Fischer, R. Pärli, and K. Ingold, “Where do Smart Cities grow? The spatial and socio-economic configurations of smart city development,” Sustain. Cities Soc., vol. 77, no. April 2021, 2022. 0.1016/j.scs.2021.103578.
  78. Z. Chen and J. Cheng, “Economic Impact of Smart City Investment: Evidence from the Smart Columbus Projects,” J. Plan. Educ. Res., vol. 43, no. 4, p. 0739456X221129173, 2022. [CrossRef]
  79. A. Vishnivetskaya and E. Alexandrova, “‘Smart city’ concept. Implementation practice,” IOP Conf. Ser. Mater. Sci. Eng., vol. 497, no. 1, pp. 1–6, 2019. [CrossRef]
  80. N. Komninos, C. Kakderi, A. Panori, and P. Tsarchopoulos, “Smart City Planning from an Evolutionary Perspective,” J. Urban Technol., vol. 26, no. 2, pp. 3–20, 2019. [CrossRef]
  81. A. Kaginalkar, S. Kumar, P. Gargava, and D. Niyogi, “Stakeholder analysis for designing an urban air quality data governance ecosystem in smart cities,” Urban Clim., vol. 48, no. January, p. 101403, 2023. [CrossRef]
  82. 82. M. Y. Salman and H. Hasar, “Review on environmental aspects in smart city concept: Water, waste, air pollution and transportation smart applications using IoT techniques,” Sustain. Cities Soc., vol. 94, no. December 2022, p. 104567, 2023. [CrossRef]
  83. M. G. M. Almihat, M. T. E. Kahn, K. Aboalez, and A. M. Almaktoof, “Energy and Sustainable Development in Smart Cities: An Overview,” Smart Cities, vol. 5, no. 4, pp. 1389–1408, 2022. [CrossRef]
  84. A. Razmjoo, A. H. Gandomi, M. Pazhoohesh, S. Mirjalili, and M. Rezaei, “The key role of clean energy and technology in smart cities development,” Energy Strateg. Rev., vol. 44, no. August, p. 100943, 2022. [CrossRef]
  85. Z. Wang, “Research on Smart City Environment Design and Planning Based on Internet of Things,” J. Sensors, vol. 2022, 2022. [CrossRef]
  86. T. O. Alshammari, “Smart City Public Transport Remodel Urban Biodiversity Management,” IOP Conf. Ser. Earth Environ. Sci., vol. 1026, no. 1, 2022. [CrossRef]
  87. M. Burlacu, R. G. Boboc, and E. V. Butilă, “Smart Cities and Transportation: Reviewing the Scientific Character of the Theories,” Sustainability, vol. 14, no. 13, 2022. 0.3390/su14138109.
Preprints 84131 g001
Figure 1. Crowdsourced data generation and information extraction.
Figure 1. Crowdsourced data generation and information extraction.
Preprints 84131 g001
Preprints 84131 g002
Figure 2. Design Science Methodology for Developing Assessment Model Adopted from Alan Hevner [43].
Figure 2. Design Science Methodology for Developing Assessment Model Adopted from Alan Hevner [43].
Preprints 84131 g002
Preprints 84131 g003
Figure 3. Research Process for Building Assessment Model for Sustainable City using Crowdsourced Data.
Figure 3. Research Process for Building Assessment Model for Sustainable City using Crowdsourced Data.
Preprints 84131 g003
Preprints 84131 g004
Figure 4. Multi Criteria Analaysis Technique for Building Taxonomy of Dimension and Indicator.
Figure 4. Multi Criteria Analaysis Technique for Building Taxonomy of Dimension and Indicator.
Preprints 84131 g004
Preprints 84131 g005
Figure 5. Procedure to Set Taxonomy of Dimension and Indicator.
Figure 5. Procedure to Set Taxonomy of Dimension and Indicator.
Preprints 84131 g005
Preprints 84131 g006
Figure 6. Smart City Reference Architecture Framework for Developing Assessment Model.
Figure 6. Smart City Reference Architecture Framework for Developing Assessment Model.
Preprints 84131 g006
Preprints 84131 g007
Figure 7. Taxonomy of Dimensions and Indicators.
Figure 7. Taxonomy of Dimensions and Indicators.
Preprints 84131 g007
Preprints 84131 g008
Figure 8. Relevance Ratings of Dimensions, Indicators and Their Corresponding Keywords.
Figure 8. Relevance Ratings of Dimensions, Indicators and Their Corresponding Keywords.
Preprints 84131 g008
Preprints 84131 g009
Figure 9. The Proposed Sustainable City Assessment Model Using Crowdsourced Data.
Figure 9. The Proposed Sustainable City Assessment Model Using Crowdsourced Data.
Preprints 84131 g009
Table 1. Expert profile.
Table 1. Expert profile.
No Position Country Expertise Experince
1 Associate Professor Indonesia Green it, e-government, smart cities, e-learning, and IT public services 10-15 years
2 Professor Indonesia Computer vision, information systems,
human factors, and smart cities.		 >20 years
3 Associate Professor Indonesia open government data, smart cities, network secuity, and digital forensics investigation. 15-20 years
4 Associate Professor Indonesia open data government, smart cities, data mining, information system, and technology adoption. 15-20 years
5 Associate Professor Malaysia user experience, human computer interaction, sustainability, and gerontechnology. 5-10 years
6 Associate Professor Malaysia information system, project management,
and sustainable governance.		 10-15 years
7 Professor Indonesia smart system platform and ecosystem, IT architecture and governance, and smart cities. >20 years
8 Professor Malaysia IT governance, urban development,
social media, data analytics, and fintech.		 >20 years
Table 3. Indicators Thema for Social Dimension.
Table 3. Indicators Thema for Social Dimension.
Indicators Thema
Assets equity [7,8].
Housing [7,8,9,12,5,60,57,61,13,10]
Social inclusion [9,12,5,60,57,61]
Price of property [11]		 Equity
Health [7,8,9,11,12,5,60,57,61,13]
Health Status [10]
Hospitals [11]
Mortality [11,13]
Nutritional regime [13]
Sanitation conditions [13]
Drinking water [13]		 Health
Education [7,8,9,12,5,60,57,61]
Educational level [13]
Literacy [13]		 Education
Security [7,8,10]
Population [7,8,13]
Safety [9,12,5,60,57,61,10]
Crime rate [11]
Unemployment [11]
Global Peace Indeks [11]
Global Slavery Indeks [11]
Government response to situations of slavery [11]
Terrorism [11]
Violence [13]		 Security
Culture [9,12,5,60,57,61]
Female workers [11]
Happiness indeks [11]
Gender equality [13]		 Culture and Equality
Table 4. Indicators Thema for Economy Dimension.
Table 4. Indicators Thema for Economy Dimension.
Indicators Thema
Entrepreneurship and innovation [7,8,14,15,16,17,9,18,12,57,5,60,61]
Ability to transform [15]
Innovation Industries [10]
Innovative spirit [15,19,21]
Innovative output [58]
Entrepreneurial enterprises [58]		 Innovation
Availability of employment finding services [17]
Employment [17,9,18,12,57,5,60,61]
GDP estimate [11]
GDP [11]
GDP per capita [11,18]
Labour Force Participation [10]
Talent Pool [10]
Human Capital [59]		 Income
Local and global connection [7,8,14,19]
ICT Infrastructure [9,12,57,5,60,61]
Physical infrastructure [9,12,57,5,60,61]
Headquarters [11]
Use of information and communication technologies [21]
Global interconnectedness [58]		 Infrastructure
Productivity [7,8,14,15,16,9,11,19,10,58,12,57,5,60,61]
Trade [9,12,57,5,60,61]
Economic image and trademarks [15]
Flexibility of labor market international embeddedness [15,19]
Economic performance [13]
Trading [13,9]
Financial status [13]
material consumption [13]
Energy consumption [13]
Economic Vitality and Planning [16]
Online services made it easy to start a new business [17]
E-commerce companies [17]
Time required to start a business [11]
Ease of starting a business [11]
Motivation for early-stage entrepreneurial activity[11]
Competitiveness [21]
Socially responsible use of resources [21]
Market size [59]
Market Attractiveness [59]
Business Environment [59]
Ease of Doing Business [59]
Public sector [12,57,5,60,61]		 Business Opportunity
Table 5. Indicators Thema for Environment Dimension.
Table 5. Indicators Thema for Environment Dimension.
Indicators Theme
PM2.5 and PM10 [11]
Pollution [15,11]
Air quality [17,59,12,57,5,60,61,13]
Availability and quality of apps for air pollution monitoring [17,16,19]
Air pollution indeks [18]
Volume of CO2 emissions [20,11]
Pollution control [21]
Quality of air and water [21,9,11,13,12,57,5,60,61]
Monitoring emissions [21]
Industrial wastewater [22]
Industrial waste gas emissions [22]
Industrial solid waste discharge [22,11]
Discharge of hazardous waste [22]
Natural Environment [59]
Noise [9,12,57,5,60,61]		 Air
Recycling [17]
Renewable energy production [17]
Energy consumption [17]
Energy management [21,16]
Energy [9,10,12,57,5,60,61]
Energy Efficiency [19]		 Energy
Attractivity of natural conditions [15]
Sustainable resource management [15]
Environmental protection [15]
Basic sanitation quality [17]
Smart building and renovation [19,10]
Urban and Resource planning [19]
Expenses for urban amenities [20]
Green area per capita [18,13]
Level of waste reuse and recycle [18]
Improvements of waste discarding [21]
House and facility management [21]
Vehicle for city environmental [22,10]
Environmental Quality/Sustainability [16,12,57,5,60,61]		 Public Facilities
Table 6. Keywords of Crowdsourced Data.
Table 6. Keywords of Crowdsourced Data.
Indicators Keywords for Crowdsourced Data
Equity equity, house, housing, apartment, property
Health health, hospital, health center, nutrition, sanitation, drinking water
Education education, literacy, schooling, campus, college
Security security, unemployment, slavery, crime, criminality, peace, violence, terrorism, terrorist, terror
Culture and equality culture, equality, population, female workers
Innovation entrepreneur, company, innovation, technology, industry, transformation
Income income, salary, employment, poverty rate, finances, talent, human capital
Infrastructure infrastructure, cooperation, connections
Business opportunity economic performance, consumption, market, trade, competitiveness, productivity, business
Air air, pollution, emissions, defilement, waste
Energy renewable energy, electricity, green industry, solar energy
Public facilities green space, parks, city parks, vehicles, public transport, environmental facilities (equipment)
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
Prerpints.org logo

Preprints.org is a free preprint server supported by MDPI in Basel, Switzerland.

facebook logotwitter logolinkedin logo
MDPI Initiatives
Important Links
Subscribe

Disclaimer

Terms of Use

Privacy Policy

© 2024 MDPI (Basel, Switzerland) unless otherwise stated