Prerpints.org logo
Preprint
Review

A Critical Review of Smart City Frameworks: New Criteria to Consider When Building Smart City Framework

Altmetrics

Downloads

164

Views

37

Comments

0

A peer-reviewed article of this preprint also exists.

This version is not peer-reviewed

Submitted:

12 July 2023

Posted:

13 July 2023

You are already at the latest version

Alerts
Abstract
With the rise of serious problems of urbanization and climate change, the call for intelligent and sustainable urban development is increasing nowadays. Therefore, smart cities have become a hot topic in academia, industry, and politics, resulting in the emergency of numerous smart city programs and projects. This study selected 26 evaluation frameworks proposed in recent years or large-scale research projects that have lasted for many years and analyzed and compared them from the 7 aspects of generalizability, comprehensiveness, availability, flexibility, scientific, transparency, and interpretability. The result indicates that the current smart city evaluation framework still needs to be upgraded and updated. These standards can not only guide the development direction and hotspot of the future smart city evaluation model but also serve as a guide for model demanders to test results and an effective tool to verify the framework designed by themselves.
Keywords: 
Subject: Social Sciences  -   Urban Studies and Planning

1. Introduction

Urbanization is a unified development trend and goal all over the world. The researchers point out that the world's urban population will double by 2050 and 70 percent of people will live in cities [1]. Cities are the centers for human activities, including political, economic, and everyday life activities. Thus, cities are expected to be sustainable, efficient, and inclusive. However, the planning and governance of cities often face many difficulties since cities are complex systems with multiple functions and interactions between humans and the environment. The need to design strategies for managing the development plan, addressing the development problem, and assessing the development status is urgent.
The concept of Smart City was first proposed in 1990 aiming to incorporate advanced information and communication technology (ICT) into urban planning [2]. With the needs of decision-makers and the development of Smart City, the concept of Smart City is linked with various other concepts such as intelligence, ubiquitous, knowledge, information, and digital city [3]. Overall, the introduction of smart cities aims to improve transparency, accountability, effectiveness, and efficiency of transactions between citizens and government through the integration of ICT technologies [3]. Although almost all smart city developments have the same final goal, to build sustainable, efficient, and happy cities for residents, the smart city development framework put forward is completely different due to different definitions of Smart City. Admittedly, due to the differences between the characteristics of cities and the nature of countries, the development framework of smart cities cannot be generalized, but there should be roughly unified principles and standards for the evaluation of smart city work. This facilitates information exchange and learning among different cities and makes the development of smart cities more oriented.
Arroub studied the paradigms of smart cities and claimed the problems in the different fields of smart cities [4]. They discussed the architecture and infrastructure of the framework based on the definition of the smart city. Parul Gupta delivered an insightful view on the popular themes of smart city assessment, indicating the unbalanced distribution of indicators [3]. Al Sharif discussed the associated risks of smart city development through the concept of smart cities and the existing smart city frameworks [5]. They demonstrated that the complexity of smart cities and the use of technology may become big risks for cities if not adequately understood and addressed. Yin C. etc. briefly reviewed the smart city frameworks and sorted out them from their definition and application for the reference of other researchers [6]. Ayyoob Sharifi also reviewed the latest smart city assessment tools and raised criteria when designing the framework [7]. However, there remains a research gap -- few works analyze the advantages and weaknesses of the current smart city frameworks and there is still no consensus on the framework design. This makes the smart city evaluation framework unstable, thus affecting the development and planning of smart cities.
Smart city assessment is a nascent field with great development potential. Therefore, it is of great significance to study and evaluate the existing smart city framework and formulate unified assessment criteria [7]. A deeper understanding of the construction principle of smart city evaluation and a common overall principle will profoundly impact the development and planning of smart cities in the future. Just like the current trend of economic globalization, the development of smart cities is gradually ushering in the future goal of homogenization. The common general principles will not affect the detailed rules of each country's smart city development but can provide guidance and improvement in the general direction. Thus, this paper introduces a new set of principles for the construction of comprehensive indicators for smart cities. When designing the smart city assessment tool, designers should consider the generalizability, comprehensiveness, flexibility, science, transparency, and interpretability of the framework. Also, the comparison of different smart city frameworks is carried out following the principles mentioned before.
This paper is structured into 4 sections. In the following section, a brief literature review of the latest smart city frameworks is presented. The methods used in this paper are illustrated in section 3. Section 4 presents the results and analysis of the smart city frameworks. Finally, the discussion and concluding remarks are delivered in the last section.

2. Methods

The content analysis method is used in this article to investigate the construction of smart city evaluation frameworks. Using this method, the details of smart city evaluation frameworks could be quantified and analyzed thoroughly.
Firstly, the keywords are decided according to the current commonly used vocabulary in the field of smart cities. Google Scholar and Web of Science are chosen as the search engine since they are the largest research search engines. Here, three main keywords are chosen: smart city index, smart city framework, and smart city assessment tools. Finally, we got 1,955,7251 and 459 search results respectively. The filtering criteria are listed as follows.
  • Relevant pieces of literature from 2000 to 2020 were screened.
  • The objective of the literature should be a comprehensive assessment of smart cities. That is, the assessment should cover all areas rather than a single area. Thus, research focusing on ‘smart cities ’is considered while research only covers one field in the smart city is not considered.
  • The literature must include one complete model, such as the indicators selected, the methods adopted, etc. Literature on conceptual revision and theoretical guidance types is not counted here.
After screening the titles and abstracts, about 100 articles were left. After carefully reading, 26 smart city evaluation models were finally selected for analysis and comparison. The models for analysis and comparison in this paper are listed as follows (Table 1).

3. Results

As mentioned above, an in-depth investigation has been made to analyze the commonality and differences among the selected 26 smart city index models. Six criteria have been discussed here concerning the construction of a smart city framework.

3.1. Generalizability

It is common for extremely elaborate and large index systems to evaluate the development of smart cities. Scholars aim to use these complex systems to evaluate all aspects and details of the target city. However, just as we talked about overfitting in statistics, these complex models may overfit the targeted city. That is, the definitions of the indicators are tightly restricted to the data source of the targeted city and the starting point of the construction of the evaluation system is also tightly constrained by the amount of data we can obtain for the target city. Thus, the generalizability of the indicators should be discussed at the beginning of constructing the framework.
To check the generalizability of the indicators among the selected smart city index models, it is first important to list the indicators and then group them according to their definitions and contents. Here, the indicators are grouped into two groups: international general and local indicators. The classification criteria are as follows:
  • It is an international research index. That is, it has a unified definition and calculation method (E.g., GDP, Gini Index, Birth rate).
  • A comprehensive global index published by international organizations or research organizations.
  • Indicators that can be converted to one another. The definition and calculation differences between these indicators are caused by national and regional differences.
A total of 1209 indicators of the selected models (6 models are excluded due to lacking details of indicators) have been investigated. The analysis in Figure 1 shows that only 22% of indicators are international and the others could only be applied locally.
The analysis in Figure 2 confirms the statement that few models would consider the generalizability of indicators. None of these models had more than 50 percent of common international indicators or even none. Thus, the models are irreproducible for other cities, or even irreproducible for the same city in the next year. This may result in standing still in the city’s development. The stronger the characteristic of the ‘designated city’ of the index, the more unfavorable the breadth and depth extension of the research.

3.2. Comprehensiveness

Couclelis stated that it is the complexity of the urban that results in the multitude of possible ways of approaching the study of the city [34]. Therefore, the connotation of the city has been endowed with a richer core. For example, urban economics, urban sociology, urban history, urban geography, urban ecology, urban transportation, urban health, urban anthropology, urban planning, etc., and now also urban informatics.
Though ‘digital cities’ or ‘ICT cities’ are considered the core of smart cities, and indeed the IT industry has transformed many urban areas economically, socially, and spatially [35,36,37], we still lack the ability to have the common language to develop amongst citizens, city staff, mayors, and the private sector [38]. Also, the term concerning ICT or digital mechanism would play down some of the underlying urban issues and problems inherent in the labeling process itself [39]. Thus, Cohen introduced the six-wheel model of Smart City, which covers six main fields that researchers and mayors work on most. The six-wheel model not only targets the comprehensive contents of smart cities, but also the comprehensive users.
Couclelis believed that smart cities will be sustainable, livable, equitable, innovative, and creative [34]. The essence of a city is to serve human beings. Therefore, the smart city is to make it more convenient for city residents to live happier lives and develop sustainably. In this context, smart solutions are expected to provide multiple socio-economic, environmental, and institutional benefits [15]. Comprehensive frameworks are thus important for assessing the smart city.
In this article, the comprehensive criterion is checked based on Cohen’s six-wheel model for smart cities. Since the frameworks are created for different purposes, the users or stakeholders are not compared. This doesn’t mean that comprehensive users or stakeholders are not important, rather, locating the purposes and users for the evaluation frameworks of smart cities is significant for designers. Statistical comparisons are not made here since each model has its corresponding audience.
As the analysis shows (Figure 3), the indicators in different fields are not balanced in the usual smart city framework. In general, among the 26 selected models, smart living, smart environment, and smart economy are three popular topics in the smart city models. Researchers tend to pay more attention to these three aspects. On the other hand, these three aspects may be the ones that are easier to get more data or information since they usually have clear coordination between departments.
For further details, the distribution of indicators is investigated in each selected model in Figure 4. It is obvious that few models could balance the indicators in 6 fields, especially for those large-scale evaluation models. Generally speaking, the index distribution of most models is consistent with the overall distribution. Smart environment, smart economy, and smart living are the three largest segments, and their proportions were adjusted slightly depending on the subject or area of study.
This brings us to a new question: should we balance the assessment indicators in various fields for the sake of fairness, or should we adjust the ratio to suit the actual situation and research objectives? It is hard to judge which statement is correct, but there is still one we should keep in mind when designing smart city frameworks to guarantee comprehensive. Although it cannot be completely guaranteed that the indicators in each field are equal during the selection of indicators, appropriate weights should be applied to each field to prevent cities that are outstanding or backward in some aspects from obtaining improper evaluation. That is, cities that are particularly outstanding in one aspect but not good in other aspects will get more scores because of more indicators in this field, and thus get higher scores. Cities that are inferior in a certain aspect will get worse scores in the same way. Then, the evaluation framework would lose fairness and rationality.

3.3. Availability

Because the same data can be used for a range of purposes, making data available can be an efficient use of limited research resources. Doing so can also improve traceability and, thus, accountability when it comes to research findings [40]. The availability here we discussed not only refers to available data for people who proposing the framework, but also for everyone who touches or uses this framework. Data availability may decide three dimensions of the model: a) whether the model makes any practical sense; b) whether the model is reproducible; c) whether the model can be extended to periodic work.
To check the availability of the framework, two criteria have been defined.
  • Data sources are public sources. i.e., public websites or annual report data
  • All data used in the framework is offered by the framework designers.
Of the models selected, only 38% provided data sources (see Figure 5). Only three of the models that provided the source of the data fully used public data (see Figure 6). Most of the models are very vague about the data sources and are often described using terms such as ‘may obtain from...’. This situation greatly affects the influence and applicability of the model.
This situation arises out of both technical and ethical difficulties. On the one hand, the popularity of electronic open data is not high, and only highly developed countries and regions have relatively mature electronic open data sets. However, even for highly developed cities, their public data sets are still undergoing form change, and the frequency of updates is not fixed. On the other hand, due to privacy data protection, not all the data required by the evaluation can be disclosed. This is especially true for models created with government or corporate funding.
However, there is no doubt that data availability will increase as technology advances. With the progress of information masking technology and data acquisition technology, and the gradual maturity of urban electronic open data set platforms, open data will become more abundant. Therefore, considering data availability when designing smart city evaluation frameworks is also an important reference for models to consider sustainability.

3.4. Flexibility

Ideally, smart city frameworks should not be affected by location, scale, and time, or they should be affected by as little sound as possible. On the one hand, a flexible framework could avoid frequent changes and thus reduce the cost for organizers and researchers. On the other hand, flexibility, generalizability, and comprehensiveness are mutually reinforcing. That is, when generalizability and comprehensiveness reach the standard, the breadth and depth of the model have reached a considerable degree. Then the fine-tuning of time and place will not have a large impact on the model. In contrast, when the model is flexible enough and not limited by place, year and scale, the model indicators must be universal and comprehensive.
In the study scale and study area differences (Figure 7), we found that the change in the study area was greatly influenced by the identity of the model designer. As the chart below shows, 10 models are models of global research, designed by a consortium of academics in a large laboratory, a collaboration between the institute and a large company, or a group project of an international organization. Models based on a single city or country are usually designed and studied by university professors. This is limited by its limited human and resource constraints. However, regional studies only focus on Europe at present, thanks to the rich public data sets and the habit of updating data at a fixed frequency in the EU. The model without disclosing the research area is basically the research of scholars on the new model structure or the weighting method, without carrying out the practice.
Among the differences in study timescales, few models were able to maintain continuous updates and provide annual reports. But there is no doubt that almost all models that have been steadily updated and reported over a long period of time have significant international influence.

3.5. Scientific

The scientificity of frameworks is the top issue to discuss for all researchers. The content for the term ‘scientificity’ is rather broad, including the indicator selection, weighting method, structure selection of framework, and so on. 'Scientific thought' is regarded here as both a type of goal-directed behavior (practice) and its product and the question of its 'nature' posed in terms of that goal and of means appropriate for achieving it [41]. Thus, we could judge scientificity from goals and means. However, the indicator selection and structure selection are personal choices to some extent, it would be hard to judge whether they are scientific or not. Since the weighting method usually relates to an exact mathematical-statistical method, the scientificity would be checked here mainly based on the weighting method. The selected models would be checked based on the following process in Figure 8.
Among the selected models, Figure 9 shows that 58 percent had a clear weighting approach, while 42 percent did not. The model of the weighted approach and the model of the unweighted approach is almost 50/50. This is because some model designers assume that the model design work is finished after providing metrics and frameworks. They believe that the selection of indicators is the focus of the model. In the selection of the model weighting method (Figure 10), 9 models chose the objective method (60%), 5 models chose the subjective method (35%), and 1 model chose the combination of the subjective and objective method (5%). Indeed, objective methods are welcomed for their variety and rapidity compared with subjective methods. However, the objective method is often closely related to the data features, which makes the data distribution pattern and data features dominate the weight too much, and sometimes the results are contrary to the designer's assumption. Although subjective methods can synthesize expert opinions, they are time-consuming and costly. The subjective and objective method combines the advantages of the two methods, which is more scientific and effective in the design process. However, it also combines the disadvantages of the two methods, which are longer and larger in time and energy consumption and increases the complexity of calculation method design.

3.6. Transparency of Process

The transparency of processing the framework is usually ignored by many researchers or organizations. Transparency means each link has a scientific detailed introduction. That is, the process of constructing the framework, including indicator selection, data sources or data collection method, weighting methods and final weights, calculation, and data processing, should all be clear for the readers.
Transparency is significant in scientific research since it decides the ultimate practicability and impact on the world. That is, it is transparency that makes the framework we build reproducible, supervised, and malleable.
To analyze the transparency of the selected models, we mainly check whether the models have detailed descriptions or separate documents for each link of the construction.
Analysis of the selected models (Figure 11) explained the lack of transparency when designing smart city frameworks. 42% of the selected models have no information about the calculation method and weights for each indicator, and 19% of the selected models only provide a brief logic of the used method, concealing the actual operation process and the use method. Only 39% of the selected models have introduced the principle of the calculation method, the calculation process, and the method of obtaining the result in detail. However, the models that are willing to provide detailed calculation processes and principles are generally from the papers of scholars. The common feature of these models is that they have not been verified in practice or can only be verified in some cities of a single country.
From the above analysis, although transparency is very important to expand the influence of the smart city model and strengthen the reference and exchange between different models, it is still a difficult task to disclose the specific details of the model in detail. Since the investors of the smart city model are often diverse, the detailed description of the model is constrained by many aspects. Academics may be reluctant to disclose the details of weighting and scoring methods, especially for models with innovative approaches. For enterprises, they do not want to reveal too much data collection and processing details, because it involves user privacy, corporate privacy, or other interests.
In short, there is still a long way to go in terms of transparency in model design and results. However, this does not affect the consideration of this principle in the process of model design, and appropriate adjustments should be made in the disclosure of the final results, in order to provide inspiration and suggestions for the subsequent model design.

3.7. Interpretability of Results

The interpretability of results is another ignorant problem when designing the smart city assessment framework. Typically, most people regard the structure and the ultimate weights we used as the ending of the framework. However, what the framework could bring to our city or society is what they are really concerned about.
To be honest, it is not easy to define whether a model has a social impact or whether it successfully explains model results. The result documents or the analysis part for the models are checked here for ensuring interpretability. Though we could only check from the design parts of the documents or papers, it still reflects the significance and importance of the interpretability of results.
The analysis in Figure 12 shows that 42% of the selected models have analysis for the final scores or the city rankings, 39% of the selected models only have final rankings or city scores, and 19% of the selected models have no results. The models that have analysis for the results are usually designed or implemented by international organizations, big companies, or schools that work with businesses. The final analyses for the selected models mainly focus on the score analysis of cities in various fields and infer the development trend and future development hotspots of smart cities in the world. In addition, part of the analysis is for a single city, from all aspects of the city to evaluate and then form a city report, which is what urban planners and builders tend to do. The results of the model, designed by schools cooperating with the government or organizations, are generally reports of each city in the study area. For international organizations or companies, it is more likely to treat rankings and predictions of general trends as the final analysis. For enterprises, the change in the ranking of cities can often bring shock news and heat, which is also what they need and expect.
In a word, the final analysis results of different evaluation frameworks are different due to the differences in purpose, but the loss of the analysis of results is obviously an incomplete evaluation framework. For the evaluation framework of smart cities, the analysis and presentation of results is an important part of the interpretation of the core of the evaluation framework. Usually, our understanding of the various areas in the framework design and expectations for the future development direction of smart cities will be revealed in our analysis of the results. In the early introduction process of indicators, each indicator can only allow decision-makers to understand the designer's understanding of the smart city from the perspective of a single indicator. After model calculation, the results can reflect the application effect of the composite index in the actual urban scene and show the spark after the collision between design and practice.

4. Discussion and Conclusion

In recent years, as the concept of Smart City has been accepted by more and more designers and scholars, more and more smart city evaluation systems have emerged and been applied in different areas. Through the evaluation and research of recent smart city models, this study proposes new design principles for smart city models. First, the macro design principles of smart city model make the model more scientific and purposeful in design. That is, the macro rules can regulate and constrain the design, but also make up for the deficiency in consideration, to effectively avoid the problem of incomplete consideration or strategic thinking. Additionally, for model demanders, this principle can also be used as a test standard for model results. On the one hand, it prevents the demander from blindly following the designer's thinking frame and failing to put forward new suggestions. On the other hand, the demander can correspond the demand to each standard one by one and can also play a role in further improving the demand.
In the overall analysis of the selected model, the following phenomena emerge. Three criteria have been considered by most of the selected models: scientific, flexibility, and comprehensive. Generally, models fully consider a wide range of indicators, even over 100 indicators in some cases, rigorous grading methods, and research areas based on different purposes. However, the other four criteria-generalizability, availability, interpretability, and transparency - performed poorly. Most models give them little consideration, and a few ignore them outright. These differences are caused by the difference between the research purpose and the model designer. Scholars who only want to explore a new framework or research method may not need the model to have realizable significance, so they usually complete the design of the evaluation model after referring to the index collection and weighting method. For businesses and governments, the availability of more data sources and funding allows the application of the framework to be implemented in a wider area. But this also means that the data is usually kept secret and the designed model cannot be migrated or copied. In short, this is usually a ‘specified consumer-owned model.
In the future, based on the design principles proposed in this study, the evaluation model of Smart City should be more portable, communicable, influential, and credible. In addition, these principles can also greatly exploit the development potential of the model. Laterally, the breadth of the model increases. That is, the areas considered by the model are increased, and the number of indicators considered is also increased. Longitudinally, the depth of the model is also expanded. That is, it can continuously observe the differences in the time series of models and the development of model structural changes, and update and upgrade accordingly.
At present, there are still many cities in the world whose smart city development is still in the embryonic or embryonic stage. The definition and development direction of smart cities has not been unified globally. The work aims to create a sustainable, portable, and potential smart city evaluation system framework for the future. It provides the development direction and designs ideas of smart city framework for cities that are developing smart cities or are still in the embryonic stage. For cities with research results or mature systems, smart city framework designers can be encouraged to reflect on and improve the existing framework.

Author Contributions

Conceptualization, Fan Shi and Wenzhong Shi; methodology, Fan Shi and Wenzhong Shi; formal analysis, Fan Shi; writing—review & editing, Fan Shi and Wenzhong Shi; visualization, Fan Shi; supervision, Wenzhong Shi; project administration, Wenzhong Shi; funding acquisition, Wenzhong Shi. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by Urban Informatics for Smart Cities, The Hong Kong Polytechnic University under Grant [1-ZVN6], and the Otto Poon Charitable Foundation Smart Cities Research Institute, Hong Kong Polytechnic University under Grant [CD03].

Data Availability Statement

No Public data used in this literature review.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. United Nations, Department of Economic and Social Affairs, World Urbanization Prospects: The 2014 Revision, Highlights, United Nations New York, 2014.
  2. Bibri, Simon Elias, and John Krogstie. "Smart sustainable cities of the future: An extensive interdisciplinary literature review." Sustainable cities and society 2017, Volume 31, pp. 183-212.
  3. Gupta, P., Chauhan, S.; Jaiswal, M. P. Classification of smart city research-a descriptive literature review and future research agenda. Information Systems Frontiers 2019, Volume 21(3), pp. 661-685.
  4. Arroub, A.; Zahi, B.; Sabir, E.; Sadik, M. A literature review on Smart Cities: Paradigms, opportunities and open problems. In 2016 International conference on wireless networks and mobile communications (WINCOM) (pp. 180-186). IEEE. (2016, October).
  5. Al Sharif, Reem, and Shaligram Pokharel. "Smart city dimensions and associated risks: Review of literature." Sustainable Cities and Society, 2022, Volume 77, pp. 103542.
  6. Yin, C., Xiong, Z.; Chen, H., Wang, J.; Cooper, D.; David, B.. A literature survey on smart cities. Science China Information Sciences, 2015, Volume 58(10), pp. 1-18.
  7. Sharifi, Ayyoob. "A critical review of selected smart city assessment tools and indicator sets." Journal of cleaner production 2019, Volume 233, pp. 1269-1283.
  8. Bannerjee, S.; Bone, J.; Finger, Y.; Haley, C.. European Digital City Index. Methodology report. Nesta Report—ISBN, 2016, (978-1), 84875-153.
  9. Shen, L.; Huang, Z.; Wong, S. W.; Liao, S.; Lou, Y. A holistic evaluation of smart city performance in the context of China. Journal of Cleaner Production, 2018, Volume 200, pp. 667-679.
  10. Bosch, P., Jongeneel, S., Rovers, V., Neumann, H. M., Airaksinen, M., & Huovila, A. CITYkeys indicators for smart city projects and smart cities. CITYkeys report, 2017.
  11. Giffinger, R., Fertner, C., Kramar, H., & Meijers, E., City-ranking of European medium-sized cities. Cent. Reg. Sci. Vienna UT, 2007, Volume 9(1), pp. 1-12.
  12. Li, C., Dai, Z., Liu, X., & Sun, W., Evaluation system: Evaluation of smart city shareable framework and its applications in China. Sustainability, 2020, Volume 12(7), pp. 2957.
  13. Mori Memorial Foundation, Global Power City Index, 2017.
  14. Sharma, K., & Tayal, S., Indian smart city ranking model using taxicab distance-based approach. Energy Systems, 2019, pp. 1-18.
  15. United for Smart Sustainable Cities, Collection methodology for key performance indicators for smart sustainable cities —ISBN, 2017, 978-92-61-25231-1.
  16. Gutman, S., & Rytova, E, Indicators for assessing the development of smart sustainable cities. In International Scientific Conference on Innovations in Digital Economy, 2019, pp. 55-73. Springer, Cham.
  17. Gokhberg, Leonid & Kutsenko, Evgeniy & Boos, Victoria & Islankina, Ekaterina & Ismagulova, Saule & Streltsova, Ekaterina & Tyurchev, Kirill, HSE Global Cities Innovation Index: 2020. 10.17323/978-5-7598-2343-8.
  18. ISO 37122:2019, Sustainable cities and communities — Indicators for smart cities, 2019.
  19. SG20, I. T. U. T. (2017). Key performance indicators for smart sustainable cities to assess the achievement of sustainable development goals. Recomm. ITU-T Y. 4903/L, 1603, 1-50.
  20. Antwi-Afari, P., Owusu-Manu, D. G., Ng, S. T., & Asumadu, G., Modeling the smartness or smart development levels of developing countries’ cities. Journal of Urban Management, 2021, Volume 10(4), pp. 369-381.
  21. Lombardi, P., Giordano, S., Farouh, H., & Yousef, W., Modelling the smart city performance. Innovation: The European Journal of Social Science Research, 2012, Volume 25(2), pp. 137-149.
  22. IESE Business School, IESE Cities in Motion Index (ST-442-E), 2017.
  23. NETWORKED SOCIETY CITY INDEX 2016, Available online: http://mb.cision.com/Public/15448/2245037/93894148bfbf1118.pdf (accessed on 31 05 2023).
  24. Zhang, Y., Liu, F., Gu, Z., Chen, Z., Shi, Y., & Li, A., Research on smart city evaluation based on hierarchy of needs. Procedia Computer Science, 2019, Volume 162, pp. 467-474.
  25. Singh, P. K., & Ohri, A., Sensitivity analysis of the smart city environmental sustainability index (SCESI), 2021.
  26. Serrano, M., Griffor, E., Wollman, D., Dunaway, M., Burns, M., Rhee, S., & Greer, C., Smart Cities and Communities: A Key Performance Indicators Framework. NIST Special Publication, 2022, Volume 1900, pp. 206.
  27. Wang, C., Li, S., Cheng, T., & Li, B., A construction of smart city evaluation system based on cloud computing platform. Evolutionary Intelligence, 2020, Volume 13(1), pp. 119-129.
  28. Canıtez, F., & Deveci, M., A SMART CITY ASSESSMENT FRAMEWORK: THE CASE OF ISTANBUL'S SMART CITY PROJECT. Economic and Social Development: Book of Proceedings, 2018, pp. 369-380.
  29. Berger, R., The Smart City Breakaway Smart City Strategy Index 2019. Think: Act, 2019.
  30. Sojda, A., Smart city index based on topsis method. Zeszyty Naukowe. Organizacja i Zarządzanie/Politechnika Śląska, 2020.
  31. IMD Smart City Index Report, Available online: https://www.imd.org/smart-city-observatory/home/ (accessed on 31 05 2023).
  32. Liu, Y., The study on smart city construction assessment based on TOPSIS—’ the Beijing-Tianjin-Tangshan City Clusters’ as the case. 2016, In 2016 International Conference on Smart City and Systems Engineering (ICSCSE) (pp. 321-325). IEEE.
  33. Theng, Y. L., Xu, X., & Kanokkorn, W., Towards the construction of smart city index for analytics (SM-CIA): pilot-testing with major cities in China using publicly available data. 2016, In 2016 49th Hawaii International Conference on System Sciences (HICSS) (pp. 2964-2973). IEEE.
  34. Couclelis, H., Conceptualizing the City of the Information Age. In: Shi, W., Goodchild, M.F., Batty, M., Kwan, MP., Zhang, A. (eds) Urban Informatics. The Urban Book Series. Springer, Singapore, 2021. [CrossRef]
  35. Graham, S. and Marvin, S. Telecommunications and the City: Electronic Spaces, Urban Places. London: Routledge, 1996.
  36. Graham, S. and Marvin, S. Splintering Urbanism: Networked Infrastructures, Technological Mobilities and the Urban Condition. London: Routledge, 2001.
  37. Florida, R. The Rise of the Creative Class: and How it’s Transforming Work, Leisure, Community and Everyday Life. New York: Basic Books, 2002.
  38. COHEN, B.: What exactly is a smart city? September 19, 2012b, Available online:: http://www.fastcoexist.com/1680538/what-exactly-is-asmart-city. (accessed on 31 05 2023).
  39. Begg, I. (ed.) Urban Competitiveness: Policies for Dynamic Cities. Cambridge: Polity Press, 2002.
  40. Huntington, H. P., Archer, E., Ashley, W. S., Cutter, S. L., Goldstein, M. A., Roncoli, C., & Spero, T. L., Data Availability Principles and Practice, Weather, Climate, and Society, 2020, Volume 12(4), pp. 647-649.
  41. Suchting, W.A. The nature of scientific thought, 1995, Sci Educ 4, 1–22. [CrossRef]
Preprints 79283 g001
Figure 1. Analysis result concerned with the criteria ‘Generalizability’.
Figure 1. Analysis result concerned with the criteria ‘Generalizability’.
Preprints 79283 g001
Preprints 79283 g002
Figure 2. Analysis result concerned with the criteria ‘Generalizability’ in the selected assessment models.
Figure 2. Analysis result concerned with the criteria ‘Generalizability’ in the selected assessment models.
Preprints 79283 g002
Preprints 79283 g003
Figure 1. Analysis result concerned with the criterion ‘Comprehensive’ in total
Figure 1. Analysis result concerned with the criterion ‘Comprehensive’ in total
Preprints 79283 g003
Preprints 79283 g004
Figure 2. Analysis result concerned with the criterion ‘Comprehensive’ in each selected model
Figure 2. Analysis result concerned with the criterion ‘Comprehensive’ in each selected model
Preprints 79283 g004
Preprints 79283 g005
Figure 3. Analysis result concerned with the criterion ‘Availability’ in total.
Figure 3. Analysis result concerned with the criterion ‘Availability’ in total.
Preprints 79283 g005
Preprints 79283 g006
Figure 4. Analysis result concerned with the criterion ‘Availability’ in each selected model.
Figure 4. Analysis result concerned with the criterion ‘Availability’ in each selected model.
Preprints 79283 g006
Preprints 79283 g007
Figure 7. Analysis result concerned with the criterion ‘Flexibility’ in total
Figure 7. Analysis result concerned with the criterion ‘Flexibility’ in total
Preprints 79283 g007
Preprints 79283 g008
Figure 8. Checking process framework for the criterion ‘Scientific’
Figure 8. Checking process framework for the criterion ‘Scientific’
Preprints 79283 g008
Preprints 79283 g009
Figure 5. Analysis result concerned with the criterion ‘Scientific’ in total
Figure 5. Analysis result concerned with the criterion ‘Scientific’ in total
Preprints 79283 g009
Preprints 79283 g010
Figure 10. Analysis result of specified weighting methods concerned with the criterion ‘Scientific’ in total.
Figure 10. Analysis result of specified weighting methods concerned with the criterion ‘Scientific’ in total.
Preprints 79283 g010
Preprints 79283 g011
Figure 6. Analysis result concerned with the criterion ‘Transparency’ in total
Figure 6. Analysis result concerned with the criterion ‘Transparency’ in total
Preprints 79283 g011
Preprints 79283 g012
Figure 7. Analysis result concerned with the criterion ‘Interpretability’ in total
Figure 7. Analysis result concerned with the criterion ‘Interpretability’ in total
Preprints 79283 g012
Table 1. List of selected models in the article
Table 1. List of selected models in the article
Number Title
1 European Digital City Index [8]
2 A holistic evaluation of smart city performance in the context of China [9]
3 CITYkeys indicators for smart city projects and smart cities [10]
4 City-ranking of European Medium-Sized Cities [11]
5 Evaluation System: Evaluation of Smart City Shareable Framework and Its Applications in China [12]
6 Global Power City Index [13]
7 Indian smart city ranking model using taxicab distance-based approach [14]
8 Smart Sustainable Cities [15]
9 Indicators for Assessing the Development of Smart Sustainable Cities [16]
10 HSE Global Cities Innovation Index [17]
11 ISO/CD 37122 Sustainable cities and communities — Indicators for smart cities [18]
12 Key performance indicators for smart sustainable cities to assess the achievement of sustainable development goals [19]
13 Modeling the smartness or smart development levels of developing countries’ cities [20]
14 Modelling the smart city performance [21]
15 IESE Cities in Motion Index [22]
16 Networked society city index [23]
17 Research on Smart City Evaluation Based on Hierarchy of Needs [24]
18 Sensitivity Analysis of the Smart City Environmental Sustainability Index (SCESI) [25]
19 Smart Cities and Communities: A Key Performance Indicators Framework [26]
20 A construction of smart city evaluation system based on cloud computing platform [27]
21 A SMART CITY ASSESSMENT FRAMEWORK: THE CASE OF ISTANBUL'S SMART CITY PROJECT [28]
22 Smart city Strategy index [29]
23 SMART CITY INDEX BASED ON TOPSIS METHOD [30]
24 Smart city index-IMD [31]
25 The study on Smart City Construction Assessment Based On TOPSIS-’the Beijing-Tianjin-Tangshan City Clusters’ as the case [32]
26 Towards the Construction of Smart City Index for Analytics (SM-CIA): Pilot-Testing with Major Cities in China Using Publicly Available Data [33]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
Prerpints.org logo

Preprints.org is a free preprint server supported by MDPI in Basel, Switzerland.

facebook logotwitter logolinkedin logo
MDPI Initiatives
Important Links
Subscribe

Disclaimer

Terms of Use

Privacy Policy

© 2024 MDPI (Basel, Switzerland) unless otherwise stated