The continuous expansion of Large Language Models (LLMs) has significantly transformed the fields of artificial intelligence (AI) and natural language processing (NLP). This paper reviews the rapidly evolving domain of language models and introduces the concept of Extremely Large Language Models (XtremeLLMs), a new category defined for models exceeding one trillion parameters. These models are monumental in scale and engineered to enhance performance across a diverse range of language tasks. This study aims to establish a comprehensive framework that explores the significant opportunities and complex challenges presented by such extensive scaling and emphasises the implications for future advancements in the field.