Among women globally, breast cancer is a major cause of cancer-related death. Accurate and timely diagnosis is essential, and results can be significantly improved. A new era in image analysis has been brought about by the emergence of artificial intelligence (AI), which has made significant progress in the diagnosis and customization of treatment plans for breast cancer possible. This study aimed to develop a comprehensive AI framework for breast cancer detection by integrating convolutional neural networks (CNNs) for image analysis with an artificial neural network (ANN) for clinical data. Using a dataset of ultrasound and pathology images, along with clinical features from 569 patients, we trained CNN models to classify breast tissue as benign or malignant, and the ANN to process clinical data for the same task. The combined approach significantly enhanced the accuracy of breast cancer diagnosis, achieving over 96% accuracy by leveraging both imaging and clinical data. Class Activation Maps (CAM) and heatmaps provided valuable interpretability for CNN predictions, improving trust and transparency in the model’s decision-making process. The results demonstrate that the fusion of CNNs and ANNs enhances diagnostic accuracy and offers a promising tool for early breast cancer detection, while providing crucial insights for clinical validation.