1) Background: Intellectual disability is more common in low- and middle-income countries than in high-income countries. Stigma and discrimination have contributed to barriers to people with intellectual disability accessing healthcare. As part of a larger study on caregiving of children with intellectual disability in urban Cape Town, South Africa, we interviewed a sub-group of families who had never used the intellectual disability services available to them, or who had stopped using them; 2) Methods: We employed a qualitative research design and conducted semi-structured interviews to explore the views and perspectives of parents and caregivers of children with intellectual disability who are not using specialised hospital services. We developed an interview guide to help explore caregivers’ and parents’ views; 3) Results: Results revealed that caregivers and parents of children with intellectual disability did not use the service due to financial difficulties, fragile care networks and opportunity costs, community stigma and lack of safety, lack of faith in services and powerlessness at effecting changes, and self-stigmatisation; 4) Conclusion: Current findings highlight a need for increased intervention at community level and collaboration with community-based projects to facilitate access to services, and engagement with broader issues of social exclusion.