Magnetic resonance elastography (MRE) has revealed sexual dimorphism in brain stiffness in healthy individuals and multiple sclerosis (MS) patients. In the animal model of MS, experimental autoimmune encephalomyelitis (EAE), we showed previously that inflammation-induced brain softening was associated with alterations of the extracellular matrix (ECM). However, it remained unclear whether the brain ECM presents sex-specific properties that can be visualized by MRE. Therefore, we aimed here at quantifying sexual dimorphism in brain viscoelasticity in association with ECM changes in healthy and inflamed brains. Multifrequency MRE was applied to the midbrain of healthy and EAE mice of both sexes to quantitatively map regional stiffness. To define differences in brain ECM composition, gene expression of the key basement membrane components laminin (Lama4, Lama5), collagen (Col4a1, Col1a1) and fibronectin (Fn1) was investigated by RT-qPCR. We showed that the healthy male cortex expressed less Lama4, Lama5, Col4a1 but more Fn1 (all p < 0.05) than the healthy female cortex, which was associated with 9% softer properties (p = 0.044) in that region. At peak EAE, cortical softening was similar in both sexes compared to healthy tissue, with an 8% difference remaining between males and females during EAE (p < 0.001). Cortical Lama4, Lama5 and Col4a1 expression increased 2 to 3-fold in EAE in both sexes while Fn1 decreased only in males (all p < 0.05). No significant sex differences in stiffness were detected in other brain regions. In conclusion, sexual dimorphism in the ECM composition of cortical tissue in the mouse brain is reflected by in vivo stiffness measured with MRE and should be considered in future studies by sex-specific reference values.