When earthquake damage occurs, it is extremely important to quickly identify the presence or absence of such damage. In this study, we developed and evaluated a new remote sensing index to automatically determine "damage" by image analysis when damage greater than partial destruction of a house by an earthquake or tsunami occurs. The accuracy of the houses estimated as "damaged" was high and sufficient for the initial assessment of damage. The results of this research are expected to be utilized for more rapid damage assessment after a disaster.