In this paper, we investigate the relationship people have with their smart devices. We use the concept of agency to capture aspects of users’ sense of mastery as they relate to their device. This study gives preliminary evidence of the existence of two independent dimensions of agency for modeling the interaction between humans and smart devices: (i) user agency and (ii) device agency. These constructs emerged from an exploratory factorial analysis conducted on a survey data collected from 587 participants. In addition, we investigate the correlation between user agency and device agency with background variables of the respondents. Finally, we argue that mapping the users’ dynamics with their device into user agency and device agency fosters a better understanding of the needs of the users and helps in designing interfaces tailored for the specific capabilities and expectations of the users.