The establishment of the symbiosis between legumes and nitrogen-fixing rhizobia is finely regulated at the transcriptional, posttranscriptional and posttranslational levels. Argonaute5 (AGO5), a protein involved in RNA silencing, is able to bind both viral RNAs and microRNAs to control plant-microbe interactions and plant physiology. For instance, AGO5 regulates the systemic resistance of Arabidopsis against Potato Virus X as well as the pigmentation of soybean (Glycine max) seeds. Here, we show that AGO5 is also playing a central role in legume nodulation based on its preferential expression in common bean (Phaseolus vulgaris) and soybean roots and nodules. We also report that the expression of AGO5 is induced after 1 hour of inoculation with rhizobia. Down-regulation of AGO5 gene in P. vulgaris and G. max causes diminished root hair curling, reduces nodule formation and interferes with the induction of three critical symbiotic genes: NUCLEAR FACTOR Y-B (NF-YB), NODULE INCEPTION (NIN) and FLOTILIN2 (FLOT2). Our findings provide evidence that the common bean and soybean AGO5 genes play an essential role in the establishment of the symbiosis with rhizobia in determinate legumes.