Extracellular vesicles (EV), including exosomes and microvesicles, are released from various cells and alter recipient cell phenotypes and fates by their biomolecules. Here we review current knowledge about tumor EVs and how they prompt malignant cell communication with tumor-associated cells, such as cancer-associated fibroblasts, tumor endothelial cells, and immune cells. We delineate the major pathways and molecular players that influence each step of cancer initiation, progression, and resistance. Of note, cancer exosomes involve immunosuppression by tumor-associated macrophages, myeloid-derived suppressor cells, and regulatory T cells. Moreover, tumor exosomes can induce the apoptosis of killer T cells and immune checkpoint of dendritic cells and attenuate natural killer cells. An in-depth understanding of EV biology is essential to ensure the clinical development of exosome/EV-based therapeutic products, which will be of benefit to exosome manipulation in cancer management.