Finite element discretizations of problems in computational physics often rely on hand-generated initial mesh and adaptive mesh refinement (AMR) to preferentially resolve regions containing important features during simulation. We propose Adaptnet, a Graph Neural Networks (GNNs) framework for learning mesh generation and adaptation. The model is composed of two GNNs: the first one, Meshnet, learns mesh parameters commonly used in open-source mesh generators, to generate an initial mesh from a Computer Aid Design (CAD) file; while the second one, Graphnet, learns mesh-based simulations to predict the components of an Hessian-based metric to perform anisotropic mesh adaptation. Our approach is tested on structural (Deforming plate - Linear elasticity) and fluid mechanics (Flow around cylinders - steady-state Stokes) problems. Our results show it can accurately predict the dynamics of the system and adapt the mesh accordingly. The adaptivity of the model supports learning resolution-independent dynamics and can scale to more complex state spaces at test time.