Physical literacy (PL) has gained considerable attention and traction in the field of health and physical education (HPE) for some time now and can thus be seen as part of the HPE discourse. However, just as advocacy for PL has grown exponentially over the last decade(s), so have the critical voices raised over the universal adoption of this concept in HPE curricula and practices. The aim of this paper is to illustrate how the influence and constitution of the concept of PL in the Swedish school subject physical education and health (PEH) curriculum and practice can be understood through a Bernsteinian lens. We argue that although the influence of the concept PL on current Swedish PEH practice to date has been limited, such an influence within a neoliberal context, risk (re)producing idealised and limited notions of what is (im)possible in becoming and being a physically literate (educated) individual in Swedish PEH and beyond. We conclude by calling for the recognition of the plurality of physical literacies in the (re)constitution of HPE/PEH practices underpinned by inclusion, diversity, equity and social justice.