Cutting-edge Human-Computer Interaction (HCI) technologies embedded with Machine Learning (ML) will cause a paradigm shift in various domains, including manufacturing and developing facilities and services for professional and personal use. ML implemented HCIs can help people overcome societal challenges brought about by the COVID-19 pandemic. We introduce a system for people to perform physical exercises at home. This system is intended to help a range of demographics, from non-critical physical therapy patients to experienced weightlifters. More specifically, we propose a method to assess the difficulty of an exercise for visual exercise tracking systems. Pose estimation tracks exercises and reinforcement learning provides autonomous feedback to the user (patient/athlete). This information is processed largely on the client side, allowing the application to run smoothly anywhere in the world.