The day that exoskeletons assist their human counterparts throughout daily life continues to draw nearer. With overall mobility correlated closely with quality of life, exoskeleton technology can drive substantial positive impacts in society. Over the past few years, exoskeleton research has boomed, with significant advances in optimizing human-exoskeleton interactions and overall device designs; however, to translate this technology into the real-world requires an intelligent controller able to assist the user throughout their day. In this work, a first-of-its-kind, task-agnostic controller is proposed, optimized, and validated. The controller commanded exoskeleton assistance based solely on the user’s underlying joint moments, which naturally vary with changes in human activity. Since human joint moments cannot be measured directly, a deep neural network was optimized and integrated into the controller to generate instantaneous user joint moment estimates. The model was validated across over 60 ambulatory and non-ambulatory conditions without any calibration or data required from the user. When deployed onboard an autonomous hip exoskeleton, the resulting controller significantly reduced human energetics during multiple ambulation modes. Additionally, when deployed on a multi-joint exoskeleton, the task-agnostic controller augmented human energetics during both cyclic and non-cyclic activities by seamlessly coordinating assistance across the hip and knee without any experimenter intervention. Thus, this work presents substantial progress in autonomous exoskeleton control, bridging the gap between in-lab exoskeleton benefits and real-world need.