Knowledge of the cycle of work and recovery is crucial for protecting employee health and well-being and preserving working capabilities. However, the daily process of effort and recovery is not well understood. This study investigated how the time spent on activities in the work and off-job domains, and the pleasure and effort experienced while engaging in these activities, affect the daily recovery process. We expected higher levels of effort at work and during off-job time to be negatively related to recovery, and higher levels of pleasure at work and during off-job time to be positively related to recovery. We also hypothesized that pleasure would act as a buffer against the negative effects of effort. Data were collected by means of a five-day diary study (three measurements daily, before and immediately after work, and at bedtime) among 120 university academic staff. Fatigue and (low) vigour were used as indicators of (lack of) recovery. Multilevel analyses showed that pleasure in the work and off-job domains had beneficial effects on recovery. An adverse association between effort expenditure and recovery was lacking. However, in the work domain, a combination of unpleasant and effortful work activities was negatively related to recovery. These findings stress the importance of engaging in pleasant activities during work and off-job time.