Local support is critical to the success and longevity of fishery management initiatives. Previous research suggests that how resource users perceive ecological changes, explain them, and cope with them, influences local support. The objectives of this study were two-fold. First, we collated local fishers' knowledge to characterize the long-term socio-ecological dynamics of the small-scale fishery of Sainte-Marie Island, in Madagascar. Second, we empirically assessed the individual- and site-level factors influencing support for fishery restrictions. Our results indicate that fishers observed a decline in fish abundance and catch sizes, especially in nearshore areas; many also perceived a reduction in fish sizes and the local disappearance of species. To maintain their catches, most fishers adapted by fishing harder and further offshore. Accordingly, fishers identified increased fishing effort (number of fishers and gear evolution) as the main cause of fishery changes. Collectively, our results highlight that the transition from a subsistence to commercial fishery, and resulting changes in the relationship between people and the fisheries, was an underlying driver of fishery changes. Additionally, we found that gender, membership to local associations, coping mechanisms, and perceptions of ecological health, were all interlinked and significantly associated with conservation-oriented attitudes. Conservation-oriented attitudes, however, were not associated with fishers' willingness to decrease fishing. In the short-term, area-based restrictions could contribute to building support for conservation. In the long-term, addressing the underlying causes of the decline will necessitate collaborations among the various groups involved to progressively build livelihood flexibility. Collectively, our study provides additional insights on the individual- and site-level factors influencing support for fishery restrictions. It also highlights the importance of dialoguing with fishers to ensure that fishery management plans are adapted to the local context.