This contribution, which reviews some broad trends in human history and in the history of fishing, argues that sustainability, however defined, rarely if ever occurred as a result of an explicit policy, but as result of our inability to access a major part of exploited stocks. With the development of industrial fishing, and the resulting invasion of the refuges previously provided by distance and depth, our interactions with fisheries resources have come to resemble the wars of extermination that newly arrived hunters conducted 40,000-50,000 years ago in Australia, and 11,000-13,000 years ago against large terrestrial mammals arrived in North America. These broad trends are documented here through a map of change in fish sizes, which displays characteristic declines, first in the nearshore waters of industrialized countries of the Northern Hemisphere, then spread offshore and to the Southern Hemisphere. This geographical extension met its natural limit in the late 1980s, when the catches from newly accessed stocks ceased to compensate for the collapse in areas accessed earlier, hence leading to a gradual decline of global landing. These trends affect developing countries more than the developed world, which have been able to meet the shortfall by increasing imports from developing countries. These trends, however, together with the rapid growth of farming of carnivorous fishes, which consumes other fishes suited for human consumption, have led to serious food security issues. This promotes urgency to the implementation of the remedies traditionally proposed to alleviate overfishing (reduction of overcapacity, enforcement of conservative total allowable catches, etc.), and to the implementation of non-conventional approaches, notably the re-establishment of the refuges (also known as marine reserves), which made possible the apparent sustainability of pre-industrial fisheries.