Deep Learning models are being applied to address plant phenotyping problems such as leaf segmentation and leaf counting. Training these models requires large annotated datasets of plant images, which, in many cases, are not readily available. We address the problem of data scarcity by proposing a data augmentation approach based on generating artificial images using conditional Generative Adversarial Networks (cGANs). Our model is trained by conditioning on the leaf segmentation mask of plants with the aim to generate corresponding, realistic, plant images. We also provide a novel method to create the input masks. The proposed system is thus capable of generating realistic images by first producing a mask, and subsequently using the mask as input to the cGANs. We evaluated the impact of the data augmentation on the leaf counting performance of the Mask R-CNN model. The average leaf counting error is reduced by 16.67% when we augment the training set with the generated data.