Plant diseases have a disastrous impact on the safety of food production, and they can cause a significant reduction in both the quality and quantity of agricultural products. In severe cases, plant diseases may even cause no grain harvest entirely. Thus, the automatic identification and diagnosis of plant diseases are highly desired in the field of agricultural information. Many methods have been proposed for solving this task, where deep learning is becoming the preferred method due to the impressive performance. In this work, we study transfer learning of the deep convolutional neural networks for the identification of plant leaf diseases and consider using the pre-trained model learned from the typical massive datasets, and then transfer to the specific task trained by our own data. The VGGNet pre-trained on ImageNet and Inception module are selected in our approach. Instead of starting the training from scratch by randomly initializing the weights, we initialize the weights using the pre-trained networks on the large labeled dataset, ImageNet. The proposed approach presents a substantial performance improvement with respect to other state-of-the-art methods; it achieves a validation accuracy of no less than 91.83% on the public dataset. Even under complex background conditions, the average accuracy of the proposed approach reaches 92.00% for the class prediction of rice plant images. Experimental results demonstrate the validity of the proposed approach, and it is accomplished efficiently for plant disease detection.