This paper presents a new on-line algorithm for creating a self-organizing fuzzy neural network (SOFNN) from sample patterns to implement a singleton or Takagi-Sugeno (TS) type fuzzy model. The SOFNN is based on ellipsoidal basis function (EBF) neurons consisting of a center vector and a width vector. New methods of the structure learning and the parameter learning, based on new adding and pruning techniques and a recursive on-line learning algorithm, are proposed and developed. A proof of the convergence of both the estimation error and the linear network parameters is also given in the paper. The proposed methods are very simple and effective and generate a fuzzy neural model with a high accuracy and compact structure. Simulation work shows that the SOFNN has the capability of self-organization to determine the structure and parameters of the network automatically.