What are the limitations of using a simple feedforward neural network as a generative model?

What are the limitations of using a simple feedforward neural network as a generative model?