Are Generative Models Mugging Parrots?

#AI #Generative_Models

Some say these complex deep learning models are mere stochastic mugging parrots. Well, what is the answer? Yes or not exactly. Let’s argue about it.

If you take one or two layers in the learning model, then you can say that yes there is some kind of mugging. But here in live models, the system works with many linear or non-linear activations (as the case may be). Hence, apart from summation, non-linear functions are applied, which takes the system to the next level. Apart from that error correction is applied, which may be MSE/loss functions, in any case, this combination doesn’t work like a mugging engine.

These are complex systems that can be modeled using higher-dimensional calculus. The higher dimension comes from the number of variables and the activations employed.

More needs to be done to replicate the model, understand it and decipher the relationships between the input and outputs.

This certainly is a black box problem right now. But can we mold it? Can we have a white box model for each deep learning problem, generative or non-generative? If it was a mugging parrot, then it would not be a black box problem.

Can we debug deep-learning models? The answer is again if you consider this as a stochastic mugging parrot, then you can debug it, right? But till now we can’t replicate it or debug it. This proves another point.

How can we represent this solution using a pictorial representation of functions? What would pictorial representation look like when it’s a mugging machine, meaning the same input — output relations?

In some cases yes, when there is an exact match, then output would be best represented as that given in input. Hence, the model behaves like a mugging parrot, but see for cases when the input is not the one given in training data. That means when some of the testing data points and the training data are the same then the output can be the one given in input, hence a mugging engine, but what about unknown inputs — No more a mugging parrot!?