I've done some work in diffusion models, which are still supposed to be state of the art for generative applications. There's a huge amount of hype for this stuff, which is really all hostage to useful training data. I had a lot of fun using Stable Diffusion, and think it would be nice to see diffusion models used for things like CAD.
the problem I see at the moment is that when the training data runs out, the models collapse swiftly into incoherence; and even when the LLMs are working fine, they keep throwing all the long and fat tails of the data, ie whatever is beyond 3 sigma. over time this will mean a Brave New World: you are only allowed to have 'approved' opinions.
isn't diffusion also liable to reinforce that issue by throwing away long and fat tails? I don't understand diffusion that well, so this is a genuine question. I have seen impressive art output from stable diffusion and dall-e, but can that be replicated over generations, or do the models deteriorate rapidly?
3 comments:
I've done some work in diffusion models, which are still supposed to be state of the art for generative applications. There's a huge amount of hype for this stuff, which is really all hostage to useful training data. I had a lot of fun using Stable Diffusion, and think it would be nice to see diffusion models used for things like CAD.
the problem I see at the moment is that when the training data runs out, the models collapse swiftly into incoherence; and even when the LLMs are working fine, they keep throwing all the long and fat tails of the data, ie whatever is beyond 3 sigma. over time this will mean a Brave New World: you are only allowed to have 'approved' opinions.
isn't diffusion also liable to reinforce that issue by throwing away long and fat tails? I don't understand diffusion that well, so this is a genuine question. I have seen impressive art output from stable diffusion and dall-e, but can that be replicated over generations, or do the models deteriorate rapidly?
Post a Comment