AIGC¶
Autoencoder¶
Ref: https://lilianweng.github.io/posts/2018-08-12-vae/
Video: https://www.youtube.com/watch?v=hZ4a4NgM3u0
Autoencoder
- Learn an identity function
- An unsupervised way
- Reconstruct the original input
- Efficient and compressed representation
- Encoder translates the original high-dimensional input into a low-dimensional code, like Principal Component Analysis (PCA) or Matrix Factorization (MF)
- Decoder recovers the data from the code
- Metrics: Cross Entropy, Mean Squared Error (MSE)
Denoising Autoencoder¶
- Risk of overfitting when #parameters > #data
- Partially corrupted by adding noises to or masking the input vector
- Motivated by the fact that humans can easily recognize an object or a scene even the view is partially occluded or corrupted
- Discover and capture relationship between dimensions
- Learning robust latent representation
Variational Autoencoder (VAE)¶
Vector Quantized Variational Autoencoder (VQ-VAE)¶
Generative Adversarial Network (GAN)¶
Ref: https://lilianweng.github.io/posts/2017-08-20-gan/
Diffusion Model¶
Denoising Diffusion Probabilistic Model (DDPM)¶
Latent Diffusion Model (LDM)¶
Ref: https://lilianweng.github.io/posts/2021-07-11-diffusion-models/