Caricature portraits diffusion model

Stable Diffusion v1.5 fine tuned on the 2D Caricature Dataset from 3D-CariGAN cropped to 512x512 and blip captioned. If you want more details on how to generate your own blip captioned dataset see this colab

Training was done using this Hugging-Face's text to image training script

Dataset and Training

Finetuned for 10,000 iterations upon runwayml/stable-diffusion-v1-5 on BLIP captioned portraits portraits using 1xA5000 GPU on my home desktop computer

Trained by @norod78

The original Caricature dataset citation credits

@article{ye2021caricature,
 author = {Ye, Zipeng and Xia, Mengfei and Sun, Yanan and Yi, Ran and Yu, Minjing and Zhang, Juyong and Lai, Yu-Kun and Liu, Yong-Jin},
 title = {3D-CariGAN: An End-to-End Solution to 3D Caricature Generation from Normal Face Photos},
 journal = {IEEE Transactions on Visualization and Computer Graphics},
 year = {2021},
 doi={10.1109/TVCG.2021.3126659},
}