SDXL ?????

Another SDXL LoRA for more ????????????????, this time it is for ?????? and ??? toys. This was a difficult concept for SD 1.5 to master, so we will see how XL can handle it.

• v0.3:

  • Better support for anime and cartoon/render/pixar styles.

  • More variety of lingerie and clothing, not always with ??????? out.

  • Works at 0.8-1.0 strength, not as overtrained.

• v0.2:

  • Epoch 93 and 99 are both pretty good, using 0.7 - 0.8 strength.

• v0.1:

  • In general, I think epoch 74 has the best results, using about 0.75 strength.

Learned Keywords:

• primary:

  • dildo for v0.3

  • sexl_dildo for v0.1 and v0.2

• others:

  • dildo: 1410

  • 1girl: 1254

  • pussy: 1114

  • solo: 1099

  • realistic: 1015

  • sex toy: 864

  • object insertion: 837

  • nude: 674

  • ass: 634

  • breasts: 610

  • masturbation: 603

  • anus: 569

  • female ????????????: 563

  • nipples: 549

  • spread legs: 534

  • long hair: 524

  • pubic hair: 510

  • vaginal: 505

  • vaginal object insertion: 488

  • female pubic hair: 424

  • lips: 360

  • lying: 354

  • brown hair: 349

  • anal: 338

  • blonde hair: 317

  • black hair: 317

  • anal object insertion: 314

Complete list of tag frequencies: https://gist.github.com/onnx-nsfw/4bd77d82a0354853bec14bc667b1ddd1

Example Captions:

• dildo, solo, ???, pantyhose, ????, ?????, indoors, ???????????, ???????, fingernails, uncensored, ????, ??? toy, ????, close-up, ????????????, female ????????????, object insertion, realistic, ?????, ?????????, ???? object insertion, ??????? object insertion, lower body, gaping, large insertion, ????? riding

• dildo, solo, ???, pantyhose, ?????, ???????????, ???????, uncensored, ????, ??? toy, ????, close-up, ????????????, female ????????????, object insertion, realistic, ?????, ?????????, ???? object insertion, ??????? object insertion, lower body, gaping, large insertion, ????? riding

• dildo, solo, ???, ????, ?????, ???????????, ???????, fingernails, uncensored, ????, ??? toy, ????, close-up, ????????????, female ????????????, object insertion, realistic, ?????, ?????????, ???? object insertion, ??????? object insertion, ????? riding

• dildo, solo, thighhighs, ?????????, panties, ???, ???????????, bra, black panties, back, ????????? only, lingerie, black bra, close-up, garter belt, head out of frame, ??? focus

• dildo, solo, thighhighs, ?????????, panties, ???, ???????????, fishnets, ??? toy, ????, garter belt, object insertion, ?????, ?????????, fishnet pantyhose, ???? object insertion, fishnet thighhighs

Training Method:

Testing this method: https://civitai.com/articles/1771. Versions 0.1 and 0.2 used the 100 epoch method with 2e-05 learning rate.

• v0.3:

  • 1458 source images (> 768 in shortest dimension)

  • 1588 random crops (1024x1024)

  • 1410 remaining after curation

    • up to 12 crops per source

    • downscaled shortest dimension to 1024

    • deduplicated with resnet 18, cosine similarity, 0.925 threshold

    • deleted partial and out of frame

    • no watermarks

  • automated captions with WD 1.4, threshold 0.45

    • incorrect tags removed

  • 2 repeats

  • 8 batch size

  • 100 epochs

  • 32 dim/32 alpha

  • AdamW8bit with constant

  • 2e-05 learning rate for both text encoder and UNet

  • shuffle captions

  • caption dropout rate 0.05

  • stop text encoder training after 30% (still not sure if this feature is implemented)

• v0.2:

  • 344 source images

  • 287 after deduplication and resize

  • 1087 random crops

  • 806 remaining after curation

    • same method as v0.1

  • automation captions with WD 1.4, threshold 0.6

    • lightly curated, incorrect tags removed

  • 3 repeats

  • 8 batch size

  • 100 epochs

  • 32 dim/32 alpha

  • AdamW8bit with constant

  • 2e-05 learning rate for both text encoder and UNet

  • shuffle captions

• v0.1:

  • about 500 source images (> 1024x1024)

  • about 1000 random crops (1024x1024)

  • 440 remaining after curation

    • deduplicated with resnet 18, cosine similarity, 0.9 threshold

    • deleted partial and out of frame

    • no watermarks

  • 1 repeats

  • 8 batch size

  • 100 epochs (models around 45-50 and 70-75 were best)

  • 32 dim/32 alpha

  • AdamW8bit with constant

  • 2e-05 learning rate for both text encoder and UNet

I am happy to share the data set but do not have a good place for hosting those.

• v0.3:

  • Includes more lingerie and clothing

• v0.2:

  • Next time I need to indicate the color of the ????? in the caption and include more of the image poses.

• v0.1:

  • Next time I think I will try removing the sexl_dildo keyword and specializing the existing ????? token.

Training Parameters:

{
  "LoRA_type": "Standard",
  "adaptive_noise_scale": 0,
  "additional_parameters": "",
  "block_alphas": "",
  "block_dims": "",
  "block_lr_zero_threshold": "",
  "bucket_no_upscale": false,
  "bucket_reso_steps": 64,
  "cache_latents": true,
  "cache_latents_to_disk": true,
  "caption_dropout_every_n_epochs": 0.0,
  "caption_dropout_rate": 0.05,
  "caption_extension": ".txt",
  "clip_skip": "1",
  "color_aug": false,
  "conv_alpha": 1,
  "conv_block_alphas": "",
  "conv_block_dims": "",
  "conv_dim": 1,
  "decompose_both": false,
  "dim_from_weights": false,
  "down_lr_weight": "",
  "enable_bucket": false,
  "epoch": 100,
  "factor": -1,
  "flip_aug": true,
  "full_bf16": false,
  "full_fp16": false,
  "gradient_accumulation_steps": "1",
  "gradient_checkpointing": true,
  "keep_tokens": "0",
  "learning_rate": 2e-05,
  "logging_dir": "",
  "lora_network_weights": "",
  "lr_scheduler": "constant",
  "lr_scheduler_args": "",
  "lr_scheduler_num_cycles": "",
  "lr_scheduler_power": "",
  "lr_warmup": 0,
  "max_bucket_reso": 2048,
  "max_data_loader_n_workers": "0",
  "max_resolution": "1024,1024",
  "max_timestep": 1000,
  "max_token_length": "75",
  "max_train_epochs": "100",
  "max_train_steps": "",
  "mem_eff_attn": false,
  "mid_lr_weight": "",
  "min_bucket_reso": 256,
  "min_snr_gamma": 5,
  "min_timestep": 0,
  "mixed_precision": "fp16",
  "model_list": "custom",
  "module_dropout": 0,
  "multires_noise_discount": 0,
  "multires_noise_iterations": 0,
  "network_alpha": 32,
  "network_dim": 32,
  "network_dropout": 0,
  "no_token_padding": false,
  "noise_offset": 0,
  "noise_offset_type": "Original",
  "num_cpu_threads_per_process": 2,
  "optimizer": "AdamW8bit",
  "optimizer_args": "",
  "output_dir": "/mnt/training/results/dildo-v03-std/",
  "output_name": "last",
  "persistent_data_loader_workers": false,
  "pretrained_model_name_or_path": "stabilityai/stable-diffusion-xl-base-1.0",
  "prior_loss_weight": 1.0,
  "random_crop": false,
  "rank_dropout": 0,
  "reg_data_dir": "",
  "resume": "",
  "sample_every_n_epochs": 1,
  "sample_every_n_steps": 0,
  "sample_prompts": "beautiful woman playing with ?????,  detailed, 8k, long hair, ???????, open mouth, multiple girls, ???, ????, ?????, ??? toy, ????, object insertion, realistic, ?????, ???? object insertion --d 5 --w 1024 --h 1024 --l 9 --s 75",
  "sample_sampler": "euler_a",
  "save_every_n_epochs": 1,
  "save_every_n_steps": 0,
  "save_last_n_steps": 0,
  "save_last_n_steps_state": 0,
  "save_model_as": "safetensors",
  "save_precision": "fp16",
  "save_state": false,
  "scale_v_pred_loss_like_noise_pred": false,
  "scale_weight_norms": 0,
  "sdxl": true,
  "sdxl_cache_text_encoder_outputs": false,
  "sdxl_no_half_vae": true,
  "seed": "",
  "shuffle_caption": true,
  "stop_text_encoder_training_pct": 30,
  "text_encoder_lr": 2e-05,
  "train_batch_size": 8,
  "train_data_dir": "/mnt/training/cropped/dildo-v03",
  "train_on_input": true,
  "training_comment": "",
  "unet_lr": 2e-05,
  "unit": 1,
  "up_lr_weight": "",
  "use_cp": false,
  "use_wandb": false,
  "v2": false,
  "v_parameterization": false,
  "v_pred_like_loss": 0,
  "vae_batch_size": 0,
  "wandb_api_key": "",
  "weighted_captions": false,
  "xformers": "xformers"
}