Originally posted to HuggingFace by ai-characters

Disclaimer

None of the images that were used to train this or any of my other models were created by me or are owned by me.

If you are the owner of some of the images and would like them removed from the model, contact me (AIGeneratedCharacters@mailbox.org) and I will train a new version of the model in question without them included. The old model and dataset version will then be deleted. A public link to the dataset used for this model can be found down below on this page.

Social Media

• kofi
• instagram
• twitter

I appreciate any donations to my KoFi to help me fund renting GPU's for further model creation and experimentation! I experiment a lot to achieve the highest possible model quality and flexbility, but this much experimentation requires me to train a lot of different models which in turn costs me a lot in GPU renting costs. So any amount of donations would help me greatly to offset some of the costs!

Tips and TricksI recommend using a UI that supports high-res fix, Img2Img, and negative prompts at the very least. Personally I use the "Automatic1111 WebUI".

• Rescale to a 1024x1024 resolution using either Img2Img or high-res fix at a 0.5 denoising strength for much better, higher quality results.
• 
  
• Use "in the artstyle of dd" as a negative prompt, if you want to generate images that are closer to what the standard StableDiffusion model would generate. Be mindful however, that due to the nature of how this model was trained the output generated via this way will still look different from the original vanilla model.
• Use "armor" and "medieval" as negative prompts to achieve more modern looking results. No guruantee that it will always work however. At times it may also reduce the output quality.
• 
  
• If you notice recurring elements in your outputs, such as every output image having sunglasses in it, make said recurring element a negative prompt. No guruantee that it will work.

Recommended settings

These are my recommended settings for quick high quality results. These are also the settings on which I tested the model.

• Sampler: Euler_a
• Steps: 20
• CFG: 7
• Resolution: 512x512, then rescaled to 1024x1024 using either high-res fix or Img2Img

Prompts and settings used to generate the examples found in the thumbnail of this page, in order

• facial closeup artwork in the artstyle of dd of a princess Steps: 20, Sampler: Euler a, CFG scale: 7, Seed: 1199794856, Size: 1024x1024, Model hash: 6ae1720a, Batch size: 2, Batch pos: 0, Denoising strength: 0.5, First pass size: 512x512
• character art in the artstyle of dd of Supergirl Steps: 20, Sampler: Euler a, CFG scale: 7, Seed: 2738426192, Size: 1024x1024, Model hash: 6ae1720a, Batch size: 2, Batch pos: 0, Denoising strength: 0.5, First pass size: 512x512
• (full-body full-length full-shot:1.3) character art in the artstyle of dd of (Zendaya:1.4) wearing helmetless armor and a cape and holding one spear Negative prompt: (helmet), (half-body, half-length, medium shot), man, male, cloak, hoodie Steps: 20, Sampler: Euler a, CFG scale: 7, Seed: 229405137, Size: 1024x1024, Model hash: 6ae1720a, Batch size: 2, Batch pos: 0, Denoising strength: 0.5, First pass size: 512x512
• artwork in the artstyle of dd of Emma Watson standing in a street in New York City wearing a tshirt and jeans Negative prompt: sunglasses Steps: 20, Sampler: Euler a, CFG scale: 7, Seed: 2287424046, Size: 1024x1024, Model hash: 6ae1720a, Batch size: 2, Batch pos: 0, Denoising strength: 0.5, First pass size: 512x512
• artwork in the artstyle of dd of a winged fairy with butterfly wings in a (sunny forest clearing:1.3) wearing fancy fairy dress happy light atmosphere Negative prompt: armor Steps: 20, Sampler: Euler a, CFG scale: 7, Seed: 761962005, Size: 1024x1024, Model hash: 6ae1720a, Batch size: 2, Batch pos: 0, Denoising strength: 0.5, First pass size: 512x512
• artwork in the artstyle of dd of a winged fairy with butterfly wings (in a dark forest clearing:1.3) wearing fancy fairy dress dark moody atmosphere Negative prompt: armor Steps: 20, Sampler: Euler a, CFG scale: 7, Seed: 2510528777, Size: 1024x1024, Model hash: 6ae1720a, Batch size: 2, Batch pos: 0, Denoising strength: 0.5, First pass size: 512x512
• character art in the artstyle of dd of Ellie from Last of Us with feathered wings Negative prompt: armor, 3d video game render, man, rifle, gun, holding Steps: 20, Sampler: Euler a, CFG scale: 7, Seed: 1202360977, Size: 1024x1024, Model hash: 6ae1720a, Batch size: 2, Batch pos: 1, Denoising strength: 0.5, First pass size: 512x512
• artwork in the artstyle of dd of the animal pig Negative prompt: human, people, person, humanoid, armor, clothing, soldier, knight Steps: 20, Sampler: Euler a, CFG scale: 7, Seed: 1357521697, Size: 1024x1024, Model hash: 6ae1720a, Batch size: 2, Batch pos: 1, Denoising strength: 0.5, First pass size: 512x512
• landscape art in the artstyle of dd of a mountain valley with a river flowing through Negative prompt: house, building, houses, buildings, city, town, civilization Steps: 20, Sampler: Euler a, CFG scale: 7, Seed: 1359049560, Size: 1024x1024, Model hash: 6ae1720a, Batch size: 2, Batch pos: 0, Denoising strength: 0.5, First pass size: 512x512
• landscape art in the artstyle of dd of a modern city with skyscrapers Negative prompt: medieval Steps: 20, Sampler: Euler a, CFG scale: 7, Seed: 3002591719, Size: 1024x1024, Model hash: 6ae1720a, Batch size: 2, Batch pos: 0, Denoising strength: 0.5, First pass size: 512x512
• artwork in the artstyle of dd of a car Steps: 20, Sampler: Euler a, CFG scale: 7, Seed: 856392436, Size: 1024x1024, Model hash: 6ae1720a, Batch size: 2, Batch pos: 0, Denoising strength: 0.5, First pass size: 512x512
• artwork in the artstyle of dd of a modern car Steps: 20, Sampler: Euler a, CFG scale: 7, Seed: 1281304113, Size: 1024x1024, Model hash: 6ae1720a, Batch size: 2, Batch pos: 1, Denoising strength: 0.5, First pass size: 512x512

Current shortcomings of the model

• The model is "infected" by the training, which means that using it to generate images not in the "dd artstyle" will generate output that will be different from what you would get in the original Stable Diffusion model
• 
  
• The model may struggle at times with more complex prompts
• The model is better at simple character art than it is at anything else (e.g. landscapes), though you can still use it for those other art types
• 
  
• No unique characters, classes, locations, or items of the game have been trained or captioned and thus cannot be deliberately prompted to appear

How I created this model and the underlying dataset (+ dataset download link!)

I wanted to test various different methods of captioning images as well as training on various different resolutions. For that I needed a model that can be created using only a small dataset so that testing times and expenses are kept low. Thus I randomly came up with this Darkest Dungeon model, which uses only 14 training images (initally only 11).

I took the images from google image search as well as Riot Pixels. I chose only the most cleanest and highest quality images. I then further edited the images to take out watermarks and texts as well as remove unneccessary noise in the image. I also cropped the images a bit.

I created many different models to complete my testing and to find the most optimal settings and caption style to create a Darkest Dungeon model with the highest possible quality.

The captioned and edited dataset that I used to train this model with can be found here!

I trained this model using the "EverDream" repo with filename captions. I captioned the images using the BatchPrompter.

For the training I used the v1-finetune_everydream.yaml file with the following settings:

• Learning rate: 1e-6
• Repeats: 250
• Epochs: 1
• Batch size: 1
• Conditional dropout: 0.05
• Resolution: 512

For a more in-depth guide on how to actually setup and use the EverDream repo, please look up other tutorials on the web, especially YouTube or Reddit, or visit the EverDream Discord!

License

This model is open access and available to all, with a modified CreativeML OpenRAIL-M license for non-commercial use only further specifying rights and usage. This modified CreativeML OpenRAIL License specifies:

1. You cannot use this model to generate so-called "NFTs" or "Non-fungible Tokens"
2. You can't host or use the model or its derivatives or inference or finetuning based on those on websites/apps/etc., from which you earn, will earn, or plan to earn revenue, unless you offer those services for the model in question for free.
3. You are free to host the model or its derivatives on completely non-commercial websites/apps/etc. Please state the full model name (Darkest-Diffusion v1.0) and include a link to the model card (https://huggingface.co/ai-characters/Darkest-Diffusion).
4. You can't use the model to deliberately produce nor share illegal or harmful outputs or content
5. The authors claims no rights on the outputs you generate, you are free to use them and are accountable for their use which must not go against the provisions set in the license
6. You may re-distribute the weights. If you do, please be aware you have to include the same use restrictions as the ones in the license and share a copy of this modified CreativeML OpenRAIL-M for non-commercial use only to all your users (please read the license entirely and carefully) Please read the full license here