normalmap token
I have been trying to use the topdown base lora and it works for the colormap and ambmap generation. However the normalmap token does not appear to trigger anything. Wanted to reach out and see if there was a different token for the topdown base lora to generate normal maps compared to the 3d lora?
Yeah not much to say here, normal maps just kinda suck for SDXL. The model has no idea what they are in the first place, and so all my attempts at adding them in just results in sorta weird looking lighting. It might be fixable with a proper fine-tune, I kinda half-addressed in my project thesis (resulting In the 3D version of the model), but, either way, throughout all of my testing it seems that normals only work correctly if you combine them with some form of controlnet, because otherwise the model just hallucinates whatever garbage it can scoop out of the latent space.
Also, the model was trained with sorta long-form captions, so phrases like "normal map", "normal texture", etc contribute to the output. Since the SDXL version of this project is long dead at this point, I have uploaded some of the crappy code I used to label and inference the model to github (https://github.com/dog-god-rus/sdxl-texture-synthesis). There you can see some extra examples of how prompting is performed and the steps needed to do full pbr inference.
Though frankly considering the state of that codebase I'd rather stay away.