Support more LoRA models (Flux and SD3.x)

[stduhpf]

Needs more testing.

Seems to work on CPU and Vulkan (unquantized) backends, on my machine at least.

Please do some testing an let me know what works and what doesn't.

Should fix #370 and fix #483

[Green-Sky]

Nice, good job @stduhpf , you have been coding like a machine :)

[stduhpf]

Nice, good job @stduhpf , you have been coding like a machine :)

Thanks, I'll probably get bored out of it eventually, but so far so good.

[stduhpf]

I was testing SD3 support with https://huggingface.co/ByteDance/Hyper-SD/tree/main?show_file_info=Hyper-SD3-8steps-CFG-lora.safetensors. I can't wrap my head around what's going on with pos_embed, the sizes don't match at all the full model...
Well it seems to work without it, but I would like to understand still.

[stduhpf]

Welp, I've already found a SD3 Lora that still doesn't work with these changes.
I think I'll need to refactor the whole thing

[stduhpf]

Ok I think Lora compatibility just got a lot better, but I still found some loras with very weird naming conventions that I don't quite understand so far.

[stduhpf]

I think this broke previously working LoRAs