Error unpacking new BFLAN Nintendo Switch

[Migushthe2nd]

I don't want to overload you with issues, but here is another error.
The common.szs (another system file that contains info just like ResidentMenu.szs) also comes with an animation folder. Some of these animations can be opened in Wexos and extracted by your kit, like SystemColorFader_In.bflan, but others, like SystemColorFader_Color.bflan, result in an error. This so far only happens with files that (I assume) contain color information: BgNml_Color.bflan, BlurBg2_Color.bflan, CntrIndicatorS_Color.bflan, SystemColorFader_Color.bflan etc.
SystemColorFader_In.bflan:
https://drive.google.com/open?id=1YaYKFYTffGlzWl7kkRnddsEwO7aUiWm_

--------SystemColorFader_Color.bflan--------
No compression
BFLAN file found
Extracting...
Traceback (most recent call last):
  File "3DSkit.py", line 249, in <module>
    result = main(args, opts)
  File "3DSkit.py", line 169, in main
    extract_files(filename, args.big, args.format, args.verbose, opts)
  File "3DSkit.py", line 112, in extract_files
    unpack.extract(filename, file, format, endian, verbose, opts)
  File "S:\Mijn Drive\Things\NSW\Themes\Very nice theme\tools\3DSkit\unpack\__init__.py", line 16, in extract
    unpacker = cls(*args)
  File "S:\Mijn Drive\Things\NSW\Themes\Very nice theme\tools\3DSkit\util\utils.py", line 18, in __new__
    return self.main(*args, **kwargs)
  File "S:\Mijn Drive\Things\NSW\Themes\Very nice theme\tools\3DSkit\unpack\BFLAN.py", line 41, in main
    self.extract_sections(ptr)
  File "S:\Mijn Drive\Things\NSW\Themes\Very nice theme\tools\3DSkit\unpack\BFLAN.py", line 63, in extract_sections
    method(ptr)
  File "S:\Mijn Drive\Things\NSW\Themes\Very nice theme\tools\3DSkit\unpack\BFLAN.py", line 131, in readpai1
    data = self.unpack_from(pai1_TAG, self.file, tagpos)
  File "S:\Mijn Drive\Things\NSW\Themes\Very nice theme\tools\3DSkit\util\rawutil.py", line 101, in unpack_from
    unpacked, ptr = _unpack(stct, data, offset, byteorder, refdata)
  File "S:\Mijn Drive\Things\NSW\Themes\Very nice theme\tools\3DSkit\util\rawutil.py", line 58, in __new__
    return ins.main(*args, **kwargs)
  File "S:\Mijn Drive\Things\NSW\Themes\Very nice theme\tools\3DSkit\util\rawutil.py", line 317, in main
    return self.unpack_file(stct)
  File "S:\Mijn Drive\Things\NSW\Themes\Very nice theme\tools\3DSkit\util\rawutil.py", line 415, in unpack_file
    final.append([self.unpack_file(el.stct)[0] for i in range(count)])
  File "S:\Mijn Drive\Things\NSW\Themes\Very nice theme\tools\3DSkit\util\rawutil.py", line 415, in <listcomp>
    final.append([self.unpack_file(el.stct)[0] for i in range(count)])
  File "S:\Mijn Drive\Things\NSW\Themes\Very nice theme\tools\3DSkit\util\rawutil.py", line 467, in unpack_file
    final += struct.unpack(substruct, subdata)
struct.error: unpack requires a buffer of 4 bytes

Idk if Wexos' error will add anything, but I'll include it anyway

Some files return a bit different error, like FooterBtn_TextColor.bflan
FooterBtn_TextColor.bflan:
https://drive.google.com/open?id=1viUh1Xuy_WFCtLbvjWfljpKfbBaBt8Qm

--------FooterBtn_TextColor.bflan--------
No compression
BFLAN file found
Extracting...
Traceback (most recent call last):
  File "3DSkit.py", line 249, in <module>
    result = main(args, opts)
  File "3DSkit.py", line 169, in main
    extract_files(filename, args.big, args.format, args.verbose, opts)
  File "3DSkit.py", line 112, in extract_files
    unpack.extract(filename, file, format, endian, verbose, opts)
  File "S:\Mijn Drive\Things\NSW\Themes\Very nice theme\tools\3DSkit\unpack\__init__.py", line 16, in extract
    unpacker = cls(*args)
  File "S:\Mijn Drive\Things\NSW\Themes\Very nice theme\tools\3DSkit\util\utils.py", line 18, in __new__
    return self.main(*args, **kwargs)
  File "S:\Mijn Drive\Things\NSW\Themes\Very nice theme\tools\3DSkit\unpack\BFLAN.py", line 41, in main
    self.extract_sections(ptr)
  File "S:\Mijn Drive\Things\NSW\Themes\Very nice theme\tools\3DSkit\unpack\BFLAN.py", line 63, in extract_sections
    method(ptr)
  File "S:\Mijn Drive\Things\NSW\Themes\Very nice theme\tools\3DSkit\unpack\BFLAN.py", line 148, in readpai1
    tagentry['type2'] = self.gettype(type2, type)
  File "S:\Mijn Drive\Things\NSW\Themes\Very nice theme\tools\3DSkit\unpack\BFLAN.py", line 161, in gettype
    el = eval('%s_types[type]' % tagtype)
ValueError: source code string cannot contain null bytes

And here's Wexos' error for that one:

Edit: I'm adding some more info

I have executed the unpack command for every file.
Here's a hastebin: https://hastebin.com/direwirose.rb
I noticed that not every color file, BgNml_Color.bflan and BlurBg2_Color.bflan, results in an error when unpacking.
Oh whoops, I forgot that unpacking BFLAN's is only partially supported and they cannot be repacked yet.

[Tyulis]

No problem, I'll take a look.
And yep, the BFLAN format is not the most well-known, so 3DSkit cannot read them completely. But I'll try to work on it after BFLYT

[Tyulis]

Okay, now that the BFLYT modules work fine, let's try on BFLAN !
I'll try to do the same thing as for BFLYT, a new and better implementation (BFLYT and BFLAN modules were in the first modules of 3DSkit...). The only problem is that the BFLAN format is not well documented at all.

[Tyulis]

I finally managed to make something usable, so you can test

[Migushthe2nd]

It indeed works, but it's still missing stuff. Great job so far!

First and only error, but those are those color files I was talking about in the first post:

BFLAN file found
Extracting...
Format version : 8.6.0.0
Number of sections : 2
Traceback (most recent call last):
  File "..\3DSkit.py", line 249, in <module>
    result = main(args, opts)
  File "..\3DSkit.py", line 169, in main
    extract_files(filename, args.big, args.format, args.verbose, opts)
  File "..\3DSkit.py", line 112, in extract_files
    unpack.extract(filename, file, format, endian, verbose, opts)
  File "S:\Mijn Drive\Things\NSW\Themes\Very nice theme\tools\3DSkit-master\unpack\__init__.py", line 16, in extract
    unpacker = cls(*args)
  File "S:\Mijn Drive\Things\NSW\Themes\Very nice theme\tools\3DSkit-master\util\utils.py", line 18, in __new__
    return self.main(*args, **kwargs)
  File "S:\Mijn Drive\Things\NSW\Themes\Very nice theme\tools\3DSkit-master\unpack\BFLAN.py", line 32, in main
    self.readdata(data)
  File "S:\Mijn Drive\Things\NSW\Themes\Very nice theme\tools\3DSkit-master\unpack\BFLAN.py", line 71, in readdata
    name, node = self.readsection(data, magic, startpos)
  File "S:\Mijn Drive\Things\NSW\Themes\Very nice theme\tools\3DSkit-master\unpack\BFLAN.py", line 79, in readsection
    name, node = self.readpai1(data, startpos)
  File "S:\Mijn Drive\Things\NSW\Themes\Very nice theme\tools\3DSkit-master\unpack\BFLAN.py", line 116, in readpai1
    tagentryoffsets = self.unpack_from('%dI' % tagentrynum, data)
  File "S:\Mijn Drive\Things\NSW\Themes\Very nice theme\tools\3DSkit-master\util\rawutil.py", line 101, in unpack_from
    unpacked, ptr = _unpack(stct, data, offset, byteorder, refdata)
  File "S:\Mijn Drive\Things\NSW\Themes\Very nice theme\tools\3DSkit-master\util\rawutil.py", line 58, in __new__
    return ins.main(*args, **kwargs)
  File "S:\Mijn Drive\Things\NSW\Themes\Very nice theme\tools\3DSkit-master\util\rawutil.py", line 317, in main
    return self.unpack_file(stct)
  File "S:\Mijn Drive\Things\NSW\Themes\Very nice theme\tools\3DSkit-master\util\rawutil.py", line 467, in unpack_file
    final += struct.unpack(substruct, subdata)
struct.error: unpack requires a buffer of 5722419480 bytes```

[Tyulis]

Okay, that's another weird version difference, because all of my 7.2.1 files worked.
Can you send one or two of these files please ?
Thanks !

[Migushthe2nd]

The FooterBtn_textColor is one of the files that doesn't work. Link is in the first post.
But here you go, another file HudSignal_Color:
https://drive.google.com/file/d/1KGyb0eI4-GaBL6YR0gXfDcegVDBl9tZv/view?usp=sharing

[Migushthe2nd]

Whoops, wrong link. I updated it in the last post

[Tyulis]

Ok, many stuff was missing because of a bad structure.
Then, is there any working file with the current commit ? I found the issue, but as it is completely incompatible with other versions it would be very strange if other similar files were processed fine.

[Migushthe2nd]

Yeah there are many files working. 192 out of 207 in the ResidentMenu

[Migushthe2nd]

(ResidentMenu bflans)

[Tyulis]

Thank you very much !
So it is not a version issue but something not implemented yet.
BTW I found some documentation about the BRLAN format (the Wii's one), and thanks to that I was able to make certain values more explicit, and to complete a few things.

[Tyulis]

Okay, now it seems to work, and with more data than before.
BTW, if you have any idea about the meaning of the "EU" in "FLEU", I'd be glad

[Migushthe2nd]

I don't really know the context, so not really. Would you know what FLEU does? If not I'll just have to try it out on my Switch and see what changes.
Edit: I'm dumb: can't repack it yet

[Tyulis]

So, I guess that HudTime_Color.bflan has something to do with some kind of clock on the HUD.
If it doesn't annoy you, can you try to replace it with these hand-modified files, to see what kinds of effects it does ? Warning, the probability that the one in unknown value will provoke a crash is quite high, I won't blame you if you prefer not to try, I don't know the consequences of a crash due to a theme.

Another time, thanks for your help !

EDIT : I say a crash, but it also may be completely ignored if there are errors, I don't know

[Migushthe2nd]

The first two files don't seem to change anything, but the unknown value indeed crashes. Perhaps try modifying the BatteryConsole_Color. Maybe the HudTime_Color is forced to always be white or something like that. The BatteryConsole changes color with normal behaviour like plugging in the charger in.
Edit oh BatteryConsole_Color is something else, BatteryConsole_BatteryColor changes the color when charging if I'm correct, but you can still try

[Tyulis]

Ok, I made similar edits but on BatteryConsole_Color.bflan

BFLAN.zip

[Migushthe2nd]

Btw. the name C_Id in BatteryConsole_Color is also in many bflyts. It says something about the data of a panel with a picture. https://hastebin.com/paxetuvami.bash Under usd1 - entries there's C_Id.

{
    "name": "C_Id",
    "type": 1,
    "unk1": 0,
    "data": [
        1
    ]
},

data is either a 1 with "usd1 - L_<random (what is L?)>" or a 0 with "usd1 - P_<random picture?>"

[Migushthe2nd]

Ok, I made similar edits but on BatteryConsole_Color.bflan
BFLAN.zip

Same as last time: doesn't seem like anything has changed

[Migushthe2nd]

Maybe the E stands for Entry? Or External
Probably not..

[Tyulis]

Theorically, it defines the animated data, like Vertex Colors for FLVC, VIsibility for FLVI, ...
So why not External (...), if it is linked with usd1s

[Tyulis]

Anyway, the unpack.BFLAN module seems to work fine, so let's start the packing one

[Migushthe2nd]

I found it out, the EU stands for 'Extended User Information'.
In total there are 10 animation types (These are the official names):

1. Pane SRT
2. Visibility
3. Vertex Color / Transparency
4. Material Color / Text Shadows
5. Alpha Test
6. Texture Pattern
7. Texture SRT
8. Per-character transformation offset time
9. Extended User Information (String, Integer value list or Real number value list)
10. Indirect

Btw. are you on Discord?

[Tyulis]

Thank you ! I'll include this in the doc and the module
And yes, I'm on discord