Error on opening co-op replays in 1.6.0

[jcfieldsdev]

Using the latest version of sc2reader available on PIP, sc2reader chokes when opening co-op replays that use factions besides the standard multiplayer ones. I did not encounter this error previously on version 1.5.0.

Here is a sample replay.

Here is a simple test script that uses that replay to demonstrate the issue:

#!/usr/bin/env python3

import sc2reader

replay = sc2reader.load_replay('./Miner Evacuation (64).SC2Replay', load_level=2)

It gives me the following error when executed:

Traceback (most recent call last):
  File "./test.py", line 5, in <module>
    replay = sc2reader.load_replay('./Miner Evacuation (64).SC2Replay', load_level=2)
  File "/Users/jcfields/Library/Python/3.8/lib/python/site-packages/sc2reader/factories/sc2factory.py", line 85, in load_replay
    return self.load(Replay, source, options, **new_options)
  File "/Users/jcfields/Library/Python/3.8/lib/python/site-packages/sc2reader/factories/sc2factory.py", line 143, in load
    return self._load(cls, resource, filename=filename, options=options)
  File "/Users/jcfields/Library/Python/3.8/lib/python/site-packages/sc2reader/factories/sc2factory.py", line 152, in _load
    obj = cls(resource, filename=filename, factory=self, **options)
  File "/Users/jcfields/Library/Python/3.8/lib/python/site-packages/sc2reader/resources.py", line 301, in __init__
    self._read_data(data_file, self._get_reader(data_file))
  File "/Users/jcfields/Library/Python/3.8/lib/python/site-packages/sc2reader/resources.py", line 902, in _read_data
    self.raw_data[data_file] = reader(data, self)
  File "/Users/jcfields/Library/Python/3.8/lib/python/site-packages/sc2reader/readers.py", line 303, in __call__
    result = [
  File "/Users/jcfields/Library/Python/3.8/lib/python/site-packages/sc2reader/readers.py", line 304, in <listcomp>
    Attribute(
  File "/Users/jcfields/Library/Python/3.8/lib/python/site-packages/sc2reader/objects.py", line 80, in __init__
    self.value = lookup[value.strip("\x00 ")[::-1]]
KeyError: 'InfT'

The problem seems to be that the "InfT" (Infested Terran) race used on the Miner Evacuation mission is not in the "attributes.json" file (which only has the standard Terran, Zerg, Protoss, and Random), so its presence in the replay file causes a key look-up error.

I looped through all of my co-op replays and found the following non-standard races that cause read errors:

• InfT
• ProZ
• PZrg
• TerH
• TerT

I don't know if this is an exhaustive list, but it's a pool of over 1,000 co-op replays.

I would guess that either adding these races to "attributes.json" or implementing some sort of fallback to prevent a fatal error if a non-standard race can't be identified would fix this issue.

Edit: Using sc2reader 1.6.0, Python 3.8.5, Mac OS 10.14.6.

[StoicLoofah]

Thanks for sharing the detailed report.

Just to confirm, you said this wasn't happening on 1.5.0 for you, right? I tried running this same replay on v1.5.0 2ee6974 and got this

>>> import sc2reader
>>> sc2reader.load_replay('./github-126.SC2Replay')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/kevin/sc2reader/sc2reader/factories/sc2factory.py", line 85, in load_replay
    return self.load(Replay, source, options, **new_options)
  File "/home/kevin/sc2reader/sc2reader/factories/sc2factory.py", line 143, in load
    return self._load(cls, resource, filename=filename, options=options)
  File "/home/kevin/sc2reader/sc2reader/factories/sc2factory.py", line 152, in _load
    obj = cls(resource, filename=filename, factory=self, **options)
  File "/home/kevin/sc2reader/sc2reader/resources.py", line 301, in __init__
    self._read_data(data_file, self._get_reader(data_file))
  File "/home/kevin/sc2reader/sc2reader/resources.py", line 893, in _read_data
    self.raw_data[data_file] = reader(data, self)
  File "/home/kevin/sc2reader/sc2reader/readers.py", line 274, in __call__
    for i in range(data.read_bits(5))
  File "/home/kevin/sc2reader/sc2reader/readers.py", line 274, in <listcomp>
    for i in range(data.read_bits(5))
  File "/home/kevin/sc2reader/sc2reader/readers.py", line 212, in <listcomp>
    for i in range(
  File "/home/kevin/sc2reader/sc2reader/decoders.py", line 210, in read_uint32
    data = self._buffer.read_uint32()
  File "/home/kevin/sc2reader/sc2reader/decoders.py", line 91, in read_uint32
    return self._unpack_int(self.read(4))[0]
struct.error: unpack requires a buffer of 4 bytes

Obviously we still need to fix this, but I just want to make sure we're seeing the same regression

[jcfieldsdev]

Yes, that's correct, I got the same error when trying to open 5.0.x co-op replays in sc2reader 1.5.0 (though 4.x.x replays worked fine, so I assume it was due to whatever changes in the replay format were introduced by that update).

[StoicLoofah]

Interesting. It looks like sc2reader/data/attributes.json may be woefully out-of-date. We updated it by hand based on easily discoverable properties, but we probably need to generate it from Blizzard data again.

Thankfully, the original maintainer wrote a script to do this for us fe2a376

The next step would be to get a .s2gs file, with instructions for that at https://tl.net/forum/starcraft-2/330926-s2gs-file-a-fun-little-mystery-for-us

Unfortunately I don't have access to the StarCraft 2 client at this very moment to do this, but I'll give it a shot later when I do

[StoicLoofah]

Potential fix in #129. Does that work for you?

[jcfieldsdev]

Yup, that did it, scanned 1,139 co-op replays with no errors. Thanks!