How to load a HF model from a locally downloaded version and not a hub copy? #2133
Unanswered
CoffeeVampir3
asked this question in
Q&A
Replies: 1 comment
-
It looks as if there's no built in way, but I did hack something together: def load_cfg_from_json(json_file):
with open(json_file, "r", encoding="utf-8") as reader:
text = reader.read()
return json.loads(text)
def load_cfg(cfg_path):
hf_config = load_cfg_from_json(cfg_path)
if 'pretrained_cfg' not in hf_config:
# old form, pull pretrain_cfg out of the base dict
pretrained_cfg = hf_config
hf_config = {}
hf_config['architecture'] = pretrained_cfg.pop('architecture')
hf_config['num_features'] = pretrained_cfg.pop('num_features', None)
if 'labels' in pretrained_cfg: # deprecated name for 'label_names'
pretrained_cfg['label_names'] = pretrained_cfg.pop('labels')
hf_config['pretrained_cfg'] = pretrained_cfg
pretrained_cfg = hf_config['pretrained_cfg']
pretrained_cfg['hf_hub_id'] = hf_config["architecture"] # insert hf_hub id for pretrained weight load during model creation
pretrained_cfg['source'] = 'hf-hub'
# model should be created with base config num_classes if its exist
if 'num_classes' in hf_config:
pretrained_cfg['num_classes'] = hf_config['num_classes']
# label meta-data in base config overrides saved pretrained_cfg on load
if 'label_names' in hf_config:
pretrained_cfg['label_names'] = hf_config.pop('label_names')
if 'label_descriptions' in hf_config:
pretrained_cfg['label_descriptions'] = hf_config.pop('label_descriptions')
model_args = hf_config.get('model_args', {})
model_name = hf_config['architecture']
return pretrained_cfg, model_name, model_args
#"/home/blackroot/Desktop/ViTagger/model/config.json"
def load_model_but_good_this_time_srsly(model_path, model_config):
pretrained_cfg, model_name, model_args = load_cfg(model_config)
model = timm.create_model(
model_name,
checkpoint_path=model_path,
pretrained_cfg=pretrained_cfg,
**model_args)
return model not sure if this is worth a PR to add support for, but with the rise of HF models and their configs this might be a worthwhile addition. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
pytorch-image-models/timm/models/_factory.py
Line 38 in 59b3d86
TIMM seems to have a loader for HF based models, but I can't see how I can simply load an existing downloaded model using the same config info, without uploading it to the hub, and then letting timm redownload it again. Surely I'm missing something here, cheers.
Beta Was this translation helpful? Give feedback.
All reactions