Good custom model evaluation but poor segmentation

[AgathePuissant]

Hello,
Thanks for this great package !
I trained a custom model to segment the 4 wings of butterflies, which gave me good results in the evaluation (mAP ~0.9), but when I try to infer on images it doesn't segment anything or sometimes it can segment something but it is always the same class (the last one).
Moreover, I tried on the test images and it still worked poorly, and when I try multiple times it gave different results.
I just don't understand the gap between the evaluation and my tests and the inconsistance of the results?

I am on tensorflow 2.5.0, torch 1.9.0

My code :

from pixellib.custom_train import instance_custom_training
train_maskrcnn = instance_custom_training()
train_maskrcnn.modelConfig(network_backbone = "resnet50", num_classes= 4, batch_size = 1)
train_maskrcnn.load_pretrained_model("C:/Users/apuissant/Desktop/mask_rcnn_coco.h5")
train_maskrcnn.load_dataset("C:/Users/apuissant/Desktop/dataset/")
train_maskrcnn.train_model(num_epochs=300,augmentation=True,path_trained_models="C:/Users/apuissant/Desktop/mask_rcnn_models")

from pixellib.instance import custom_segmentation

segment_image = custom_segmentation()
segment_image.inferConfig(num_classes= 4, class_names= ["BG","ant_d","post_d","ant_g","post_g"], detection_threshold=0.2)
segment_image.load_model("C:/Users/apuissant/Desktop/mask_rcnn_models/mask_rcnn_model.262-0.076744.h5")
segment_image.segmentImage("imtest.jpg", show_bboxes=True, output_image_name="imtest_out.jpg")

[AgathePuissant]

Just in case someone runs into the same issue, I have found the origin of the problem... I changed the backbone and did a few tests to resnet101 and did a few tests and it seems to work well. There may be an issue with resnet50 though!