Error at ocsf modeling during Azure execution

[pedrooot]

Steps to Reproduce

{"timestamp": "2025-01-27 12:53:14,040", "filename": "network_public_ip_shodan.py:35", "level": "ERROR", "module": "network_public_ip_shodan", "message": "No Shodan API Key -- Please input a Shodan API Key with -N/--shodan or in config.yaml"}
{"timestamp": "2025-01-27 12:53:20,683", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'Permission' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,683", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'Permission' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,684", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'Permission' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,684", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'Permission' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,685", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'Permission' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,686", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'Permission' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,686", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'Permission' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,687", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'Permission' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,687", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'Permission' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,688", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'Permission' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,688", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'Permission' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,689", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'Permission' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,689", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'Permission' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,690", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'Permission' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,690", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'Permission' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,691", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'Permission' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,691", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'Permission' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,692", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'Permission' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,693", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'Permission' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,693", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'Permission' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,694", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'Permission' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,694", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'Permission' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,695", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'Permission' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,695", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'Permission' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,728", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'NetworkRuleSet' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,729", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'NetworkRuleSet' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,729", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'NetworkRuleSet' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,730", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'NetworkRuleSet' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,730", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'NetworkRuleSet' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,731", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'NetworkRuleSet' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,731", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'NetworkRuleSet' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,732", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'NetworkRuleSet' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,733", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'NetworkRuleSet' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,733", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'NetworkRuleSet' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,734", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'NetworkRuleSet' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,734", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'NetworkRuleSet' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,735", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'NetworkRuleSet' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,736", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'NetworkRuleSet' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,736", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'NetworkRuleSet' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,737", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'NetworkRuleSet' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,737", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'NetworkRuleSet' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,738", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'NetworkRuleSet' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,738", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'NetworkRuleSet' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,739", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'NetworkRuleSet' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,740", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'NetworkRuleSet' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,740", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'NetworkRuleSet' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,741", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'NetworkRuleSet' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,741", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'NetworkRuleSet' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,742", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'NetworkRuleSet' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,742", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'NetworkRuleSet' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,743", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'NetworkRuleSet' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,744", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'NetworkRuleSet' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,744", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'NetworkRuleSet' is not JSON serializable"}
{"timestamp": "2025-01-27 12:53:20,745", "filename": "ocsf.py:211", "level": "ERROR", "module": "ocsf", "message": "TypeError[207]: Object of type 'NetworkRuleSet' is not JSON serializable"}

Expected behavior

NO error

Actual Result with Screenshots or Logs

How did you install Prowler?

Cloning the repository from github.com (git clone)

Environment Resource

OS used

Prowler version

master - 5.3.0

Pip version

Context

No response

[garym-krrv]

I am seeing same issue here also against azure on 5.2.3
HTML File seems ok, just the CSV and JSON files

[puchy22]

Hi @garym-krrv,

This is the PR (#6880) that fixes the issue with Azure models serialization. If you could test it and confirm whether it resolves your errors, it would be very helpful for us.

Thanks!

[garym-krrv]

Will get round to it this evening. Thanks

[garym-krrv]

I recieve a lot of these errors @puchy22
Critical findings are still missing from the JSON file, but are included in the HTML and CSV

2025-02-11 11:44:35,676 [File: _serialization.py:484] [Module: _serialization] WARNING: Discriminator source is absent or null, use base class ResourceDetails.

2025-02-11 11:44:35,676 [File: _serialization.py:484] [Module: _serialization] WARNING: Discriminator source is absent or null, use base class ResourceDetails.

2025-02-11 11:44:35,677 [File: _serialization.py:484] [Module: _serialization] WARNING: Discriminator source is absent or null, use base class ResourceDetails.

2025-02-11 11:44:35,677 [File: _serialization.py:484] [Module: _serialization] WARNING: Discriminator source is absent or null, use base class ResourceDetails.

and several of these

2025-02-11 11:44:59,887 [File: ocsf.py:211] [Module: ocsf] ERROR: TypeError[207]: Object of type 'ItemPaged' is not JSON serializable

2025-02-11 11:44:59,887 [File: ocsf.py:211] [Module: ocsf] ERROR: TypeError[207]: Object of type 'ItemPaged' is not JSON serializable

2025-02-11 11:44:59,888 [File: ocsf.py:211] [Module: ocsf] ERROR: TypeError[207]: Object of type 'ItemPaged' is not JSON serializable

2025-02-11 11:44:59,888 [File: ocsf.py:211] [Module: ocsf] ERROR: TypeError[207]: Object of type 'ItemPaged' is not JSON serializable

2025-02-11 11:52:18,682 [File: ocsf.py:211] [Module: ocsf] ERROR: TypeError[207]: Object of type 'ManagedServiceIdentity' is not JSON serializable

2025-02-11 11:52:18,888 [File: ocsf.py:211] [Module: ocsf] ERROR: TypeError[207]: Object of type 'ItemPaged' is not JSON serializable

2025-02-11 11:52:18,901 [File: ocsf.py:211] [Module: ocsf] ERROR: TypeError[207]: Object of type 'SecurityRule' is not JSON serializable

2025-02-11 11:52:18,902 [File: ocsf.py:211] [Module: ocsf] ERROR: TypeError[207]: Object of type 'SecurityRule' is not JSON serializable

2025-02-11 11:52:18,902 [File: ocsf.py:211] [Module: ocsf] ERROR: TypeError[207]: Object of type 'SecurityRule' is not JSON serializable

2025-02-11 11:52:18,903 [File: ocsf.py:211] [Module: ocsf] ERROR: TypeError[207]: Object of type 'SecurityRule' is not JSON serializable

2025-02-11 11:52:18,903 [File: ocsf.py:211] [Module: ocsf] ERROR: TypeError[207]: Object of type 'SecurityRule' is not JSON serializable

2025-02-11 11:52:18,904 [File: ocsf.py:211] [Module: ocsf] ERROR: TypeError[207]: Object of type 'SecurityRule' is not JSON serializable

Gary

[garym-krrv]

The current critical findings that are missing for me from the JSON file are related to
check_id: sqlserver_unrestricted_inbound_access

There may be others but this is one of the checks not outputting the data

@puchy22

[garym-krrv]

After recreating my powetry environment I have more errors. Critical findings are still missing from the JSON file but I have attached the log file now. We can ignore the access denied messages.

issue-6698.json

[puchy22]

@garym-krrv, thank you very much. We truly appreciate your feedback it is incredibly valuable to us. I will begin working immediately to address all the errors. Once I have completed the necessary corrections, I will inform you. Thank you again.

[garym-krrv]

@garym-krrv, thank you very much. We truly appreciate your feedback it is incredibly valuable to us. I will begin working immediately to address all the errors. Once I have completed the necessary corrections, I will inform you. Thank you again.

Anytime.. Happy to assist and be part of community..
Ping me and ill pull any updates and try again..
Cheers @puchy22

[puchy22]

Hi @garym-krrv,
I haven’t been able to reproduce all errors. However, I believe I’ve fixed several issues. If you encounter any serious problems, let me know, and I’ll investigate further.

[garym-krrv]

Thanks @puchy22 I'll pull latest commits and try again.

[garym-krrv]

Hey @puchy22

Seems I have issues still. Critical findings are still missing from the JSON file.
I added 2 print lines to 2 files locally to ensure that im running the latest commits

I know @pedrooot raised this issue, worth double checking with him?

issue-6698-20250512.json

BTW, Im in the Prowler Slack Workspace if you need to DM me directly to take a look.

[garym-krrv]

@puchy22 Ive managed to grab the raw JSON out of main.py when its trying to create the json file and there may be some issues within it.

The following are some of the Key Values. Are these expected? Again its the CheckID='sqlserver_unrestricted_inbound_access'

Severity=Severity.critical:'critical',
status=Status.PASS:'PASS',
'firewall_rules':<iterator object azure.core.paging.ItemPaged at 0x7f175f3c5910>

The errors about ItemPaged seems to correlate to the JSON Issues above ^^ The firewall_rules is the issue in the Json by the looks of it. The 3 errors below MATCH the 3 findings for sqlserver_unrestricted_inbound_access

2025-02-13 00:06:18,682 [File: ocsf.py:211] [Module: ocsf] ERROR: TypeError[207]: Object of type 'ItemPaged' is not JSON serializable

2025-02-13 00:06:18,682 [File: ocsf.py:211] [Module: ocsf] ERROR: TypeError[207]: Object of type 'ItemPaged' is not JSON serializable

2025-02-13 00:06:18,682 [File: ocsf.py:211] [Module: ocsf] ERROR: TypeError[207]: Object of type 'ItemPaged' is not JSON serializable

[garym-krrv]

All looks good now after testing the branch