Skip to content

Commit

Permalink
[CAPI] improve property implement to achieve better compatibility (op…
Browse files Browse the repository at this point in the history 
…envinotoolkit#12887)

* Solve some code style issues

Change-Id: I06c31ad9513463c758db96dbe33031983a8fd7a5

* Refine ov_dimension with C style structure

Use C style definition to replace opaque C++ handle:
 1). Improve partial shape
 2). Improve dimension/rank
 3). Improve shape

Change-Id: I680fd3e76c63d0c06d9d8b07c67af0440fb15bb7

* Some minor update

Change-Id: I237490a765e6ecd6f3896b1bd75d5f962304f43b

* add partial_shape_is_dynamic and update doc

Change-Id: Ief358f27de8386e35485d08847c8243d1490157f

* Rename property and add basic test cases

Change-Id: Ifbed0bbf7a5e836d2cab186763d7e5f8f2cb7d44

* address reviewer's comments

Change-Id: I9a5d1ede6d086906f3777c3d29237986a505db52

* Fix compile error

Change-Id: I72267a8dda32dd8f43c1ea1522d07b392052358f

* Use ov_any_t to replace ov_property_value_t

Change-Id: Ia4062d0da0941c901990bf5f16497ae3c8f2eb0f

* update doc for ov_model

Change-Id: I80183394f16dd1e07ff998993e26f8b903f2402e

* Property key apply string variables replace enum variable

Change-Id: I0e1699862de3913ccdc06ded5b8f8ac5a2bf8773

* Fix compiling error in windows

Change-Id: I4f9c7c746e218db94491c5f4bc5aeda1b3fde62b

* Change init/deinit method to create/free, remove unnecessary methods

Change-Id: Ibc5de039da94e92db494583ab1ff1a63e0586c73

* Pass by value for single C structure data

Change-Id: I289eb56e2ba1ab90942c8c801c585ab0f792952f

* fix dllexport issue in windows

Change-Id: I1632375e5e651fdfa076cdab2d8d1c3e83986549

* Fix static build error

Change-Id: I6633fe4887edad0b3a8b7e10730fb9d36a6ac0cd

* Use pass by value for all transparent structure

Change-Id: I652f6e70b420ef7c5af5cff3f5d482911cf0d75a

* Unify parameter order in  C APIs, input first and then output parameters

Change-Id: Ib89c77d81c314e3a81bfcec9917630abbc896f5e

* Rename preprocess api to be more readable

Change-Id: I0a8ecd005e73e4ba8e90a8f25bedf0739fe31836

* Simplfy property implement for set and get property

Change-Id: Icad41606368cef9520b1623f6207784c295f1b9f

* check string type property issue

Change-Id: Ie8ee6995dda401f1db0cc098e061ffcfd24de92a

* Fix memory leak

Change-Id: Ife8980736be320314eca6fb3f3605e019663e1da

* Check string property data

Change-Id: I542a136ff874cb345c67f7d3c1dbc74981a909df

* Fix issues caused by rebase

Change-Id: I3d675a16cf76980a6414e369a83464a46dec1e49

* Remove some redundant code

Change-Id: Iab6bf7f355ec7a7424d884e620319ed9db80ecf5

* Remove property type from interface

ov_core_set_property will have 2 cases:

1) enum type:  <const char* property_key, ov_property_xxx_e property_value>
2) other: <const char* property_key, const char* property_value>

Change-Id: Id7dec1ac40948d45c4f8baee687652aee92ea75d

* Remove redundant code

Change-Id: I66865d3313e305b0109cb63bd049772a01c0688f

* Remove exception log and update propery size argument

Change-Id: Ia1a60c07c9adf08d175d94a7d5e4298adb27bf80

* Apply string for enum property

Change-Id: I591b1db453a52b5ea9000e0cc3e11edf0ef5f790

* Add test case for MULTI_DEVICE_PRIORITIES

Change-Id: I59e4307564a2560724b4dacdb83c3ed45ea1ce8d

* Fix test issues and add more negative test case

Change-Id: Ib849fb8c098e487a537c1b1ab881f5e905871f73
  • Loading branch information
@riverlijunjie
riverlijunjie authored Sep 20, 2022
1 parent 08e397d commit ed31a46
Show file tree
Hide file tree
Showing 16 changed files with 569 additions and 586 deletions.
3 changes: 1 addition & 2 deletions samples/c/hello_classification/main.c
View file
Original file line number Diff line number Diff line change
Expand Up @@ -201,8 +201,7 @@ int main(int argc, char** argv) {
CHECK_STATUS(ov_preprocess_prepostprocessor_build(preprocess, &new_model));

// -------- Step 5. Loading a model to the device --------
ov_properties_t* property = NULL;
CHECK_STATUS(ov_core_compile_model(core, new_model, device_name, property, &compiled_model));
CHECK_STATUS(ov_core_compile_model(core, new_model, device_name, 0, &compiled_model));

// -------- Step 6. Create an infer request --------
CHECK_STATUS(ov_compiled_model_create_infer_request(compiled_model, &infer_request));
Expand Down
2 changes: 1 addition & 1 deletion samples/c/hello_nv12_input_classification/main.c
View file
Original file line number Diff line number Diff line change
Expand Up @@ -264,7 +264,7 @@ int main(int argc, char** argv) {
CHECK_STATUS(ov_preprocess_prepostprocessor_build(preprocess, &new_model));

// -------- Step 4. Loading a model to the device --------
CHECK_STATUS(ov_core_compile_model(core, new_model, device_name, NULL, &compiled_model));
CHECK_STATUS(ov_core_compile_model(core, new_model, device_name, 0, &compiled_model));

// -------- Step 5. Create an infer request --------
CHECK_STATUS(ov_compiled_model_create_infer_request(compiled_model, &infer_request));
Expand Down
34 changes: 2 additions & 32 deletions src/bindings/c/include/openvino/c/ov_common.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,8 @@ typedef enum {
*/
INVALID_C_PARAM = -14,
UNKNOWN_C_ERROR = -15,
NOT_IMPLEMENT_C_METHOD = -16,
UNKNOW_EXCEPTION = -17,
} ov_status_e;

/**
Expand Down Expand Up @@ -102,38 +104,6 @@ typedef enum {
U64, //!< u64 element type
} ov_element_type_e;

/**
* @enum ov_any_type_e
* @brief Enum to define ov_any data type.
*/
typedef enum {
BOOL = 0U, //!< boolean data
CHAR, //!< char data
INT32, //!< int32 data
UINT32, //!< uint32 data
INT64, //!< int64 data
UINT64, //!< uint64 data
ENUM, //!< enum data, must define U32 data for enumeration
FLOAT, //!< float data
DOUBLE, //!< double data
} ov_any_type_e;

/**
* @struct ov_any_t
* @brief Represent a property value
*/
typedef struct {
void* ptr;
size_t size;
ov_any_type_e type;
} ov_any_t;

/**
* @brief Free ov_any data.
* @param value The ov_any data will be freed.
*/
OPENVINO_C_API(void) ov_any_free(ov_any_t* value);

/**
* @brief Print the error info.
* @param ov_status_e a status code.
Expand Down
14 changes: 9 additions & 5 deletions src/bindings/c/include/openvino/c/ov_compiled_model.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -139,25 +139,29 @@ OPENVINO_C_API(ov_status_e)
ov_compiled_model_create_infer_request(const ov_compiled_model_t* compiled_model, ov_infer_request_t** infer_request);

/**
* @brief Sets properties for the current compiled model.
* @brief Sets properties for a device, acceptable keys can be found in ov_property_key_xxx.
* @ingroup compiled_model
* @param compiled_model A pointer to the ov_compiled_model_t.
* @param property ov_properties_t.
* @param property_key The property key.
* @param ... variadic paramaters The format is <char *property_key, char* property_value>.
* Supported property key please see ov_property.h.
* @return Status code of the operation: OK(0) for success.
*/
OPENVINO_C_API(ov_status_e)
ov_compiled_model_set_property(const ov_compiled_model_t* compiled_model, const ov_properties_t* property);
ov_compiled_model_set_property(const ov_compiled_model_t* compiled_model, ...);

/**
* @brief Gets properties for current compiled model.
* @ingroup compiled_model
* @param compiled_model A pointer to the ov_compiled_model_t.
* @param property_name Property name.
* @param property_key Property key.
* @param property_value A pointer to property value.
* @return Status code of the operation: OK(0) for success.
*/
OPENVINO_C_API(ov_status_e)
ov_compiled_model_get_property(const ov_compiled_model_t* compiled_model, const char* key, ov_any_t* value);
ov_compiled_model_get_property(const ov_compiled_model_t* compiled_model,
const char* property_key,
char** property_value);

/**
* @brief Exports the current compiled model to an output stream `std::ostream`.
Expand Down
37 changes: 20 additions & 17 deletions src/bindings/c/include/openvino/c/ov_core.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -151,17 +151,19 @@ ov_core_read_model_from_memory(const ov_core_t* core,
* @param core A pointer to the ie_core_t instance.
* @param model Model object acquired from Core::read_model.
* @param device_name Name of a device to load a model to.
* @param property Optional pack of pairs: (property name, property value) relevant only for this load operation
* operation.
* @param property_args_size How many properties args will be passed, each property contains 2 args: key and value.
* @param compiled_model A pointer to the newly created compiled_model.
* @param property paramater: Optional pack of pairs: <char* property_key, char* property_value> relevant only
* for this load operation operation. Supported property key please see ov_property.h.
* @return Status code of the operation: OK(0) for success.
*/
OPENVINO_C_API(ov_status_e)
ov_core_compile_model(const ov_core_t* core,
const ov_model_t* model,
const char* device_name,
const ov_properties_t* property,
ov_compiled_model_t** compiled_model);
const size_t property_args_size,
ov_compiled_model_t** compiled_model,
...);

/**
* @brief Reads a model and creates a compiled model from the IR/ONNX/PDPD file.
Expand All @@ -171,44 +173,45 @@ ov_core_compile_model(const ov_core_t* core,
* @param core A pointer to the ie_core_t instance.
* @param model_path Path to a model.
* @param device_name Name of a device to load a model to.
* @param property Optional pack of pairs: (property name, property value) relevant only for this load operation
* operation.
* @param property_args_size How many properties args will be passed, each property contains 2 args: key and value.
* @param compiled_model A pointer to the newly created compiled_model.
* @param property paramater: Optional pack of pairs: <char* property_key, char* property_value> relevant only
* for this load operation operation. Supported property key please see ov_property.h.
* @return Status code of the operation: OK(0) for success.
*/
OPENVINO_C_API(ov_status_e)
ov_core_compile_model_from_file(const ov_core_t* core,
const char* model_path,
const char* device_name,
const ov_properties_t* property,
ov_compiled_model_t** compiled_model);
const size_t property_args_size,
ov_compiled_model_t** compiled_model,
...);

/**
* @brief Sets properties for a device, acceptable keys can be found in ov_property_key_e.
* @brief Sets properties for a device, acceptable keys can be found in ov_property_key_xxx.
* @ingroup Core
* @param core A pointer to the ie_core_t instance.
* @param device_name Name of a device.
* @param property ov_property propertys.
* @param property_key The property key.
* @param ... variadic paramaters The format is <char* property_key, char* property_value>.
* Supported property key please see ov_property.h.
* @return Status code of the operation: OK(0) for success.
*/
OPENVINO_C_API(ov_status_e)
ov_core_set_property(const ov_core_t* core, const char* device_name, const ov_properties_t* property);
ov_core_set_property(const ov_core_t* core, const char* device_name, ...);

/**
* @brief Gets properties related to device behaviour.
* The method extracts information that can be set via the set_property method.
* @ingroup Core
* @param core A pointer to the ie_core_t instance.
* @param device_name Name of a device to get a property value.
* @param property_name Property name.
* @param property_value A pointer to property value.
* @param property_key Property key.
* @param property_value A pointer to property value with string format.
* @return Status code of the operation: OK(0) for success.
*/
OPENVINO_C_API(ov_status_e)
ov_core_get_property(const ov_core_t* core,
const char* device_name,
const char* property_name,
ov_any_t* property_value);
ov_core_get_property(const ov_core_t* core, const char* device_name, const char* property_key, char** property_value);

/**
* @brief Returns devices available for inference.
Expand Down
168 changes: 115 additions & 53 deletions src/bindings/c/include/openvino/c/ov_property.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,60 +13,139 @@

#include "openvino/c/ov_common.h"

typedef struct ov_property {
const char* key;
ov_any_t value;
} ov_property_t;

typedef struct ov_properties {
ov_property_t* list;
size_t size;
} ov_properties_t;

/**
* @enum ov_performance_mode_e
* @brief Enum to define possible performance mode hints
* @brief This represents OpenVINO 2.0 ov::hint::PerformanceMode entity.
*
*/
typedef enum {
UNDEFINED_MODE = -1, //!< Undefined value, performance setting may vary from device to device
LATENCY = 1, //!< Optimize for latency
THROUGHPUT = 2, //!< Optimize for throughput
CUMULATIVE_THROUGHPUT = 3, //!< Optimize for cumulative throughput
} ov_performance_mode_e;

/**
* @enum ov_affinity_e
* @brief Enum to define possible affinity patterns
* @brief property key
*/
typedef enum {
NONE = -1, //!< Disable threads affinity pinning
CORE = 0, //!< Pin threads to cores, best for static benchmarks
NUMA = 1, //!< Pin threads to NUMA nodes, best for real-life, contented cases. On the Windows and MacOS* this
//!< option behaves as CORE
HYBRID_AWARE = 2, //!< Let the runtime to do pinning to the cores types, e.g. prefer the "big" cores for latency
//!< tasks. On the hybrid CPUs this option is default
} ov_affinity_e;

//!< Read-only property<string> to get a string list of supported read-only properties.
OPENVINO_C_VAR(const char*) ov_property_key_supported_properties;

//!< Read-only property<string> to get a list of available device IDs
OPENVINO_C_VAR(const char*) ov_property_key_available_devices;

//!< Read-only property<uint32_t string> to get an unsigned integer value of optimaln
//!< number of compiled model infer requests.
OPENVINO_C_VAR(const char*) ov_property_key_optimal_number_of_infer_requests;

//!< Read-only property<string(unsigned int, unsigned int, unsigned int)> to provide a
//!< hint for a range for number of async infer requests. If device supports
//!< streams, the metric provides range for number of IRs per stream.
OPENVINO_C_VAR(const char*) ov_property_key_range_for_async_infer_requests;

//!< Read-only property<string(unsigned int, unsigned int)> to provide information about a range for
//!< streams on platforms where streams are supported
OPENVINO_C_VAR(const char*) ov_property_key_range_for_streams;

//!< Read-only property<string> to get a string value representing a full device name.
OPENVINO_C_VAR(const char*) ov_property_key_device_full_name;

//!< Read-only property<string> to get a string list of capabilities options per device.
OPENVINO_C_VAR(const char*) ov_property_key_device_capabilities;

//!< Read-only property<string> to get a name of name of a model
OPENVINO_C_VAR(const char*) ov_property_key_model_name;

//!< Read-only property<uint32_t string> to query information optimal batch size for the given device
//!< and the network
OPENVINO_C_VAR(const char*) ov_property_key_optimal_batch_size;

//!< Read-only property to get maximum batch size which does not cause performance degradation due
//!< to memory swap impact.
OPENVINO_C_VAR(const char*) ov_property_key_max_batch_size;

//!< Read-write property<string> to set/get the directory which will be used to store any data cached
//!< by plugins.
OPENVINO_C_VAR(const char*) ov_property_key_cache_dir;

//!< Read-write property<uint32_t string> to set/get the number of executor logical partitions.
OPENVINO_C_VAR(const char*) ov_property_key_num_streams;

//!< Read-write property to set/get the name for setting CPU affinity per thread option.
//!< All property values are string format, below are optional value:
//!< "NONE" - Disable threads affinity pinning
//!< "CORE" - Pin threads to cores, best for static benchmarks
//!< "NUMA" - Pin threads to NUMA nodes, best for real-life, contented cases. On the Windows and MacOS* this
//!< option behaves as CORE
//!< "HYBRID_AWARE" - Let the runtime to do pinning to the cores types, e.g. prefer the "big" cores for latency
//!< tasks. On the hybrid CPUs this option is default.
OPENVINO_C_VAR(const char*) ov_property_key_affinity;

//!< Read-write property<int32_t string> to set/get the maximum number of threads that can be used
//!< for inference tasks.
OPENVINO_C_VAR(const char*) ov_property_key_inference_num_threads;

//!< Read-write property, it is high-level OpenVINO Performance Hints
//!< unlike low-level properties that are individual (per-device), the hints are something that
//!< every device accepts and turns into device-specific settings detail.
//!< All property values are string, below are optional value:
//!< "UNDEFINED_MODE" - Undefined value, performance setting may vary from device to device
//!< "LATENCY" - Optimize for latency
//!< "THROUGHPUT" - Optimize for throughput
//!< "CUMULATIVE_THROUGHPUT" - Optimize for cumulative throughput
OPENVINO_C_VAR(const char*) ov_property_key_hint_performance_mode;
OPENVINO_C_VAR(const char*) ov_property_key_model_name;

//!< Read-write property<ov_element_type_e> to set the hint for device to use specified
//!< precision for inference
//!< All property values are string, below are optional value:
//!< "UNDEFINED" - Undefined element type
//!< "DYNAMIC" - Dynamic element type
//!< "BOOLEAN" - boolean element type
//!< "BF16" - bf16 element type
//!< "F16" - f16 element type
//!< "F32" - f32 element type
//!< "F64" - f64 element type
//!< "I4" - i4 element type
//!< "I8" - i8 element type
//!< "I16" - i16 element type
//!< "I32" - i32 element type
//!< "I64" - i64 element type
//!< "U1" - binary element type
//!< "U4" - u4 element type
//!< "U8" - u8 element type
//!< "U16" - u16 element type
//!< "U32" - u32 element type
//!< "U64" - u64 element type
OPENVINO_C_VAR(const char*) ov_property_key_hint_inference_precision;
OPENVINO_C_VAR(const char*) ov_property_key_optimal_batch_size;
OPENVINO_C_VAR(const char*) ov_property_key_max_batch_size;

//!< (Optional) Read-write property<uint32_t string> that backs the Performance Hints by giving
//!< additional information on how many inference requests the application will be
//!< keeping in flight usually this value comes from the actual use-case (e.g.
//!< number of video-cameras, or other sources of inputs)
OPENVINO_C_VAR(const char*) ov_property_key_hint_num_requests;

//!< Read-write property<string> for setting desirable log level.
//!< All property values are string, below are optional value:
//!< "NO" - disable any logging
//!< "ERR" - error events that might still allow the application to continue running
//!< "WARNING" - potentially harmful situations which may further lead to ERROR
//!< "INFO" - informational messages that display the progress of the application at coarse-grained level
//!< "DEBUG" - fine-grained events that are most useful to debug an application.
//!< "TRACE" - finer-grained informational events than the DEBUG
OPENVINO_C_VAR(const char*) ov_property_key_log_level;

//!< Read-write property, high-level OpenVINO model priority hint.
//!< Defines what model should be provided with more performant bounded resource first.
//!< All property value are string format, below is optional value:
//!< "LOW" - Low priority
//!< "MEDIUM" - Medium priority
//!< "HIGH" - High priority
//!< "DEFAULT" - Default priority is MEDIUM
OPENVINO_C_VAR(const char*) ov_property_key_hint_model_priority;

//!< Read-write property<string> for setting performance counters option.
//!< All property values are string, below are optional value:
//!< "YES" - true
//!< "NO" - false
OPENVINO_C_VAR(const char*) ov_property_key_enable_profiling;

//!< Read-write property<std::pair<std::string, Any>>, device Priorities config option, with comma-separated devices
//!< listed in the desired priority
//!< Some optional values for MULTI device:
//! "CPU,GPU"
//! "GPU,CPU"
//! Note: CPU plugin is not implement.
OPENVINO_C_VAR(const char*) ov_property_key_device_priorities;

// Property
/**
* @defgroup Property Property
Expand All @@ -75,21 +154,4 @@ OPENVINO_C_VAR(const char*) ov_property_key_hint_num_requests;
* @{
*/

/**
* @brief Initialize a properties list object.
* @ingroup property
* @param property The properties list will be initialized.
* @param size The list size.
* @return ov_status_e a status code, return OK if successful
*/
OPENVINO_C_API(ov_status_e) ov_properties_create(ov_properties_t* property, size_t size);

/**
* @brief Deinitialized properties list.
* properties->list[i].value.ptr need be managed by user.
* @ingroup property
* @param property The properties list object will be deinitialized.
*/
OPENVINO_C_API(void) ov_properties_free(ov_properties_t* property);

/** @} */ // end of Property
8 changes: 7 additions & 1 deletion src/bindings/c/src/common.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,15 @@
CATCH_OV_EXCEPTION(NETWORK_NOT_READ, NetworkNotRead) \
CATCH_OV_EXCEPTION(INFER_CANCELLED, InferCancelled) \
catch (...) { \
return ov_status_e::UNEXPECTED; \
return ov_status_e::UNKNOW_EXCEPTION; \
}

#define GET_PROPERTY_FROM_ARGS_LIST \
std::string property_key = va_arg(args_ptr, char*); \
std::string _value = va_arg(args_ptr, char*); \
ov::Any value = _value; \
property[property_key] = value;

/**
* @struct ov_core
* @brief This struct represents OpenVINO Core entity.
Expand Down
Loading

0 comments on commit ed31a46

Please sign in to comment.