Add grpc to enhance exporter performance

[sharkpc138]

Summary

• To verify the target of the log export, files are traversed to establish the chunk metadata for logs.
  However, if there is a delay during this process, it becomes difficult to obtain the most up-to-date information.
• To quickly construct chunk information, instead of directly traversing the files, the exporter (gRPC client) queries the cache information from the store (gRPC server) to build a fast metadata map.
• When multiple log export rules exist, transmission delays can cause other log export operations to fall further from the latest information. Therefore, this process is supplemented by updating the information before performing the log export.

case review

• When there is a large amount of logs, file rotation occurs frequently, which can lead to more frequent mismatches between the file modification time and the loaded file status

normal

• log: I0313 11:09:36.611161 1 kafka.go:81] [kafka][took 0.009061s] upload 272722 bytes to topic xxxg for {"labels":[{"app":"istio-ingressgateway"}],"namespace":"some","pod":"some_pod","pod_uid":"some_uid","container":"__emptydir__","source":{"type":"emptydir","path":"some.log"}, "start":"1741831769","end":"1741831774"}
• request start: Thursday, March 13, 2025 11:09:29 AM GMT+09:00
• request end: Thursday, March 13, 2025 11:09:34 AM GMT+09:00
• expected files to lookup: temp.log
• file modtime: 2025-03-13 11:09:41.439243488
• result: It is expected behavior for the file to be changed after the request's end time

abnormal - mismatch file status

• log: No logs are remained, but the log export process may have started at I0313 11:09:36.611161 + 5 seconds
• request start: Thursday, March 13, 2025 11:09:34 AM GMT+09:00
• request end: Thursday, March 13, 2025 11:09:39 AM GMT+09:00
• expected files to lookup: temp.log or 2025-03-13T11:09:26.205+09:00_2025-03-13T11:09:41.187+09:00_1735_0.log
• file modtime: 2025-03-13 11:09:41.439243488
• result:
  • The expected time for the query attempt is similar to the file modification time(11:09:41)
  • The time difference between the end time and the query attempt is 2 seconds (41 - 39),
    which suggests that if there is a significant time gap between the chunk loading and the query attempt,
    the current chunk information may not reflect the most up-to-date data