Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

string_byte = any byte excluding '"' and '\' is not restrictive enough for parsing streams #9

Open
andsens opened this issue Jul 30, 2021 · 0 comments

Comments

@andsens
Copy link

andsens commented Jul 30, 2021

The spec specifies that values containing bytes less than or equal to " " (\x20) must be quoted but not escaped.
Specifically with regards to newlines this is rather problematic when parsing incomplete logstreams. Take this example:

...
msg=" level=info time=2021-07-30T08:24:01.604Z
msg="
...

Apologies for the rather contrived scenario, but it shows the ambiguity when parsing a logstream rather well. The point here is you don't know whether a newline is the beginning, middle, or end of a log entry. The complete output for the above record(s) could look like this:

msg=" level=info time=2021-07-30T08:24:01.604Z
msg="

So: {msg: " level=info time=2021-07-30T08:24:01.604Z\nmsg="}
Or this:

out="
msg=" level=info time=2021-07-30T08:24:01.604Z
msg="
" level=error time=2021-07-30T08:24:05.004Z

So: {out: "\nmsg=", level: "info", time: "2021-07-30T08:24:01.604Z"} and {msg: "\n", level: "error", time: "2021-07-30T08:24:05.004Z"}

The bigger issue here is that there is no way to get back to a "clean slate". The second version could have variations that never cause a parsing error and the parser would keep on having an off-by-one error:

...
moreLines x=1 f="
out="
msg=" level=info time=2021-07-30T08:24:01.604Z
msg="
" level=error time=2021-07-30T08:24:05.004Z
x="is this quoted, who knows
" it might be. It all depends on the first line.

The above could be a long string value, or it could be a bunch of keys with the value true

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant