Use IRIs for resource identification

[jakubklimek]

Is there a reason IRIs (Internationalized Resource Identifiers) (RFC3987) are not used for resource identification even though they are part of the RDF 1.1 specification and, in fact, they are already widely used?

I understand that on the level of HTTP, which may be the base of the Solid protocol, only URIs are used. However, I have the feeling the relationship of URIs and IRIs should be addressed somehow. In RFC3987 there is a mapping (percent-encoding) of IRIs to URIs and back. However, e.g. in RDF Turtle, IRI and its percent-encoded URI counterpart, are treated as two different IRIs, therefore some issues could arise.

Maybe this was already discussed, but I cannot see any mention of IRIs in the specification.

[damooo]

Currently CSS also assume identifiers as URIs.
in .acl resources for a resource, CSS uses percent-encoded URI of that resource.

[joachimvh]

Currently CSS also assume identifiers as URIs. in .acl resources for a resource, CSS uses percent-encoded URI of that resource.

There is an open issue on what to do with this at solid/web-access-control-spec#93

Also related: #22

[kjetilk]

I don't know if it is actually the concern, but IRIs expand the attack surface for homograph attacks quite significantly. It makes sense to tread carefully in this area.

[damooo]

I don't know if it is actually the concern, but IRIs expand the attack surface for homograph attacks quite significantly. It makes sense to tread carefully in this area.

that is concern only for case of domain-names. not w.r.t resource identifiers in given domain. Even that case is not a technical vulnerability per se, but of naive usage. This need not prevent us in embracing iris, which enable international users name resources in their scripts.

[elf-pavlik]

which enable international users name resources in their scripts

Human-readable labels should be part of the resource description. IRI preferably stays machine-readable, especially everything after the domain name. Most mobile browsers will not even show anything else to the user.

[kjetilk]

that is concern only for case of domain-names. not w.r.t resource identifiers in given domain.

Is it? I agree that if there is only one storage per authority, or every storage has the same owner, it is a likely a marginal problem, but that is not necessarily the case.

Even that case is not a technical vulnerability per se, but of naive usage.

Yes, but as most attacks are not on technical vulnerabilities, I think it is an important concern nevertheless. The problem is that people tend to rely on a bunch of flawed heuristics to determine actions that have security implications, like looking at the URL, I see this in my own family members. We should ensure that they don't need to, they should be able to rely on a few security mechanisms that can be easily understood by lay people, but I believe we're not there, IMHO. Until we have that, homograph attacks look pretty scary to me, but admittedly, I do not have numbers to back that up.

This need not prevent us in embracing iris, which enable international users name resources in their scripts.

I believe there is nothing in the technical infrastructure that prevents us from adopting IRIs. It is actually not clear to me why we haven't.

[damooo]

which enable international users name resources in their scripts

Human-readable labels should be part of the resource description. IRI preferably stays machine-readable, especially everything after the domain name. Most mobile browsers will not even show anything else to the user.

IRIs are not about end-user friendly labels. But they enable to have identifiers too internationalized, which in some cases cannot be expressed in ergonomic uris. for example, if we want to have identifier for an entry in lexicon of a language, or concepts of certain culture, with out complexities of lossy-translation or transliteration, etc, or simply unicode-filenames. Of-cource we can use uris with percent-encoding in this case, but it gives above issues, and non-ergonomic.

[damooo]

@kjetilk

I believe there is nothing in the technical infrastructure that prevents us from adopting IRIs. It is actually not clear to me why we haven't.

There may be actually few things solid need to specify if to support iris. IRI support will be non-trivial for following reasons.

1. Solid is based on http. HTTP doesn't support IRIs as identifiers.
2. RDF identifier match is strictly literal comparison of iri-strings.

Say, in an identifier, we want to have a segment राम, whose percent-encoding is %E0%A4%B0%E0%A4%BE%E0%A4%AE. There are two ways prevalent in community.

1. one is direct iri, like <example:राम>. dbpedia, and many other datasets use this approach, in their international datasets. like <<http://hi.dbpedia.org/resource/राम>.
2. another practice is to, have percent-encoded uris as identifier. lexvo.org, and others uses this approach. lexvo's identifiers both for concept-resources, and information resources, follow this approach.

Now issue is, for both of above approaches, if we want those identifiers in http, and also dereferencable (as in case of solid), http uri will be same. Thus rdf mandates to distinguish between <example:राम>, and example:%E0%A4%B0%E0%A4%BE%E0%A4%AE`, both practices adopted by communities, but http identifiers cannot carry that distinction when we are dereferencing.

Thus if we want to PUT representation at <example/path/to/राम>, or <example/path/to/%E0%A4%B0%E0%A4%BE%E0%A4%AE>, solid-http-server gets PUT request for <example:path/to/%E0%A4%B0%E0%A4%BE%E0%A4%AE>, in both cases. we can't be sure of intended iri. we have to convey that loss of information through other mechanisms. Same issue on GET.

Solid is interesting case for iris, as it have to support both rdf, and http concerns for dereferencable-information-resources.

[damooo]

contd from above...

There seems two straight-forward behaviours possible to address this case.

1. Solid should specify, it only supports uris for information resources it handles, and it takes uris recieved through request literally with out any decoding. thus percent-encoded-uris will be only mechanism to represent unicode names.
2. Solid should specify, it supports iris for information resources it handles, and percent-decodes uris recieved through http request, to compute intended iri. If one want to have percent-encoded uri itself as identifier as in lexvo.org case, they should percent-encode already percent-encoded-identifier. This way has advantage of supporting both cases, and supports most prevelant case straight forwardly.

2nd behaviour seems better for me.

[kjetilk]

OK, thank you very much for your perspective, @damooo , it is very much appreciated. I believe we should address this in the future. It is very good that you already have concrete proposals to resolve this!

[damooo]

@kjetilk , @elf-pavlik

I believe we should address this in the future.

I am not sure if gravity of this issue is clearly understood. Solid doesn't specify whether to decode or not http uri, to compute identifier of resource. It has potential to cause solid based linked-data apps doesn't work same with all solid servers, and will fail in many basic tasks, when ever an unicode char enter an identifier.

I just tested with css, nss, ess. And indeed this is the case. They interpret spec differently, among themselves, and with-in themselves at different places. And few apps break with one or other server, when unicode char enter in identifier. For demo, in inrupt's ess create a container with unicode name, and then try to browse that folder in inrupt's pod-browser app. It crashes.

I give here PUT + GET request flow for a resource on both NSS and ESS, for demonstrating technicle issue.

let's say there is already a container with id <http://example.org/public/>, and we want to create a sub container with unicode-segment Germānus (note ā).

In NSS:

1. PUT <http://example.org/public/Germānus/> or <http://example.org/public/Germ%C4%81nus/> (both will be normalized to <http://example.org/public/Germ%C4%81nus/> by http client.

Request:

PUT /public/Germ%C4%81nus/ HTTP/1.1
Accept: text/turtle
Content-Type: text/turtle
Link: <http://www.w3.org/ns/ldp#BasicContainer>; rel="type"

Response:

HTTP/1.1 201 Created
X-Powered-By: solid-server/5.6.16
Content-Type: text/plain; charset=utf-8

2. GET on <http://example.org/public/Germānus> or <http://example.org/public/Germ%C4%81nus/> after above PUT

Request:

GET /public/Germ%C4%81nus/ HTTP/1.1

Response:

HTTP/1.1 200 OK
X-Powered-By: solid-server/5.6.16
Link: <.acl>; rel="acl", <.meta>; rel="describedBy", <http://www.w3.org/ns/ldp#Container>; rel="type", <http://www.w3.org/ns/ldp#BasicContainer>; rel="type"
Content-Type: text/turtle

@prefix : <#>.
@prefix dct: <http://purl.org/dc/terms/>.
@prefix ldp: <http://www.w3.org/ns/ldp#>.
@prefix stat: <http://www.w3.org/ns/posix/stat#>.
@prefix xsd: <http://www.w3.org/2001/XMLSchema#>.

<http://example.org/public/Germ%C4%81nus/>:
    a ldp:BasicContainer, ldp:Container;
    dct:modified "2021-11-17T02:47:27Z"^^xsd:dateTime;
    stat:mtime 1637117247.421;
    stat:size 4096.

Here we can see NSS creates a resource with literal identifier http://example.org/public/Germ%C4%81nus/> All rdf descriptions uses that percent-encoded identifier.

In ESS:

1. PUT <http://example.org/public/Germānus/> or <http://example.org/public/Germ%C4%81nus/>

Request:

PUT /public/Germ%C4%81nus/ HTTP/1.1
Accept: text/turtle
Content-Type: text/turtle
Link: <http://www.w3.org/ns/ldp#BasicContainer>; rel="type"

Response:

HTTP/2 201 Created
content-location: https://pod.inrupt.com/damodara/public/Germ%C4%81nus/
link: <http://www.w3.org/ns/ldp#Resource>; rel="type"
link: <http://www.w3.org/ns/ldp#BasicContainer>; rel="type"
link: <http://www.w3.org/ns/ldp#RDFSource>; rel="type"
link: <http://www.w3.org/ns/ldp#Container>; rel="type"

Here it also sent content-location header, that resource identifier is https://pod.inrupt.com/damodara/public/Germ%C4%81nus/ (encoded-form)

2. GET on <http://example.org/public/Germānus> or <http://example.org/public/Germ%C4%81nus/> after above PUT

Request:

GET /public/Germ%C4%81nus/ HTTP/1.1

Response:

HTTP/2 200 OK
content-type: text/turtle; charset=UTF-8
link: <http://www.w3.org/ns/ldp#Resource>; rel="type"
link: <http://www.w3.org/ns/ldp#BasicContainer>; rel="type"
link: <http://www.w3.org/ns/ldp#RDFSource>; rel="type"/terms#podOwner"
link: <http://www.w3.org/ns/ldp#Container>; rel="type"

@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix ldp: <http://www.w3.org/ns/ldp#> .

<http://example.org/public/Germānus/>
        rdf:type  ldp:BasicContainer .

Here we can see though it uses iri <http://example.org/public/Germānus/> as identifier to resource created contrary to NSS behaviour of encoded-uri ** for same put request **.

Thus different servers uses different identifiers in rdf-descriptions for a resource created and dereferencable by same http uri. apps cannot know which one to lookup as subject here after if they want to read rdf-description about them. Thus they will fail on one or other server for any such rdf usage. Issue get worse for further sub containers, contained resources, and for usage of relative-uris, etc.

It is too fundamental an aspect that solid must specify at very begining, evn if it is to constrain it to uris.
As identifiers get persisted in rdf-docs, fixing this later will be impossible, as data has to be modified.

[damooo]

contd from above...

There seems two straight-forward behaviours possible to address this case.

1. Solid should specify, it only supports uris for information resources it handles, and it takes uris recieved through request **literally** with out any decoding. thus percent-encoded-uris will be only mechanism to represent unicode names.

2. Solid should specify, it supports iris for information resources it handles, and percent-decodes uris recieved through http request, to compute intended iri. If one want to have percent-encoded uri itself as identifier as in `lexvo.org` case, they should percent-encode already percent-encoded-identifier. This way has advantage of supporting both cases, and supports most prevelant case straight forwardly.

2nd behaviour seems better for me.

2nd behaviour to percent-decode uris complicates few other things. specifically resolving relative-identifiers, base-identifiers etc will be nightmare to specify. There may be many other issues that will popup from different standards and operations.

Thus solid should better go with first option, and must specify it only supports uris as identifiers for information-resources it manages. And should mandate to use literal uris as identifiers for resources it manages in rdf-docs. ESS goes against this as mentioned in above comment. NSS, CSS are inline.

[kjetilk]

My apologies, @damooo , for not giving this issue the attention it deserves. My mental queue is filled since we have a deadline for the current milestone today. So, please take the following as little more than loud-thinking:

On the backend, I suspect that IRIs would be prevalent, as RDF is defined in those terms. In the case of NSS, I found that it stores filenames on disc with UTF-8:

00000000   62 6C C3 A5  62 C3 A6 72  73 79 6C 74  65 74 C3 B8  79 0A                     bl..b..rsyltet..y.

("blåbærsyltetøy" = "blueberry jam" has all three non-ascii Norwegian characters in one word, and is thus my favorite word for looking into such problems :-) )

Thus, the entire problem seems to be in the upper layer of the server implementations. I don't have the bandwidth to understand the implications, but given that we could potentially have a SPARQL Endpoint towards the stored data, it doesn't seem quite attractive to me to only have percent-encoded URIs, but there is also the homograph attack problem...

However, since NSS is in line with option 1., and the short term goal for 0.9 is to describe NSS behavior, then it nevertheless seems like what we should do in the short term. Yet, there seems to be potential for a more sophisticated approach in longer term.

[damooo]

@kjetilk , thanks for response

On the backend, I suspect that IRIs would be prevalent, as RDF is defined in those terms. In the case of NSS, I found that it stores filenames on disc with UTF-8:

I want to clarify technical issue raised.
It is not about implementation-detail of which name server used for a representation persistnce on disk. But is about which identifier server assigns and uses to refer that resource in same/other rdf-documents describing/pointing to that resource in their content. like in description of container/ in acl resource, in metadata aux-resources, and in general any other. That is well-part of their public api. They differ in api at fundamental layer of resource identification, naming thus fragmenting eco-system (irreversably-in-lonterm). not just about implementation.

[kjetilk]

It is not about implementation-detail of which name server used for a representation persistnce on disk.

Certainly! I just wanted to clarify that those implementation details are not what is holding us back.

[csarven]

All, great issue and feedback.

• The motivation and applicability of RFC3987 is clear.
• The INTERNATIONAL-SPECS includes recommendations for resources identifiers, identifiers in documents, and (potential) visibility of identifiers to users.
• RDF11-CONCEPTS describes the use of IRIs.

All Solid specifications (not only SOLID-PROTOCOL) should clarify their requirements and considerations pertaining to internationalization. There is already some work on this, including spec content and open issues - for reviews - so improvements are very welcome.

---

As per AWWW, the "situation" in Solid Protocol is that while the Interaction component necessitates the use of URIs, the Identification and Data Formats components, for the most part, necessitates the use of IRIs.

I suggest that we clarify the situations in which converting IRIs to URIs, and vice versa, could happen and where new recommendations or considerations may be necessary. If so, there needs to be coherent round-tripping IRI->URI->IRI.

Note: The table below is to document and discuss. It is NOT an exhaustive list of situations and the notes are not necessarily correct (or implementable). Implementations may want to experiment or provide feedback. Test authors should ignore this.

✔: existing requirement.
?: potential requirement.

Situation	URI-to-IRI	IRI-to-URI	Notes
Sending HTTP request		x	✔ RFC7230 client [1]
Process HTTP request identifier	x		? SOLID-PROTOCOL server [2]
Writing resource	x		? SOLID-PROTOCOL server [3]
Updating RDF (server)	x		? SOLID-PROTOCOL server [4]
Updating RDF (client)	x		? SOLID-PROTOCOL client [5]
Reading RDF			? SOLID-PROTOCOL server/client [6]
Responding HTTP request		x	✔ RFC7230 server[7]

[1] Current transmission on the wire.
[2] For canonical Identification.
[3] Normalize to IRI.
[4] Typical case is when updating containment statements. Servers should use the IRI form.
[5] When a client requests to update a resource description, the server should not process the payload by converting the identifiers. Clients should use the IRI form when possible.
[6] Do not convert. Read identifier-string as is.
[7] Current transmission on the wire (e.g., Content-Location = absolute-URI / partial-URI)

[woutermont]

@csarven, I propose to take this issue (and the related #22) up in the milestone for v0.11. As long as we specify Solid's handling of identifiers, there should be no problem, so it is important to do so.

I will do a proposal based on the relevant specs and the idea above, to get the conversation going again.

[csarven]

Sure. The "idea above" being #347 (comment) ?