HTTP Directory – What is the Purpose of the .well-known Folder?

directoryhttplog-files

If've found a new error message in our log-files and would like to know, for what this .well_known-folder stands for.

Which application-client would need to access such a folder and which application would create files inside it?

Here are some entries of the PHP Error log of one of my domain. (I removed date, ip and target-domains).

0000/00/00 00:00:00 [error] 851#0: *88611 access forbidden by rule, client: xxx.xxx.xxx.xxx, server: example.com, request: "GET /.well-known/apple-app-site-association HTTP/1.1", host: "exampleA.com"
0000/00/00 00:00:00 [error] 850#0: *89749 access forbidden by rule, client: xxx.xxx.xxx.xxx, server: example.com, request: "GET /.well-known/assetlinks.json HTTP/1.1", host: "exampleA.com"
0000/00/00 00:00:00 [error] 850#0: *89767 access forbidden by rule, client: xxx.xxx.xxx.xxx, server: example.com, request: "GET /.well-known/assetlinks.json HTTP/1.1", host: "exampleB.com"
0000/00/00 00:00:00 [error] 853#0: *90120 access forbidden by rule, client: xxx.xxx.xxx.xxx, server: example.com, request: "GET /.well-known/apple-app-site-association HTTP/1.1", host: "exampleB.com"
0000/00/00 00:00:00 [error] 853#0: *90622 access forbidden by rule, client: xxx.xxx.xxx.xxx, server: example.com, request: "GET /.well-known/apple-app-site-association HTTP/1.1", host: "www.exampleB.com"
0000/00/00 00:00:00 [error] 853#0: *90926 access forbidden by rule, client: xxx.xxx.xxx.xxx, server: example.com, request: "GET /.well-known/assetlinks.json HTTP/1.1", host: "www.exampleA.com"
0000/00/00 00:00:00 [error] 854#0: *91780 access forbidden by rule, client: xxx.xxx.xxx.xxx, server: example.com, request: "GET /.well-known/apple-app-site-association HTTP/1.1", host: "exampleA.com"

First I thought that I could be the one who generated this, but at the times I wasn't accessing/working these domains. And these access requests comes from 3 of our domains. (with different web-applications)

INFO1: It seems the IP is from the Google-Bot (Crawler) But what are
so important to access these files? (we don't have these files in the folders, checked for hidden in all domain-root-directorys.)

Best Answer

That /.well-known/ subdirectory is defined by RFC 5785 RFC 8615

It is increasingly common for Web-based protocols to require the discovery of policy or other information about a host ("site-wide metadata") before making a request. For example, the Robots Exclusion Protocol http://www.robotstxt.org/ specifies a way for automated processes to obtain permission to access resources; likewise, the Platform for Privacy Preferences [W3C.REC-P3P-20020416] tells user-agents how to discover privacy policy beforehand.

While there are several ways to access per-resource metadata (e.g., HTTP headers, WebDAV's PROPFIND [RFC4918]), the perceived overhead (either in terms of client-perceived latency and/or deployment difficulties) associated with them often precludes their use in these scenarios.

When this happens, it is common to designate a "well-known location" for such data, so that it can be easily located. However, this approach has the drawback of risking collisions, both with other such designated "well-known locations" and with pre-existing resources.

To address this, this memo defines a path prefix in HTTP(S) URIs for these "well-known locations", /.well-known/. Future specifications that need to define a resource for such site-wide metadata can register their use to avoid collisions and minimise impingement upon sites' URI space.

The reason that you see access forbidden errors might be the result of a blanket block on requests for hidden files/folders (paths starting with a dot .).
In case you do have useful content in /.well-known, this Q&A may be of interest.

Locations within that directory are then used for specific purposes,

Both of which support a similar purpose, they allow the site operator to instruct a visitor to open the site in an associated app, rather than in the (mobile) browser.

IANA maintains a comprehensive list of assigned well-known locations on www.iana.org/assignments/well-known-uris/well-known-uris.xhtml and a similar list on Wikipedia also includes a few different URI's that are not officially assigned and registered by IANA.

Related Topic