proposal: path/filepath: add Resolve, replacing EvalSymlinks

[ianlancetaylor]

This is a new proposal to replace #37113, which was closed for non-technical reasons.

Paraphrasing @rsc, the proposal is a new function in the path/filepath package:

// Resolve returns the path name as an absolute path that does not contain any symlinks.
// Resolve calls Clean on the result.
func Resolve(path string) string

The expectation is that on Unix systems this will be essentially filepath.Abs(filepath.EvalSymlinks(path)) and on Windows it will essentially acquire a handle for the path and call GetFinalPathNameByHandle.

Objections to this approach (in my own words, apologies if I misrepresent some position):

• We should instead make EvalSymlinks work better on Windows, such that filepath.Abs(filepath.EvalSymlinks(path)) will suffice on both Unix and Windows systems. This may involve changing EvalSymlinks to call GetFinalPathNameByHandle. However, any such change to EvalSymlinks on Windows may break programs that currently work on Windows.
• It will be tempting to think that the proposed Resolve function will return a canonical path, but it will not, neither on Unix nor Windows (on Unix it will not be canonical due to hard links and multiple mounts). Therefore this function will mislead people into writing buggy programs. In particular, os.SameFile can return true for two different paths returned by Resolve.

[rasky]

I have a couple of other reasons to vote against this proposal:

• GetFinalPathNameByHandle has 4 different modes where it produces different "canonical pathnames". The previous proposal was originated from git-lfs so the author needed the Go function to do exactly what git happens to do (in fact, even on Linux, git calls realpath(3), which I think it should return mostly the same result as Abs(EvalSymlinks(path)), but if it doesn't, we're back to step one). I disagree that filepath.Resolve should have an implementation which is matched against what another project does; if somebody needs absolute compatibility with something else, they should probably reimplement it outside std.
• Resolve and EvalSymlinks would basically be duplicates. It would be very hard to explain in the documentation why we need two similar functions, and it would confuse users. If Abs(EvalSymlinks(path)) on Windows is different from Resolve, we should document how and why, which would basically be an implementation detail with no real semantic meaning.

[mvdan]

It would be very hard to explain in the documentation why we need two similar functions, and it would confuse users.

To me, this is probably the most important drawback of the proposal.

[ericwj]

I can add to @rasky's comment that some modes do not produce any result, depending on the link type and mode used, e.g. see #39786.

I also have objections, which I have documented in #40180 and those objections carry over to this proposal for most part because EvalSymlinks is implemented with GetFinalPathNameByHandle. All links will be resolved using that for the implementation.

For these reasons I would like to see a proposal which allows application code to determine which links in a path get resolved, and for that to be ported to Unix instead of attempting to sort of port realpath(3) to Windows, such that go on Unix gains functionality that adheres to the objections listed in #40180, too.

[networkimprov]

We need a new API because EvalSymlinks is broken on Windows (how is described in #37113 & #40180), and cannot be fixed without breaking some existing callers. EvalSymlinks would be deprecated.

I agree with Eric that it's not correct to always resolve all symlinks in a path, but I also think we must rely on the WinApi to implement this on Windows. I don't see a way to do both. We should include in the Resolve docs Eric's suggestions from #40180 re best practices for handling paths not created by the application which resolves them.

As to the GetFinalPathNameByHandle mode, we'd use VOLUME_NAME_NT "Return the path with the volume device path" (since drive letters are mount points). This produces \\?\Device\device_name\path\file.ext (pls correct me if not).

https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-getfinalpathnamebyhandlew
https://docs.microsoft.com/en-us/windows/win32/fileio/displaying-volume-paths

[ericwj]

we must rely on the WinApi to implement this on Windows. I don't see a way to do both.

There are other ways besides GetFinalPathNameByHandle to resolve links, for example there is WNetGetUniversalName to translate paths with drive letters to UNC syntax. I'm sure research into the topic will yield similar API's for reparse points and the example above already hints at volume mount points and mounted folders. I don't think there are many other types of links at all, but I'm not sure.

This produces \\?\Device\device_name\path\file.ext (pls correct me if not).

I didn't get the prefix \\?, but that doesn't mean that cannot appear. But such paths are useless or almost useless for applications and probably very confusing, definitely without additional API's that accept such a path. Also the same file yields \Device\Mup\ComputerName\ShareName\file.ext when accessed over the SMB share \\ComputerName\ShareName, so it still doesn't canonicalize.

[networkimprov]

What code did you find that doesn't prepend \\? to \Device\...?

Are you saying that some WinApis can't accept paths starting \\?\Device? If not, why would they be useless?

I assume a Volume GUID prefix wouldn't be helpful since that differs on two hosts with otherwise identical configuration.

I think we've concluded that there's no such thing as a "canonical" path.

[rasky]

It would be very hard to explain in the documentation why we need two similar functions, and it would confuse users.

To me, this is probably the most important drawback of the proposal.

I note that this concern could be prevented if the proposal would also include a plan to eventually deprecate EvalSymlinks. I now notice that the issue title mentions "replacing EvalSymlinks", but there is no mention of "replacing" in the proposal text. @ianlancetaylor, can you please clarify this, by making the title and the description match one way or the other?

[ericwj]

What code did you find that doesn't prepend \? to \Device...?

Are you saying that some WinApis can't accept paths starting \?\Device? If not, why would they be useless?

They are not filesystem paths in the first place. They work with the API's used in the example - volume management API's, perhaps they also work with other kernel API's that deal with devices or with namespaces, but you cannot use them with regular file system API's like Chdir and so forth. I ran the API and didn't get \\?\. I don't know whether they can ever appear.

I assume a Volume GUID prefix wouldn't be helpful since that differs on two hosts with otherwise identical configuration.

On GPT disks they are the unique identifier listed in the partition entry. They should be stable and unique unless we're talking about cloned disks. But there are paths for which asking for the volume GUID does not yield a result. The one case I am aware of is accessing files over SMB. I don't have a computer around that doesn't have/use UEFI/GPT and I haven't tested with external disks which are usually not GPT, or any disks formatted with the old school MBR partitioning scheme. If Windows offers a volume GUID at all, it'll be a constructed one which isn't unique and maybe not stable.

[networkimprov]

So are you suggesting VOLUME_NAME_GUID? It seems to me that VOLUME_NAME_DOS is more generally useful.

[ericwj]

So are you suggesting VOLUME_NAME_GUID? It seems to me that VOLUME_NAME_DOS is more generally useful.

No, I agree with your previous statement.

I think we've concluded that there's no such thing as a "canonical" path.

VOLUME_NAME_GUID and VOLUME_NAME_DOS produce errors depending on the link type and the configuration of the target volume or share (whether it is a share or has a DOS device name associated with it) and none of them can be compared with each other. VOLUME_NAME_NT I think always returns a result, but still doesn't work for files on the network - not even if the host is localhost (I tested that) - since the result reveals that the target was accessed over SMB, not it's original location.

[networkimprov]

Well if VOLUME_NAME_DOS returns an error, can't we fall back to VOLUME_NAME_GUID?