Paths under %appdata% are written to the local cache instead

[Bip901]

Bug report

Bug description:

All paths under C:\Users\username\AppData\REST-OF-THE-PATH are actually treated as C:\Users\username\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\REST-OF-THE-PATH.

This happens regardless of working in a venv or not, and regardless of the library call you're using (i.e. pathlib and open behave the same)

with open(R"C:\Users\your-username\AppData\Test.txt", "w") as f:
  f.write("Test")

Expected Result

A file with the text "Test" is created under C:\Users\your-username\AppData\Test.txt

Actual Result

The file is created under C:\Users\your-username\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\Test.txt

Additional Information

• I'm using Python 3.12 installed from the Microsoft Store
• Python 3.12.4 (tags/v3.12.4:8e8a4ba, Jun 6 2024, 19:30:16) [MSC v.1940 64 bit (AMD64)] on win32
• This issue was first reported at least a year ago in this Stack Overflow post

CPython versions tested on:

3.12

Operating systems tested on:

Windows

[Bip901]

This only seems to happen in the Windows Store version of Python.

[eryksun]

The store app is isolated for certain filesystem and registry paths. The intent is to prevent apps from interfering with each other and to support clean removal of uninstalled apps. The redirections are implemented by drivers in the Windows kernel, so I don't think there's anything we can do about it.

[Bip901]

I assume there's a well-known list of redirected directories - perhaps the Windows Store version can be modified to emit a warning when trying to access them? The current behavior is very misleading (I'm surprised Microsoft themselves decided to silently redirect instead of returning a permissions error).

[zooba]

This is the intended design of the operating system.

I suspect you actually ran into this via a different issue than what was described in the original post. Files that Python creates under AppData are for Python to access, and it can always successfully access them, which suggests you were trying to write files for a different app?

Perhaps if you describe the actual issue then we can find a workaround.

[Bip901]

There are 2 main use cases for AppData:

• Modifying another app's files. As a scripting language, Python is a good candidate for doing things like that. It should be clear to the user that this is not allowed by the Microsoft store version, instead of fooling them that something happened.
• Writing a Python app that uses best Windows practices. The Windows standard is to write app-specific files to %AppData%/AppName, which not only provides a standard user experience (they expect to find configuration there) but also behaves differently to %LocalAppData% - it roams in organizational networks.

[zooba]

Both of those cases should work with the Store version, as more recent (in the last 3-4 years) versions of Windows changed the shadowing behaviour to only affect newly created files. If you try to read another app's files, or open for writing, you should get the actual file.

In particular, the Store package does actually use best Windows practices. Those practices are defined by Windows, not by users, and the behaviour you're seeing here is what Windows says is best practice for apps (which specifically includes "don't use other apps' directories under AppData").

[Bip901]

This still doesn't solve creating new files, which is sometimes required (e.g. some apps have configuration subdirectories that files should be added to to define certain settings)

And most importantly- %LocalAppData% doesn't roam!

[zooba]

If the app's folder already exists, new files will be created in the existing one. It's just things created directly in AppData\Local or AppData\Roaming that go to the app's own folder.

App data stored in the app-specific directory roams, backs-up/restores, resets, and uninstalls far more reliably than the old mechanisms. If you think it doesn't, you should report it to Windows, because it's their system and not ours.

[StrawberryStego]

As a workaround, using pathlib.Path.resolve(strict=False), where pathlib.Path is a path that refers to something in /AppData/, will refer to the actual location, i.e., .../LocalCache/... This'll be helpful if you actually need to refer to the location of your file.

You'll have to create the file/folder first, otherwise resolve( ) will refer to .../LocalCache/... which doesn't exist yet.

[zooba]

We aren't doing Store app releases anymore - runtimes installed using the Python install manager won't have this defence against breaking user machineslimitation.