gh-156707: Do not follow junctions in os_helper.rmtree() on Windows

[serhiy-storchaka]

os.lstat() reports a junction as a directory, so os_helper.rmtree() descended into junctions and could remove files outside of the removed tree.

os.walk() and shutil.rmtree() already skip junctions, so this only makes os_helper.rmtree() agree with them.

• Issue: Python test suite (test_ntpath?) tries to removes system directories in the volume root directory ("System Volume Information") #156707

[vstinner]

os.walk() and shutil.rmtree() already skip junctions, so this only makes os_helper.rmtree() agree with them.

For os.walk(), I found the line:

is_dir = entry.is_dir(follow_symlinks=False) and not entry.is_junction()

For shutil.rmtree(), I found the code:

lstat = srcentry.stat(follow_symlinks=False)
if lstat.st_reparse_tag == stat.IO_REPARSE_TAG_MOUNT_POINT:
    is_symlink = False

So yeah, fixing os_helper.rmtree() for junction sounds like the right fix to me.

[vstinner]

LGTM. Checking for isjunction() to decide if we should do a recursive call is the right fix.

[miss-islington-app]

Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.13, 3.14, 3.15.
🐍🍒⛏🤖

[bedevere-app]

GH-156714 is a backport of this pull request to the 3.15 branch.

[bedevere-app]

GH-156715 is a backport of this pull request to the 3.14 branch.

[bedevere-app]

GH-156716 is a backport of this pull request to the 3.13 branch.

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot aarch64 Fedora Stable LTO + PGO 3.13 (tier-2) has failed when building commit d4f5fdf.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1433/builds/1314) and take a look at the build logs.
4. Check if the failure is related to this commit (d4f5fdf) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1433/builds/1314

Summary of the results of the build (if available):

Click to see traceback logs

Note: switching to 'd4f5fdf123295427ef0cdf19548376c3b2ff65f5'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at d4f5fdf123295 [3.13] gh-156707: Do not follow junctions in os_helper.rmtree() on Windows (GH-156710) (GH-156716)
Switched to and reset branch '3.13'

find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
make[2]: [Makefile:3132: clean-retain-profile] Error 1 (ignored)
In function ‘hashtable_key_from_2_strings’,
    inlined from ‘_extensions_cache_find_unlocked’ at Python/import.c:1284:17:
Python/import.c:1197:5: warning: ‘strncpy’ output truncated before terminating nul copying as many bytes from a string as its length [-Wstringop-truncation]
 1197 |     strncpy(key, str1_data, str1_len);
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Python/import.c:1183:27: note: length computed here
 1183 |     Py_ssize_t str1_len = strlen(str1_data);
      |                           ^~~~~~~~~~~~~~~~~
./Modules/socketmodule.c: In function ‘getsockaddrarg’:
./Modules/socketmodule.c:2528:9: warning: ‘strncpy’ specified bound 64 equals destination size [-Wstringop-truncation]
 2528 |         strncpy((char *)sa->salg_name, name, sizeof(sa->salg_name));
      |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/ccVvH8gJ.s: Assembler messages:
/tmp/ccVvH8gJ.s: Fatal error: can't write 3919 bytes to section .rodata of Python/frozen.o: 'No space left on device'
/tmp/ccVvH8gJ.s: Fatal error: Python/frozen.o: No such file or directory
make[2]: *** [Makefile:3042: Python/frozen.o] Error 1
make[1]: *** [Makefile:895: profile-gen-stamp] Error 2
make: *** [Makefile:907: profile-run-stamp] Error 2

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot aarch64 Fedora Stable LTO 3.13 (tier-2) has failed when building commit d4f5fdf.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1399/builds/1275) and take a look at the build logs.
4. Check if the failure is related to this commit (d4f5fdf) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1399/builds/1275

Summary of the results of the build (if available):

Click to see traceback logs

Note: switching to 'd4f5fdf123295427ef0cdf19548376c3b2ff65f5'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at d4f5fdf12329 [3.13] gh-156707: Do not follow junctions in os_helper.rmtree() on Windows (GH-156710) (GH-156716)
Switched to and reset branch '3.13'

In function ‘hashtable_key_from_2_strings’,
    inlined from ‘_extensions_cache_find_unlocked’ at Python/import.c:1284:17:
Python/import.c:1197:5: warning: ‘strncpy’ output truncated before terminating nul copying as many bytes from a string as its length [-Wstringop-truncation]
 1197 |     strncpy(key, str1_data, str1_len);
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Python/import.c:1183:27: note: length computed here
 1183 |     Py_ssize_t str1_len = strlen(str1_data);
      |                           ^~~~~~~~~~~~~~~~~
/usr/bin/ld: final link failed: No space left on device
collect2: error: ld returned 1 exit status
make: *** [Makefile:1696: Programs/_freeze_module] Error 1