Skip to content

libs/template: don't create directories that a later {{skip}} names - #6348

Open
TangoEnSkai wants to merge 1 commit into
databricks:mainfrom
TangoEnSkai:fix/template-skip-dir-created-empty
Open

libs/template: don't create directories that a later {{skip}} names#6348
TangoEnSkai wants to merge 1 commit into
databricks:mainfrom
TangoEnSkai:fix/template-skip-dir-created-empty

Conversation

@TangoEnSkai

Copy link
Copy Markdown

Changes

renderer.walk records every directory it visits, and persistToDisk creates
them all so a template's structure survives even where files were skipped.
That is deliberate. But the walk tests the skip patterns only at the moment it
reaches a directory.

{{skip}} directives are registered while rendering files, so a file
processed after a directory has already been visited can name that directory.
By then the walk has recorded it, and the directory is created anyway — empty,
because its own files do match the pattern and are dropped.

This re-tests the accumulated patterns when directories are materialized.

Directories whose files merely happened to be skipped are unaffected: those
patterns (dir/*) match the files, not the directory, so dir is still
created. That keeps the behaviour the surrounding comment asks for — "Only
explicit {{skip}} directives should prevent directory creation" — which is
exactly what is not true today.

Why

The current behaviour contradicts that comment: an explicit {{skip}} naming
a directory is honoured only if it happens to be registered before the walk
reaches it. Whether it is depends on BFS ordering — which sibling directory
sorts first, and which file the directive lives in — so the same directive
works or silently leaves an empty directory depending on where it is written.

This looks like the mechanism behind #6142, which reports bundle init
leaving a stray empty directory next to the correctly generated bundle. I
could not confirm that end to end, since the template in that report is not
public, so I have not marked this as fixing it — but the symptom matches, and
the ordering dependence above would explain why it reproduces with some
template variants and not others.

Tests

go test ./libs/template/... ./cmd/bundle/... — all green.

Added TestRendererSkipDirectoryVisitedBeforeSkipDirective with a fixture
where dir2/file2.tmpl skips dir1, which the BFS walk visits first. The
test asserts dir1 is absent and dir2/file2 is written; it fails on main
with directory ".../dir1" exists.

The walk records each directory it visits and persistToDisk creates them all,
so a template ends up with the directory structure even where every file was
skipped. That is deliberate, but it tests the skip patterns only at the moment
the walk reaches a directory.

{{skip}} directives are registered while rendering files, so a file processed
after a directory has been visited can name that directory. The walk has
already recorded it by then and the directory is created regardless -- empty,
because its own files do match the pattern and are dropped.

Re-test the accumulated patterns when materializing directories. Directories
whose files merely happened to be skipped are unaffected: those patterns match
the files, not the directory, which keeps the existing behaviour of preserving
a template's structure.
@github-actions

Copy link
Copy Markdown
Contributor

Approval status: pending

/libs/template/ - needs approval

4 files changed
Suggested: @lennartkats-db
Also eligible: @janniklasrose, @pietern, @denik, @andrewnester, @anton-107, @shreyas-goenka

General files (require maintainer)

Files: .nextchanges/cli/template-skip-dir-not-created.md
Based on git history:

  • @lennartkats-db -- recent work in libs/template/, .nextchanges/cli/

Any maintainer (@andrewnester, @anton-107, @denik, @pietern, @shreyas-goenka, @simonfaltum, @renaudhartert-db, @janniklasrose, @lennartkats-db, @rugpanov, @rclarey) can approve all areas.
See OWNERS for ownership rules.

@github-actions

Copy link
Copy Markdown
Contributor

An authorized user can trigger integration tests manually by following the instructions below:

Trigger:
go/deco-tests-run/cli

Inputs:

  • PR number: 6348
  • Commit SHA: b957cab2e03a11d195c3a96c76d73f24b942f3bc

Checks will be approved automatically on success.

@TangoEnSkai

Copy link
Copy Markdown
Author

@lennartkats-db tagging you as the bot's suggestion for libs/template/.

The one judgement call worth your eyes: I deliberately kept the "create directories whose files were all skipped" behaviour intact, and only suppress a directory that a {{skip}} pattern names directly. That reading comes from the comment already in persistToDisk — "Only explicit {{skip}} directives should prevent directory creation" — which is what the code does not currently honour, since the directive is respected only when BFS ordering happens to register it before the walk reaches the directory.

If you would rather that a directory be dropped whenever every one of its files was skipped, that is a different and larger behaviour change and I would leave it out of this PR.

The new test fails on main with directory ".../dir1" exists, so it does pin the regression rather than just describing it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant