Always squash when pulling a different subrepo branch?

[svelez]

While implementing some processes, I noticed that I pull doesn't behave as I expect it to... perhaps only when switching the branch tracked by the subrepo.

Instead of having all commits from the super-repos subrepo folder applied to the subrepo history on commit, I get a single, squashed commit.

To demonstrate this, I have published https://github.com/svelez/problem-superrepo-demo.

Once that is cloned, issue git subrepo pull --branch working sub

There is a conflict in that pull which lets me more easily demonstrate the resulting subrepo history:

$ git -C .git/tmp/subrepo/sub log --oneline
ee7e336 (HEAD -> subrepo/sub, refs/subrepo/sub/branch) Silly change
10c91b3 (refs/subrepo/sub/commit) making changes
ff1e95f init commit

Which shows just one change made in the superrepo after the subrepo clone when there were actually two.

Contrarily, if I simply execute the branch command, I get this:

$ git subrepo branch sub
Created branch 'subrepo/sub' and worktree '.git/tmp/subrepo/sub'.
$ git -C .git/tmp/subrepo/sub log --oneline
872e81e (HEAD -> subrepo/sub, refs/subrepo/sub/branch) Silly change
7d58f36 Adding file that is in other branch
10c91b3 (refs/subrepo/sub/commit) making changes
ff1e95f init commit

Which is what I would expect to see when resolving conflicts doing the pull.

Is this the intended behavior?

After some investigation, it appears to me that the current, actual behavior is caused by this line and if I comment it out, I get the behavior that I would expect.

However, I don't understand enough about the system to know if there is ever a valid case for hard-coding HEAD's parent.

Any insight you could provide would be appreciated. Is there a more correct method to do what I want to do (have the super repo start tracking a different branch of the subrepo)

I am working off of the release/0.4.0_rebase_to_master branch

Thanks

[grimmySwe]

Hi @svelez, could you change to branch release/0.4.0, just to verify that the problem isn't already solved.

If you want to track another branch, you can git subrepo --force -b new_branch clone <repo> then you will use that state instead.

[svelez]

Hello @grimmySwe, apparently I was mistaken about my base-branch and this issue is reproducible in release/0.4.0.

Using --force clone does allow the operation to complete, but I am afraid it is worse than the squash I am getting now because all of the edits to the subrepo that happened in the super repo are lost.

[grimmySwe]

@svelez If you want to switch tracking branch AND keep some local changes I suggest that you first push your local changes to you current tracking branch. Then you clone the actual subrepo as a standalone repo, merge the branches in question in that repo. There you will have full control of what you want to do. Then you simply clone the result back into your master repo.

In git lingo:

cd /parent_repo
git subrepo push <subrepo>
cd /somewhere/else
git clone <subrepo>
cd subrepo_clone
git checkout target_branch
git merge other_branch
cd /parent_repo
git subrepo clone --force -b target_branch <subrepo>

Currently git subrepo doesn't support pulling in changes from other branches as it can't determine the relation of these changes as the .gitrepo file only contains the last known information.

[svelez]

That's disappointing. However, I am having a hard time understanding why this should be necessary if the upstream branch you are switching to has shared history with the upstream branch you are switching from.

Assuming that subrepo pull is just a sequence of suprepo fetch, subrepo branch, merge, then given a graph like this

                D1 - E1 - F1 <- subrepo/sub/branch
              /
A - B - C - D - E - F <- subrepo/master
          \
            G - H - I - J <- subrepo/working

I am not sure what is or should be the fundamental difference between subrepo fetch getting the master or working branches.

In fact, the issue I am experiencing is with the creation of subrepo/sub/branch during the subrepo branch operation because it looks like this:

A - B - C - D - F1 <- subrepo/branch

But only when the subrepo branch command is executed on my behalf as part of subrepo pull.

If I do the steps manually, it appears to me that the process works "Correctly" and would imply a defect in subrepo pull. In spite of the title that I chose for this issue, it would seem that it also happens when pulling the currently tracked branch as long as there is more than one commit in the subrepo dir from the super repo.

For comparison...

pull:

$ git subrepo pull sub

Subrepo 'sub' pulled from 'https://github.com/svelez/problem-subrepo-demo.git' (master).
$ git log --oneline subrepo/sub
*   5c8c1c5 (refs/subrepo/sub/commit, refs/subrepo/sub/branch, subrepo/sub) Merge commit 'refs/subrepo/sub/fetch' into subrepo/sub
|\
| * 3045d98 (refs/subrepo/sub/fetch) Progressing upstream
* | f76d04a Silly change
|/
* 10c91b3 making changes
* ff1e95f init commit

manual pull:

$ git reset --hard HEAD^
$ git subrepo clean sub
Removed branch 'subrepo/sub'.
$ git subrepo fetch sub
Fetched 'sub' from 'https://github.com/svelez/problem-subrepo-demo.git' (master).
$ git subrepo branch sub
Created branch 'subrepo/sub' and worktree '.git/tmp/subrepo/sub'.
$ git -C .git/tmp/subrepo/sub merge refs/subrepo/sub/fetch
Merge made by the 'recursive' strategy.
 newfileonmaster | 0
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 newfileonmaster
$ git subrepo commit sub
Subrepo commit 'subrepo/sub' committed as
subdir 'sub/' to branch 'master'.
$ git log --oneline subrepo/sub
*   d208ee6 (refs/subrepo/sub/commit, subrepo/sub) Merge commit 'refs/subrepo/sub/fetch' into subrepo/sub
|\
| * 3045d98 (refs/subrepo/sub/fetch) Progressing upstream
* | 7371c42 (refs/subrepo/sub/branch) Silly change
* | 1b692de Adding file that is in other branch
|/
* 10c91b3 making changes
* ff1e95f init commit

Why are the subrepo history graphs different in these two cases?

[grimmySwe]

@svelez Hi Steven, thanks for your feedback, I will see if i can look into this more during the week. I made a quick test and it looks like you are on to something.

[grimmySwe]

@svelez For the first part of your question:
Yes, it should be possible to do something smart here allowing you to use the -b flag to merge in changes. I'll see if I can make that work as expected.

[svelez]

Great. Thanks for the update.

[grimmySwe]

Hi @svelez, I have spent a couple of hours considering the options with the -b flag. With the current version of git-subrepo, there is unfortunately no easy way to do this. Why?

Git-subrepo is based on a .gitrepo file that contains the current status of the subrepo within the parent repo. As it is now, we only have one reference into the subrepo here. That reference tells us the last known commit and is used to see if we need to update our main repo and in case of a push we use it to create a virtual reference to connect our commits so we can merge them.

Consider the use case:

1$ git subrepo clone <repo> bar
2$ git subrepo pull -b another_branch bar
3$ git subrepo pull bar

In my opinion this should pull in changes from another_branch in step 2, but in step 3 it should have knowledge that it was up to date on the tracking branch.

With the current version of git-subrepo we have to decide if we want to reference the latest commit that we pulled (from another_branch) and in that case forget the latest commit on our tracking branch. So in 3 that would cause us to try and pull the original changes again. On the other hand we could ignore updating our latest commit, but then the changes from another_branch would be "cherry-picked" and if we push, there will be no trace of that operation.

I do see that we could add another parameter to the .gitrepo file so that we can store both the current tracking position AND the last pull. That way we could recreate the correct tree when we push so that we don't mess up the subrepo.

I will create a new issue for this, so that we can work on this for future releases.

For the other part I am investigating and it looks like you are correct that we set the parent wrong. I will see if I can solve that.

[svelez]

OK. Thanks for the explanation (and the fix). I have further questions, but will follow up in #328

[grimmySwe]

I will reopen this issue and keep it open until release/0.4.0 hits master.