How does git submodule add -b
work?
After adding a submodule with a specific branch, a new cloned repository (after git submodule update --init
) will be at a specific commit, not the branch itself (git status
on the submodule shows "Not currently on any branch").
I can't find any information on .gitmodules
or .git/config
about the submodule's branch or any specific commit, so how does Git figure it out?
Also, is it possible to specify a tag instead of a branch?
I'm using version 1.6.5.2.
I'd like to add an answer here that is really just a conglomerate of other answers, but I think it may be more complete.
You know you have a Git submodule when you have these two things.
Your .gitmodules has an entry like so: [submodule "SubmoduleTestRepo"] path = SubmoduleTestRepo url = https://github.com/jzaccone/SubmoduleTestRepo.git You have a submodule object (named SubmoduleTestRepo in this example) in your Git repository. GitHub shows these as "submodule" objects. Or do git submodule status from a command line. Git submodule objects are special kinds of Git objects, and they hold the SHA information for a specific commit. Whenever you do a git submodule update, it will populate your submodule with content from the commit. It knows where to find the commit because of the information in the .gitmodules. Now, all the -b does is add one line in your .gitmodules file. So following the same example, it would look like this: [submodule "SubmoduleTestRepo"] path = SubmoduleTestRepo url = https://github.com/jzaccone/SubmoduleTestRepo.git branch = master Note: only branch name is supported in a .gitmodules file, but SHA and TAG are not supported! (instead of that, the branch's commit of each module can be tracked and updated using "git add .", for example like git add ./SubmoduleTestRepo, and you do not need to change the .gitmodules file each time) The submodule object is still pointing at a specific commit. The only thing that the -b option buys you is the ability to add a --remote flag to your update as per Vogella's answer: git submodule update --remote Instead of populating the content of the submodule to the commit pointed to by the submodule, it replaces that commit with the latest commit on the master branch, THEN it populates the submodule with that commit. This can be done in two steps by djacobs7 answer. Since you have now updated the commit the submodule object is pointing to, you have to commit the changed submodule object into your Git repository. git submodule add -b is not some magically way to keep everything up to date with a branch. It is simply adds information about a branch in the .gitmodules file and gives you the option to update the submodule object to the latest commit of a specified branch before populating it.
Note: Git 1.8.2 added the possibility to track branches. See some of the answers below.
It's a little confusing to get used to this, but submodules are not on a branch. They are, like you say, just a pointer to a particular commit of the submodule's repository.
This means, when someone else checks out your repository, or pulls your code, and does git submodule update, the submodule is checked out to that particular commit.
This is great for a submodule that does not change often, because then everyone on the project can have the submodule at the same commit.
If you want to move the submodule to a particular tag:
cd submodule_directory
git checkout v1.0
cd ..
git add submodule_directory
git commit -m "moved submodule to v1.0"
git push
Then, another developer who wants to have submodule_directory changed to that tag, does this
git pull
git submodule update --init
git pull
changes which commit their submodule directory points to. git submodule update
actually merges in the new code.
cd my_submodule; git checkout [ref in submodule's repository
yields fatal: reference is not a tree: ...
. It's as if git
will only operate on the parent repository.
git checkout v1.0
a branch or a tag?
(Git 2.22, Q2 2019, has introduced git submodule set-branch --branch aBranch -- <submodule_path>
)
Note that if you have an existing submodule which isn't tracking a branch yet, then (if you have git 1.8.2+):
Make sure the parent repo knows that its submodule now tracks a branch: cd /path/to/your/parent/repo
git config -f .gitmodules submodule.
Make sure your submodule is actually at the latest of that branch: cd path/to/your/submodule git checkout -b branch --track origin/branch # if the master branch already exist: git branch -u origin/master master
(with 'origin' being the name of the upstream remote repo the submodule has been cloned from.
A git remote -v
inside that submodule will display it. Usually, it is 'origin')
Don't forget to record the new state of your submodule in your parent repo: cd /path/to/your/parent/repo git add path/to/your/submodule git commit -m "Make submodule tracking a branch"
Subsequent update for that submodule will have to use the --remote option: # update your submodule # --remote will also fetch and ensure that # the latest commit from the branch is used git submodule update --remote # to avoid fetching use git submodule update --remote --no-fetch
Note that with Git 2.10+ (Q3 2016), you can use '.
' as a branch name:
The name of the branch is recorded as submodule.
But, as commented by LubosD
With git checkout, if the branch name to follow is ".", it will kill your uncommitted work! Use git switch instead.
That means Git 2.23 (August 2019) or more.
See "Confused by git checkout
"
If you want to update all your submodules following a branch:
git submodule update --recursive --remote
Note that the result, for each updated submodule, will almost always be a detached HEAD, as Dan Cameron note in his answer.
(Clintm notes in the comments that, if you run git submodule update --remote
and the resulting sha1 is the same as the branch the submodule is currently on, it won't do anything and leave the submodule still "on that branch" and not in detached head state.)
To ensure the branch is actually checked out (and that won't modify the SHA1 of the special entry representing the submodule for the parent repo), he suggests:
git submodule foreach -q --recursive 'branch="$(git config -f $toplevel/.gitmodules submodule.$name.branch)"; git switch $branch'
Each submodule will still reference the same SHA1, but if you do make new commits, you will be able to push them because they will be referenced by the branch you want the submodule to track. After that push within a submodule, don't forget to go back to the parent repo, add, commit and push the new SHA1 for those modified submodules.
Note the use of $toplevel
, recommended in the comments by Alexander Pogrebnyak.
$toplevel
was introduced in git1.7.2 in May 2010: commit f030c96.
it contains the absolute path of the top level directory (where .gitmodules is).
dtmland
adds in the comments:
The foreach script will fail to checkout submodules that are not following a branch. However, this command gives you both:
git submodule foreach -q --recursive 'branch="$(git config -f $toplevel/.gitmodules submodule.$name.branch)"; [ "$branch" = "" ] && git checkout master || git switch $branch' –
The same command but easier to read:
git submodule foreach -q --recursive \
'branch="$(git config -f $toplevel/.gitmodules submodule.$name.branch)"; \
[ "$branch" = "" ] && \
git checkout master || git switch $branch' –
umläute refines dtmland's command with a simplified version in the comments:
git submodule foreach -q --recursive 'git switch $(git config -f $toplevel/.gitmodules submodule.$name.branch || echo master)'
multiple lines:
git submodule foreach -q --recursive \
'git switch \
$(git config -f $toplevel/.gitmodules submodule.$name.branch || echo master)'
Before Git 2.26 (Q1 2020), a fetch that is told to recursively fetch updates in submodules inevitably produces reams of output, and it becomes hard to spot error messages.
The command has been taught to enumerate submodules that had errors at the end of the operation.
See commit 0222540 (16 Jan 2020) by Emily Shaffer (nasamuffin
).
(Merged by Junio C Hamano -- gitster
-- in commit b5c71cc, 05 Feb 2020)
fetch: emphasize failure during submodule fetch Signed-off-by: Emily Shaffer
In cases when a submodule fetch fails when there are many submodules, the error from the lone failing submodule fetch is buried under activity on the other submodules if more than one fetch fell back on fetch-by-oid. Call out a failure late so the user is aware that something went wrong, and where.
Because fetch_finish() is only called synchronously by run_processes_parallel, mutexing is not required around submodules_with_errors.
Note that, with Git 2.28 (Q3 2020), Rewrite of parts of the scripted "git submodule" Porcelain command continues; this time it is "git submodule set-branch
" subcommand's turn.
See commit 2964d6e (02 Jun 2020) by Shourya Shukla (periperidip
).
(Merged by Junio C Hamano -- gitster
-- in commit 1046282, 25 Jun 2020)
submodule: port subcommand 'set-branch' from shell to C Mentored-by: Christian Couder Mentored-by: Kaartic Sivaraam Helped-by: Denton Liu Helped-by: Eric Sunshine Helped-by: Đoàn Trần Công Danh Signed-off-by: Shourya Shukla
Convert submodule subcommand 'set-branch' to a builtin and call it via git submodule.sh.
foreach
script will not depend on the hardcoded <path>
, if you substitute <path>
with $toplevel/
.
foreach
script will fail to checkout submodules that are not following a branch. However, this command gives you both: git submodule foreach -q --recursive 'branch="$(git config -f $toplevel/.gitmodules submodule.$name.branch)"; [ "$branch" = "" ] && git checkout master || git checkout $branch'
git submodule foreach -q --recursive 'git checkout $(git config -f $toplevel/.gitmodules submodule.$name.branch || echo master)'
git submodule update --remote --merge
or git submodule update --remote --rebase
. These commands do the tracking of the remote branch.
Git 1.8.2 added the possibility to track branches.
# add submodule to track branch_name branch
git submodule add -b branch_name URL_to_Git_repo optional_directory_rename
# update your submodule
git submodule update --remote
See also Git submodules
.gitmodules
file?
git submodule add -b tags/<sometag> <url>
which you can see as the line branch = tags/<sometag>
in .gitmodules
An example of how I use Git submodules.
Create a new repository Then clone another repository as a submodule Then we have that submodule use a tag called V3.1.2 And then we commit.
And that looks a little bit like this:
git init
vi README
git add README
git commit
git submodule add git://github.com/XXXXX/xxx.yyyy.git stm32_std_lib
git status
git submodule init
git submodule update
cd stm32_std_lib/
git reset --hard V3.1.2
cd ..
git commit -a
git submodule status
Maybe it helps (even though I use a tag and not a branch)?
git reset --hard V3.1.2
? I just get a "nothing to commit" with a git status
of the parent directory.
In my experience switching branches in the superproject or future checkouts will still cause detached HEADs of submodules regardless if the submodule is properly added and tracked (i.e. @djacobs7 and @Johnny Z answers).
And instead of manually checking out the correct branch manually or through a script git submodule foreach can be used.
This will check the submodule config file for the branch property and checkout the set branch.
git submodule foreach -q --recursive 'branch="$(git config -f $toplevel.gitmodules submodule.$name.branch)"; git checkout $branch'
Git submodules are a little bit strange - they're always in "detached head" mode - they don't update to the latest commit on a branch like you might expect.
This does make some sense when you think about it, though. Let's say I create repository foo with submodule bar. I push my changes and tell you to check out commit a7402be from repository foo.
Then imagine that someone commits a change to repository bar before you can make your clone.
When you check out commit a7402be from repository foo, you expect to get the same code I pushed. That's why submodules don't update until you tell them to explicitly and then make a new commit.
Personally I think submodules are the most confusing part of Git. There are lots of places that can explain submodules better than I can. I recommend Pro Git by Scott Chacon.
git clone git://github.com/git/git.git
and push that feature...? =D
git
10000000 times a day is still preferrable to the beastiality other versioning tools deem sensible.
To switch branch for a submodule (assuming you already have the submodule as part of the repository):
cd to root of your repository containing the submodules
Open .gitmodules for editing
Add line below path = ... and url = ... that says branch = your-branch, for each submodule; save file .gitmodules.
then without changing directory do $ git submodule update --remote
...this should pull in the latest commits on the specified branch, for each submodule thus modified.
Existing answers have the second step missing and overloaded with details.
To switch an existing submodule to track a new remote url and/or a new branch:
Edit the source of truth in .gitmodules.
For example, from
[submodule "api"]
path = api
url = https://github.com/<original_repo>/api.git
to
[submodule "api"]
path = api
url = https://github.com/<another_repo>/api.git
branch = work-in-progress
git submodule sync: Updates the description of submodules cached by git in .git/modules from the just-edited source of truth specified in .gitmodules. git submodule update --init --recursive --remote: Updates the checked-out submodules in the working copy. Commit the changes.
I have this in my .gitconfig
file. It is still a draft, but proved useful as of now. It helps me to always reattach the submodules to their branch.
[alias]
######################
#
# Submodules aliases
#
######################
# git sm-trackbranch: places all submodules on their respective branch specified in .gitmodules
# This works if submodules are configured to track a branch, i.e if .gitmodules looks like:
# [submodule "my-submodule"]
# path = my-submodule
# url = git@wherever.you.like/my-submodule.git
# branch = my-branch
sm-trackbranch = "! git submodule foreach -q --recursive 'branch=\"$(git config -f $toplevel/.gitmodules submodule.$name.branch)\"; git checkout $branch'"
# sm-pullrebase:
# - pull --rebase on the master repo
# - sm-trackbranch on every submodule
# - pull --rebase on each submodule
#
# Important note:
# - have a clean master repo and subrepos before doing this!
# - this is *not* equivalent to getting the last committed
# master repo + its submodules: if some submodules are tracking branches
# that have evolved since the last commit in the master repo,
# they will be using those more recent commits!
#
# (Note: On the contrary, git submodule update will stick
# to the last committed SHA1 in the master repo)
sm-pullrebase = "! git pull --rebase; git submodule update; git sm-trackbranch ; git submodule foreach 'git pull --rebase' "
# git sm-diff will diff the master repo *and* its submodules
sm-diff = "! git diff && git submodule foreach 'git diff' "
# git sm-push will ask to push also submodules
sm-push = push --recurse-submodules=on-demand
# git alias: list all aliases
# useful in order to learn git syntax
alias = "!git config -l | grep alias | cut -c 7-"
We use Quack to pull a specific module from another Git repository. We need to pull code without the whole code base of the provided repository - we need a very specific module / file from that huge repository and should be updated every time we run update.
So we achieved it in this way:
Create configuration
name: Project Name
modules:
local/path:
repository: https://github.com/<username>/<repo>.git
path: repo/path
branch: dev
other/local/path/filename.txt:
repository: https://github.com/<username>/<repo>.git
hexsha: 9e3e9642cfea36f4ae216d27df100134920143b9
path: repo/path/filename.txt
profiles:
init:
tasks: ['modules']
With the above configuration, it creates one directory from the provided GitHub repository as specified in first module configuration, and the other one is to pull and create a file from the given repository.
Other developers just need to run
$ quack
And it pulls the code from the above configurations.
The only effect of choosing a branch for a submodule is that, whenever you pass the --remote
option in the git submodule update
command line, Git will check out in detached HEAD mode (if the default --checkout
behavior is selected) the latest commit of that selected remote branch.
You must be particularly careful when using this remote branch tracking feature for Git submodules if you work with shallow clones of submodules. The branch you choose for this purpose in submodule settings IS NOT the one that will be cloned during git submodule update --remote
. If you pass also the --depth
parameter and you do not instruct Git about which branch you want to clone -- and actually you cannot in the git submodule update
command line!! -- , it will implicitly behave like explained in the git-clone(1)
documentation for git clone --single-branch
when the explicit --branch
parameter is missing, and therefore it will clone the primary branch only.
With no surprise, after the clone stage performed by the git submodule update
command, it will finally try to check out the latest commit for the remote branch you previously set up for the submodule, and, if this is not the primary one, it is not part of your local shallow clone, and therefore it will fail with
fatal: Needed a single revision Unable to find current origin/NotThePrimaryBranch revision in submodule path 'mySubmodule'
git submodule add -b develop --name branch-name -- https://branch.git
git submodule add -b master --name master -- https://github.com/nlohmann/json.git libs/json
Cheers for the tip!
Success story sharing
.gitmodules
and after doing$ git submodule update --init --remote TestModule
I got an error sayingfatal: Needed a single revision
andUnable to find current origin/TestTag revision in submodule path 'TestModule'
. When doing it with a real branch it works. Is there anyway to specify a tag in.gitmodules
without having to specify the exact commit?.gitmodules
and rangit submodule update
and nothing happened?