Submodules

Often in a project, you want to include libraries and other resources. The manual way is to simply download the necessary code files, copy them to your project, and commit the new files into your Git repository.

While this is a valid approach, it's not the cleanest one. By casually throwing those library files into your project, we're inviting a couple of problems:

  • This mixes external code with our own, unique project files. The library, actually, is a project of itself and should be kept separate from our work. There's no need to keep these files in the same version control context as our project.
  • Should the library change (because bugs were fixed or new features added), we'll have a hard time updating the library code. Again, we need to download the raw files and replace the original items.

Since these are quite common problems in everyday projects, Git of course offers a solution: Submodules.

Repositories Inside Repositories

A "Submodule" is just a standard Git repository. The only specialty is that it is nested inside a parent repository. In the common case of including a code library, you can simply add the library as a Submodule in your main project.

A Submodule remains a fully functional Git repository: you can modify files, commit, pull, push, etc. from inside it like with any other repository.

Let's see how to work with Submodules in practice.

Adding a Submodule

In our sample project, we want to add a little Javascript library called "ToProgress". We create a new "lib" folder to host this (and future) library code; and a new "ToProgress" folder inside.

Clicking the little "+" button on the bottom left of the Tower window, we choose "Add Submodule...". In the dialog, we paste the remote URL of the library and choose our new and empty destination folder:

Let's see what happened after confirming this dialog:

  • (1) The command started a simple cloning process of the specified Git repository.

  • (2) Of course, this is reflected in our file structure: our project now contains the library code in our "lib/ToProgress" directory. As you can see from the ".git" subfolder contained herein, this is a fully-featured Git repository.
Concept

It's important to understand that the actual contents of a Submodule are NOT stored in its parent repository. Only its remote URL, the local path inside the main project and the checked out revision are stored by the main repository.

Of course, the Submodule's working files are placed inside the specified directory in your project - in the end, you want to use the library's files! But they are not part of the parent project's version control contents.

  • (3) A new ".gitmodules" file was created. This is where Git keeps track of our Submodules and their configuration:

    [submodule "lib/ToProgress"]
        path = lib/ToProgress
        url = https://github.com/djyde/ToProgress
  1. In case you're interested in the inner workings of Git: besides the ".gitmodules" configuration file, Git also keeps record of the Submodule in your local ".git/config" file. Finally, it also keeps a copy of each Submodule's .git repository in its internal ".git/modules" folder.
Concept

Git's internal management of Submodules are quite complex (as you can already guess from all the .gitmodules, .git/config, and .git/modules entries...). Therefore, it's highly recommended not to mess with configuration files and values manually. Please do yourself a favor and always use manage Submodules through Tower.



Let's have a look at our project's status:

Git regards adding a Submodule as a modification like any other - and requests you to commit it to the repository. After that, we have successfully added a Submodule to our main project! Before we look at a couple of use cases, let's see how you can clone a project that already has Submodules added.

Cloning a Project with Submodules

You already know that a project repository does not contain its Submodules' files; the parent repository only saves the Submodules' configurations as part of version control.
This shows when you clone a project that contains Submodules: by default, Git only downloads the project itself. Our "lib" folder, however, would stay empty.

You have two options to end up with a populated "lib" folder (or wherever else you choose to save your Submodules; "lib" is just an example):

  • (a) You can check the "Initialize Submodules" option in the "Clone Remote Repository" dialog in Tower; this tells Tower to also initialize all Submodules when the cloning is finished.

  • (b) If you haven't used this option, you need to initialize the Submodules afterwards: in Tower's sidebar, right-click the Submodule items and choose the "Initialize" option.

Checking Out a Revision

A Git repository can have countless committed versions, but only one version's files can be in your working directory. Therefore, like with any Git repository, you have to decide which revision of your Submodule shall be checked out.

Concept

Unlike normal Git repositories, Submodules always point to a specific commit - not a branch. This is because the contents of a branch can change over time, as new commits arrive. Pointing at a specific revision, on the other hand, guarantees that the correct code is always present.

Let's say we want to have an older version of our "ToProgress" library in our project. First, we'll have a look at the library's commit history. Simply double-click the Submodule item in Tower's sidebar. This opens it for working in Tower; any actions you perform now will happen in the context of the Submodule, not its parent repository.

Now, in the log output, we spot a commit that is tagged "0.1.1":

This is the version we want to have in our project. To start with, we can simply check out this commit. Right-click it in the list and select "Check Out 3557a0e0".

Let's see what our parent repository thinks about all this. Simply select it in the navigation bar below the button toolbar to return to the main project.

On the right, we see that the Subproject is now checked out at the commit we've just chosen. This makes sense - since we just changed the checked out revision to the commit tagged "0.1.1".

In the "Status" list in the middle column, we see that Git regards moving the Submodule's pointer as a change like any other. We need to commit this to the repository in order to make it official.

Updating a Submodule with a Moved Pointer

We just saw how to check out a Submodule at a specific revision. But what if one of our teammates does this in our project? Let's say we integrate his changes (through pull, merge, or rebase for example) after he has moved the Submodule pointer to a different revision.

Now, after completing the "pull" operation, Tower detects that the Submodule pointer was moved - the version we currently have checked out in our project is not the one that is "officially" committed.

If you choose to let Tower automatically update, it does the heavy lifting for you in the background. "Update", when referring to Submodules, means that Git checks out the Submodule revision that is currently committed in the parent repository.

We now have the same version of the Submodule checked out that our teammate had committed to the repository.

Checking for New Changes in the Submodule

Normally, you don't want library code to change very often: you'll want to use a version of the Submodule that you've tested and which you know works flawlessly with your own code.
However, one of the best things about Submodules is that you can easily keep up with new releases (or minor new improvements).

Let's see if there's new code available in the Submodule: simply double-click the Submodule item in the sidebar to open it in Tower and perform a "Fetch".

It seems we're lucky because the Fetch reveals a new commit on "origin/master":

Concept

Before we go ahead and integrate these changes, I'd like to stress an important point once more. When looking at the BRANCHES section in Tower's sidebar, you see that the Submodule repository is currently checked out a specific commit - not a branch! This is what's called a detached HEAD in Git.

Normally, in Git, you always have a certain branch checked out. However, you can also choose to check out a specific commit (one that is not the tip of a branch). This is a rather rare case in Git and should normally be avoided.

However, when working with Submodules, it is the normal state to have a certain commit (and not a branch) checked out. You want to make sure you have an exact, static commit checked out in your project - not a branch which, by its nature, moves on with newer commits.

Now, let's check out our Submodule at this new commit. In this case, you can simply perform a "pull" from "origin/master" to make this commit your new HEAD.

We're done working in our Submodule; let's move back into our main project by selecting it in the navigation bar breadcrumb. Since we've just moved the Submodule pointer to a different revision, we need to commit this change to the main repository to make it official.

Working in a Submodule

In some cases, you might want to make some custom changes to a Submodule. You've already seen that working in a Submodule is like working in any other Git repository: any Git commands that you perform inside a Submodule are executed in the context of that sub-repository.

Let's say you want to change a tiny bit in a Submodule; you make your changes in the corresponding files, add them to the staging area and commit them.

This might already be the first banana skin: you should make sure you currently have a branch checked out in the Submodule before you commit. That's because if you're in a detached HEAD situation, your commit will easily get lost: it's not attached to any branch and will be gone as soon as you check out anything else.

Apart from that, everything else you've already learned still applies: in the main project, Tower will tell you that the Submodule pointer was moved and that you'll have to commit the move.

By the way: in case you have uncommitted local changes inside a Submodule, Tower will also tell you in the main project. A badge next to the Submodule in the sidebar and a description text in its detail view help you stay on top of things:

Make sure to always keep a clean state in your Submodules.

Deleting a Submodule

Rather seldomly will you want to remove a Submodule from your project. But if you really want to do this, please don't do this manually: trying to mess with all the configuration files in a correct way will almost inevitably cause problems.

Simply right-click the Submodule in the sidedbar and choose "Delete". This will allow you to have both the configuration and the actual Submodule files removed.
As with any change you'll have to commit the removal to record it in the repository.

Get our Free Cheat Sheet Package

About Us

As the makers of Tower, the best Git client for Mac and Windows, we help over 80,000 users in companies like Apple, Google, Amazon, Twitter, and Ebay get the most out of Git.

Just like with Tower, our mission with this platform is to help people become better professionals.

That's why we provide our guides, videos, and cheat sheets (about version control with Git and lots of other topics) for free.