Version control with Git#
Introduction#
Version control, also known as revision control or source control, is the management and tracking of changes to computer code and other certain other types of data in an automated way.
Any project (collections of files in directories) under version control has changes and additions/deletions to its files and directories recorded and archived over time so that you can recall specific versions later.
Version control is in fact the technology embedded in the versioning of various word processor and spreadsheet applications (e.g., Google Docs, Overleaf).

Fig. 6 An overview of how version control works. Some of the project mainline’s branches may eventually be merged back into the mainline if they are successful in achieving some objective that is worth integrating into the mainline.#
Why Version Control?#
With version control of biological computing projects, you can:
record all changes made to a set of files and directories, including text (usually ASCII) data files, so that you can access any previous version of the files
“roll back” data, code, documents that are in plain text format (other file formats can also be versioned; see section on binary files below).
collaborate more easily with others on developing new code or writing documents – branch (and merge) projects
back up your project (but git is not a backup software - see sections on binary and large files below).

Fig. 7 This will very likely happen if you don’t use Version Control.
(Source: maktoons.blogspot.com).#
Why git ?#
We will use git, developed by Linus Torvalds, the “Linu” in Linux. This is currently the most popular tool for version control. It is a distributed version control system designed to handle everything from small to very large projects with speed and efficiency. It allows multiple developers to work on a project simultaneously without overwriting each other’s changes, facilitating collaboration and maintaining a history of all modifications.
The man advantages of Git are:
Distributed System: Every developer has a complete copy of the project on their computer, including its history.
Efficiency: Fast performance for both local and remote operations.
Data Integrity: Every file and project snapshot is checked for data integrity.
Flexibility: Supports “non-linear” development workflows through branching and merging.
Collaboration: Facilitates teamwork by allowing multiple people to work on different parts (typically on separate branches) simultaneously.
Let’s get started!#
In git, each user stores a complete local copy of the project, including the history and all versions (the “repository”). So you do not rely as much on a centralized (“remote” - somewhere else, accessible through the internet) server.
The Git repository is a directory that contains your project’s files and the entire history of changes made to those files. Here is a graphical outline of the git repository, workflow and command structure:

Fig. 8 The git repository, workflow and command structure. - The “workspace” is your working directory - the actual files and folders you see and edit. The index” is your staging area (more on this below) - a place where you can group changes you want to commit
together. The “local repository” contains the database of all objects (commits, files (“blobs”), directories (“trees”), etc.) in your repository along with their metadata. The index and the actual data (the objects) associated with each repo snapshot you commit are all all inside a hidden directory named .git
that sits within your project directory.#
There are several core git concepts and commands that you need to understand. But first things first - let’s first install get and set up your first git repository!
Installing git#
Let’s install and configure git
. On Ubuntu:
sudo apt-get install git
Then, some configurations:
git config --global user.name "Your Name"
git config --global user.email "your.login@imperial.ac.uk"
git config --global init.defaultBranch main
Now check your git configurations:
git config --list
This output displays the configuration variables and their corresponding values that are currently active in your Git environment. As you are just getting started, the output will consist of just a few lines.
Tip
Git configurations are stored in different files at three levels:
System Level (e.g.,
/etc/gitconfig
): Applies to all users on the system.Global Level (e.g.,
~/.gitconfig
or~/.config/git/config
): Specific to your user account.Local Level (
.git/config in the repository
): Specific to a single repository.
git config
reads these config files in order, with local settings overriding global and system settings. If you want to know which of your configurations are local vs global, use this:
git config --list --show-origin
Your first repository#
Time to bring your TMQB coursework under version control. First, let’s navigate to it:
Then initialize your local git repository:
git init
Initialised empty Git repository in /home/mhasoba/Documents/CMEECourseWork/.git/
Create your README file (as a markdown document, so with the .md
extension):
echo "My CMEE Coursework Repository" > README.md
Check again git config --list
to see if anything has changed.
Check the files and directories that have been created:
ll #same as "ls -al"
Note
Note the hidden .git
directory! (More on this below).
Add the README (AKA “stage” it for for a git commit):
git add README.md
Check the current status of your repository:
git status
On branch main
No commits yet
Changes to be committed:
(use "git rm --cached <file>..." to unstage)
new file: README.md
Untracked files:
(use "git add <file>..." to include in what will be committed)
week1/
week2/
This tells you that there are local changes that need to be committed.
git commit -m "Added README file."
[main (root-commit) b2a73ef] Added README file.
1 file changed, 1 insertion(+)
create mode 100644 README.md
Tip
In git commands, you can also combine flags like in base UNIX commands; so you can both, add (-a
flag) and commit (-m
flag) at one go, with -am
.
Check status again:
git status
On branch main
Untracked files:
(use "git add <file>..." to include in what will be committed)
week1/
week2/
nothing added to commit but untracked files present (use "git add" to track)
What does it say now? - Compare with the above output of git status
.
You can also add changes matching a particular pattern.
For example
git add *.txt
will add all text files that were changed since last commit.
Note that this adds ALL changes that match the pattern, recursively (through all sub-directories).
Next you can commit the rest of these files inside your current coursework (e.g., week 1
) with another message, just like you did for readme
above:
Warning
Before making your first commit, please read the section about “Ignoring files” below.
git commit -m "Full first commit of new project"
On branch main
Untracked files:
(use "git add <file>..." to include in what will be committed)
week1/
week2/
nothing added to commit but untracked files present (use "git add" to track)
git status
On branch main
Untracked files:
(use "git add <file>..." to include in what will be committed)
week1/
week2/
nothing added to commit but untracked files present (use "git add" to track)

Fig. 9 Make meaningful comments in your git commit. Please don’t neglect to make each commit message meaningful. And use this mantra: “commit often, comment always”. The correct meaning of “often” in this context comes with experience.
(Source: XKCD)#
Commits#
A commit is a snapshot of your repository at a specific point in time. Each commit records:
The changes made (differences from the previous commit).
Metadata such as the author, date, and a commit message.
A unique hash that identifies the commit.
Note
The unique hash (SHA-1 checksum) in Git is a 40-character hexadecimal string. It looks like this:
e83c5163316f89bfbde7d9ab23ca2e25604af290
(You will soon see examples of this below)
It is generated by applying the SHA-1 (Secure Hash Algorithm 1) cryptographic function to the contents and metadata of a Git object. OK, that was some serious Gobbledygook! All you need to know is that this hash acts as a unique identifier for all commits, files, directories, and other objects within a Git repo so that you can track specific snapshots of the repo.
How Git Stores Data#
Git doesn’t store data as a series of changes or differences, but as a series of snapshots of the entire repository. When you make a commit
, Git stores a reference to the staged snapshot and a pointer to the commit that came immediately before it.
Git uses four types of Objects to store data:
Blobs: Stores file data.
Trees: Represents directories and contains pointers to blobs and other trees.
Commits: Points to a tree object (snapshot of the working directory), parent commits, and includes metadata.
Tags: Points to another object (usually a commit) and includes metadata.
These objects are stored in the .git/objects
directory and are identified by their SHA-1 hash.
Basic git commands#
Here are some fundamentally important git commands (please make sure you read more about them):
Command |
What it does |
---|---|
|
Initialize a new repository |
|
Download a repository from a remote server |
|
Show the current status |
|
Show differences between commits |
|
Blame somebody for the changes! |
|
Show commit history |
|
Commit changes to current branch |
|
Show branches |
|
Create new branch |
|
Switch to a different commit/branch called |
|
Gather commits info from a remote branch to your current local branch without merging them |
|
Merge two or more versions of one or more files |
|
Upload from remote repository (this is effectively the same as |
|
Send changes to remote repository |
There are more git commands which you will also learn, but it is important that you familiarize yourself with these ones in particular first.
The most common sequence of your git work and commands will be:
Edit Files: Modify files in your working directory.
Stage Changes: Use
git add <file>
to add changes to the staging area.Committ Changes: Use
git commit -m "commit message"
to save changes to the repository.
But before you do that, you will always want to
Use
git status
to see the status of your working directory and staging area.Use
git log
to view the commit history.
Your remote repository#
Remote repositories are versions of your project hosted on the internet or another network. Common platforms include GitHub, GitLab, and Bitbucket.
Nothing has been sent to the remote server yet (more on the remote server below).
So let’s go to your online git service (e.g., github) and set up. Note that github and bitbucket both give you unlimited free private repositories if you register with an academic email. Not a big deal if you will not be writing private code, handy if you are (can you think of examples when you would need to write private code?).
So let’s proceed with connecting your local git repository to your remote server:
Login to your github or bitbucket account
Set up your
ssh
-based access. SSH (S
ecureS
ocket Sh
ell) is a protocol that allows you to connect to and interact with remote servers. Here are two sets of guidelines (you can use whichever seems easier to you, irrespective of whether you are using githib or bitbucket):
Note
When accessing remote Git repositories, you have the option to use either HTTPS or SSH protocols. Both methods have their own advantages and disadvantages, and the better choice depends on your specific needs and circumstances.
In general, use SSH if:
You interact with Git repositories frequently.
You prefer a secure, key-based authentication method.
You’re setting up automated scripts or development workflows.
Use HTTPS if:
You’re in a restrictive network environment where SSH is blocked.
You prefer a simpler setup without managing SSH keys.
You’re accessing repositories infrequently or from different machines.
We will use SSH beause it is generally better for regular code development due to its security and convenience after the initial setup. You may need to use HTTPS in situations where network restrictions apply.
Next, create a new repository on your remote service with the same name as your local project (e.g.,
CMEECourseWork
), and push your new project to this newly created remote git repository. Instructions for this step are here:
Note that you have already done the git init
step, so no need to repeat those bits.
You are done. Now you can really start to use git!
The first step after having created your remote repository and added your ssh key to it, is to link the remote to your local repo (as the instructions in web pages linked above will already have told you).
The command will look like this:
git remote add origin git@github.com:YourGithubUsername/CMEECourseWork.git
Then list the URLs of the remote repositories associated with your local Git repository
git remote -v
The output would look like this
origin git@github.com:YourGithubUsername/CMEECourseWork.git (fetch)
origin git@github.com:YourGithubUsername/CMEECourseWork.git (push)
The output provides both fetch and push URLs for each remote, showing where Git pulls updates from and where it pushes updates to. This is helpful for understanding which remote repository is set as the target for various Git commands.
In the above example output:
origin
is the name of the remote (the default name for the main remote when cloning the repository).https://github.com/username/repo-name.git
is the URL for the remote repository on GitHub.In this case
fetch
andpush
have that the same URL for both fetching updates and pushing commits.
No you can git push
all your local commits:
git push origin master
This pushes the (committed) changes in your local repository up to the remote repository you specified as the origin
. Note that master
refers to the branch (you currently only have one). More on branching below.
Note
Only when you push
or fetch
do you need an internet connection, because before that you are only archiving in a local (hidden) repository (that sits in a hidden .git
directory within your project).
General command sequence for connecting to Remotes#
Here the a general sequence of commands for connecting a new remote
Add the Remote: Use
git remote add <name> <url>
to connect your local repository to a remote one.Fetch Changes: Use
git fetch <remote>
to download commits, files, and references from a remote repository.Pull Changes: Use
git pull
to fetch and integrate changes from a remote repository into your current branch.Push Changes: Use
git push <remote> <branch>
to upload your commits to a remote repository.
Branching#
Imagine you want to try something out, but you are not sure it will work well.
For example, say you want to rewrite the Introduction of your
git branch anexperiment
git branch
anexperiment
* main
git tells you that you have two branches.
Warning
If you are not seeing this specific output (with two branches) at this stage of your git chapter work, scroll back and re-check / rerun the above sequence of commands again! Note that we had renamed all master
branches as main
above (if you had not done that, your main
branch will be called master
instead).
git checkout anexperiment
Switched to branch 'anexperiment'
git branch
* anexperiment
main
git tells you that you have successfully switched to the new branch
Or if you want to create and switch to a new branch immediately, you can use the -b
flag:
git checkout -b anexperiment
Now make a change (update the Readme to record the new naughty thing you have started doing!):
echo "I am going to try this new naughty thing on this experimental branch!" >> README.md
cat README.md
My CMEE Coursework Repository
I am going to try this new naughty thing on this experimental branch!
git commit -am “Testing experimental branch”
The -am
flag performs the commit with a message while also automatically staging certain changes before committing.
Specifically, the a
part of the flag stages all changes to tracked files (files that have already been committed at least once). This flag does not stage new, untracked files; you must use git add
to stage those separately.
Now if you decide to merge the new branch after modifying it:
git checkout main
M README.md
Switched to branch 'main'
git merge anexperiment
Already up-to-date.
cat README.md
My CMEE Coursework Repository
I am going to try this new naughty thing on this experimental branch!
Unless there are conflicts, i.e., some other files that you changed locally had diverged from those files in the main
branch in the meantime (say, due to new changes pushed by another collaborator), you are done, and you can delete the branch:
git branch -d anexperiment
Deleted branch anexperiment (was 3fbab16).
If instead you are not satisfied with the result, and you want to abandon the branch:
git branch -D anexperiment
Note
If there are conflicts between branches, Git will notify you. You’ll need to resolve these conflicts manually in the files and then stage the resolved files with git add
, followed by a git commit
to complete the merge.
Obvously, we have been just messing around, so you don’t really want the Readme of your main
branch to say what it is currently saying!
To fix this, you now have two options: edit the Readme back to the state you wanted (and then add
, commit
and push
), or use the git reset --hard
command. However, use the latter with caution. Before you do anything to reverse the merge, read about resetting and revertinf below.
Tip
When you want to test something out, always branch! Reverting changes, especially in code, is typically painful. Merging can be tricky, especially if multiple people have simultaneously worked on a particular document. In the worst-case scenario, you may want to delete the local copy and re-clone the remote repository.

Fig. 10 Try not to do this. But most of us mortals will have, at some point!
(Source: XKCD)#
Note
You can perform Git branching and merging entirely within a local repository without needing a remote.
Teamwork using git and common branch
ing mistakes#
As you can see, branchhing is the cornerstone of collaborative coding and software development using git.
Here are common mistakes to avoid with git branching and merging in team projects:
Working Directly on the Main Branch
Issue: Making changes directly to the
main
ormaster
branch without using feature branches.Why It’s a Problem: This practice can lead to a cluttered commit history and increases the risk of introducing unstable code into the main codebase, affecting all team members.
Not Keeping Branches Up-to-Date
Issue: Failing to regularly merge or rebase the main branch into feature branches.
Why It’s a Problem: Leads to large, complex merge conflicts when it’s time to integrate changes back into the main branch.
Not Pulling Latest Changes Before Starting Work
Issue: Forgetting to fetch and merge the latest changes from the remote repository before starting new work.
Why It’s a Problem: Leads to working on outdated code and causes significant merge conflicts when pushing changes later.
Improper Branching Practices
Issue: Not creating separate branches for new features or bug fixes, or using inconsistent branch naming conventions.
Why It’s a Problem: Makes it difficult to manage code changes, track progress, and collaborate effectively.
Poor Commit Hygiene
Issue: Making large, infrequent commits with vague messages like “Fixed stuff” or “Changes.”
Why It’s a Problem: Hinders the ability to trace specific changes, debug issues, and understand the project’s evolution.
Not Handling Merge Conflicts Properly
Issue: Overwriting code, deleting others’ work, or incorrectly resolving conflicts without understanding the implications.
Why It’s a Problem: Can introduce bugs, cause loss of important code, and disrupt the functionality of the application.
Overwriting Team Members’ Changes
Issue: Using
git push --force
or not merging the latest remote changes before pushing local commits.Why It’s a Problem: Leads to loss of others’ work and creates confusion in the team’s codebase.
Ignoring the .gitignore File
Issue: Committing unnecessary files like binaries, temporary files, or sensitive information.
Why It’s a Problem: Clutters the repository, increases clone/download times, and poses security risks.
Misusing Git Commands
Issue: Incorrectly using commands like
git reset
,git revert
, or rebasing public branches.Why It’s a Problem: Can rewrite commit history, cause loss of work, and confuse team members due to unexpected repository states.
Lack of Communication and Coordination
Issue: Not informing team members about new branches, changes, or issues encountered.
Why It’s a Problem: Leads to duplicated work, merge conflicts, and a lack of cohesion in development efforts.
Pushing Broken or Untested Code
Issue: Committing untested code.
Why It’s a Problem* Disrupts the workflow for others, especially if continuous integration pipelines fail due to the broken code.
Confusion Between Local and Remote Branches
Issue: Forgetting to push local branches to the remote repository or misunderstanding the state of branches.
Why It’s a Problem: Causes discrepancies between local and remote repositories, leading to confusion and potential loss of work.
Conflict Between Personal and Team Workflows
Issue: Using personal Git configurations or workflows that don’t align with the team’s agreed-upon practices.
Why It’s a Problem: Creates inconsistency and can cause issues with code integration and collaboration.
Over-Reliance on GUI Tools Without Understanding Git Basics
Issue: Using graphical interfaces without grasping the underlying Git commands.
Why It’s a Problem: Limits the ability to troubleshoot issues and understand what’s happening behind the scenes, leading to mistakes.
Ignoring Merge Warnings and Errors
Issue: Proceeding with merges without addressing warnings or errors.
Why It’s a Problem: Can result in incomplete merges, broken code, and unresolved conflicts that affect the entire team.
Mismanaging Merge Strategies
Issue: Inappropriately using
git merge
vs.git rebase
without understanding the consequences.Why It’s a Problem: Can create a confusing commit history and complicate collaboration efforts.
Deleting Branches Prematurely
Issue: Removing branches that others are still using or that contain unmerged changes.
Why It’s a Problem: Causes loss of work and disrupts team members who are relying on those branches.
Tips to Avoid These Mistakes:
Educate Yourself: Take time to learn Git fundamentals, including branching, merging, and resolving conflicts.
Use Feature Branches: Create a new branch for each feature or bug fix to isolate your work.
Commit Often with Clear Messages: Make small, incremental commits with descriptive messages.
Pull Frequently: Regularly fetch and merge changes from the remote repository to stay up-to-date.
Communicate: Keep open lines of communication with your team about what you’re working on.
Test Before Committing: Ensure your code compiles and passes tests before pushing.
Understand Git Commands: Before using advanced commands like
git reset
orgit rebase
, make sure you understand their effects.Use .gitignore Wisely: Configure your
.gitignore
file to exclude unnecessary or sensitive files.Review Pull Requests: Participate in code reviews to catch issues early and share knowledge.
Follow Team Conventions: Adhere to agreed-upon workflows, branch naming conventions, and commit message formats.
Tip
Cloning vs. Forking: Forking a git repository allows you to create a personal copy of someone else’s repository on your own GitHub account. This is ideal if you want to contribute to the original repository. You can make changes in your forked copy and then submit those changes back to the original repository through pull requests. Merely Cloning on the other hand, only creates a local copy on your machine. It does not provide a way to contribute changes back to the original repository without special permissions. Forking is a common approach in open-source projects because it doesn’t require permission from the original repository’s owner to make changes. You can work on your copy and submit a pull request without needing write access to the original repository.
The README file#
A README (like the README.md
that you created in your git repo above) is a text file that introduces and explains a project. It contains information that is required to understand what the project is about and how to use or run it.
While READMEs can be written in any text file format, Markdown (saved as an .md
file) is most commonly used, as it allows you to add simple text formatting easily. Two other formats that you might most often see are plain text and reStructuredText (saved as an .rst
file, common in Python projects).
You can find many README file suggestions (and templates) online. Essentially, it should ideally have the following features/content:
Project name / title
Brief description: what your project does and/or is for. Provide context and add links to any references to help new visitors.
Languages: List language(s) and their versions used in the project
Dependencies: What special packages (which are not part of standard libraries of the language(s) used) might be needed for a new user to run your project
Installation: Guidelines for installing the project (if applicable), including dependencies.
Project structure and Usage: How the project is structured and how to run/use it. Explain, if relevant, what specific files do. No need to list every file, such as data or experimental ones (like the ones in
sandbox
).Author name and contact
In addition, you may want to include(but not necessary for your current coursework), License, Acknowledgments, and instructions for Contributing.
Ignoring Files#
You will have some files you don’t want to track (log files, temporary files, executables, etc). You can ignore entire classes of files with .gitignore
.
CMEECourseWork
)):
echo -e "*~ \n*.tmp" > .gitignore
cat .gitignore
*~
*.tmp
git add .gitignore
touch temporary.tmp
Then,
git add *
You can also create a global gitignore file that lists rules for files to be ignored in every Git repository on your computer.
Tip
Templates for .gitignore: You can find standard .gitignore
templates online. For example, search for “.gitignore templates”.
Directory and file patterns for gitignoring#
It’s important that you use the correct patterns for what to ignore in your .gitignore
, to make sure that all the correct directories and files are being excluded. Note that you can put additonal .gitignore
files in sub-directories if you need more fine-grained control over what is to be ignored, bt it is not necessary, because git-ignoring works recursively (including thorugh sub-directories) as long as you include the correct pattern.
Here are the basic patterns (rules) :
Pattern |
gitignore result |
---|---|
|
This is a comment - this gitignore entry will be ignored (avoid having any file or directory with |
|
Every file OR directory (because lack of trailing |
|
Every directory (due to the trailing |
|
Every file OR directory named |
|
Every directory named target in the top-most directory (because of the leading and trailing |
|
Every file or directory ending with the extension |
|
Every file or directory ending in |
There are additonal patterns - please see the Readings and Resources section.
.gitignore
-ng after commit
-ing#
If you find that a file or directory belonging to a pattern that you incuded in your .gitignore
fails to be ignored (still comes under version control), it most likely means that you gitignored it AFTER committing and pushing it. In this scenario, you need to use
git rm --cached <file>
for a file, and
git rm -r --cached <folder>
for a directory.
While these commands will not remove the physical file from your local repository, it will remove the files from other locals on their next git pull
.
General good practice for gitignoring#
When writing a gitignore it is best to ignore any file that does not contain plaintext code. This type of file is what git was designed for!
If you do need to include a small csv file, image, or output (such as when creating reports or including test data) it is a good idea to do this using exceptions (aka the ! symbol)
For example the following gitignore file would ignore ALL csv files except for pushabledata.csv
*.csv
!pushabledata.csv
Tip
As git reads gitignore files top-to-bottom and can overwrite contradicticting rules with others, it is important to make sure that these are in the correct order.
Whitelist principle#
Gitignore files are usually used as a blacklist: where anything you add to your git repository is allowed unless you specifically outlaw it in the gitignore.
An alternative, safer technique (though one which is definitely more work to maintain) is a whitelist-based approach.
To do this, you firstly ignore every file with the pattern *.*
and then explicitly allow files and file types that you know you want. For example:
*.*
!.gitignore
!*.R
!*.py
!*.sh
!*.tex
!*.bib
!Week1/results/myfirstlatexdoc.tex
This is not necessarily a better or worse way to do things, you are in the end trading your time and effort for increased safety and cleanliness of your repository.
Dealing with binary files#
A binary file is computer-readable but not human-readable, that is, it cannot be read by opening them in a text viewer. Examples of binary files include compiled executables, zip files, images, word documents and videos. In contrast, text files are stored in a form (usually ASCII) that is human-readable by opening in a text reader (e.g., gedit). Without some git extensions and configurations (coming up next), binary files cannot be properly version-controlled because each version of the entire file is saved as is in a hidden directory in the repository (.git
).
However, with some more effort, git can be made to work for binary formats like *.docx
or image formats such as *.jpeg
, but it is harder to compare versions; have a look at this and this1, and also, this.
Dealing with large files#
As such, git was designed for version control of workflows and software projects, not large files (say, >100mb) (which may be plain-text or binary). Binary files are particularly problematic because each version of the file is saved as is in .git
, when you have a large number of versions it means that there are the same number of binary files in the hidden directory (for example 100
So please do not keep large files (especially binary files) under version control2. For example, if you are doing GIS work, you may have to handle large raster image files. Do not bring such files under version control. We suggest that you include files larger than some size in your .gitignore
. For example, you can use the following bash command:
find . -size +100M | cat >> .gitignore
The 100M means 100 mb – you can reset it to whatever you want.
Then what about code that needs large files? For this, the best approach is write code that scales up with data size. If it works on a 1 mb file, it should also work on a 1000 mb file! If you have written such code, then you can include a smaller file as a MWE (minimum working example).
Tip
And how do you back up your large data files? Remember, version control software like git are not meant for backing up data. The solution is to back up separately, either to an external hard drive or a cloud service. rsync
is a great Linux utility for making such backups. Look it up on the ‘net!
You may also explore alternatives such as git-annex
(e.g., see this), and git-lfs
(e.g., see this).
Tip
Checking size of your git repo: You have two options in Linux/UNIX to check the size of your git repo. You can use (cd
to your repo first) du -sh .git
, or for more detailed information about what’s using the space, use git count-objects -vH
(this will work across platforms as this is a git command).
Pre-Commit Git Hook#
pre-commit
is a framework that manages the installation and execution of pre-commit hooks across multiple programming languages (including Python and R). These hooks are scripts that run automatically every time a git commit
is made, aborting the commit if the script fails. These can be used to enforce standards in code format and style.
So, how would this be useful in practice?
To avoid the occurrence of a bloated git history, i.e. by mistakenly committing a large file to your repository, a good option is to use the check-added-large-files
hook from the pre-commit-hooks
package.
This will abort a commit if the file size is too large (default size limit = 500kB).
Here’s how you can install and configure check-added-large-files
on Linux:
pip install pre-commit
Create new configuration file
.pre-commit-config.yaml
for configuring thecheck-added-large-files
hook; its contents should look like:
# .pre-commit-config.yaml
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v5.0.0 # or use the latest version
hooks:
- id: check-added-large-files
args: ['--maxkb=500'] # optional: set a custom size limit in KB
Here,
repo
is the GitHub repository where thepre-commit-hooks
package is hosted.rev
is the version of thepre-commit-hooks
repository to use. You can specify a tag (likev5.0.0
) or usemaster
for the latest version, though pinning to a specific version is best for stability.hooks
are a list of hooks to configure:id
: Specifies the hook to use, in this case,check-added-large-files
.args
: Optional argument to set the maximum file size limit (in KB). By default, it is 500 KB, but you can adjust this as needed with--maxkb=<size_in_kb>
.
This configuration will ensure that no added files exceed the specified size limit before a commit is allowed.
Note: By default, only files that have been staged for addition to the repository via git add
will be checked by the check-added-large-files
hook.
To allow the hook to check all files in the repo (both staged and unstaged), add --enforce-all
as an argument by modifying the above configuration yaml file with
args: ['--maxkb=1000', '--enforce-all']
The .pre-commit-config.yaml
file should be stored in the root directory of your repository. This allows pre-commit
to detect the configuration automatically when you run the pre-commit install
command, which sets up the hooks for the specific repository. Once set up, pre-commit
will refer to this configuration file each time you attempt to make a commit.
To set up the hooks after creating this file, run the following command in the root of your repository:
pre-commit install
This installs the hooks locally, ensuring they run on every commit.
Removing files#
To remove a file (i.e., stop version controlling it) use git rm
:
echo "Text in a file to remove" > FileToRem.txt
git add FileToRem.txt
git commit -am "added a new file that we'll remove later"
[master 706f40d] added a new file that we'll remove later
2 files changed, 3 insertions(+)
create mode 100644 .gitignore
create mode 100644 FileToRem.txt
git rm FileToRem.txt
rm 'FileToRem.txt'
git commit -am "removed the file"
[master 284c62e] removed the file
1 file changed, 1 deletion(-)
delete mode 100644 FileToRem.txt
I typically just make all my changes and then just use git add -A
for the whole directory (and it’s subdirectories; -A
is recursive).
Un-tracking files#
.gitignore
will prevent untracked files from being added to the set of files tracked by git. However, git will continue to track any files that are already being tracked. To stop tracking a file you need to remove it from the index. This can be achieved with this command.
git rm --cached <file>
The removal of the file from the head revision will happen on the next commit.
Accessing history of the repository#
To see particular changes introduced, read the repo’s log :
git log
For a more detailed version, add -p
at the end.
Reverting to a previous version#
If things go horribly wrong with new changes, you can revert to the previous, “pristine” state:
git reset --hard
git commit -am "returned to previous state" #Note I used -am here
If instead you want to move back in time (temporarily), first find the “hash” for the commit you want to revert to, and then check-out:
git status
And then,
git log
Then, you can
git checkout *version number*
e.g, git checkout 95f7d0
Now you can play around. However, if you do want to commit changes, you create a “branch” (see below). To go back to the future, type
git checkout master
Running git commands on a different directory#
Since git
version 1.8.5, you can run git directly on a different directory than the current one using absolute or relative paths. For example, using a relative path, you can do:
git -C ../SomeDir/ status
Tip
Cloning the MulQuaBio repository: You can clone the master repository of this book on your computer and regularly git pull it to keep it updated. That way the these notes and the associated code and data files are easily and locally available on your computer. DO NOT clone the repository into your own coursework repository though!
Running git commands on multiple repositories at once#
For git pulling in multiple subdirectories (each a separate repository), here is an example:
find . -mindepth 1 -maxdepth 1 -type d -print -exec git -C {} pull \;
Breaking down these commands one by one,
find .
searches the current directory
-type d
finds directories, not files
-mindepth 1
sets min search depth to one sub-directory
-maxdepth 1
sets max search depth to one sub-directory
-exec git -C {} pull \
runs a custom git command one on every git repo found
Using git through a GUI#
There are many nice git GUI’s (Graphical User Interfaces) out there, such as gitKraken. Or if you are using a code editor like Visual Studio Code, there are nice extensions that will give you considerable GUI functionality.
Tip
Checking git status: Always run the git status
command on a repository before pulling/fetching from, or pushing to a remote repository!
A friendly concluding note on git#
Git can feel hard and unintuitive. It is likely that at some point you will make a mistake and lose some work as a consequence.
THIS IS NORMAL
Don’t worry about it, this has happened to pretty much everyone who has ever used git.
When things like this do happen though, it is imperative that you work out what went wrong so you can make sure it doesn’t happen again.
Questions to ask yourself include:
Did I use
git status
to check what I was adding?Did I commit regularly enough, whenever I had completed a logical section of work?
Did I make sure to only commit code rather than committing data files and outputs too?
Did I make sure to gitignore the things I needed to?
When you get used to working with git, it becomes as natural as juggling chainsaws: i.e. it will always feel a bit odd and strange but it won’t be as scary as it once was.
Next Steps:
Learn Commands: Familiarize yourself with common Git commands.
Explore Branching: Practice creating, switching, and merging branches.
Use Hosting Services: Try using platforms like GitHub to collaborate with others.
Advanced Features: Delve into rebasing, cherry-picking, and submodules for more complex workflows.
Practicals#
Instructions#
Don’t modify anything (or refer to any files) in your local copy of the git repository of these notes (on Github). Copy whatever you need from the master repository to your own repository.
Git commit and push every time you do some significant amount of coding work (after testing it), and then again before the given deadline (this will be announced in class). This includes UnixPrac1.txt
from the Unix Chapter.
Push your git repository
The only practical submission for git is pushing your coursework git repository,
.gitignore
andreadme
files included. Make sure your.gitignore
has meaningful exclusions, and yourreadme
has useful information (as explained in the section above). Also search online “readme good practices” or something like that to find additional tips/info.Also, invite your assessor to your coursework repository (e.g,
CMEECourseWork
) repository with write privileges (AKA push privilege in GitHub). The current assessor is s.pawar@imperial.ac.uk (or “mhasoba” on both bitbucket and github).
Readings & Resources#
General#
Excellent book on Git: http://git-scm.com/book
Look up the Bitbucket Git resources
Really great git tutorials: https://try.github.io
Markdown#
A cheatsheet for github-flavored Markdown: adam-p/markdown-here
Committing practices#
Some tips for good commit practices: https://chris.beams.io/posts/git-commit/
.gitignore#
Guidelines for the
.gitignore
file: https://labs.consol.de/development/git/2017/02/22/gitignore.html, https://docs.github.com/en/free-pro-team@latest/github/using-git/ignoring-files
Branching#
Guidelines for brancing: https://gist.github.com/digitaljhelms/4287848
Footnotes
1: There you will find the following phrase: “…one of the most annoying problems known to humanity: version-controlling Microsoft Word documents.”. LOL!
2: None of the computing weeks assessments will require you to use such large files anyway