ideas: Sort various topics by relative order of importance.

[tutorial.git] / ideas

* Introduction

** Intended audience
People already familiar with an SCM (CVS/Subversion).  Developers.
Slides intended to be more standalone than presentation-based.

** Normal SCM workflow
Centralized server from which people checkout working copies and commit
changes.  (diagram)

** Distributed SCM
Usually a centralized server (reference point) which people clone.
What makes a truly distributed SCM:
  - The ability to exchange revisions without the centralized server.
  - Anybody can become the reference point of someone else.
(diagram)

** Revision numbers
Incremental numbers no longer make sense in DSCM.  (explain with a diagram)

* Why Git is a good (D)SCM, why do the switch?

** Problems with traditional SCMs
Common problems:
  - Branch management
  - Commit policy

Branch management: branches are costly to create and nightmare to maintain.
Merging branches is utterly cumbersome.  Typical example with SVN: one must
manually track the merges as to not merge the same thing twice.  Merging
flattens the history (loose the commit logs, makes the history extremely
hard to review, prevents "blame" from working properly, doesn't handle file
renames).

Commit policy: without commit (write) access, one can't work without extra
tools (e.g., quilt).  Who must be given write access?  What can be
committed?  Typical example: "trunk" (main development line) must always
work (compile, pass the testsuite...).  Problems: "Idea.  Hack, hack, hack.
Almost good but not quite perfect.  Some problems must be solved before the
commit.  Hack, hack, hack.  Send a huge commit."  Does not encourage code
review.

** What's good in Git
Very lightweight branches where merging is made trivial.

Encourages code review (will demonstrate this later in the workflows.)

Cryptographically secure.  100% reliable.  GPG-signed tags.

Feature rich (git-grep, bisect, GUIs, gitweb, rerere, submodules, etc.)

Blazingly fast.  Space efficient.

Patches are first class objects.  Very good integration with emails (to
send/receive and apply patches).

Interaction with other SCMs (SVN in particular).

The ability to work offline.  Exchange commits with others.

Made the UNIX way: lots of small programs.  Easily scriptable.

Huge and quickly growing community, many active developers, project moving
at a fast pace and quickly spreading.

Cherry on the cake: colourful :)

** What's not so good with Git
Windows port is on its way but not as good as that of CVS/SVN.

Git is obviously harder to learn than SVN, because it has more concepts.

* The basics

** The git commands
"git-foo" vs "git foo" (completion in dumb shells, whether you have 1
command in your PATH or 137 "git-*")

"git foo --help" = "man git-foo"

Git has two types of commands: Plumbing commands and Porcelain.  The
"plumbings" are more low-level commands on top of which "porcelain" are
written.  Git used to be very low-level and several tools came on top of it
to make it a friendly SCM (e.g. the deprecated Cogito).  Since then, Git has
evolved a lot and its numerous "porcelain" commands make it a real SCM.

** Telling git who you are
  $ git config --global user.name "Your Name"
  $ git config --global user.email you@example.domain.com
Stored in ~/.gitconfig

** Creating a working copy
Creating a repository (git init).  This operation will create an empty
directory with a single .git directory in it.  That's where everything will
be and there will be a single .git directory (unlike the .svn directories).
Unlike SVN's .svn directories, you are allowed to look at the files in the
.git directory, they're not like a "very-internal implementation detail"
that you shouldn't use.  In particular, there is a file .git/config which
contains configuration options specific for this repository (which override
any global configuration).

Cloning a (remote) repository (git clone). Supported protocols: HTTP(S)
(read-only without DAV), rsync (r/w, but writing deprecated because
non-atomic), SSH (needs Git installed in PATH on the remote host, can be
restricted with git-shell), local paths (r/w), native Git protocol (ro).

** Basic operations
Importing a tree:
  $ git init
  $ git add .
  $ git commit -m "Initial import."

Standard operations:
  $ git add file
  $ git rm file
  $ git mv foo bar
  $ git diff
  $ git-gui

Sending changes
  $ git commit
    -s (Signed-by)
    --amend
    --author "Name <mail@foo.com>" (committer not necessarily author)
A commit is identified by its sha1 which verifies the entire tree, contents,
history and everything that let to this commit.  Revisions can be
abbreviated by providing only the first few characters.  If they uniquely
match a commit, Git will figure out.  You can also use the "commit^" syntax
to express "the first parent of commit" (e.g.: "HEAD^") or "commit^N" for
"the Nth parent of commit".  So if you want to 3 revisions back, you can use
"HEAD^^^" or "HEAD~3" which is more convenient.

Reviewing changes
  $ git log
    -Swhat
  $ git shortlog
  $ gitk
  $ git blame
    -Swhat
  $ git diff 'revision^!' == git diff revision^ revision

Undoing changes:
  $ git reset (warning, this is the equivalent of svn revert)
  $ git revert (create a commit that is the opposite of another, unlike svn
                revert)

Setting up aliases:
  $ git config alias.st status
  $ git config alias.diffstat 'diff --stat'
  $ git config alias.diffw 'diff --ignore-all-space'
You might want to setup many CVS/SVN-like aliases for ci, co, st, etc.
Aliases can't override builtin commands, e.g.:
  $ git config alias.diff 'diff --patch-with-stat'
If you then run "git diff", the alias will be silently ignored.

** Understanding the index
With SVN, HEAD is replicated in the ".svn" in each directory.  Operations
such as diff can be done without network access.  The ".svn" serve as a
cache.  The next commit is built by looking at the differences between the
working copy and the version in the ".svn".  With CVS, there must be a
network access but the result is identical.

Git has a single ".git" at the root and it also has a similar caching
mechanism called the "index" in the binary file ".git/index".  When running
"git diff" the working copy is compared to the tree stored in ".git/index".
What is different with Git is that the index is used as a staging area to
build the tree of the next commit.  By default, "git commit" doesn't do a
full-tree commit like in other SCMs (can be done with -a however).  It only
commits the content of the index.  Thus, you must first add your changes to
the index before committing them.  Adding changes to the index can be done
with "git add" (which has thus 2 roles: telling Git about new files,
updating the content of the index).  This can look weird or impractical at
first sight but ends up being very useful.  With git-gui or git-add
--interactive, one can even select the hunks to schedule in the next commit.

** Understanding the basics of Git internals
It helps to understand how Git works.  Let's dive in the .git directory.
The "HEAD" file specifies the current HEAD ("ref: refs/heads/master", sort
of symlink to the file "refs/heads/master" relative to the HEAD file).

The "objects" directory which contains the real data of the repository.  By
data we mean: file contents, trees (list of contents associated with names),
commits (tree with attributes), and tags.  All these are identified by a
(hopefully) unique sha1 sum.  Objects are immutable, and Git will never
delete them unless explicitly asked so.

The "refs" directory contains references to these objects.  It has 2
sub-directories: "heads" and "tags" to store the HEAD of the different
branches and tags.  They basically contains files (possibly in
sub-directories) which contain a 40-byte hex sha1 (plus a \n).

Creating a commit is thus a matter of building a tree in the index and then
adding the associated attributes (commit message, author, parent commits,
etc.).

** Working with the index
Adding/removing things from the index.  Diffing the index against the
working copy or against HEAD.

** Creating tags
local (private tags) are only refs, annotated/signed tags are real objects.

* Branches

** The basics
A branch is nothing more than a pointer to a point in history stored under
.git/refs/heads/branch-name.  The convention is that "master" is the default
branch and is thus assimilated to SVN's "trunk".  There can be
sub-directories under .git/refs/heads/ to express different namespaces of
branches.

"git branch" shows the (local) branches and the current branch (which can
also be known by looking at .git/HEAD).  Switching branches is as easy as
doing "git checkout branch-name" and creating a branch is a matter of
"git branch branch-name" or "git checkout -b new-branch branch-name" to
create a new branch from branch-name's HEAD and switch to that new-branch.

<sample branch setup>
<view in gitk --all>

** Merging branches
Example of a merge with conflicts.  View the history.  The merge point is a
commit with two parents.  Graphical viewers come handy.  Of course, Git
remembers the merge points.

** Viewing branches
GUIs, git-show-branch.

** Remote branches
It's possible to import branches from remote repositories.  They live under
.git/refs/remote and are not meant to be changed locally.  "git-fetch" is
used to retrieve the state of remote branches in the local repositories.  In
order to change a remote branch, it must be forked first by creating a local
branch: "git checkout -b my-branch remote-branch".  Once you have that
remote branch, you just git-merge with it.  Since it's a very common
operation, git-pull does them both: fetch+merge in the current local
branch.  By default when cloning a remote repository, a remote branch named
"origin/HEAD" is created.  It's possible to add/remove remote branches with
git-remote.  In particular, it's possible to fetch remote branches from
multiple different repositories.

** How do merges work?
Git first finds a common ancestor between the two branches and then uses a
standard 3-way merge algorithm.  It's possible to define custom merge
drivers to automatically merge specific files (e.g. merge Open Document
files or tarballs).  It is also possible to define custom diff drivers.
Anyways, once the common ancestor is found, it is read and written in the
index, along with the HEAD of the two branches to merge.  Each of these 3
trees is stored in a different "stage" of the index.  Stages also the index
to contain multiple trees.  In particular, the stage 0 is used to build the
next commit (that's where git-add puts its stuff).
<look at the output of git-diff during a merge>

* Workflows
It's possible to use Git in an SVN-way but it's not what's most efficient.
Here are some typical workflows that perform well in Git.

** Simple private project (with 1 developer)
Only one repository where changes are committed (trivial).

** Simple published project (with 1 developer)
Repository on a public server.  <explain "bare" repositories>.  The
developer publish changes with "git-push" <example>.  Warning "git-push" is
NOT the opposite of "git-pull" because pull = fetch + merge whereas "push"
is actually the opposite of "fetch".
FIXME: pitfalls of git-push (it pushes all refs, stashes, etc).

** Small project (~10 developers)
One public "reference" repository.  One maintainer (the "integrator") in
charge of it.  Developers can either directly push to the public reference
repository (CVS-like usage of Git, not recommended) or send their changes to
the maintainer for review (git-format-patch / git-send-email for the
developer, and git-am for the maintainer, or push to another per-developer
public repository and ask the maintainer to pull from it).

The hierarchy is informal and can easily be changed or forked.  Nothing in
Git requires it.

** Large project (many developers)
One public "reference" repository.  One maintainer (the "integrator") and
several sub-maintainers.  The developers first send their changes to the
sub-maintainer in charge of the module they're changing.  The sub-maintainer
approves the change by signing it and pushing it in his own public
repository.  Every once in a while, the integrator will pull from the
sub-maintainers because he trusts them.

The hierarchy is based on trust between people and is easy to adjust as
people join/leave the project.

* Working with Git on a daily basis: How to take most benefits?
Here, we address typical questions or problems that arise during daily
development and explain how Git helps to solve them faster.  Various topics
are discussed and sorted in what seemed to be from the most useful to the
least useful.

** Housekeeping
As we'll see, you can easily rewrite the history with Git.  We already
said that the objects are immutable and will never be deleted unless you
explicitly asked so.  Moreover, the fact that each object (file content,
tree, commit) has its own file under .git/objects will quickly lead to
degraded performances.  Git can pack the objects together so that they will
consume much less space on the disk and accessing them will be faster.
Therefore, you should repack your repository every once in a while.  The
command "git-gc" does all the required housekeeping for you by packing
together all the objects into one single big pack.  If you want to remove
unreferenced loose objects, you can pass it the "--prune" option.  Beware
though that using this option is not safe if someone else is working on the
repository.  If you want to repack harder, you can use "git gc --aggressive"
which will take more time but produce a much thinner pack (I've seen 200M
.git repositories shrunk down to 12M!).
Alternatively, you can run "git repack" to produce a smaller incremental
pack.
If you never repack, or if you end up having lots of small incremental
packs, the performances will degrade.  Some people did not repack in several
months and observed slow downs where git would take several seconds to
perform basic operations (whereas it would do them instantaneously with
nicely packed data).
As of today, Git's head contains a mechanism called "git-gc --auto" which is
automatically invoked by some commands to do a minimal housekeeping in your
back and warn you if your repository gets insanely unpacked, but this is
only available in Git's HEAD (so you will have to wait until the next
release).

** Setting up a public repository
FIXME

** Working from multiple places
Often people need to leave and pick up their work where it stopped, later
and from another place.  It's the "Going back home" commit syndrom.  With
Git it can be addressed by pushing a private branch to one's public
repository or by using git-bundle.

** Topic branches
Because it's so common to work on multiple independent changes at the same
time and because Git fosters branches, it's very common to have many local
(typically private) branches to work on different ideas.  They are called
topic branches and usually work as follows:
  - Fork the current development line (usually "master"):
    git checkout -b fix-something master
    (the branch is named after the topic it will be about)
  - Hack, commit, hack, commit, hack.
  - Meanwhile, the "master" branch has changed.  2 scenarios:
      1. test the current code against the changes in master, just to see if
         it still works.
      2. "rebase" the current branch, that is, forward-port local commits to
          the updated upstream head.
  - once the topic is finished, simply fast-forward merge it back in
    "master" (after making sure "master" is up-to-date and having rebased
    the topic branch) and delete the topic branch.

** Detached head
If you ever checkout a given revision that is not the HEAD of a branch (such
as a tag or by directly using the sha1 of a commit) you will be "detached"
because it's not on any branch.  Be careful because any change you make
there will not be referenced by any branch (it won't get lost, you can still
give it a name with "git branch").

** Doing an urgent fix
You're in the middle of something and your boss comes to demand that you fix
something *immediately*.  You could put your stuff in a temporary branch to
put your changes away and then switch branch to do the emergency fix.  This
involves many operations is tedious.  git-stash will help by saving all your
changes (whether they were saved in the index or still in "dirty" state in
the working copy) in a stash and restore the working copy to HEAD
(git-reset --hard).

<example>

git-stash can also be used to pull in a dirty tree
<example>

** Since when is it broken?
FIXME: git-bisect / git-blame

** Submodules
FIXME

** Git as a better SVN client
If you have SVN installed with its Perl bindings, you will be able to use
git-svn.  On Debian at least, git-svn comes in its own package, separate
from git-core.  git-svn enables you to work with SVN repositories
transparently.  It will basically clone an SVN repository (by checking out
all its revisions or only a given subset) and let you fetch new revisions
from and push your own commits to the SVN repository.  One of the goals of
git-svn is to be truly transparent, no-one should be able to tell whether
you're using SVN or Git.
  <examples>

** Rewriting the history
From what we've said on rebase earlier, it's clear that what happens in this
case is that the history must be rewritten.  What rebase does is that it
removes all the commits to be rebased and saves somewhere.  Then it
fast-forwards the current HEAD to the new HEAD.  Finally, it re-applies all
the commits that were removed.  If there is a conflict, it will stop and let
you fix the problem and do the commit.  Then you will have to invoke
"git rebase --continue" to process with the remaining commits.  This will
not delete commits from the repository, they will still be there but no
longer reachable by this line of development.  If no other branch reference
them, they will be "dangling" and will have to be pruned (see later).

The history must never be rewritten once it has been made public, because
people who took a copy of it (clone) will run into troubles upon their next
fetch.  If the history has been published, use git-revert instead.

The last commit (HEAD) can be easily changed by using "git commit --amend".
What this will do is that it will simply merge the current index in the
previous commit. <example>.  Once again, the previous HEAD will not be
deleted from the repository.

Now rebase can be used to fix a mistake done in earlier (unpublished)
commits (suppose the current branch is "master":
  $ git tag bad HEAD~5
  $ git checkout bad
  $ <hack>
  $ git commit -a --amend
  $ git rebase --onto HEAD bad master
  $ git tag -d bad

** Recovering from mistakes
// FIXME: reflogs
// Explain that stashes are implemented with the reflogs?

** Splitting a repository
Frequently, a project starts out as a main application, and after some time,
some parts of it emerge as being rather independent and often useful to
other projects.  Thus, it's often a good idea to extract these independent
parts and make them live in their own repositories so they can be re-used as
git-submodules.  The best thing to do, in order to preserve the full history
of this part of the project, is to extract the part of the history that has
to do with the given module and put it in its own repository.  This can be
done with git-filter-branch, which has been specifically designed for huge
history rewrites.

FIXME: What needs to be done exactly?

** Rerere (Reuse recorded resolution)
If you have a topic branch which frequently needs to be checked against the
latest changes in, say, master but which you don't want to merge with master
until it's finished (and you don't want to rebase it) you will frequently do
this:
  $ git merge master
  $ fix conflicts
  $ git commit
  $ <test>
  $ git reset --hard # get rid of the merge
The problem is that you will have the same conflicts to solve over and over
again whenever you want to test your work against the changes in master.
rerere will record the way you solve conflicts so that whenever you get into
the same conflict again, it will re-use the recorded resolution.  All you
need is to run "git config rerere.enabled true" to enable the recording and
re-using of conflict resolutions.



 LocalWords:  SCM CVS LocalWords workflow DSCM SCMs testsuite workflows GUIs mv
 LocalWords:  SVN gitweb rerere submodules submodule scriptable gitconfig init
 LocalWords:  gui shortlog gitk svn symlink rebase rebased
 Local Variables:
 mode: outline
 End: