1 hg-fast-export.sh - mercurial to git converter using git-fast-import
2 =========================================================================
7 Most hg-* scripts are licensed under the [MIT license] and were written
8 by Rocco Rutte <pdmef@gmx.net> with hints and help from the git list and
9 \#mercurial on freenode. hg-reset.py is licensed under GPLv2 since it
10 copies some code from the mercurial sources.
12 The current maintainer is Frej Drejhammar <frej.drejhammar@gmail.com>.
14 [MIT license]: http://www.opensource.org/licenses/mit-license.php
19 If you have problems with hg-fast-export or have found a bug, please
20 create an issue at the [github issue tracker]. Before creating a new
21 issue, check that your problem has not already been addressed in an
22 already closed issue. Do not contact the maintainer directly unless
23 you want to report a security bug. That way the next person having the
24 same problem can benefit from the time spent solving the problem the
27 [github issue tracker]: https://github.com/frej/fast-export/issues
32 This project depends on Python (>=3.7) and the Mercurial package (>=
33 5.2). If Python is not installed, install it before proceeding. The
34 Mercurial package can be installed with `pip install mercurial`.
36 On windows the bash that comes with "Git for Windows" is known to work
42 Using hg-fast-export is quite simple for a mercurial repository <repo>:
45 git init repo-git # or whatever
47 hg-fast-export.sh -r <local-repo>
51 Please note that hg-fast-export does not automatically check out the
52 newly imported repository. You probably want to follow up the import
53 with a `git checkout`-command.
55 Incremental imports to track hg repos is supported, too.
57 Using hg-reset it is quite simple within a git repository that is
58 hg-fast-export'ed from mercurial:
61 hg-reset.sh -R <revision>
64 will give hints on which branches need adjustment for starting over
67 When a mercurial repository does not use utf-8 for encoding author
68 strings and commit messages the `-e <encoding>` command line option
69 can be used to force fast-export to convert incoming meta data from
70 <encoding> to utf-8. This encoding option is also applied to file names.
72 In some locales Mercurial uses different encodings for commit messages
73 and file names. In that case, you can use `--fe <encoding>` command line
74 option which overrides the -e option for file names.
76 As mercurial appears to be much less picky about the syntax of the
77 author information than git, an author mapping file can be given to
78 hg-fast-export to fix up malformed author strings. The file is
79 specified using the -A option. The file should contain lines of the
80 form `"<key>"="<value>"`. Inside the key and value strings, all escape
81 sequences understood by the python `unicode_escape` encoding are
82 supported; strings are otherwise assumed to be UTF8-encoded.
83 (Versions of fast-export prior to v171002 had a different syntax, the
84 old syntax can be enabled by the flag `--mappings-are-raw`.)
86 The example authors.map below will translate `User
87 <garbage<tab><user@example.com>` to `User <user@example.com>`.
90 -- Start of authors.map --
91 "User <garbage\t<user@example.com>"="User <user@example.com>"
92 -- End of authors.map --
95 If you have many Mercurial repositories, Chris J Billington's
96 [hg-export-tool] allows you to batch convert them.
101 As Git and Mercurial have differ in what is a valid branch and tag
102 name the -B and -T options allow a mapping file to be specified to
103 rename branches and tags (respectively). The syntax of the mapping
104 file is the same as for the author mapping.
106 When the -B and -T flags are used, you will probably want to use the
107 -n flag to disable the built-in (broken in many cases) sanitizing of
108 branch/tag names. In the future -n will become the default, but in
109 order to not break existing incremental conversions, the default
110 remains with the old behavior.
112 By default, the `default` mercurial branch is renamed to the `master`
113 branch on git. If your mercurial repo contains both `default` and
114 `master` branches, you'll need to override this behavior. Use
115 `-M <newName>` to specify what name to give the `default` branch.
120 hg-fast-export supports filtering the content of exported files.
121 The filter is supplied to the --filter-contents option. hg-fast-export
122 runs the filter for each exported file, pipes its content to the filter's
123 standard input, and uses the filter's standard output in place
124 of the file's original content. The prototypical use of this feature
125 is to convert line endings in text files from CRLF to git's preferred LF:
128 -- Start of crlf-filter.sh --
130 # $1 = pathname of exported file relative to the root of the repo
131 # $2 = Mercurial's hash of the file
132 # $3 = "1" if Mercurial reports the file as binary, otherwise "0"
134 if [ "$3" == "1" ]; then cat; else dos2unix -q; fi
135 # -q option in call to dos2unix allows to avoid returning an
136 # error code when handling non-ascii based text files (like UTF-16
137 # encoded text files)
138 -- End of crlf-filter.sh --
145 hg-fast-export supports plugins to manipulate the file data and commit
146 metadata. The plugins are enabled with the --plugin option. The value
147 of said option is a plugin name (by folder in the plugins directory),
148 and optionally, and equals-sign followed by an initialization string.
150 There is a readme accompanying each of the bundled plugins, with a
151 description of the usage. To create a new plugin, one must simply
152 add a new folder under the `plugins` directory, with the name of the
153 new plugin. Inside, there must be an `__init__.py` file, which contains
157 def build_filter(args):
161 def __init__(self, args):
163 #Or don't pass, if you want to do some init code here
166 Beyond the boilerplate initialization, you can see the two different
167 defined filter methods in the [dos2unix](./plugins/dos2unix) and
168 [branch_name_in_commit](./plugins/branch_name_in_commit) plugins.
171 commit_data = {'branch': branch, 'parents': parents, 'author': author, 'desc': desc, 'revision': revision, 'hg_hash': hg_hash, 'committer': 'committer', 'extra': extra}
173 def commit_message_filter(self,commit_data):
175 The `commit_message_filter` method is called for each commit, after parsing
176 from hg, but before outputting to git. The dictionary `commit_data` contains the
177 above attributes about the commit, and can be modified by any filter. The
178 values in the dictionary after filters have been run are used to create the git
182 file_data = {'filename':filename,'file_ctx':file_ctx,'data':file_contents}
184 def file_data_filter(self,file_data):
186 The `file_data_filter` method is called for each file within each commit.
187 The dictionary `file_data` contains the above attributes about the file, and
188 can be modified by any filter. `file_ctx` is the filecontext from the
189 mercurial python library. After all filters have been run, the values
190 are used to add the file to the git commit.
192 The `file_data_filter` method is also called when files are deleted,
193 but in this case the `data` and `file_ctx` keys map to None. This is
194 so that a filter which modifies file names can apply the same name
195 transformations when files are deleted.
199 See README-SUBMODULES.md for how to convert subrepositories into git
205 hg-fast-export supports multiple branches but only named branches with
206 exactly one head each. Otherwise commits to the tip of these heads
207 within the branch will get flattened into merge commits. There are a
208 few options to deal with this:
209 1. Chris J Billington's [hg-export-tool] can help you to handle branches with
211 2. Use the [head2branch plugin](./plugins/head2branch) to create a new named
212 branch from an unnamed head.
213 3. You can ignore unnamed heads with the `--ignore-unnamed-heads` option, which
214 is appropriate in situations such as the extra heads being close commits
215 (abandoned, unmerged changes).
217 hg-fast-export will ignore any files or directories tracked by mercurial
218 called `.git`, and will print a warning if it encounters one. Git cannot
219 track such files or directories. This is not to be confused with submodules,
220 which are described in README-SUBMODULES.md.
222 As each git-fast-import run creates a new pack file, it may be
223 required to repack the repository quite often for incremental imports
224 (especially when importing a small number of changesets per
227 The way the hg API and remote access protocol is designed it is not
228 possible to use hg-fast-export on remote repositories
229 (http/ssh). First clone the repository, then convert it.
234 hg-fast-export was designed in a way that doesn't require a 2-pass
235 mechanism or any prior repository analysis: it just feeds what it
236 finds into git-fast-import. This also implies that it heavily relies
237 on strictly linear ordering of changesets from hg, i.e. its
238 append-only storage model so that changesets hg-fast-export already
239 saw never get modified.
244 Please create a pull request at
245 [Github](https://github.com/frej/fast-export/pulls) to submit patches.
247 When submitting a patch make sure the commits in your pull request:
249 * Have good commit messages
251 Please read Chris Beams' blog post [How to Write a Git Commit
252 Message](https://chris.beams.io/posts/git-commit/) on how to write a
253 good commit message. Although the article recommends at most 50
254 characters for the subject, up to 72 characters are frequently
255 accepted for fast-export.
257 * Adhere to good [commit
258 hygiene](http://www.ericbmerritt.com/2011/09/21/commit-hygiene-and-git.html)
260 When developing a pull request for hg-fast-export, base your work on
261 the current `master` branch and rebase your work if it no longer can
262 be merged into the current `master` without conflicts. Never merge
263 `master` into your development branch, rebase if your work needs
264 updates from `master`.
266 When a pull request is modified due to review feedback, please
267 incorporate the changes into the proper commit. A good reference on
268 how to modify history is in the [Pro Git book, Section
269 7.6](https://git-scm.com/book/en/v2/Git-Tools-Rewriting-History).
271 Please do not submit a pull request if you are not willing to spend
272 the time required to address review comments or revise the patch until
273 it follows the guidelines above. A _take it or leave it_ approach to
274 contributing wastes both your and the maintainer's time.
279 * git fast-import crashes with: `error: cannot lock ref 'refs/heads/...`
281 Branch names in git behave as file names (as they are just files and
282 sub-directories under `refs/heads/`, and a path cannot name both a
283 file and a directory, i.e. the branches `a` and `a/b` can never
284 exist at the same time in a git repo.
286 Use a mapping file to rename the troublesome branch names.
288 * `Branch [<branch-name>] modified outside hg-fast-export` but I have
289 not touched the repo!
291 If you are running fast-export on a case-preserving but
292 case-insensitive file system (Windows and OSX), this will make git
293 treat `A` and `a` as the same branch. The solution is to use a
294 mapping file to rename branches which only differ in case.
296 * My mapping file does not seem to work when I rename the branch `git
297 fast-import` crashes on!
299 fast-export (imperfectly) mangles branch names it thinks won't be
300 valid. The mechanism cannot be removed as it would break already
301 existing incremental imports that expects it. When fast export
302 mangles a name, it prints out a warning of the form `Warning:
303 sanitized branch [<unmangled>] to [<mangled>]`. If `git fast-import`
304 crashes on `<mangled>`, you need to put `<unmangled>` into the
307 * fast-import mangles valid git branch names which I have remapped!
309 Use the `-n` flag to hg-fast-export.sh.
311 * `git status` reports that all files are scheduled for deletion after
312 the initial conversion.
314 By design fast export does not touch your working directory, so to
315 git it looks like you have deleted all files, when in fact they have
316 never been checked out. Just do a checkout of the branch you want.
318 * `Error: repository has at least one unnamed head: hg r<N>`
320 By design, hg-fast-export cannot deal with extra heads on a branch.
321 There are a few options depending on whether the extra heads are
322 in-use/open or normally closed. See [Notes/Limitations](#noteslimitations)
323 section for more details.
325 [hg-export-tool]: https://github.com/chrisjbillington/hg-export-tool