Wednesday, July 30, 2008

Migrating a Subversion repository to a remote Git repository

Migrating an existing SVN repository to git is pretty easy; here’s a quick step-by-step guide (and reminder to myself).

Before we start, let’s make sure git and git-svn are installed:

$ sudo apt-get install git git-core git-svn

1. Getting the code out of Subversion

First, use git-svn to get a copy of the remote Subversion repository you want to migrate. For instance:

$ git-svn clone --authors-file=git-svn-users.txt --stdlayout https://my.svn.repository.com/MyProject

A couple of words on these options:

  • clone gets a copy of the entire repository, including all commit history, tags and branches
  • –stdlayout tells git-svn that you have laid your SVN repository as per the SVN recommendations, namely with top-level folders for trunk, tags and branches. If not, you can specify the paths for these relative to the repository root using the –trunk, –tags and –branches switches respectively (instead of –stdlayout)
  • the –authors-file option allows you to map SVN users to git users, for the commit history; the file should be in this format:
    svnUserId = First Last

This process will take a little while, especially for large repositories. When that’s done, git-svn will have created a new MyProject folder containing the full repository.

We can verify its contents:

$ cd MyProject
$ ls -la 
$ cat .git/config
    repositoryformatversion = 0
    filemode = true
    bare = false
    logallrefupdates = true
[svn-remote "svn"]
    url = https://my.svn.repository.com/MyProject
    fetch = trunk:refs/remotes/trunk
    branches = branches/*:refs/remotes/*
    tags = tags/*:refs/remotes/tags/*

Note that the location of our remote SVN repository is stored in the .git/config - this allows git-svn to commit any changes you make back to the main SVN repository if you want to (how to do this is well explained in the introduction to git-svn for Subversion deserters).

2. Creating a remote git repository

However, our aim is to replace our existing remote SVN repository with a remote git repository; to do so, we first need to create an empty git repository on the remote machine, which we will then upload the repository we just created locally. This assumes that git-core is installed on the remote machine:

$ ssh me@remote-machine.com
$me@remote>: mkdir git
$me@remote>: cd git
$me@remote>: mkdir MyProject.git
$me@remote>: cd MyProject.git
$me@remote>: pwd
$me@remote>: exit

Note the output of pwd - this will be the path we will use to access the remote repository from our local box.

3. Configure the local repository

Now that we have created the remote repository, we can upload our local repository to its remote location. First, back on our local machine, we need to tell git where the remote repository is located:

$ git remote add origin ssh://me@remote-machine.com/home/me/git/MyProject.git


  • remote add: adds a remote location
  • origin: the name of the remote location

Next, we need to tell git who we are:

$ git config --add user.name "First Last"
$ git config --add user.email me@domain.com

Let’s have a quick look at our configuration:

$ cat .git/config
    repositoryformatversion = 0
    filemode = true
    bare = false
    logallrefupdates = true
[svn-remote "svn"]
    url = https://secure2.svnrepository.com/s_zcrar70/MetisseV2
    fetch = trunk:refs/remotes/trunk
    branches = branches/*:refs/remotes/*
    tags = tags/*:refs/remotes/tags/*
    name = First Last
    email = me@domain.com
[remote "origin"]
    url = ssh://me@remote-machine.com/home/me/git/MyProject.git
    fetch = +refs/heads/*:refs/remotes/origin/*

Note the user information and the remote location have been added to the config file.

4. Uploading to the remote git repository

That should be it - we’re ready to upload the local repository to the remote location. It’s as simple as:

$ git push origin master

This will upload the contents of the master branch to the origin remote repository. We will also want to upload any tags and branches from subversion:

$git branches -r
$git push origin branch1:refs/heads/branch1
$git push origin branch2:refs/heads/branch2
$git push origin tags/tag1:refs/tags/tag1
$git push origin tags/tag2:refs/tags/tag2

A note on the commands here:

  • git branches -r: list all remote branches
  • git push origin :/refs/heads/: push the local branch to the remote repository, creating a new remote branch of the same name
  • git push origin tags/:/refs/tags/: push the local tag to the remote repository, creating a new remote tag of the same name

5. Accessing our new remote git repository

The contents of our local git repository were retrieved from SVN; now that we’ve uploaded that to our new remote git repository, we can get rid of the intermediary local repository we were working on, and begin using with our remote git repository in earnest:

$ cd ..
$ rm -rf MyProject
$ mkdir MyProject
$ cd MyProject
$ git init
$ git config --add user.name "First Last"
$ git config --add user.email me@domain.com
$ git remote add origin ssh://me@remote-machine.com/home/me/git/MyProject.git
$ git pull origin master
$ git branch -a
* master
$ cat .git/config
    repositoryformatversion = 0
    filemode = true
    bare = false
    logallrefupdates = true
    name = First Last
    email = me@domain.com
[remote "origin"]
    url = ssh://me@remote-machine.com/home/me/git/MyProject.git
    fetch = +refs/heads/*:refs/remotes/origin/*

That’s it! We’ve just pulled the master branch from our remote git repository, and at this stage, we are ready to start working with git locally. Note that when we ask git to tell us about all of the branches it knows about, it no longer mentions the SVN branches that we are no longer interested in.

We can now edit files, commit changes, and then push those changes to our remote repository; a good place for SVN users to start learning how to do this is the Git-SVN Crash Course

Note: if you have git-svn installed on the remote machine, you can use git-svn to clone the repository from SVN directly on the remote machine using the –bare option (which gets just the repository, without the files). You can then use git pull to get a local copy once the remote information has been set up; this involves less steps, and would likely be a little quicker. Unfortunately, I didn’t have access to git-svn remotely, or access to install it, so this wasn’t an option for me.

The following guides were helpful to me:
Setting up a new remote git repository - Toolman Tim
An introduction to git-svn for Subversion deserters
Converting a Subversion repository to git - Ruby Forums
Converting Subversion repositories to Git - Redline Software

posted by Nick at 9:02 pm - filed in linux  

Sunday, July 27, 2008

WSE 2.0 in Visual Studio 2008

A slightly older .Net 2 application I wrote for a client to upload files to Amazon S3 makes use of the Amazon S3 SOAP API. The SOAP API has the advantage over the REST API of being able to stream large files as DIME attachements when uploading to the S3 servers, rather than trying to load them up in one go, which is handy when uploading large files. The S3 WS API requires Microsoft’s Web Services Enhancements (WSE) 2.0 though, rather than the later WSE 3.0, because in WSE 3.0 DIME is replaced by MTOM (which Amazon doesn’t support - in typical WS-* way).

However, when I tried to install this, I got the following error:

Unknown error in CorBindToRuntimeHost

Error 1001.InstallUtilLib.dll: Unknown error in CorBindToRuntimeHost

The workaround (a reminder to myself) is to use the WSE 2.0 SP3 installer rather than the older versions that come up when searching for WSE 2.0 on google.

posted by Nick at 6:24 pm - filed in .net, windows  

Saturday, March 15, 2008

Dvorak keyboard in Ubuntu/Xubuntu

For my own reference, a couple of ways of setting up the Dvorak keyboard in Linux. In Gnome:

System > Preferences > Keyboard > Layouts tab > Add button > Dvorak

For a single terminal session:

loadkeys dvorak

Adding that line to .bashrc should result in it applying to all your terminal sessions. For all X sessions (including the GDM logon screen and Gnome), add Option "XbdLayout" "dvorak" to /etc/X11/xorg.conf as such:

Section "InputDevice"
  Identifier "Generic Keyboard"
  Driver "kbd"
  Option "CoreKeyboard"
  Option "XkbRules" "xorg"
  Option "XkbModel" "pc105"
  Option “XkbLayout” “dvorak,gb”

This allows switching between the Dvorak and GB layouts by pressing both shift keys at once.

posted by Nick at 2:20 am - filed in dvorak, linux  

Saturday, January 12, 2008

Paul Graham’s “On Lisp” - in a single HTML page

Paul Graham kindly made this available free of charge in PDF and Postscript formats (here). However, because I wanted to change the printing format (to print it in booklet format, which is two A5 pages per A4 sheet), I was looking for something I could change the layout of, which proved hard to do in PDF (unsurprisingly, since the aim of the PDF and PS formats is fixing layout for print). I found a multi-page HTML file here; this wasn’t too convenient for printing though, so I concatenated it to a single file. I’m posting it here (860k) in case anyone else finds it useful.

posted by Nick at 5:01 pm - filed in lisp  

Monday, December 17, 2007

Vi! Vi! Vi!

I’ve been using Vim for about 4 months now, and I must admit that I’m not sure how I did without it. Modal editing seems like a natural paradigm to me (similar to lifting your hands off the keyboard to perform operations with the mouse, except without lifting hands from the keyboard :-), and the movement commands seemed intuitive and somehow logical, much easier than remember all the different keybindings Emacs has. Vim is a great text editor; however, it isn’t as well suited to more specific tasks, such as being an IDE or a document editor, as dedicated software such as IntelliJ IDEA or Word/OpenOffice are. The problem with those, in turn, is that text editing isn’t half as smooth as with Vim.

So, I was quite pleased when I stumbled upon this:


A Netbeans module for Vi-like editing in Netbeans. I’ve been playing with it a bit and it definitely looks promising; this goes at least some way to resolving the IDE question, as I’ve found Netbeans to be great for Ruby/Rails and second only to IntelliJ as a Java IDE. For work, I’m thinking of splashing out on this:


I’ve had a go with the demo and it works great. They have a version for Word/Outlook too, which is definitely very enticing - even though I already find editing text with Vim a lot more efficient than with a traditional editor, I still feel like I could be quite a bit more productive, and the more I get to practice, the easier this should hopefully become. Now if only I could find an equivalent for Open Office and Thunderbird…

posted by Nick at 11:15 pm - filed in linux, windows  

Monday, October 1, 2007

Dvorak touchtyping - week 4

Well, I broke the 55WPM mark on the home row this week, but things go downhill pretty rapidly from there. On other exercises I hover between 35 and 45WPM, and on a general typing test I fare much worse (around 33-35WPM on this typing test). So, I’m still pretty far off the mark from my ambitious ‘60WPW in a month’ aim, but I guess the idea was to aim loftily, and then reassess after a month…

I’ll definitely need a lot more practice to reach 60WPM. My overall speed is still slower than my quick hunt-and-peck QWERTY skills, which was probably to be expected but is still disappointing. I also still have to think about where letters are on the keyboard when I type most of the time, and in the real world (i.e. not typing tests) I still make a lot of mistakes even on the keys I’ve practiced the most on. On the positive side, I can actually type mostly without looking at the keyboard, and I’ve more or less assimilated using the basic copy-cut-paste keyboard shortcuts with the DVORAK layout.

The key to faster speeds will obviously be more practice (which I’d let up on a little in the past couple of weeks) - with regular practice, I’m hoping to make 60WPM overall in the next month or so.

posted by Nick at 6:19 pm - filed in dvorak  

Friday, September 28, 2007

Using SSL/HTTPS for Gmail and Google Apps

Today, I was thinking about Google Apps and wondering whether it used SSL to encrypt email. Now the Gmail POP/SMTP facilities use encryption for any data transferred between the mail client and Google’s mail servers, so I would have thought it natural that SSL was used to encrypt any email sent via the web interface as well. As it turns out though, that isn’t the case; firing up my trusty copy of Fiddler, I sent an email to myself using the Google Apps web interface and monitored the resulting HTTP exchange. This is the result (this is the POST request transcript - the relevant bits are in bold).

The same applies to Gmail itself. I know for most people this is a non-issue as email is generally insecure (as it gets transmitted unencrypted from mail server to mail server). I was thinking about local malicious users or sysadmins viewing the email I am sending or receiving. There are quite a few cases I can think of when I’d like to know local sysadmins can’t read my mail: a public internet cafe (in China for example, or another country free speech not a given), a corporate VPN, a non-encrypted (or WEP-encrypted) WiFi hotspot, etc… 

With Gmail, the solution is to access Gmail via https://mail.google.com rather than https://mail.google.com. In this case, a secure session is maintained throughout and all communication between Gmail and the browser is encrypted.

With Google Apps, the solution is to log in using a URL like this:


However, I couldn’t get https://mail.mydomain.com to forward to the above, which is a shame as it’s my main gateway to Google Apps (and others’ too I suspect - and therefore possibly a minor security issue for sysadmins administering domains using Google Apps…).

posted by Nick at 3:50 pm - filed in Uncategorized  

Saturday, September 22, 2007

Dvorak touchtyping - week 3

Well, better but still far from perfect. I broke the 50WPM barrier on the home row this week, at 98% accuracy too, but I also let up on the regular practise a little (I must have practised two or three times this week for an hour or so at a time, down from 6 the two first weeks at an hour and a half or so each). I’m still nowhere near as good at any of the other exercises though, and I still catch myself looking at the keyboard quite regularly.

Interestingly, I noticed myself being rather verbose in some of by emails this week, presumably as a subconscious reaction to the joy of finally being able to type at something approaching reasonable speeds. I’m also starting to get a feel for the positions of the keys other than the home row without having to look at them too, although I’m noticing that I frequently get the ‘e’ and the ‘o’ mixed up. Another interesting thing is that I seem to be in a different mindset when typing freely and doing the exercises; for some reason, some of the keys I remember well typing freely don’t seem to make it over to the exercises - I guess the different visuals (the terminal for dvorakng and Windows for daytime work) disassociate the two in my mind, which is a shame…

posted by Nick at 2:48 pm - filed in dvorak  

Sunday, September 16, 2007

Dvorak touchtyping - week 2

Well, there has been progress, but it has been slow. I’m on around 40-45WPM on the home row (which accounts for 70% of keys in English), but my error rate is still high at round 95-96%. The WPM drops dramatically once I leave the home row though, at around 30-35WPM for anything involving the top row (I haven’t even tried any exercises involving the bottom row). At least the frustration of typing letters at what seemed like 2WPM has abated, so I can stop resorting to using the mouse as much as possible and actually write emails without crying…

posted by Nick at 2:47 pm - filed in dvorak  

Saturday, September 8, 2007

Dvorak touchtyping - week 1

So I’m going to try to learn how to touch-type again; this will be my second attempt, the first being about three months ago. The difference is that this time I’m switching to Dvorak cold-turkey; the last time I had also tried to learn Dvorak, but I think that the constant swiching between Dvorak and Qwerty was what ended being discouraging, as progress was very slow.

It’s been just over a week now, and the first few days were among the most frustrating I remember having in a long time. The double-switch of Dvorak and touch-typing proved very difficult. On the first day, it took me almost 20 mins to write a 5-line email sheerly due to my typing speed; it got to the stage that I was not emailing people back because the frustration at not being able to type was so great. It’s not so bad now; I can type at a reasonable speed, I’m guessing around 10-20 WPM, which is still horrendously slow but at least workable. I had set myself an ambitious goal of 80 WPM in a month, which right now doesn’t seem achievable, especially given my still very high error rate - still worth trying though…

posted by Nick at 12:49 am - filed in dvorak  
Next Page �