Lance Albertson Musings of a UNIX SysAdmin, jazz lover, and wine/beer snob

27Apr/095

Adding a simple progress bar to dd

I recently ran into an issue where I wanted to move several KVM based virtual machines from one server to another server. There's several ways you can accomplish this depending on what you want to do. In my case I was using LVM for the disk backend, so simply copying the disk image files wasn't an option. It boiled down to two basic options.

  • Put system in single-user mode, rsync the contents over, and reinstall grub
  • Use dd and copy the whole LVM volume over piped through ssh

The advantage using the rsync method is that you're only copying the files you need over, thus less data transfer happens. But then you run into needing to re-run grub (which generally isn't a problem). In addition, if you're using LVM within the LVM volume for the VM and the volume group is named the same, you run into some interesting issues. The advantage for using dd is that you can get a literal copy of the disk image and just start the VM back up without any other steps. Of course, this will only work if the volumes are the same on both ends.

So I decided to go with dd but ran into a problem of seeing the progress of a 15G volume copy. I did some digging around and found a blog post that mentioned using a command line application called 'bar' so I decided to give it a shot! Its a fairly simple application that just creates a basic progress bar based on the data being piped into it. If you're running Gentoo, the package is called app-admin/bar.

Here's the command I ended up running:

$ dd if=/dev/lvm/cholula-disk | bar -s 15g | \
    ssh -c arcfour $host "dd of=/dev/lvm/cholula-disk"

When ran, it gives you output similar to:

6.0GB at   17.9MB/s  eta:   0:08:32   40 [=========================                 ]

The downside is that you need to specify the block device size before hand, but for something simple like this its quite nice. Of course I could just use one of the many dd forks out there which include progress bars but this is quick, dirty, and simple!

I used the arcfour cipher mainly to reduce the CPU overhead and increase the throughput, but you should probably never use this cipher on an untrusted network as it does have weaknesses. I didn't try doing throughput tests on other ciphers, but it would be interesting. It took me approximately 10-12 minutes to copy a 15G volume over a gigabit network which isn't too bad.

Another trick you can do is utilitize the LVM snapshot feature and create a snapshot of the running volume. If any data changes on the volume, it won't be copied over obviously, but it will at least let you do a cold "live" migration of sorts.

Tagged as: , , , , 5 Comments
3Apr/095

Why you should go to Beaver BarCamp 3

Beaver BarCamp is tomorrow in Corvallis and you should go to it! Why?

  • Its completely free and open to EVERYONE
  • Free food and drinks!
  • You get a free T-Shirt (if you register)
  • Cool and interesting topics & ideas will be discussed!
  • Meet other cool people from the area

But what is a BarCamp exactly?

BarCamp is an ad-hoc gathering born from the desire for people to share and learn in an open environment. It is an intense event with discussions, demos and interaction from participants who are the main actors of the event. -- barcamp.org

beaver barcampThis is the third incarnation of Beaver BarCamp in Corvallis and is bound to be the largest thus far. Many people I have talked to have a hard time understanding what happens at a barcamp, and if its only for technical people. In the past, our barcamp has been tech focused, but that's been primarily because the outreach for the event has been mostly directed at OSU EECS students. Barcamp's are designed so that the people who attend also are the presenters. The more people who attend, the more variety you'll have at the event. You don't need to have a full-fledge presentation prepared, just an idea, a room (which we'll provide), and people to talk to!

How do you give a session? Make sure you get to Kelly Engineering at 10AM, put your presentation and name on a sticky note, pick a room & time and stick the wall. We'll adjust rooms based on the popularity of the session to ensure there is enough room. We have all of the meeting and classrooms available in Kelly essentially, so we shouldn't have any problems finding a room! Food and drinks will be provided for free thanks to the sponsorship of the OSEL for breakfast, lunch, and dinner. We'll also have a snack break in the afternoon!

Some fun non-technical examples you could see at a BarCamp:

  • How to kayake
  • How to brew beer
  • How to grow chickens in your backyard

Don't feel like giving a session yourself? That's OK! We won't pressure you at all. You can come and attend just one session to see what its like and don't need to spend all day there.

I plan on giving sessions about the following tomorrow:

  • What is Calagator and why Corvallis needs it
  • What's the next step for the Corvallis Social Tech scene?
  • Ignite Corvallis brainstorm / roundtable discussion

I hope to see you there tomorrow!