Using DRBD block level replication on Windows

WDRBD or Windows DRBD

DRBD is a well known distributed replicated storage system for Linux. Recently a company has ported DRBD kernel driver and userspace utilities on Windows, so it’s now possible to setup DRBD resources on a Windows machine. DRBD is block level storage replication system  (similar to RAID-1) used on highly available storage setups. You can use both Desktop and Server Windows O/S, but it’s recommended  to use a Server version if this is intended for production use.

What you will need:
– 2 x Windows Server machines (Win2012 R2 in my case)
– DRBD binaries from here
– A dedicated volume (disk) to be replicated by DRBD. You can also use a NTFS volume, with existing data. You can use this method to replicate for example an existing Windows file server on a second Windows server. However, in this case you will need to resize (shrink) server’s partition in order to create a second, small partition needed for DRBD meta-data.
– Optionally a dedicated network for DRBD replication.

Configuration:

You must follow these steps on both nodes.

– Setup both Windows machines with static IP addresses. In my case I will use 10.10.10.1 for node1 and 10.10.10.2 for node2. Also provide a meaningful hostname on each server since you will need this during DRBD configuration. In my case node1: wdrbd1 and node2: wdrbd2 .
– Install DRBD binaries by double clicking on setup file and following the wizard. Finally reboot both servers.
– Navigate to “Program Files\drbd\etc” and “Program Files\drbd\etc\drbd.d”  folder and rename (or create a copy) the following files:

drbd.conf.sample –> drbd.conf
   global_common.conf.sample –> global_common.conf

(Note: For this test we do not need to modify the content of the above files. However it may be needed to do so in different scenarios.)

– Create a resource config file in “Program Files\drbd\etc\drbd.d”

r0.res (you can copy the contents from the existing sample config file)

A simple resource config file should look like this:

resource r0 {
      on wdrbd1 {
            device          e   minor 2;
            disk            e;
            meta-disk       f;
            address      10.10.10.1:7789;
      }

      on wdrbd2 {
              device        e   minor 2;
              disk          e;
              meta-disk     f;
              address       10.10.10.2:7789;
    }
}

“minor 2” means volume index number. (c: volume is minor 0, d: volume is minor 1, and e: is minor 2).

– Partition the hard drive for DRBD use. In my case I have a dedicated 40GB disk to be used for DRBD replication. I will use Disk Management to partition/format the hard drive.
I will need 2 partitions, 1st partition will be the data partition(device e above) and 2nd partition will be the meta-data partition(device f above). So let’s create the partition 1 and format it in NTFS.The size of this partition (e) in my case will be 39.68GB. The rest of free space will be dedicated for the meta-data partition (f), 200MB. To calculate the size of the meta-data properly please use the following link from Linbit DRBD documentation site.
The disk layout should look like this:
Please note that the data partition (E:) has a filesystem, NTFS,  but the meta-data partition (F:) does not, so it must be a RAW partition.

– Once finished with the above on both nodes, open a command prompt (as an Administrator)  and use the following commands to prepare DRBD:

  •  drbdadm create-md r0    (on each nodes)
Initial Sync
  • drbdadm up r0   (on node1)
  • drbdadm up r0   (on node2)
  • drbdadm status r0  (on node1)

You should see something like the following:

C:\Users\wdrbd>drbdadm status r0
  r0 role:Secondary
    disk:Inconsistent
    wdrbd2 role:Secondary
        peer-disk:Inconsistent

Initiate a full sync on node1:

  • drbdadm primary –force r0

After the sync is completed you should get the following:

C:\Users\wdrbd>drbdadm status r0
  r0 role:Primary
    disk:UpToDate
    wdrbd2 role:Secondary
          peer-disk:UpToDate

The disk state on both nodes should be in UpToDate state. As you can see the primary node in this case it’s node1. This means that node1 is the only node which can access the E: drive to read/write data into it. Remember that NTFS is not a clustered file system, meaning that it cannot be opened for read/write access concurrently on both nodes. Our DRBD configuration in our scenario prevents dual Primary mode in order to avoid corruption of the file system.

Switching the roles:

If you want to make node2 the Primary and node1 the Secondary, you can do so by doing the following (make sure there are no any active read/write sessions on node1 since DRBD will have to force close them):

  • On node1: drbdadm secondary r0
  • On node2: drbdadm primary r0

After issuing the above commands, you should be able to access the E: drive on node2 this time. Feel free to experiment and don’t forget to report any bugs to the project’s github web page!

DRBD9 | How to setup a basic three node cluster

DRBD9 | How to setup a basic three node cluster

This guide assumes that you already have DRBD9 kernel module, drbd-utils and drbdmanage installed on the servers.
In this setup, we will need 3 identical (as much as possible) servers with a dedicated network and storage backend for the DRBD replication.

HostA: drbd-host1
ip addr: 10.1.1.3
netmask: 255.255.255.0

HostB: drbd-host2
ip addr: 10.1.1.4
netmask: 255.255.255.0

HostC: drbd-host3
ip addr: 10.1.1.5
netmask: 255.255.255.0

* make sure you setup /etc/hosts on each node to reflect the ip addresses of the other nodes. This is a requirement for DRBD.

* all hosts must be able to connect to each other via ssh without password. To do so execute the following commands on each node:

ssh-keygen # Follow the wizard, make sure you don’t set a passphrase!
ssh-copy-id <node name> # where <node name> is the hostname of the opposite node, e.g if you are on drbd-host1 then the opposite hosts should be drbd-host2 and drbd-host3. Do the same on all 3 nodes.

* make sure you can connect to each node password less:

drbd-host1:~# ssh drbd-host2 && ssh drbd-host3

Ok, now that you have access to each node without needing to enter password let’s configure drbd.

First, we must select the underlying storage that we will use for drbd. In this example each host has /dev/sdb volume which we dedicate for drbd, where /dev/sdb corresponds to an RAID10 disk array on each host.

Now let’s connect to the first host, drbd-host1 and create needed LVM VG. We should name this VG ‘drbdpool’ which is how drbd will recognise it and use it to allocate the space:

drbd-host1~# pvcreate /dev/sdb
drbd-host1~# vgcreate drbdpool /dev/sdb
drbd-host1~# lvcreate -L 4T –thinpool drbdthinpool drbdpool # We create a Thin provisioned LV inside drbdpool VG. It is necessary to call it drbdthinpool otherwise operations later will fail!

* repeat steps above to the rest of nodes (drbd-host2, drbd-host3).

Now will have to use drbdmanage utility to initiate the drbd cluster. One drbd-host1 execute the following:

drbdmanage init 10.1.1.3 # where 10.1.1.3 is the ip address on drbd-host1 which is dedicated for drbd replication (see at the top of this guide).

If successful then proceed adding the rest 2 nodes in the cluster (again from drbd-host1!)

drbdmanage add-node drbd-host2 10.1.1.4 # note that here you need to specify node’s hostname as well!. You should be able to auto complete each parameter (even the ip) by pressing the TAB button.

drbdmanage add-node drbd-host3 10.1.1.5

Now verify that everything is good:

drbdmanage list-nodes # all nodes should appear with OK status

Next, let’s create the first resource with a test volume within it:

drbdmanage add-resource res01

drbdmanage add-volume vol01 40G –deploy 3 # additionally here we specify on how many nodes the volume should reside. In this case we set 3 nodes.

Verify that the above are created successfully:

ls /dev/drbd* #Here you should see /dev/drbd0, /dev/drbd1 which belong to the control volumes that drbdmanage automatically creates during “drbdmanage init“. Additionally there should be /dev/drbd100 which corresponds to the vol01 volume we created above. You can handle this as a usual block device, e.g fdisk it then create a fs with mkfs and finally mount and write data to it. All writes will be automatically replicated to the rest of nodes.

drbdmanage list-volumes

drbdmanage list-resources

drbd-overview # Shows the current status of the cluster. Please take a note of the node which is elected to be in Primary state. That is the only node which can mount the newly created drbd volume!

drbdadm status

Proxmox VE users: If you setup the DRBD9 cluster on PVE nodes make sure you add the following entries in /etc/pve/storage.cfg , Note that you don’t need to create volumes manually like we did previously. Proxmox storage plugin will create those automatically per each VM you create :

drbd: drbd9-stor1  # where drbd9-stor1 can be any arbitrary label to identify the storage
content images
redundancy 3   # This is the volume redundancy level, in this case it’s 3.

Ok, so if everything went right you should now have a working 3 node DRBD9 cluster. Of course you will need to spend a lot of time to familiarise yourself with drbd utils command line and drbd in general. Have fun!