Install Notes

25 Jun

DRBD Notes
Compilation
System:
0.6.1 DRBD
7.3 Red Hat
2.4.18-3 Kernel

1. Make sure the kernel source is in /usr/src
– Use the kernel-source rpm for your kernel to install the source if necessary
2. Try to ‘make’ DRBD
• If there are errors like:
drbd_syncer.c: In function ‘drbd_syncer’:
drbd_syncer.c:394 Structure has no member named ‘nice’
Then go to drbd/drbd_syncer.c:
At about line 325,
Comment out: ‘define drbd_set_user_nice(current,x) (current-> nice=(x))’
Put in: ‘define drbd_set_user_nice(current,x) set_user_nice(current,(x))’

Configuration
1. drbd.conf: fsck-cmd should really be fsckcmd
For a journalled file system set this to: fsckcmd=/bin/true
This command will be exec’d when a disks goes down (i.e. useful in ext2)
2. make sure the directory /var/lib/drbd exists
3. make sure drbd.conf is the same on both machines
4. make sure that /data does not get mounted on boot (you are going to let heartbeat control this). So comment out the /data mount in /etc/fstab
Put in a line in /etc/fstab so that /etc/init.d/datadisk will be able to mount the drbd partition:
/dev/nb0 /data ext3 noauto 0 0
5. use /etc/init.d/drbd start to start drbd. Make sure to start the primary node first – when you start the first node it will ask “do you want this machine to be a primary?”, don’t answer but start the second node, drbd will sort out the rest
6. when drbd is up, you should be able to cat /proc/drbd on both machines to see the disks syncing up. The values will be changing, but a big resync (>gigs) may take a long time
7. you can only mount /dev/nb0 on one machine at a time AND only the primary can mount it – make sure you are the primary by checking cat /proc/drbd. To become the primary issue the command drbdsetup /dev/nb0 primary
8. You can format a disk whilst it is syncing, or after. Otherwise you can force the sync to stop using: drbdsetup /dev/nbX disconnect

Issues
1. After doing a mkfs -t ext3 /dev/nb0, the disks will have to re-sync, so be careful. A 12 gig disk do a full re-sync on a 10T network takes about 4 hours (800K/sec), on a 100T network it takes about 30 minutes (7Meg/sec).

Reference
Protocol Description
A A write operation is complete as soon as the data is written to disk and sent to the network.
B A write operation is complete as soon as a reception acknowledgement arrives.
C A write operation is complete as soon as a write acknowledgement arrives.

Heartbeat
Gotchas
Note that you MUST have the drbd service running before heartbeat. (i.e. cat /proc/drbd HAS to give output first, start drbd with /etc/init.d/drbd start). Only then can you start heartbeat. If you start heartbeat before doing this, you’ll end up serving /data on the web which isn’t actually your mounted LAN RAID 1 but just a directory.

If you start heartbeat before drbd, you must STOP heartbeat, then start drbd then re-start heartbeat for the drbd directories to be mounted correctly.

If you stop both DRBD servers, it will have to do a complete ‘SyncingAll’ when you restart – this seems like strange behaviour.

Files on a DRBD are associated with UID’s not usernames or group names. This means that usernames/groups may change when the Primary node changes and the drbd share re-mounted. To get around this problem, make sure that the /etc/passwd files are synchronised for users who are going to be writing to the drbd.
– use usermod command to alter user UID’s

Heartbeat + Drbd Startup
1. Start DRBD on Primary: service drbd start
2. Start DRBD on Secondary: service drbd start
3. Make sure that the Primary is actually primary – issue the drbdsetup /dev/nb0 primary command on the primary node to make sure it is the primary. Check the output of cat /proc/drbd to see which node is the primary.
4.
5. Start Heartbeat on the Primary service heartbeat start
6. Start Heartbeat on the Secondary service heartbeat start

/data should be automatically mounted on the primary
httpd should automatically be started on the primary
Heartbeat + Drbd Shutdown
1. Stop heartbeat on the secondary: service drbd stop
2. Stop heartbeat on the primary: service drbd stop
[Note: this will umount /data for you and stop all services heartbeat is providing]
3. Before stopping drbd, make sure that both nodes are in the Secondary state. I.e. the output of cat /proc/drbd gives Secondary/Secondary
If either note is not in this state, make it so by issuing the command: drbdsetup /dev/nb0 secondary on the node
4. Stop drbd on both nodes: service drbd stop
[Note if you do not shut down drbd in this manner, the disks will do a ‘SyncingAll’ when they are next brought up..]

Advertisements

One Response to “Install Notes”

  1. jtsoong December 23, 2005 at 12:52 pm #

    drbd 0.7.15 notes on Fedora Core 4 (fc4)

    Download kernel source and drbd tar ball.

    Copy /boot/config- to /usr/src/kernels//.config

    $> export CC=gcc32
    $> make oldconfig; make oldconfig # in kernel directory

    Edit kernel Makefile and change the ‘version’ line to match your kernel as specified by ‘uname -r’.

    Sym link /usr/src/kernels/ to /lib/modules//build
    e.g.: ln -s /usr/src/kernels/2.6.14-1.1653_FC4-i686/ /lib/modules/2.6.14-1.1653_FC4smp/build

    Untar drbd and run “make clean all”. If this works then you can try “make rpm”.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: