Archive | June, 2004

Install Notes

25 Jun

DRBD Notes
Compilation
System:
0.6.1 DRBD
7.3 Red Hat
2.4.18-3 Kernel

1. Make sure the kernel source is in /usr/src
– Use the kernel-source rpm for your kernel to install the source if necessary
2. Try to ‘make’ DRBD
• If there are errors like:
drbd_syncer.c: In function ‘drbd_syncer’:
drbd_syncer.c:394 Structure has no member named ‘nice’
Then go to drbd/drbd_syncer.c:
At about line 325,
Comment out: ‘define drbd_set_user_nice(current,x) (current-> nice=(x))’
Put in: ‘define drbd_set_user_nice(current,x) set_user_nice(current,(x))’

Configuration
1. drbd.conf: fsck-cmd should really be fsckcmd
For a journalled file system set this to: fsckcmd=/bin/true
This command will be exec’d when a disks goes down (i.e. useful in ext2)
2. make sure the directory /var/lib/drbd exists
3. make sure drbd.conf is the same on both machines
4. make sure that /data does not get mounted on boot (you are going to let heartbeat control this). So comment out the /data mount in /etc/fstab
Put in a line in /etc/fstab so that /etc/init.d/datadisk will be able to mount the drbd partition:
/dev/nb0 /data ext3 noauto 0 0
5. use /etc/init.d/drbd start to start drbd. Make sure to start the primary node first – when you start the first node it will ask “do you want this machine to be a primary?”, don’t answer but start the second node, drbd will sort out the rest
6. when drbd is up, you should be able to cat /proc/drbd on both machines to see the disks syncing up. The values will be changing, but a big resync (>gigs) may take a long time
7. you can only mount /dev/nb0 on one machine at a time AND only the primary can mount it – make sure you are the primary by checking cat /proc/drbd. To become the primary issue the command drbdsetup /dev/nb0 primary
8. You can format a disk whilst it is syncing, or after. Otherwise you can force the sync to stop using: drbdsetup /dev/nbX disconnect

Issues
1. After doing a mkfs -t ext3 /dev/nb0, the disks will have to re-sync, so be careful. A 12 gig disk do a full re-sync on a 10T network takes about 4 hours (800K/sec), on a 100T network it takes about 30 minutes (7Meg/sec).

Reference
Protocol Description
A A write operation is complete as soon as the data is written to disk and sent to the network.
B A write operation is complete as soon as a reception acknowledgement arrives.
C A write operation is complete as soon as a write acknowledgement arrives.

Heartbeat
Gotchas
Note that you MUST have the drbd service running before heartbeat. (i.e. cat /proc/drbd HAS to give output first, start drbd with /etc/init.d/drbd start). Only then can you start heartbeat. If you start heartbeat before doing this, you’ll end up serving /data on the web which isn’t actually your mounted LAN RAID 1 but just a directory.

If you start heartbeat before drbd, you must STOP heartbeat, then start drbd then re-start heartbeat for the drbd directories to be mounted correctly.

If you stop both DRBD servers, it will have to do a complete ‘SyncingAll’ when you restart – this seems like strange behaviour.

Files on a DRBD are associated with UID’s not usernames or group names. This means that usernames/groups may change when the Primary node changes and the drbd share re-mounted. To get around this problem, make sure that the /etc/passwd files are synchronised for users who are going to be writing to the drbd.
– use usermod command to alter user UID’s

Heartbeat + Drbd Startup
1. Start DRBD on Primary: service drbd start
2. Start DRBD on Secondary: service drbd start
3. Make sure that the Primary is actually primary – issue the drbdsetup /dev/nb0 primary command on the primary node to make sure it is the primary. Check the output of cat /proc/drbd to see which node is the primary.
4.
5. Start Heartbeat on the Primary service heartbeat start
6. Start Heartbeat on the Secondary service heartbeat start

/data should be automatically mounted on the primary
httpd should automatically be started on the primary
Heartbeat + Drbd Shutdown
1. Stop heartbeat on the secondary: service drbd stop
2. Stop heartbeat on the primary: service drbd stop
[Note: this will umount /data for you and stop all services heartbeat is providing]
3. Before stopping drbd, make sure that both nodes are in the Secondary state. I.e. the output of cat /proc/drbd gives Secondary/Secondary
If either note is not in this state, make it so by issuing the command: drbdsetup /dev/nb0 secondary on the node
4. Stop drbd on both nodes: service drbd stop
[Note if you do not shut down drbd in this manner, the disks will do a ‘SyncingAll’ when they are next brought up..]

Advertisements

gawk awk awkiness

23 Jun

gawk can be used to add numbers together and average them:

gawk ‘{a=a+$1}END{print a/NR}’

I used this to get average load values:
sar -q | awk ‘{print $6}’ | grep -e “^[0-9].*” | gawk ‘{a=a+$1}END{print a/NR}’

Heres some more awkiness:
for i in `ls`; do cat $i | egrep -v “It was submit|—” | awk -F: ‘BEGIN{ORS=””} {print $2; print “,”; print $3; print “\n”}’ ; done
– The “for i in `ls`” is a bash loop
– The “-F:” specifies that a colon (“:”) will be the field sepeartor
– The “ORS=””” specifies that the Output Record Separtor will be nothing (usually it is a new line)
– The “done” is the end of the bash loop

Disk Performance

22 Jun

Different ways to check disk performance
Continue reading

Postfix

22 Jun

Adding a transport.

If you want mail to be routed to a particular server for a particular domain just put a line like:

sms.imvs.sa.gov.au smtp:smsgw.imvs.sa.gov.au:25

into /etc/postfix/transport and run /usr/sbin/postmap on the file.

How to make ssh keys

16 Jun

SSH keys allow password-less logins
Continue reading

Solaris Packages

4 Jun

See: http://www.sunfreeware.com/ for Sun OS/Solaris Packages.

Need to download package, e.g. zlib-1.1.4-sol25-sparc-local.gz

$> cd /tmp/zlib
## download package
$> gunzip zlib-1.1.4-sol25-sparc-local.gz
$> pkgadd -d zlib-1.1.4-sol25-sparc-local

Should add the packages properly!

To remove packages:
$> pkgrm

For information:
$> pkginfo -l