grep answers unix

X11 tunneling, ssh and su

2011-09-23T17:30:00.005+01:00

It's pretty well known that ssh can be used to tunnel X11 sessions, typically from a server to your local workstation. This is extremely handy if you need to run an X program on a remote system on which you only have terminal/shell access. If you're a windows user, I recommend using Putty, and you can enable X11 forwarding very simply.

From the initial putty menu, note the options on the left, and expand the + next to 'SSH', under the 'Connection' category:

On the next page, you should tick the 'Enable X11 forwarding' tick box:

As soon as you log into the server, you should see a message saying that a new authority file has been created:

This has now set a magic cookie (no really) to authorise the server you have just logged into. If you're wondering what you've just authorised - you've just authorised the server to send X windows back through the ssl session to display on your computer. Naturally, you need an Xwin server such as Cygwin-X running on your system.

If everything is working correctly, your session should have an environment variable (DISPLAY) set to something very similar to localhost:10.0, and if you run /usr/openwin/bin/xclock, it should appear magically on your desktop:

While all of this is really great and helpful in the extreme, it is covered in many places and isn't new. What I recently worked out, and want to share, is the solution to the following two problems:

Having logged in to a system, and set up your X11 tunneling, you lose your X11 tunnel if you:

1) su to another user
2) ssh to a new system

Both of these are dead handy things to be able to do. If you have to assume root privileges, for example, to run the command that generates an X session, you'll hit problem 1. If you need to switch to a user on a different system, because you don't have a direct connection to that server, you'll hit problem 2.

1 - How to maintain your X11 tunnel while su-ing to a new user

a) log in to your system as described above
b) check your environment DISPLAY variable:

-bash-3.00$ echo $DISPLAY
localhost:10.0

c) Discover what your session magic cookie is:

-bash-3.00$ /usr/openwin/bin/xauth list
hostname/unix:10 MIT-MAGIC-COOKIE-1 fca50d4504788a86d4b680f3eda4628e

d) su to new user:

-bash-3.00$ su -
Password:

e) set your DISPLAY variable:

root@hostname # DISPLAY=localhost:10.0
root@hostname # export DISPLAY

f) set your magic cookie:

-bash-3.00$ /usr/openwin/bin/xauth add hostname/unix:10 MIT-MAGIC-COOKIE-1 fca50d4504788a86d4b680f3eda4628e

Congratulations - you should now be authorised again to send X11 commands to your local machine.

2 - How to maintain your X11 tunnel while ssh-ing to a new system

This is really easy - either of the following ssh commands works.

ssh -o "ForwardX11 yes" username@remotesystem

ssh -X -A username@remotesystem

Very slow log in to Solaris Server

2011-08-01T19:00:00.004+01:00

Error keywords seen: None - just exceptionally slow login

Operating System: Solaris variants

Software: default ssh daemon

Keywords: solaris stop suppress reverse dns lookup ssh

I've googled many times to find the answer to this one, and read through many man pages.

It's very easy to discover that slow log ins are almost always due to sshd trying to do an nslookup on the login client. This is a perfectly reasonable security measure, but if the lookup fails it can take a while to timeout, giving the impression that the login is very slow. In Linux, you change sshd_config and add 'UseDNS no' - this I found recommended many times.

In Solaris, the answer is just as easy, though harder to find:

Add the following line to /etc/ssh/sshd_config:

LookupClientHostnames=no

Restart sshd:

svcadm restart ssh

Check that ssh is running okay - do this before you log out:

svcs -l ssh

If sshd is in maintenance mode, revert your changes and restart ssh:

svcadm clear ssh

Check your change for a typo, and debug as usual.

You should now be able to log in much faster.

Solaris/OpenIndiana NFS server to Ubuntu NFS host - 4294967294 problem

2011-04-10T18:16:00.002+01:00

Error keywords seen: User and Group set to 4294967294 instead of UID/GID expected

Operating System: Open Indiana oi_148 (Solaris) and Ubuntu 11.04 (Natty Narwhal)

Software: NFS v4

When mounting an nfs shared directory from an OpenIndiana home server, the directory ownership was set to 4294967294:4294967294, despite the ownership on the server being 1000:1000, and the equivalent UID / GID being set up on the client machine.

The solution is to edit the config file /etc/default/nfs-common - the two lines required are:

NEED_STATD="no"
NEED_IDMAPD="yes"

This is enough to change the reported ownership from 4294967294 to 'nobody:nogroup' - which is progress, of a sort.

Our next requirement is to make sure that the nfs client and the nfs server are both using the same domain name. On the client, change /etc/idmapd.conf so that the domain parameter is correct - in my case, 'homenetwork'.

Domain = homenetwork

Secondly, on the server, make sure that the domainname is correctly set. As this is running a Solaris(-based) OS, it's very easy - just create or edit the contents of /etc/defaultdomain so that it contains (nothing more than) the correct domain:

homenetwork

And you're done - reboot both sides for luck, and everything should now appear as you expect.

Configure a lofs (LOopback virtual FileSystem) shared area between zones

2011-03-07T10:43:00.003Z

Operating System: Solaris 10 & variants

Create area to be shared on global zone, e.g. /share

mkdir /share

Configure the zones to use the shared area:

# zonecfg -z zone1
zonecfg:zone1> add fs
zonecfg:zone1> set dir=/share
zonecfg:zone1> set special=/share/sharex
zonecfg:zone1> set type=lofs
zonecfg:zone1> end
zonecfg:zone1> verify
zonecfg:zone1> commit
zonecfg:zone1> exit
# zonecfg -z zone2
zonecfg:zone2> add fs
zonecfg:zone2> set dir=/share
zonecfg:zone2> set special=/share/sharex
zonecfg:zone2> set type=lofs
zonecfg:zone2> end
zonecfg:zone2> verify
zonecfg:zone2> commit
zonecfg:zone2> exit
#

Reboot both zones

# zoneadm -z zone1 reboot
# zoneadm -z zone2 reboot

The filesystems will now be available and fully writeable for both zones. There is no limit to the number of zones that can share the filesystem.

Perl: how to inherit and modify the constructor of parent class

2011-02-10T13:49:00.001Z

This reminder post will be by example, but basically talks about how to inherit a constructor from a parent class, then modify it slightly. It's also a good case for remembering the reason for this blog - which is to serve as assistance to my memory, and isn't in any way a guidebook or even set of suggestions for how to do things. This is particularly true of my forays into perl hackery.

Right then. For this example, we'll start with two simple classes, "Parent" and "Child". The inheritance relationship works exactly as you'd expect from the names:

Code for inheritance1.pl

#!/usr/bin/perl -w

use strict;

#--------------------------------

package Parent;
sub new {
    my $class=shift;
    my $string1=shift;
    print("1) Called $class constructor with: " . $string1 . "\n");
    my $self={
        string1 => $string1,
        };
    bless($self, $class);
    return $self;
    }

1;

#--------------------------------

package Child;
our @ISA='Parent';
sub new {
    my $class=shift;
    my $string1=shift;
    my $self=Parent->new($string1);
    print("2) Called $class constructor with: " . $string1 . "\n");
    bless($self,$class);
    return $self;
    }

1;

#--------------------------------

my $testObject1=Parent->new("Parent's String");
my $testObject2=Child->new("Child's String");

print("Parent object has stored: ",$testObject1->{string1},"\n");
print("Child object has stored: ",$testObject2->{string1},"\n");

exit 0;

This produces the output:
1) Called Parent constructor with: Parent's String
1) Called Parent constructor with: Child's String
2) Called Child constructor with: Child's String
Parent object has stored: Parent's String
Child object has stored: Child's String

The line of interest here is:
    my $self=Parent->new($string1);

Here the constructor for Child inherits the properties of the constructor for Parent. As a result, whenever we call the constructor for Child, we see that code from the parent constructor is run. Additional code from the Child constructor is also run. However we've really done nothing useful here, as we haven't added any great extra feature to the Child constructor. So what if we want to store a new parameter into our new object?

Code for inheritance2.pl

#!/usr/bin/perl -w

use strict;

#--------------------------------

package Parent;
sub new {
    my $class=shift;
    my $string1=shift;
    print("1) Called $class constructor with: " . $string1 . "\n");
    my $self={
        string1 => $string1,
        };
    bless($self, $class);
    return $self;
    }

1;

#--------------------------------

package Child;
our @ISA='Parent';
sub new {
    my $class=shift;
    my $string1=shift;
    my $string2=shift;
    my $self=Parent->new($string1);
    print("2) Called $class constructor with: " . $string2 . "\n");
    $self={
        string2 => $string2,
        };
    bless($self,$class);
    return $self;
    }

1;

#--------------------------------

my $testObject1=Parent->new("Parent's String");
my $testObject2=Child->new("Child's 1st String","Child's 2nd String");

print("Parent object has stored: ",$testObject1->{string1},"\n");
print("Child object has stored: ",$testObject2->{string1},"\n");
print("Child object has stored: ",$testObject2->{string2},"\n");

exit 0;

This produces the output:
1) Called Parent constructor with: Parent's String
1) Called Parent constructor with: Child's 1st String
2) Called Child constructor with: Child's 2nd String
Parent object has stored: Parent's String
Use of uninitialized value in print at ./inheritance2.pl line 47.
Child object has stored:
Child object has stored: Child's 2nd String

That didn't work at all! That shouldn't be too much of a surprise, really, as we've overwritten the declaration of the $self hash. We could expand it to include everything that the Parent class has, and in this case, it wouldn't take a lot of extra typing, but it's hardly making use of our Object Oriented design, and if there was lots of exciting configuration defined by the $self declaration, we'd lose it all, or have to recreate it. Better is:

Code for inheritance3.pl
#!/usr/bin/perl -w

use strict;

#--------------------------------

package Parent;
sub new {
    my $class=shift;
    my $string1=shift;
    print("1) Called $class constructor with: " . $string1 . "\n");
    my $self={
        string1 => $string1,
        };
    bless($self, $class);
    return $self;
    }

1;

#--------------------------------

package Child;
our @ISA='Parent';
sub new {
    my $class=shift;
    my $string1=shift;
    my $string2=shift;
    my $self=Parent->new($string1);
    $self->{string2} = $string2;
    print("2) Called $class constructor with: " . $string2 . "\n");
    bless($self,$class);
    return $self;
    }

1;

#--------------------------------

my $testObject1=Parent->new("Parent's String");
my $testObject2=Child->new("Child's 1st String","Child's 2nd String");

print("Parent object has stored: ",$testObject1->{string1},"\n");
print("Child object has stored: ",$testObject2->{string1},"\n");
print("Child object has stored: ",$testObject2->{string2},"\n");

exit 0;

This produces the output:
1) Called Parent constructor with: Parent's String
1) Called Parent constructor with: Child's 1st String
2) Called Child constructor with: Child's 2nd String
Parent object has stored: Parent's String
Child object has stored: Child's 1st String
Child object has stored: Child's 2nd String

And that's just as we expect.

CPAN problem with Compress::Zlib

2011-02-09T09:54:00.001Z

Perl Version: 5.8.4

Operating System: OpenIndiana oi_148

Error Message(s) seen:

Undefined subroutine &Compress::Zlib::gzopen called at /usr/perl5/5.8.4/lib/CPAN/Tarzip.pm line 122

This error showed up while installing a module called Config::Crontab. It potentially could have caused problems with installing any module though, as I'm pretty sure that it's down to missing dependencies for Compress::Zlib.

The error presented itself as follows:

cpan[2]> install Config::Crontab
Running install for module 'Config::Crontab'
Running make for S/SC/SCOTTW/Config-Crontab-1.30.tar.gz
CPAN: LWP::UserAgent loaded ok (v5.835)
Fetching with LWP:
ftp://mirror.ox.ac.uk/sites/www.cpan.org/authors/id/S/SC/SCOTTW/Config-Crontab-1.30.tar.gz
CPAN: Digest::SHA loaded ok (v5.50)
Fetching with LWP:
ftp://mirror.ox.ac.uk/sites/www.cpan.org/authors/id/S/SC/SCOTTW/CHECKSUMS
Catching error: "Undefined subroutine &Compress::Zlib::gzopen called at /usr/perl5/5.8.4/lib/CPAN/Tarzip.pm line 122.\cJ" at /usr/perl5/5.8.4/lib/CPAN.pm line 391
CPAN::shell() called at -e line 1

As this suggests a problem with Compress::Zlib, I tried to update it:

cpan[3]> install Compress::Zlib
Compress::Zlib is up to date (2.033).j

The solution was to remove Compress::Zlib and reinstall it, which then picked up missing dependencies and installed them:

root@ph_opensolaris:/root# ls /usr/perl5/site_perl/5.8.4/Compress/
Zlib.pm
root@ph_opensolaris:/root# rm /usr/perl5/site_perl/5.8.4/Compress/Zlib.pm

cpan[1]> install Compress::Zlib

Problems using CPAN / Perl Modules with OpenSolaris (& OpenIndiana)

2011-02-09T09:20:00.002Z

Perl version: 5.8.4
Operating System: OpenIndiana oi_148 (I imagine that OpenSolaris also sees these problems)

Error Message(s) seen:

cc: unrecognized option `-KPIC'
cc: language ildoff not recognized
cc: SHA.c: linker input file unused because linking not done

[...] 

cc: SHA.o: No such file or directory
cc: no input files

Make had returned bad status, install seems impossible

The key errors here are the 'unrecognised option' and 'language not recognised' messages, which are coming from cc. cc on OpenIndiana is a link from /usr/gnu/bin/cc to /usr/sfw/bin/gcc - and gcc is not recognising some of the flags that are passed to it.

These are specified in this file:

/usr/perl5/5.8.4/lib/i86pc-solaris-64int/Config.pm

by the lines:

cccdlflags='-KPIC'

and,

optimize='-xO3 -xspace -xildoff'

To solve the problem, we need to remove (all) the offending entries, so that these two lines read:

cccdlflags=''

and,

optimize=''

Migrating a zone between two global hosts

2010-10-27T10:40:00.004+01:00

One feature of Solaris zones is the ability to migrate a non-global zone from one global host to another. In the case of zones built on zfs filesystems, this can be done as follows. (If you're using UFS, substitute tar commands for the zfs commands).

The rationale for this move is that the zone has an interface on VLAN101, but needs a second interface on VLAN102. The global host it is currently attached to (mars) does not have an interface on VLAN102. Another global host (venus) has interfaces on both VLANs, and is therefore an appropriate (in fact the only) candidate for receiving the zone.

Global1=mars
Global2=venus
zone=scorpio

First, let's check the Solaris 10 release level on each box.

(cat /etc/release)

mars: Solaris 10 10/08 s10s_u6wos_07b SPARC
venus: Solaris 10 10/09 s10s_u8wos_08a SPARC

And architecture:

(uname -m)

mars: sun4v
venus: sun4v

Moving from a lower release to a higher one will certainly involve some packages being upgraded, but Solaris will handle most of that automatically. Hopefully.

So, first, let's halt the zone and detach it:

mars# zoneadm -z scorpio halt
mars# zoneadm -z scorpio detach

The zone is now ready for transfer, which we will do with the convenient zfs commands available. If you are not using zfs, tar and ssh will work just as well.

First we'll take a snapshot of the zone filesystem:

mars# zfs snapshot -r rootpool/zones/scorpio@mars

We also have a zfs dataset allocated to the zone, so we'll snapshot that too:

mars# zfs snapshot -r datapool/zones/scorpio@mars

Just quickly check those snapshots have been taken:

mars# zfs list -t snapshot|grep scorpio
rootpool/zones/scorpio@mars
datapool/zones/scorpio@mars
datapool/zones/scorpio/zonedata@mars
datapool/zones/scorpio/zonedata/opt@mars

Great. Now for sending them. Firstly, we'll need to allow a root login (temporarily) on venus:

venus# vi /etc/ssh/sshd_config
change
PermitRootLogin no
to
PermitRootLogin yes
venus# svcadm restart ssh:default

let's send them:

mars# zfs send rootpool/zones/scorpio@mars | ssh root@venus "zfs recv rootpool/zones/scorpio"
Password:
mars# zfs send datapool/zones/scorpio/zonedata@mars | ssh root@venus "zfs recv rootpool/zones/scorpio/zonedata"
Password:
mars# zfs send datapool/zones/scorpio/zonedata/opt@mars | ssh root@venus "zfs recv rootpool/zones/scorpio/zonedata/opt"
Password:

Note that they're being received to the rootpool on venus, as it does not have a datapool configured.

Obviously now change the root login permission back to 'no':

venus# vi /etc/ssh/sshd_config
change
PermitRootLogin yes
to
PermitRootLogin no
venus# svcadm restart ssh:default

Now the datasets are transferred, we can attach the detached zone to the new global host. This is done with zonecfg, as with creating a new zone, but using create -a to specify that we are attaching a zone.

venus# zonecfg -z scorpio
scorpio: No such zone configured
Use 'create' to begin configuring a new zone.
zonecfg:scorpio; create -a /zones/scorpio

Now let's check that has imported all the zone config:

zonecfg:scorpio; info
zonename: scorpio
zonepath: /zones/scorpio
brand: native
autoboot: true
bootargs: 
pool: 
limitpriv: 
scheduling-class: FSS
ip-type: shared
[cpu-shares: 1]
inherit-pkg-dir:
dir: /lib
inherit-pkg-dir:
dir: /platform
inherit-pkg-dir:
dir: /sbin
inherit-pkg-dir:
dir: /usr
net:
address: 10.278.28.35
physical: e1000g101001
defrouter not specified
capped-memory:
physical: 4G
[swap: 4G]
dataset:
name: datapool/zones/scorpio/zonedata
rctl:
name: zone.max-swap
value: (priv=privileged,limit=2147483648,action=deny)
rctl:
name: zone.cpu-shares
value: (priv=privileged,limit=1,action=none)

We'll need to make some changes here, as the zfs dataset is no longer part of datapool, but rootpool, because the interface on VLAN 101 is named differently on venus, and because we need the zone to have a new interface on VLAN 102.

Change interface name:

zonecfg:scorpio; select net address=10.278.28.35
zonecfg:scorpio:net; info
net:
address: 10.278.28.35
physical: e1000g101001
defrouter not specified
zonecfg:scorpio:net; set physical=aggr101001
zonecfg:scorpio:net; end

Add new interface:

zonecfg:scorpio; add net
zonecfg:scorpio:net; set address=10.278.38.35
zonecfg:scorpio:net; set physical=aggr102001
zonecfg:scorpio:net; end

Change dataset name:

zonecfg:scorpio:dataset; set dataset rootpool/zones/scorpio/zonedata/opt@mars
zonecfg:scorpio:dataset; set name=rootpool/zones/scorpio/zonedata/opt
zonecfg:scorpio:dataset; end

Commit these changes and exit:

zonecfg:scorpio; commit
zonecfg:scorpio; exit

The zone is now configured and ready to be attached. But (as we read in the zoneadm manpages):

By default, zoneadm checks package and patch  levels  on
the  machine to which the zone is to be attached. If the
packages/patches that the zone depends on from the  glo-
bal zone are different (have different revision numbers)
from  the  dependent  packages/patches  on  the   source
machine,  zoneadm  reports  these conflicts and does not
perform the attach.

Adding a '-u' flag tells Solaris to attempt to upgrade any packages/patches at a lower version in the new zone, though it cannot downgrade packages/patches.

Running without '-u' gives you a long list of package and patch inconsistencies, as we are moving from update 6 to update 8.

root@venus# zoneadm -z scorpio attach -u
zoneadm: zone 'scorpio': ERROR: attempt to downgrade package SUNWlucfg, the source had patch 121430-43 but this system only has 121430-42
zoneadm: zone 'scorpio': ERROR: attempt to downgrade package SUNWlur, the source had patch 121430-43 but this system only has 121430-42
zoneadm: zone 'scorpio': ERROR: attempt to downgrade package SUNWluu, the source had patch 121430-43 but this system only has 121430-42
zoneadm: zone 'scorpio': ERROR: attempt to downgrade package SUNWservicetagr 1.1.4,REV=2008.04.25.09.06 to version 1.0,REV=2007.05.21.20.36
zoneadm: zone 'scorpio': ERROR: attempt to downgrade package SUNWservicetagu 1.1.4,REV=2008.04.25.09.06 to version 1.0,REV=2007.05.21.20.36
zoneadm: zone 'scorpio': ERROR: attempt to downgrade package SUNWstosreg 1.1.4,REV=2008.04.25.09.06 to version 1.0,REV=2007.05.21.20.36

Somewhere along the line, someone has upgraded these packages, and patch 121430 on the u6 box, and they are at a higher level on the zone than they are on the new target global_host.

If this happens to you, it's likely that you'll have a completely different set of patches and packages, and you should proceed accordingly. Here, the Services Tools Bundle has been installed, and the aforementioned patch 121430. I downloaded and applied these to venus.

# zoneadm -z scorpio attach -u                            
Getting the list of files to remove
Removing 326 files
Remove 24 of 24 packages
Installing 189 files
Add 15 of 15 packages
Updating editable files
The file  within the zone contains a log of the zone update.

# zoneadm list -iv | grep scorpio
- scorpio installed /zones/scorpio  native shared

# zoneadm -z scorpio boot
# zoneadm list -iv | grep scorpio
81 scorpio running /zones/scorpio  native shared

Success!

Java /temp directory - set in catalina.properties

2010-10-26T17:37:00.003+01:00

Error keywords seen:

javax.imageio.IIOException: Can't create cache file!

java.io.IOException: No such file or directory

Operating System: Solaris 10

Software: Tomcat (/tcServer)

Customers complained that they could not use an image upload feature of the site. The application logs were filled with the following error:

javax.imageio.IIOException: Can't create cache file!

at javax.imageio.ImageIO.createImageInputStream(ImageIO.java:335)

at javax.imageio.ImageIO.read(ImageIO.java:1325)

at uk.co.site.utils.ImageUtility.resizeImage(ImageUtility.java:24)

at uk.co.site.utils.ImageUtility.resizeImage(ImageUtility.java:19)

...

at org.springframework.web.servlet.mvc.SimpleFormController.processFormSubmission(SimpleFormController.java:267)

...

at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)

at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)

...

Caused by: java.io.IOException: No such file or directory

at java.io.UnixFileSystem.createFileExclusively(Native Method)

at java.io.File.checkAndCreate(File.java:1704)

at java.io.File.createTempFile(File.java:1792)

at javax.imageio.stream.FileCacheImageInputStream.(FileCacheImageInputStream.java:84)
at com.sun.imageio.spi.InputStreamImageInputStreamSpi.createInputStreamInstance(InputStreamImageInputStreamSpi.java:51)
at javax.imageio.ImageIO.createImageInputStream(ImageIO.java:331)
...

This is telling us that tomcat can't write to its temporary directory. Tomcat writes to this directory when it needs to write out transient temporary files - here, while resizing an image.

The temporary directory is by [catalina_base]/temp by default - which is not always created on install. Therefore, 99 times out of 100, the thing to do is create [catalina_base]/temp, and check that the tomcat user has permissions to write to it.

When I first saw this problem, it was the 1 time out of 100 where something else was going on. The location of the temporary directory can be overridden - normally by specifying a JVM argument in setenv.sh:

-Djava.io.tmpdir=[some_location]
(Onto APPLICATION_OPTS)

When I checked setenv.sh (and the arguments of the running process too) - java.io.tmpdir was set as expected. So what was going on? The answer lay in catalina.properties - it is also possible to override the directory structure here, and this takes precedence over the JVM arguments (something I found surprising). It is set by:

tomcat/conf/catalina.properties:
java.io.tmpdir=[some_location]

Editing this and restarting is the only way to redirect the temporary files in this case.

transferring files across firewalls / piping through ssh

2010-09-02T09:20:00.005+01:00

Operating System: All Unix variants

Dependencies/assumptions: Presence of gzip/gunzip

Problem:

transferring files across firewalls /

transferring files using ssh tunneling

A problem issue in a tiered, firewalled network architecure, is how to transfer files, especially large files, between servers in different parts of the network. There are lots of ways to do this - get your friendly network administrator to open up a temporary hole in the firewall, connect up a cable between the two servers, and transfer that way, etc. Potentially, however, this will break your security procedures, or it might be impossible to get physical access, or your network administrator may not be available, or going through the request process would be too slow, and so on.

It's quite usual, however, for the two networks to be administered via ssh from a third network, or sometimes single ip address. They cannot speak to each other, but this third location can speak to them both. So the usual sidestep to the problem is to transfer the file to the third location (which is probably your desktop), then upload it to the new location. Which is certainly fine for a small file, but when it is many gigabytes (or many hundreds of gigabytes) this may not be feasible, and certainly doubles the transfer time.

So what can be done? The answer is to use an ssh pipe - transfer the file as a stream from the first server to the desktop, pipe this straight to an ssh connection to its new location. The script below makes this simple (and also assumes that you'd like to gzip and gunzip the file in the stream, reducing the amount of data to be transferred (though increasing processor load).

#!/bin/ksh

# sshtransfer expects parameters:

# sshtransfer user@server1:file user@server2:file

usage() {

    print "sshtransfer expects user1@server1:file1 user2@server2:file2\n"

    exit 1

    }

testssh() {

    RESULT=failure

    RESULT=`ssh -o Batchmode=yes $1 'echo success'`

    if [[ $RESULT == "success" ]];then

        return 0

    fi

    print "Could not connect to $1 - could be connection or ssh key error.\n"

    return 1

    }

if [[ $# -ne 2 ]];then

    usage

fi

CONNECT1=`echo $1 | awk -F: '{print $1}'`

CONNECT2=`echo $2 | awk -F: '{print $1}'`

FILE1=`echo $1 | awk -F: '{print $2'}`

FILE2=`echo $2 | awk -F: '{print $2'}`

testssh $CONNECT1 && testssh $CONNECT2 && eval ssh -o Batchmode=yes $CONNECT1 \'gzip -c ${FILE1}\' 2\>/dev/null \| ssh -o Batchmode=yes $CONNECT2 \'gunzip \> $FILE2 \' 2>/dev/null

exit 0

Using perl (CGI.pm) to generate a meta refresh tag with start_html

2010-07-27T16:50:00.000+01:00

Perl version: 5.8.4
Operating System: Solaris 10 (should be irrelevant)
Problem: It's irritatingly difficult to get perl to generate a refresh tag...

This should have been simple, but wasn't. Sounds like a good topic for a quick blog post.

How to generate <meta http-equiv="refresh" content="5" /> in your header tag with Perl.pm:

print start_html(
      -head=>meta({-http_equiv => 'refresh',
        -content => '5' }),
          );

Easy. If you found this with two minutes googling, feel the warm, satisfying glow of knowing that it took me about two hours to figure that one out.

Adding password security to a website with Apache (without using .htaccess)

2009-08-05T11:56:00.003+01:00

To password protect your site, without using a .htaccess file, follow the following instructions:

/usr/local/apache/bin/htpasswd -c /usr/local/apache/conf/htpasswd [username]
This will prompt for a password to be specified.
For each new user required, type in:
/usr/local/apache/bin/htpasswd /usr/local/apache/conf/htpasswd [username]

Edit /usr/local/apache/conf/httpd.conf :
vi /usr/local/apache/conf/httpd.conf

For the directory that needs to be protected (probably the same as the DocumentRoot directory) -

AuthUserFile /usr/local/apache/conf/htpasswd
AuthGroupFile /dev/null
AuthName "Password required"
AuthType Basic
require valid-user

Save httpd.conf and gracefully restart apache:
/usr/local/apache/bin/apachectl graceful

File in wrong format error: compiling Apache on Redhat 5

2009-06-23T16:28:00.004+01:00

Apache version: 2.0.63
Operating System: Redhat 5 (2.6.18-53.el5)
Error keywords seen:
/usr/lib/libexpat.so: could not read symbols: File in wrong format
collect2: ld returned 1 exit status
(in make step)

A few people seem to have experienced this problem when compiling Apache on Redhat, so I thought I'd share my fix.

The problem is that /usr/lib/libexpat.so links to a 32-bit binary library, where a 64-bit library is expected:
# ls -l /usr/lib/libexpat.so
lrwxrwxrwx 1 root root 27 May 8 2008 /usr/lib/libexpat.so -> ../../lib/libexpat.so.0.5.0
# file /lib/libexpat.so.0.5.0
/lib/libexpat.so.0.5.0: ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), stripped

The correct library can be found at /lib64/libexpat.so.0.5.0:
# file /lib64/libexpat.so.0.5.0
/lib64/libexpat.so.0.5.0: ELF 64-bit LSB shared object, AMD x86-64, version 1 (SYSV), stripped

We can easily see that this /lib64 directory is not referenced during the compile:

#gcc -print-search-dirs
install: /usr/lib/gcc/x86_64-redhat-linux/4.1.2/
programs: =/usr/libexec/gcc/x86_64-redhat-linux/4.1.2/:/usr/libexec/gcc/x86_64-redhat-linux/4.1.2/:/usr/libexec/gcc/x86_64-redhat-linux/:/usr/lib/gcc/x86_64-redhat-linux/4.1.2/:/usr/lib/gcc/x86_64-redhat-linux/:/usr/libexec/gcc/x86_64-redhat-linux/4.1.2/:/usr/libexec/gcc/x86_64-redhat-linux/:/usr/lib/gcc/x86_64-redhat-linux/4.1.2/:/usr/lib/gcc/x86_64-redhat-linux/:/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../x86_64-redhat-linux/bin/x86_64-redhat-linux/4.1.2/:/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../x86_64-redhat-linux/bin/
libraries: =/usr/lib/gcc/x86_64-redhat-linux/4.1.2/:/usr/lib/gcc/x86_64-redhat-linux/4.1.2/:/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../x86_64-redhat-linux/lib/x86_64-redhat-linux/4.1.2/:/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../x86_64-redhat-linux/lib/:/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../x86_64-redhat-linux/4.1.2/:/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../:/lib/x86_64-redhat-linux/4.1.2/:/lib/:/usr/lib/x86_64-redhat-linux/4.1.2/:/usr/lib/

The question of course, is how to ensure that the correct library is picked up. You could recreate the link from /usr/lib to the 64-bit library, but a better way is to set the library in the LIBRARY_PATH variable:

# export LIBRARY_PATH=/lib64

Now we can see that the /lib64 directory will be searched:

# gcc -print-search-dirs
install: /usr/lib/gcc/x86_64-redhat-linux/4.1.2/
programs: =/usr/libexec/gcc/x86_64-redhat-linux/4.1.2/:/usr/libexec/gcc/x86_64-redhat-linux/4.1.2/:/usr/libexec/gcc/x86_64-redhat-linux/:/usr/lib/gcc/x86_64-redhat-linux/4.1.2/:/usr/lib/gcc/x86_64-redhat-linux/:/usr/libexec/gcc/x86_64-redhat-linux/4.1.2/:/usr/libexec/gcc/x86_64-redhat-linux/:/usr/lib/gcc/x86_64-redhat-linux/4.1.2/:/usr/lib/gcc/x86_64-redhat-linux/:/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../x86_64-redhat-linux/bin/x86_64-redhat-linux/4.1.2/:/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../x86_64-redhat-linux/bin/
libraries: =/lib64/x86_64-redhat-linux/4.1.2/:/lib64/:/usr/lib/gcc/x86_64-redhat-linux/4.1.2/:/usr/lib/gcc/x86_64-redhat-linux/4.1.2/:/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../x86_64-redhat-linux/lib/x86_64-redhat-linux/4.1.2/:/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../x86_64-redhat-linux/lib/:/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../x86_64-redhat-linux/4.1.2/:/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../:/lib/x86_64-redhat-linux/4.1.2/:/lib/:/usr/lib/x86_64-redhat-linux/4.1.2/:/usr/lib/

When we rerun configure and make, the compile now completes as it should.

Reconstructing user directories from a passwd file

2009-05-18T14:38:00.003+01:00

Sometimes I'm asked to set up a box with the same users as another box. The easiest way for me to do this is to copy the /etc/passwd, /etc/group and /etc/shadow files over, and then run the following shell fragment:

for LINE in `cat /etc/passwd|sed s/' '/'SPACE'/g`;do
DIR=`echo $LINE|awk -F: '{print $6}'|sed s/'SPACE'/' '/g`
USER=`echo $LINE|awk -F: '{print $1}'|sed s/'SPACE'/' '/g`
GROUP_NUM=`echo $LINE|awk -F: '{print $4}'`
GROUP=`grep ":${GROUP_NUM}:" /etc/group|awk -F: '{print $1}'`
if [[ ! -d ${DIR} ]];then
echo $DIR doesn\'t exist
echo mkdir ${DIR}
mkdir ${DIR}
echo cp -r /export/home/appman/* ${DIR}
cp -r /export/home/appman/* ${DIR}
echo cp -r /export/home/appman/.profile ${DIR}
cp -r /export/home/appman/.profile ${DIR}
echo chown -R ${USER}:${GROUP} ${DIR}
chown -R ${USER}:${GROUP} ${DIR}
fi
done

Compiling mod_security

2009-04-02T15:55:00.008+01:00

Apache version: 2.0.63 / 2.2.11
mod_security version: 2.5.9
Operating System: Solaris 10 (Generic_120011-14 on Sun T2000) non-global zone
Error keywords seen:
configure: *** pcre library not found.;
configure: error: pcre library is required;
configure: *** apr library not found.;
configure: error: apr library is required;
configure: *** apu library not found.;
configure: error: apu library is required;
./configure: line 5518: /app/apache_2.0.63/bin/apr-config/apr-config: Not a directory

Installing mod_security as a shared object should be a simple, straight-forward task - and indeed, the last time I had to do it, I had no problems. However, today I had to compile mod_security on a Solaris 10 zone, and I ran into some unusual problems - partly due to an error in the configure script.

Running through the basic instructions, everything proceeded as expected:

download modsecurity-apache_2.5.9.tar.gz from http://sourceforge.net/project/showfiles.php?group_id=68846&package_id=67646

cp download modsecurity-apache_2.5.9.tar.gz /tmp
gunzip modsecurity-apache_2.5.9.tar.gz
tar -xf modesecurity-apache_2.5.9.tar
cd /tmp/modsecurity-apache_2.5.9

Check that mod_unique_id is installed and that /usr/lib/libxml2.so is present. Ensure that gcc and ar are present in the PATH.

Okay - ready to run configure:

# ./configure --with-apxs=/app/apache_2.0.63/bin/apxs
checking for g++... g++
...
REDACTED
...
configure: looking for Apache module support via DSO through APXS
configure: found apxs at /app/apache_2.0.63/bin/apxs
configure: checking httpd version
configure: httpd is recent enough
Use of uninitialized value in concatenation (.) or string at /app/apache_2.0.63/bin/apxs line 237.

checking for libpcre config script... no
configure: *** pcre library not found.
configure: error: pcre library is required

So - we pop over to sunfreeware, download pcre-7.8-sol10-sparc-local.gz - transfer this to the global zone, and as root install it:

pkgadd -d ./pcre-7.8-sol10-sparc-local

On with the show:

# ./configure --with-apxs=/app/apache_2.0.63/bin/apxs
checking for g++... g++
...
REDACTED
...
Use of uninitialized value in concatenation (.) or string at /app/apache_2.0.63/bin/apxs line 237.
checking for libpcre config script... /usr/local/bin/pcre-config
configure: using '-L/usr/local/lib -R/usr/local/lib -lpcre' for pcre Library
checking for libapr config script... /app/apache_2.0.63/bin/apxs/apxs
./configure: line 5509: /app/apache_2.0.63/bin/apxs/apxs: Not a directory
./configure: line 5512: /app/apache_2.0.63/bin/apxs/apxs: Not a directory
./configure: line 5515: /app/apache_2.0.63/bin/apxs/apxs: Not a directory
./configure: line 5518: /app/apache_2.0.63/bin/apxs/apxs: Not a directory
configure: *** apr library not found.
configure: error: apr library is required

What's that? Whoever said anything about /app/apache_2.0.63/bin/apxs/apxs? Where did that come from?

Well, let's address the missing apr library issue first, but adding --with-apr=/app/apache_2.0.63/bin/apr-config to the configure settings:

# ./configure --with-apxs=/app/apache_2.0.63/bin/apxs --with-apr=/app/apache_2.0.63/bin/apr-config
checking for g++... g++
...
REDACTED
...
./configure: line 5509: /app/apache_2.0.63/bin/apr-config/apr-config: Not a directory
configure: apr CFLAGS:
./configure: line 5512: /app/apache_2.0.63/bin/apr-config/apr-config: Not a directory
configure: apr LDFLAGS:
./configure: line 5515: /app/apache_2.0.63/bin/apr-config/apr-config: Not a directory
configure: apr LIBS:
./configure: line 5518: /app/apache_2.0.63/bin/apr-config/apr-config: Not a directory
configure: apr LINK_LD:
configure: *** apr library not found.
configure: error: apr library is required

And we have it again - we've got a doubled apr-config/apr-config - where did that come from?

So I had a look at configure, and we can fix the problem by finding the line:

APR_CONFIG="${with_apr}/${APR_CONFIG}"

and replace with,

APR_CONFIG="${with_apr}"

while we're at it, we note the the same is true of apu-config, so:

find the line:

APU_CONFIG="${with_apu}/${APU_CONFIG}"

and replace with:

APU_CONFIG="${with_apu}"

The bug for this is actually earlier in the code, where the ${withval} parameter isn't stripped down right, but this fix will work well enough for our requirements.

Now we run configure as:

./configure --with-apxs=/app/apache_2.0.63/bin/apxs --with-apr=/app/apache_2.0.63/bin/apr-config --with-apu=/app/apache_2.0.63/bin/apu-config

and it works.

make && make install

- will now finish the installation

------

I've reported this bug to the mod_security project, (https://www.modsecurity.org/tracker/browse/MODSEC-51) - and the bug is known and will be fixed in the 2.5.10 release.

Compiling a new sendmail.cf

2009-03-06T16:51:00.003Z

Have you spent hours of your life, trying to compile a new sendmail.cf file from a sendmail.mc?

All the instructions tell you to use m4 to compile it, but without explanation. All you get is a short file that really doesn't look anything like a sendmail.cf ...

So try this:

/usr/ccs/bin/m4 /etc/mail/cf/m4/cf.m4 mysendmail.mc > mysendmail.cf

dup2: Bad file descriptor

2009-02-04T13:32:00.004Z

Operating System: Redhat Enterprise Linux 4
Linux Kernel: 2.4.21-37.ELsmp
Error keywords seen:
dup2: Bad file descriptor
/dev/null: Read-only filesystem

This is one of the scariest errors I've ever seen. Mainly because it appeared on boot-up after reracking one of our more important Oracle database servers. One of those 'blood runs cold' moments when you realise that the simple, 'turn off, move, turn on' plan has gone wrong, and you're looking at a halted system with an error message you've never seen before.

Left with a prompt inviting me to log in as root, and fix the problem, the first, obvious, thing to do is fsck the drive. fsck (-f) uncovered and repaired a handful of unreferenced inodes and a few missing blocks. But this didn't work - the same error message appeared after a reboot.

Google's great though, and thanks to this page I was able to fix the error quite quickly.

I'm not convinced the error is caused by device inode permission problems, but certainly some form of corruption was affecting /dev/null

The root filesystem had been mounted as read-only, so the first task is to remount this with a write option:

mount -n -o remount,rw /

Next we remove the offending /dev/null entry:

rm -f /dev/null

Now we create a new /dev/null

mknod -m 666 /dev/null c 1 3

Happily, the system now rebooted without a problem.

How to discover whether Solaris is running in 32- or 64-bit mode

2009-01-27T14:08:00.002Z

The command is:

isainfo -v

If the following line is present, then Solaris is running in 64-bit mode:

64-bit sparcv9 applications

If only the following line is present, Solaris is running in 32-bit mode:

32-bit sparc applications

Delegating root privileges with RBAC

2009-01-23T18:28:00.002Z

Operating System: Solaris 8/9/10
Related software: Websphere, IBM IHS, Apache, EdgeSuite Testing Server, etc.

The default HTTP port is always 80, which causes a problem on Unix servers. All ports less than 1024 are privileged, and only root can bind processes to them.

Since webservers therefore frequently have to be run as root, this can lead to a conflict between development teams (for example), who might want to start and stop Apache with impunity, and Systems Administrators, who might object to giving out the root password, or being bombarded with requests to stop and start the webserver.

There are a number of ways to devolve root authority to individual users, not least sudo, but on Solaris the preferred method (mine, at least) is RBAC.

RBAC (Role-Based Access Control) is a long and complicated topic - so let's just focus on what you need to do to safely devolve the root authority.

First we create an authority for the delegation of control, called org.apache.httpd.control by editing the /etc/security/auth_attr file to contain the following line:

org.apache.httpd.control:::Control Apache webserver::help=RtApache.html

The RtApache.html part refers to a helpfile held in the directory /usr/lib/help/profiles/locale/C - you can choose to create this file or not, as you choose.

Second we create a profile for this delegation of control, ApacheCTL, by editing the /etc/security/prof_attr file to contain the following line:

ApacheCTL:::Permit stop and start of Apache webserver:help=RtApache.html;auths=org.apache.httpd.control

Third we must create two entries in /etc/security/exec_attr, corresponding to the commands that we wish to run as root:

ApacheCTL:suser:cmd:::/usr/local/apache/bin/httpd:euid=0;egid=0;uid=0;gid=0
ApacheCTL:suser:cmd:::/usr/local/apache/bin/apachectl:euid=0;egid=0;uid=0;gid=0

Next, assuming that we have a user ppetrelli, we must edit the /etc/user_attr file to include a line for ppetrelli as follows (or to edit the ppetrelli line to reflect the following):

ppetrelli::::type=user;profiles=ApacheCTL;auths=org.apache.httpd.control

Finally, we must change the default login shell for ppetrelli so that it uses one of the profile checking shells pfksh, pfcsh or pfsh. These are identical to ksh, csh or sh, with the additional feature that they check the RBAC profiles and can devolve root authority as described. This should be done by editing the /etc/passwd file.

It may also be necessary to change the first line of /usr/local/apache/bin/apachectl to also use this new shell - so change #!/bin/sh to #!/bin/pfsh - but often this is not required.

This will now allow ppetrelli to stop and start apache using his own user account. Only the two commands apachectl and httpd are run as root.

Cascading HP IP Console (KVM) Switches (AF601A)

2009-01-22T10:30:00.004Z

Hardware: HP IP Console Switch AF601A
Error: Unable to create cascade configuration

I've had a rubbish couple of days trying to configure a number of HP Console switches. Despite them all working on their own, I couldn't get one of them - a new HP IP Console switch (model AF601A) to work as part of the cascade. None of the suggested diagnostic steps helped (including updating all the firmware versions!)

I checked the documentation, and the HP websites - all of which confirmed that these switches should work in a cascade configuration, allowing management of up to 4096 servers (e.g. see here).

But don't believe it. After two days work, I uncovered one line, on one webpage (this one) saying:

HP IP Console Switches are only supported at the top tier of a cascaded configuration.

And that's the answer. You cannot use these switches to create a cascade configuration, unless you're using them as the controller.

Happily I was able to juggle the switches with others in our server room, and now have a working configuration - but be warned!

Creating a new imapd.pem self-signed certificate (with openssl)

2009-01-08T14:34:00.002Z

openssl req -x509 -newkey rsa:1024 -keyout /usr/local/ssl_0.9.7g/certs/imapd.pem -out imapd.pem -days 3650

for unencrypted private key:
openssl req -x509 -newkey rsa:1024 -keyout /usr/local/ssl_0.9.7g/certs/imapd.pem -out imapd.pem -days 3650 -nodes

That's it.

NFS mount a remote directory with solaris

2009-01-08T11:17:00.005Z

Let's say we have a directory /hiro on server nakamura, which we need to mount as directory /claire on server bennet. We're going to assume that both servers are aware of the other by name (either as an entry in the /etc/hosts file, or dns, etc.) - but if not, you can replace the server names by ip addresses.

If this is a one-off requirement, then on nakamura, we issue the command:
share -F nfs -o rw,public,anon=root /hiro

This will make the directory /hiro available for nfs-mounting.

On bennet, we issue the command:
mount -F nfs nakamura:/hiro /claire

This will make the /hiro directory from nakamura available as the /claire directory on bennet.

However, all of this is transient, and will be lost on a reboot. To make this permanent, edit /etc/nfs/dfstab on nakamura, and add the line:

share -F nfs -o rw,public,anon=root /hiro

You can now share the directory as before, or by the command shareall

On bennet, edit /etc/vfstab and add in the line:

nakamura:/hiro - /claire nfs - yes rw,hard,bg

You can now mount the directory as before, or by the command mount -a, or mount /claire

How to rename a zone

2009-01-06T11:47:00.003Z

Operating System: Solaris 10 (Generic_125100-10 on Sun T2000) global zone

# zoneadm -z oldname halt
# zonecfg -z oldname
zonecfg:oldname> set zonename=newname
zonecfg:oldname> commit
zonecfg:oldname> exit
# zoneadm -z newname boot

To a great extent, that's it - you're done. But it is a good idea also to edit the /etc/hosts and the /etc/inet/ipnodes files, and update them with the new name also.

Recovering a box without the root password or a user account

2008-12-30T17:11:00.004Z

How to rescue a Linux box where you've lost the root password, and you don't have an account. (This is equivalent to saying, how to take control of a Unix system)

We'll do this in 4 stages:

1) prepare a temporary account to use to log into the system
2) boot the target system using an alternative boot media
3) remove the root password, and set up the temporary account
4) boot the system normally, and log in via the temporary account, then switch to the root user, and assume control of the system

1) prepare a temporary account to use to log into the account

On a separate Unix system, add a temporary user via your preferred method, and give it a password. Now extract the hashed password from /etc/shadow on that system. Assuming that you have called the account 'temp', this command will extract it in one line:

grep ^temp: /etc/shadow|awk -F: '{print $2}'

It'll look something horribly unfriendly like $1$ndbiqpq5$4DVeD3KsBmlHZv8jre3S31 - make a careful note of it. There's no escaping this bit.

2) Set up your boot media - be it a CD, a DVD, a PXE kernel, a bootable data stick - whatever you want to use, insert it as appropriate in the machine and turn it on. So long as you can get a terminal session, it'll do. The Redhat install CDs even mount the system for you to /mnt/sysimage - but otherwise do this manually.

3) On the target system, find and edit the /etc/passwd file, you're looking for the root entry, which should be right at the top. It'll look something like this:

root:x:0:0:root:/root:/bin/ksh

Delete out the 'x' in the second column, so it now reads:

root::0:0:root:/root:/bin/ksh

You'll also need to add in a line for a temporary user:

temp:x:9999:10:Temporary - remove after use!:/home/temp:/bin/ksh

Now find and edit the /etc/shadow file. Again, you'll need to edit the root entry, which will look something like:

root:$1$ndbiqpq5$4DVeD3KsBmlHZv8jre3S31:13244:0:99999:7:::

Replace this completely with the line:

root::::

Also add a line for the temporary user such as:

temp:$1$ndbiqpq5$4DVeD3KsBmlHZv8jre3S31:14243:0:99999:7:::

Be sure to replace the second column with the password hash that you know, rather than the example one I use here.

4) Boot the system normally. Log in using the temporary account and password, and acquire a terminal session. Switch to the root user (su -) - you should not be prompted for a password and should simply become the root user. With this power, you should now edit the /etc/passwd file once more, and add back the x you removed earlier, so that:

root::0:0:root:/root:/bin/ksh

becomes:

root:x:0:0:root:/root:/bin/ksh

Now set a new root password - 'passwd' or 'passwd root', and remove the temporary account 'userdel temp'

You now have root access to the system, and can add new accounts as required.

How to stop a Belkin Router blocking webpages

2008-11-23T17:32:00.004Z

Sometimes Belkin routers can wrongly blacklist webpages. If you can't get through to a webpage, but suspect the router is blocking it, log into the router using your browser of choice, and check the Security Log. If you see something like...

Nov.23.2008 17:34:43 security:752802.541 Blocked Prot=6, 65.54.153.254:80 > 36.168.14.24:3694, AS Seq=442762075, Ack=474151315 -Black List Defense

Then the chances are your router is blocking the page. Run an nslookup on the website that's being blocked - if the same IP address shows up in the security log, that's what's happening.

Fix:

telnet to the router (here it's 192.168.0.1, but more usually it'll be 192.168.2.1)

$ telnet 192.168.0.1
Trying 192.168.0.1...
Connected to 192.168.0.1.
Escape character is '^]'.

                ,vvvdP9P???^   ,,,
              vvd###P^`^         vvvvv v
         vv#####?^                  ????####vv,
      vv####??     ,vvvdP???^  ,,,        ??##^
     v#####?    ,vvd##P?^        #?#v#vvv
   v#####?    v###P^    ,vvv,        '?#?,
  ######?   ####?^ ,vd#P?^     `???##
  #####?   v####  ,d##P^           ''
 ######   v####  ]###L                   _   _          _                  ___
 #####?   v####  ]##L                   /   / \  |\ |  |_  \/   /\   |\ |   |
 ######    ####  ]###L                  \_  \_/  | \|  |_  /\  /--\  | \|   |
 ?#####v   ####v  ]##h,            ,,
  ?#####    ?###h,  `9#hv,     ,vv###
    ######    #####L    ]###L        ,v#v'
    ?#####vv    ?9##hv,        ,,vvvv###'
       ?#####vv     `??9P\vv,   ^         vv##,
          ######                       #######L
            ??###hvv,          ,vvv#?##?????
                `????9hdhvv,

Login: admin
Password: ****************
Login successful
--> firewall set IDS blacklist disable   
-->

And that's that. Unfortunately, if you reset the router (and with a Belkin router? You're going to reset it a lot) - then you'll need to do this again.

How to change the name of a Solaris server

2008-11-21T16:26:00.003Z

This script was written for Solaris 10, but it checks for the presence of all the files it wants to change and allows you to quit or continue if it doesn't find all of them. It was also written for a T2000, hence the interface name being e1000g0, but you can change that easily.

#Change oldname -> newname in the following files:
#
#/etc/hosts
#/etc/hostname.e1000g0 (or bge0)
#/etc/nodename
#/etc/dumpadm.conf
#/etc/inet/ipnodes
#/etc/inet/hosts
#/etc/mnttab
#/etc/sysidcfg

#Accomplish this with...

OLDNAME=old_name_of_server
NEWNAME=new_name_of_server
INTERFACE=e1000g0 (or bge0 and so on)

for FILE in /etc/hosts /etc/hostname.$INTERFACE /etc/nodename /etc/dumpadm.conf /etc/inet/ipnodes /etc/inet/hosts /etc/mnttab /etc/sysidcfg; do

count=0
if [[ ! -f $FILE ]]; then
echo $FILE not found
$count=$(( $count + 1 ))
fi

if [[ $count -gt 0 ]];then
GO="false"
while [[ $GO == "false" ]];do
print -n 'Continue anyway? y/n'
read ANSWER
if [[ "$ANSWER" == "y" ]] || [[ "$ANSWER" == "Y" ]]; then
echo "Okay."
GO="true"
elsif [[ "$ANSWER" == "n" ]] || [[ "$ANSWER" == "Y" ]]; then
echo "Quitting."
exit 0
else
echo "Sorry?"
fi
done
fi

sed s/$OLDNAME/$NEWNAME/g /etc/hosts > /etc/hosts_new
sed s/$OLDNAME/$NEWNAME/g /etc/hostname.$INTERFACE > /etc/hostname.$INTERFACE_new
sed s/$OLDNAME/$NEWNAME/g /etc/nodename > /etc/nodename_new
sed s/$OLDNAME/$NEWNAME/g /etc/dumpadm.conf > /etc/dumpadm.conf_new
sed s/$OLDNAME/$NEWNAME/g /etc/inet/ipnodes > /etc/inet/ipnodes_new
sed s/$OLDNAME/$NEWNAME/g /etc/inet/hosts > /etc/inet/hosts_new
sed s/$OLDNAME/$NEWNAME/g /etc/mnttab > /etc/mnttab_new
sed s/$OLDNAME/$NEWNAME/g /etc/sysidcfg > /etc/sysidcfg_new

#Make sure you back up the current settings...

cp hosts hosts_orig
cp hostname.$INTERFACE hostname.$INTERFACE_orig
cp nodename nodename_orig
cp dumpadm.conf dumpadm.conf_orig
cp inet/ipnodes inet/ipnodes_orig
cp inet/hosts inet/hosts_orig
cp mnttab mnttab_orig
cp sysidcfg sysidcfg_orig

#Feeling brave?

mv hosts_new hosts
mv hostname.$INTERFACE_new hostname.$INTERFACE
mv nodename_new nodename
mv dumpadm.conf_new dumpadm.conf
mv inet/ipnodes_new /inet/ipnodes
mv inet/hosts_new inet/hosts
mv mnttab_new mnttab
mv sysidcfg_new sysidcfg

#Now set the new name:

uname -S $NEWNAME

#Finally, restart:

shutdown -i 6 -g 0 -y

crontab

2008-11-21T14:14:00.001Z

Never worry about how crontab works again, just put this at the bottom of every crontab:

#* * * * * command +options
#| | | | |
#| | | | .----- day of week (Sun=0-6)
#| | | .------- month (1-12)
#| | .--------- day of month (1-31)
#| .----------- hour (0-23)
#.------------- min (0-59)

It works for me, anyway.

add ssh key pair for ssh logins without password authentication

2008-11-21T12:16:00.001Z

This isn't really recommended unless you require it for scripting purposes. However, if you are trying to write a script that needs to log in to a number of remote servers, and are trying to avoid typing in passwords (or hard-coding passwords in an expect script, or similar) - this could be a lifesaver.

Create a key:

$ ssh-keygen -t dsa -f ~/.ssh/id_dsa -C "you@example.com"
Generating DSA keys: Key generation complete.
Enter passphrase (empty for no passphrase): USE-A-PASSPHRASE
Enter same passphrase again: USE-A-PASSPHRASE
Your identification has been saved in ~/.ssh/id_dsa
Your public key is:
1024 35 [really long string] you@example.com
Your public key has been saved in ~/.ssh/id_dsa.pub

For some servers, it's ssh-keygen -d

Create a list of servers (serverlist) you want to be able to log into freely.

for server in `cat serverlist`;do
if [[ "`echo $server|cut -c 1`" == "#" ]]; then
continue
fi
if [[ "`echo $server|cut -c 1`" == "&" ]]; then
break
fi
echo $server
cat ~/.ssh/id_dsa.pub | ssh pharries@$server 'cat - >> ~/.ssh/authorized_keys'
done

Now you should be able to ssh to the box without being prompted for a password.

How to add mod_deflate to apache without recompiling

2008-11-20T13:39:00.000Z

Apache version: 2.0.63
mod_deflate version: from here
Operating System: Solaris 10 (Generic_125100-10 on Sun T2000) non-global zone
Error keywords seen:
Cannot load mod_deflate.so into server: ld.so.1: httpd: fatal: relocation error: file mod_deflate.so: symbol deflate: referenced symbol not found
gcc: command not found

A requirement came up today to add the mod_deflate module to Apache. (From apache.org: The mod_deflate module provides the DEFLATE output filter that allows output from your server to be compressed before being sent to the client over the network.)

This shouldn't be tricky, and as I generally compile apache with mod_so enabled, it shouldn't be a problem to add it in as a Dynamic Shared Object (DSO), following the process described here.

So let's have a go and see what happens:

# cp mod_deflate.c /tmp
# cd /app/apache/bin
# ./apxs -c /tmp/mod_deflate.c
/app/apache_2.0.63/build/libtool --silent --mode=compile gcc -prefer-pic -DAP_HAVE_DESIGNATED_INITIALIZER -DSOLARIS2=10 -D_POSIX_PTHREAD_SEMANTICS -D_REENTRANT -g -O2 -I/app/apache_2.0.63/include -I/app/apache_2.0.63/include -I/app/apache_2.0.63/include -I/usr/local/include -c -o /tmp/mod_deflate.lo /tmp/mod_deflate.c && touch /tmp/mod_deflate.slo
/app/apache_2.0.63/build/libtool: line 1279: gcc: command not found
apxs:Error: Command failed with rc=65536

----------------------------------------------------------------------------------------------------------------------

Okay - this is my fault, I'm an idiot, and I always do this. I've forgotten that the root PATH doesn't include /usr/sfw/bin by default. Again.

# which gcc
no gcc in /usr/sbin /usr/bin
# PATH=$PATH:/usr/sfw/bin

# ./apxs -c /tmp/mod_deflate.c
/app/apache_2.0.63/build/libtool --silent --mode=compile gcc -prefer-pic -DAP_HAVE_DESIGNATED_INITIALIZER -DSOLARIS2=10 -D_POSIX_PTHREAD_SEMANTICS -D_REENTRANT -g -O2 -I/app/apache_2.0.63/include -I/app/apache_2.0.63/include -I/app/apache_2.0.63/include -I/usr/local/include -c -o /tmp/mod_deflate.lo /tmp/mod_deflate.c && touch /tmp/mod_deflate.slo
/app/apache_2.0.63/build/libtool --silent --mode=link gcc -o /tmp/mod_deflate.la -rpath /app/apache_2.0.63/modules -module -avoid-version /tmp/mod_deflate.lo

----------------------------------------------------------------------------------------------------------------------

That's more like it.

APXS is the Apache Extension Tool. Sorry, I mean the APache eXtenSion tool, and here it has compiled mod_deflate.c into a libtool control file, mod_deflate.la, which we'll now turn into an Apache shared object. (If you're not following this, don't worry, neither am I.)

So we follow the process:

# ./apxs -i -a -n deflate /tmp/mod_deflate.la
/app/apache_2.0.63/build/instdso.sh SH_LIBTOOL='/app/apache_2.0.63/build/libtool' /tmp/mod_deflate.la /app/apache_2.0.63/modules
/app/apache_2.0.63/build/libtool --mode=install cp /tmp/mod_deflate.la /app/apache_2.0.63/modules/
cp /tmp/.libs/mod_deflate.so /app/apache_2.0.63/modules/mod_deflate.so
chmod +x /app/apache_2.0.63/modules/mod_deflate.so
cp /tmp/.libs/mod_deflate.lai /app/apache_2.0.63/modules/mod_deflate.la
cp /tmp/.libs/mod_deflate.a /app/apache_2.0.63/modules/mod_deflate.a
chmod 644 /app/apache_2.0.63/modules/mod_deflate.a
ranlib /app/apache_2.0.63/modules/mod_deflate.a
----------------------------------------------------------------------
Libraries have been installed in:
/app/apache_2.0.63/modules

If you ever happen to want to link against installed libraries
in a given directory, LIBDIR, you must either use libtool, and
specify the full pathname of the library, or use the `-LLIBDIR'
flag during linking and do at least one of the following:
- add LIBDIR to the `LD_LIBRARY_PATH' environment variable
during execution
- use the `-RLIBDIR' linker flag

See any operating system documentation about shared libraries for
more information, such as the ld(1) and ld.so(8) manual pages.
----------------------------------------------------------------------
chmod 755 /app/apache_2.0.63/modules/mod_deflate.so
[activating module `deflate' in /app/apache_2.0.63/conf/httpd.conf]

----------------------------------------------------------------------------------------------------------------------

And yep, here we can see the module has been created and added to the modules subdirectory, and the appropriate LoadModule line is present in httpd.conf:

# cd ../modules
# ls
httpd.exp mod_deflate.so mod_jk.so

# cd ../conf
# cat httpd.conf
…
LoadModule deflate_module modules/mod_deflate.so
…

----------------------------------------------------------------------------------------------------------------------

Restart apache, and we’re in business, right? Wrong.

# ../bin/apachectl restart
Syntax error on line 1056 of /app/apache_2.0.63/conf/httpd.conf:
Cannot load /app/apache_2.0.63/modules/mod_deflate.so into server: ld.so.1: httpd: fatal: relocation error: file /app/apache_2.0.63/modules/mod_deflate.so: symbol deflate: referenced symbol not found

----------------------------------------------------------------------------------------------------------------------

Oh dear. What does that mean? Well, with a bit of googling – and most credit goes to Matty at prefetch.net – it becomes clear that mod_deflate is missing the library with the zip compression tools, libz.so, which is present on the system.

We can easily confirm that mod_deflate.so is not being linked to this library with ldd:

# ldd /app/apache/modules/mod_deflate.so
libc.so.1 => /lib/libc.so.1
libgcc_s.so.1 => /usr/sfw/lib/libgcc_s.so.1
libm.so.2 => /lib/libm.so.2
/platform/SUNW,Sun-Fire-T200/lib/libc_psr.so.1

----------------------------------------------------------------------------------------------------------------------

According to gnu.org (here), we need to call gcc with -lz (just as to link to libxml2.so we would need to call gcc with -lxml2, and so on) – and we can do this with the variable LDFLAGS (“flags to be used by libtool when it links a program”). Armed with this information, let’s see what happens:

# export LDFLAGS="-lz";./apr-config –ldflags

#
#

----------------------------------------------------------------------------------------------------------------------

We get nothing. Apache ignores the LDFLAGS parameter. Diving into the apr-config file, we can se that this is because the following line wipes anything we pass to it:

LDFLAGS=""

Well, it’s a small deal to change this to,

LDFLAGS="-lz"

And now we get the expected response:

# ./apr-config –ldflags
-lz

----------------------------------------------------------------------------------------------------------------------

So we go through the procedure again:

# ./apxs –c /tmp/mod_deflate.c
…
# ./apxs -i -a -n deflate /tmp/mod_deflate.la
…

----------------------------------------------------------------------------------------------------------------------

Let’s check to see that this has now linked to the desired libz.so:

# ldd /app/apache_2.0.63/modules/mod_deflate.so
libz.so.1 => /usr/lib/libz.so.1
libc.so.1 => /lib/libc.so.1
libgcc_s.so.1 => /usr/sfw/lib/libgcc_s.so.1
libm.so.2 => /lib/libm.so.2
/platform/SUNW,Sun-Fire-T200/lib/libc_psr.so.1

----------------------------------------------------------------------------------------------------------------------

It has, and happily we now see no errors:

# ./apachectl restart
#

What this is, and what this isn't

2008-11-20T12:09:00.002Z

What this is

The output of my professional life, in no particular order. I have a terrible memory, and this is to remind me how I fixed things in the past. If it helps anyone else out in the process, that's serendipitous.

What this isn't

Any attempt to set myself up as an authority on anything whatsoever, or a guaranteed set of support instructions for anything, ever, in any way shape or form. At all.

(A vague disclaimer is nobody's friend. Have fun!)