Talk is cheap. Show me the code.: 2011

Benchmark of String Comparison Methods

The fastest way of identifying (non)empty strings in Perl is the direct method ($foo eq ''), the slowest is the regular expression ($foo =~ /^$/):

	#!/usr/bin/env perl

	use strict;
	use warnings;
	use Benchmark ( 'cmpthese' );

	my $foo = '';

	cmpthese( 10000000, {
	'direct' => sub { return $foo eq '' },
	'length' => sub { return length $foo },
	'regexp' => sub { return $foo =~ /^$/ },
	});

	$foo = 'something';

	cmpthese( 10000000, {
	'direct' => sub { return $foo eq '' },
	'length' => sub { return length $foo },
	'regexp' => sub { return $foo =~ /^$/ },
	});

view raw benchmark-empty-strings.pl hosted with ❤ by GitHub

My own results:

             Rate regexp direct length
regexp  4219409/s     --   -76%   -78%
direct 17543860/s   316%     --    -7%
length 18867925/s   347%     8%     --
             Rate regexp length direct
regexp  5128205/s     --   -73%   -74%
length 18867925/s   268%     --    -6%
direct 20000000/s   290%     6%     --

Colors in svn

Install colorsvn for colorizing svn output and "replace" svn with colorsvn, e.g.:

alias svn='colorsvn'

Install colordiff for colorizing svn diff output and configure it to "replace" builtin diff command by editing ~/.colordiffrc, e.g.:

[helpers]
diff-cmd = colordiff

Forward and Reverse DNS Lookups

Hostname to IP:

perl -MSocket -lne 'print "$_: ", eval { inet_ntoa inet_aton $_ }' sites.txt

IP to hostname:

perl -MSocket -lne 'print "$_: ", scalar gethostbyaddr( inet_aton( $_ ), AF_INET )' ips.txt

US International Keyboard

To enable typing of international characters with AltGr key while using US layout:

setxkbmap -rules evdev -model evdev -layout us -variant altgr-intl

To make the changes persistant create /etc/X11/xorg.conf.d/10-keyboard-layout.conf file:

Section "InputClass"
        Identifier "keyboard-layout"
        Driver "evdev"
        MatchIsKeyboard "yes"
        Option "XkbLayout"  "us"
        Option "XkbVariant" "altgr-intl"
EndSection

Some useful key combinations:

á ← AltGr+a
ä ← AltGr+q
č ← Shift+AltGr+.+c
ď ← Shift+AltGr+.+d
é ← AltGr+e
í ← AltGr+i
ĺ ← AltGr+'+l
ľ ← Shift+AltGr+.+l
ň ← Shift+AltGr+.+n
ó ← AltGr+o
ô ← AltGr+6+o
ŕ ← AltGr+'+r
š ← Shift+AltGr+.+s
ť ← Shift+AltGr+.+t
ú ← AltGr+u
ý ← AltGr+'+y
ž ← Shift+AltGr+.+z
£ ← Shift+AltGr+4
€ ← AltGr+5
¥ ← AltGr+-
© ← AltGr+c
° ← Shift+AltGr+;
¶ ← AltGr+;

Add New CA Certificate

System wide update of CA certificates (used by for example wget, curl, git):

mkdir -p /usr/local/share/ca-certificates
wget -O /usr/local/share/ca-certificates/ca_cert.pem https://example.com/ca_cert.pem
update-ca-certificates

Transform TAR Archived File Names

Prepend mypackage-0.1 to each archived file and compress the archive using bzip2 in one round:

tar --xform 's,^,mypackage-0.1/,' -cjf mypackage-0.1.tar.bz2 bin/ conf/ doc/ log/ Makefile

Unix Timestamp

Display current or specified date in Unix epoch:

date --utc +%s
date --utc +%s -d '2012-01-01 00:00:00'

Partition Cloning Over Network

Copy disk layout stored in MBR if needed:

Start Netcat receiver on the destination (10.0.0.1) and reread MBR after copying is finished:

nc -l -p 1234 > /dev/sda
partprobe /dev/sda

And send first 512 bytes from the source:

dd if=/dev/sda bs=512 count=1 | nc 10.0.0.1 1234

Copy desired partition:

Start Netcat receiver on the destination again:

nc -l -p 1234 > /dev/sda1

And send partition raw data over the wire:

dd if=/dev/sda1 | nc 10.0.0.1 1234

Reread MBR

Apply changes in disk layout:

partprobe

Copy Whole Disk over Network

rsync -axv -e ssh root@source:/ /mnt/root

Disk Encryption Using LUKS

Load device mapper, encrypt partition, optionally add second passphrase, open encrypted partition and format it using ReiserFS:

moprobe dm-mod
cryptsetup -c aes-xts-plain -y -s 512 luksFormat /dev/sda1
cryptsetup luksAddKey /dev/sda1
cryptsetup luksOpen /dev/sda1 root
mkfs.reiserfs -l ROOT /dev/mapper/root

PostgreSQL Timestamp and Unix Time Conversions

PostgreSQL timestamp to Unix time:

postgres=# select extract(epoch from timestamp with time zone '2011-01-02 03:04:05+06');
 date_part  
------------
 1293915845
(1 row)

Unix time to PostgreSQL timestamp:

postgres=# select to_timestamp(1293915845);
      to_timestamp      
------------------------
 2011-01-01 22:04:05+01
(1 row)

Capture and Replay Network Traffic

Example of capturing and replaying captured syslog traffic.

Capture:

tcpdump -v -s 0 -w syslog.dump -nn udp dst port 514

Rewrite:

tcprewrite --fixcsum \
           --enet-dmac=00:11:22:33:44:55 \
           --dstipmap=0.0.0.0/0:1.2.3.4/32 \
           --infile=syslog.dump \
           --outfile=syslog.redump

Replay:

tcpreplay -i eth0 --topspeed syslog.redump
tcpreplay -i eth0 --mbps=5.0 syslog.redump

POE::Component::Client::DNS

Examples of resolving a "large" number of domains in parallel using asynchronous DNS calls.

Version 3 API

	#!/usr/bin/env perl

	use strict;
	use warnings;

	use POE;
	use POE::Component::Client::DNS;

	my @ips = qw( 127.0.0.1 10.0.0.1 192.168.0.1 192.168.1.1 1.2.3.4
	8.8.4.4 8.8.8.8 208.67.220.220 208.67.222.222 );

	my $resolver = POE::Component::Client::DNS->spawn(
	Alias => 'resolver',
	Timeout => 10,
	);

	POE::Session->create(
	inline_states => {
	_start => sub { $_[KERNEL]->yield('lookup') for 1..5; },
	lookup => sub {
	my $ip = shift @ips;

	return unless defined $ip;

	# Example of using POE::Component::Client::DNS Version 3 API
	my $response = $resolver->resolve(
	event => 'response',
	host => $ip,
	type => 'PTR',
	context => { },
	);
	$_[KERNEL]->yield(response => $response) if $response;
	},
	response => sub {
	my $request_address = $_[ARG0]->{'host'};
	my $net_dns_packet = $_[ARG0]->{'response'};
	my $net_dns_errorstring = $_[ARG0]->{'error'};

	unless ( defined $net_dns_packet ) {
	print "$request_address: error ($net_dns_errorstring)\n";
	$_[KERNEL]->yield('lookup');
	return;
	}

	my @net_dns_answers = $net_dns_packet->answer();

	unless ( scalar @net_dns_answers ) {
	print "$request_address: no answer\n";
	$_[KERNEL]->yield('lookup');
	return;
	}

	foreach ( @net_dns_answers ) {
	if ( $_->type eq "PTR" ) {
	( my $fqdn = $_->rdatastr ) =~ s/\.$//;
	print "$request_address: $fqdn\n";
	last;
	}
	}

	$_[KERNEL]->yield('lookup');
	},
	}
	);

	$poe_kernel->run();
	exit 0;

view raw dns-3.pl hosted with ❤ by GitHub

Version 2 API

	#!/usr/bin/env perl

	use strict;
	use warnings;

	use POE;
	use POE::Component::Client::DNS;

	my @ips = qw( 127.0.0.1 10.0.0.1 192.168.0.1 192.168.1.1 1.2.3.4
	8.8.4.4 8.8.8.8 208.67.220.220 208.67.222.222 );

	my $resolver = POE::Component::Client::DNS->spawn(
	Alias => 'resolver',
	Timeout => 10,
	);

	POE::Session->create(
	inline_states => {
	_start => sub { $_[KERNEL]->yield('lookup') for 1..5; },
	lookup => sub {
	my $ip = shift @ips;

	return unless defined $ip;

	# Example of using POE::Component::Client::DNS Version 2 API
	$_[KERNEL]->post( 'resolver',
	'resolve',
	'response',
	$ip, 'PTR', 'IN' );
	},
	response => sub {
	my $request_address = $_[ARG0]->[0];
	my ( $net_dns_packet, $net_dns_errorstring ) = @{ $_[ARG1] };

	unless ( defined $net_dns_packet ) {
	print "$request_address: error ($net_dns_errorstring)\n";
	$_[KERNEL]->yield('lookup');
	return;
	}

	my @net_dns_answers = $net_dns_packet->answer();

	unless ( scalar @net_dns_answers ) {
	print "$request_address: no answer\n";
	$_[KERNEL]->yield('lookup');
	return;
	}

	foreach ( @net_dns_answers ) {
	if ( $_->type eq 'PTR' ) {
	( my $fqdn = $_->rdatastr ) =~ s/\.$//;
	print "$request_address: $fqdn\n";
	last;
	}
	}

	$_[KERNEL]->yield('lookup');
	},
	}
	);

	$poe_kernel->run();
	exit 0;

view raw dns-2.pl hosted with ❤ by GitHub

SVN Relocate

svn: Repository moved temporarily to 'https://svn.example.com/project/trunk'; please relocate

can be solved by

svn switch --relocate http://svn.example.com/project/trunk \
https://svn.example.com/project/trunk

Find out installed EPEL RPM packages

yum list installed | grep @epel

Global Subversion Settings for CA Certificates

How to avoid

Error validating server certificate for 'https://host.example.com:443':

when doing

svn co https://svn.example.com/project/trun

in a system-wide manner:

wget -O /etc/ssl/certs/your_cer.pem https://example.com/your_cert.pem
mkdir /etc/subversion
cat <<EOF > /etc/subversion/servers
[global]
ssl-authority-files = /etc/ssl/certs/your_cer.pem
EOF

Change Oracle User Password

$ su - oracle
$ sqlplus / as sysdba
SQL> alter user USER identified by "PASSWORD";

VirtualBox 4 USB support in Linux

Download VirtualBox Oracle VM VirtualBox Extension Pack and install it by VirtualBox → File → Preferences... → Extensions → Add package.

Then grant vboxusers groups access to the USB device filesystem:

$ grep 108 /etc/group
vboxusers:x:108:
$ vim /etc/fstab
usbfs  /proc/bus/usb  usbfs  auto,busgid=108,busmode=0775,devgid=108,devmode=0664  0  0

Password protected GRUB Legacy

Add password into the main GRUB section to password protect the interactive operations (i.e. editing menu entries and entering the command-line interface). Optionally add lock into a menu entry to prevent it from executing if no valid password is provided by the user.

$ grub-md5-crypt 
Password: 
Retype password: 
$1$K0Kh10$OmQNQOthH8jppDFQ5TYx5/

$ chmod 600 /boot/grub/menu.lst
$ vim /boot/grub/menu.lst
default 0
...
password --md5sum $1$K0Kh10$OmQNQOthH8jppDFQ5TYx5/

title Linux
kernel ...

title OpenBSD
lock
kernel ...

Create a svn tag from trunk

svn mkdir http://svn.example.com/project/tags -m "release directory"
svn copy http://svn.example.com/project/trunk \
         http://svn.example.com/project/tags/1.0 -m "release 1.0"

Nicer Oracle Date Output

Oracle date output including full year, month, day, hours, minutes and seconds:

SQL> alter session set nls_date_format = 'yyyy-mm-dd hh24:mi:ss';
SQL> alter session set nls_timestamp_format = 'yyyy-mm-dd hh24:mi:ss';
SQL> select foo_date from bar_table where rownum <= 1;

FOO_DATE
-------------------
2012-01-01 01:02:03

Tunneling VNC connection

Connect to a VNC server listening on localhost on port 5901 (+1 than usual) using tunneling:

vncviewer -via yourusername@remotemachine localhost:1

Identify old RPM packages in a directory

	#!/usr/bin/env perl
	# [2011-03-15] viliam.pucik@gmail.com
	# Prints all old (SRC) RPM packages from specified directory.
	# Assumed package format: name-version-release.distribution_optionalpatch.suffix.rpm
	#
	# Use case:
	#
	# $ ls test/
	# glibc-2.12-1.7.el6_0.3.src.rpm glibc-2.12-1.7.el6_0.4.src.rpm glibc-2.12-1.7.el6.src.rpm
	# $ ./oldrpm.pl test/
	# glibc-2.12-1.7.el6.src.rpm glibc-2.12-1.7.el6_0.3.src.rpm

	use strict;
	use warnings;

	if ( scalar @ARGV < 1 ) {
	print "usage: oldrpm.pl <dir>\n";
	exit 1;
	}

	opendir my $dh, $ARGV[0] or die $!;
	my @rpms = grep { /\.rpm$/ } readdir $dh;
	close $dh;

	my %names;
	map { /([\w\-\+]+)-[^-]+-[\d\.]+/, push @{ $names{$1} }, $_ } @rpms;

	for ( sort keys %names ) {
	next if scalar @{ $names{$_} } < 2;
	my $pkg = my $version = my $release = my $patch = undef;
	for ( @{ $names{$_} } ) {
	/[\w\-\+]+-([^-]+)-([\d\.]+)[^_]+_([\d\.]+)/;
	my $_3 = ( defined $3 ) ? $3 : '';
	/[\w\-\+]+-([^-]+)-([\d\.]+)/;
	if ( !defined $pkg
	\|\| $version lt $1
	\|\| ( $version eq $1 && $release lt $2 )
	\|\| ( $version eq $1 && $release eq $2 && $patch lt $_3 ) ) {
	$pkg = $_;
	$version = $1;
	$release = $2;
	$patch = $_3;
	}
	}
	# commented alternative output
	#print "-"x20, "\n";
	for ( @{ $names{$_} } ) {
	#print;
	#print " *" if ( $_ eq $pkg );
	#print "\n";
	print "$_ " if $_ ne $pkg;
	}
	}

view raw oldrpm.pl hosted with ❤ by GitHub

How to build RPM packages as a regular user

Install the necessary packages as root:

sudo yum install rpmdevtools rpm-build redhat-rpm-config

Initialize the directory hierarchy for your custom build RPM packages as a regular user:

rpmdev-setuptree

Optionally add additional attributes into ~/.rpmmacros, e.g.:

echo '%packager Your Name <your.name@foo.bar>' >> ~/.rpmmacros

Use rpmbuild to build RPM from a source RPM or from a SPEC file:

rpmbuild --rebuild package.src.rpm
rpmbuild -bb package.spec