Microservices, Monitoring, Infrastructure as Code & Test Automation

An interesting option I just came accross while trying to FTP a complete site down to my linux clinet from the command line Instead of using the ftp client or mget etc you can use wget -r ftp://username:password@www.yourdomain.com/ But I think this may only recurse directories down to a level of 5 So using the switch -l 0 This solves that problem wget -r  -l 0 ftp://username:password@www.yourdomain.com/

Hadoop can also be run on a single-node in a pseudo-distributed mode where each Hadoop daemon runs in a separate Java process. 1. Install Java SDK if not already java -version In my case java version "1.6.0_22" was available So whereis javac If you don't have a recent java sdk installed download jdk and install wget http://download.oracle.com/otn-pub/java/jdk/6u30-b12/jdk-6u30-linux-i586-rpm.bin Make sure to note the location where you install for use later in setting JAVA_HOME in my case JAVA_HOME was set to /usr 2. Do some basic tasks to make life easy mkdir /hadoop useradd hadoop passwd hadoop groupadd hadoop 3. Get stable release of hadoop, install in to /hadoop/hadoop cd /hadoop wget http://ftp.heanet.ie/mirrors/www.apache.org/dist/hadoop/common/stable/hadoop-1.0.3.tar.gz tar -xvf  hadoop-1.0.3.tar.gz mv hadoop-1.0.3 hadoop export HADOOP_INSTALL=/hadoop/hadoop export PATH=$PATH:$HADOOP_INSTALL/bin Set these up in your logi

Some useful grep / sed / cut commands to help you monitor your log files and detect possible attacks to your system. Mainly based on CentOS 5.4 but can be tailored for other distros 1. Search through /var/log/secure and determine what IPs have the highest failed attempts to your FTP daemon in this instance vsftpd grep "authentication failure" secure.* | grep vsftpd   | sed s/::ffff://g  | cut -d' ' -f14 |sort|uniq -c|sort -nr|more 2. Search through /var/log/secure and determine what IPs have the highest failed attempts to your SSHD daemon grep "refused connect from" secure.*  | grep "sshd" | sed s/::ffff://g  | cut -d' ' -f9 |sort|uniq -c|sort -nr|more