main 9.0
Loading...
Searching...
No Matches
HOWTO Filter - The Basics

What is a filter and how is the filter used for.

INTRODUCTION

"

BI-DIRECTIONAL FILTER

The following code is based on the JAVA example example/java/Filter3.java:

package example;

create the class, implement the the server interface:

class Filter3 extends MqContextC implements IServerSetup {
public Filter3(MqContextC tmpl) {
super(tmpl);
}
MqContextC(MqContextC *tmpl=NULL)

define the ServerSetup function required by the interface

public void ServerSetup() {

get the filter-context to implement the proxy later

MqContextC * SlaveGetFilter()

define a Generig-Handler (+ALL) for the current context

ServiceProxy("+ALL");
void ServiceProxy(MQ_TOK token, MQ_SLAVE_ID id=MQ_SLAVE_OTHER)

and the filter context

ftr.ServiceProxy("+ALL");

define the main function, the factory and create the initial context:

public static void main(String[] argv) {
MqContextC srv = MqFactoryC.Add(Filter3.class,"Filter3").New();

process the command-line arguments and start the initial link:

try {
srv.LinkCreate(argv);

finally start the event-loop and wait for an incoming service-request:

srv.ProcessEvent(MqWaitOnEventE.FOREVER);
MqWaitOnEventE

on exit delete the context and finish the process:

} catch (Throwable e) {
srv.ErrorCatch(e);
} finally {
srv.Exit();
}
}
}

ONE-DIRECTIONAL FILTER

The following code is based on the JAVA example example/java/Filter1.java:

package example;
import java.util.ArrayList;

create a class:

class Filter1 extends MqContextC {
private ArrayList<ArrayList<String>> data = new ArrayList<ArrayList<String>>();
public Filter1(MqContextC tmpl) {
super(tmpl);
}

define a service-handler for the +FTR token:

public static class FTR implements ICallback {
public void Callback(MqContextC ctx) {
ArrayList<String> d = new ArrayList<String>();
while (ctx.ReadGetNumItems() != 0) {
d.add("<" + ctx.ReadC() + ">");
}
((Filter1)ctx).data.add(d);
ctx.SendRETURN();
}
}

define a service-handler for the +EOF token: The EOF is the end-of-data and is used to finally send out all the collected items at once. This is necessary when filtering the data as a whole, not by row.

public static class EOF implements ICallback {

get the filter target (slave 0)

public void Callback(MqContextC ctx) {
MqContextC ftr = ctx.SlaveGetFilter();

loop over data and send ever item

for (ArrayList<String> d: ((Filter1)ctx).data) {
ftr.SendSTART();
for (String s: d) {
ftr.SendC(s);
}
ftr.SendEND_AND_WAIT("+FTR");
}

finally send the EOF to signal \€ end-of-data

ftr.SendSTART();
ftr.SendEND_AND_WAIT("+EOF");
ctx.SendRETURN();
}
}

at the main create initial factory and context:

public static void main(String[] argv) {
MqContextC srv = MqFactoryC.Add(Filter1.class).New();

configure as server:

try {
srv.ConfigSetIsServer(true);

create the link:

srv.LinkCreate(argv);

and setup the both filter-callback:

srv.ServiceCreate("+FTR", new Filter1.FTR());
srv.ServiceCreate("+EOF", new Filter1.EOF());

finally start the event.loop and wait for data:

srv.ProcessEvent(MqWaitOnEventE.FOREVER);

on error catch the message

} catch (Throwable e) {
srv.ErrorCatch(e);

and on exit cleanup the application

} finally {
srv.Exit();
}
}
}

FILTER - SHELL - INTEGRATION

libmqmsgque was designed to act as a glue between different applications. For an overview about the basic concepts we are using the good old shell and using libmqmsgque to extend the usability of the well known pipe '|' syntax.

basic shell behaviour

A shell command-line is a collection of one or more commands linked together using the '|' symbol:

command1 | command2 | command3

command1, command2 and command3 are started by the shell and the stdout of command1 is the stdin of command2 and the stdout of command2 is the stdin of command3. The data send through the pipeline are strings and every command in the pipeline have to parse the string output of the previous command to extract the information's needed.

  • advantage:
    • easy to use and human readable interface based on strings
  • disadvantage:
    • every command has to re-parse the output of the previous command

additional shell behaviour using the libmqmsgque syntax

libmqmsgque is adding an additional link character '@' to the shell and the example from above looks like this:

alfacmd1 @ alfacmd2 @ alfacmd3

Only alfacmd1 is started by the shell and gets '@ alfacmd2 @ alfacmd3' as command-line arguments. libmqmsgque will start the both commands alfacmd2 and alfacmd3 and setup the message-queues:

  1. alfacmd1 -> alfacmd2
  2. alfacmd2 -> alfacmd3

alfacmd2 receiving the output from alfacmd1 and alfacmd3 receiving the output from alfacmd2 without re-parsing the data again.

  • advantage:
    • single parsing of output and direct access to columns and rows
    • the alfacmdX commands are independent processes and only connected by message queues
  • disadvantage:
    • every command using the alfa syntax have to use the libmqmsgque library

interface between shell commands and alfa commands

For full integration of alfa commands into the shell syntax 2 additional interfaces are necessary

  • advantage:
    • direct integration between shell and alfa commands
    • reusing of already available shell commands and adding new features by alfa commands

interface: shellcmd | alfacmd

To connect a shell with an alfa command the special alfa command split is used:

    shellcmd | atool split @ ...

The split command expect input data from stdin and is sending output data as package to an alfa command. For every input data string an output package is created by splitting the input string into output objects using the the delimiter -d.

interface: alfacmd | shellcmd

To connect an alfa with a shell command the special alfa command join is used.
If the libmqmsgque object was created by atool :

    ... @ join | shellcmd

or if the libmqmsgque object was not created by atool :

    ... @ atool join | shellcmd

The join tool expect data from a msgque client as input and create for every input package an stdout output string by joining the objects of the input package together using the delimiter -d .

command pipelines using multiple hosts

starthost: alfacmd1 --tcp --host rmthost --port rmtport
endhost: alfacmd2 --tcp --port rmtport

By default libmqmsgque is using unix-domain sockets (UDS) for communication but inet (TCP) sockets can be used as well. The data-flow is the same as above except that two hosts are involved using libmqmsgque over tcp sockets for connection. The tcp connection is buildup between alfacmd1 and alfacmd2.

  • advantage:
    • multiple hosts can be used

a collection of examples should help to understand the software

example: this is a list of commands already available in this distribution

  • atool - a tool to work like a swiss-knife for command-line operations
  • aexec - a tool to setup a remote function call client and server

example 1 : just the famous hello world example

echo 'hello world with text' | atool split -d " " @ cut -f 0,1 @ join -D ":"

return: hello:world

example 2 : use tcl to create a smart filter

The following tcl code total.tcl does 2 things:

  1. convert the currencies into dollar ($)
  2. calculate the total amount
package require tclmsgque::MqMsgque

set total 0
array set exchange {
  euro    1.3541
  pound   1.9896
  dollar  1
}

proc FTR {ctx} {
  set ftr [$ctx SlaveGetFilter]
  foreach {position amount currency} [$ctx ReadLIST] break
  set amount [expr {$amount * $::exchange($currency)}]
  set currency dollar
  set ::total [expr {$::total + $amount}]
  $ftr Send "W" "+FTR:CD" $position $amount
  $ctx SendRETURN
}

proc EOF {ctx} {
  set ftr [$ctx SlaveGetFilter]
  $ftr Send "W" "+FTR:CD" total $::total
  $ftr Send "W" "+EOF"
  $ctx SendRETURN
}

tclmsgque::MqMsgque Main {
  tclmsgque::MqMsgque::MqContextC create srv
  srv ConfigSetIsServer yes
  srv ConfigSetName total
  try {
    srv LinkCreate {*}$argv
    srv ServiceCreate "+FTR" FTR
    srv ServiceCreate "+EOF" EOF
    srv ProcessEvent MQ_WAIT_FOREVER
  } on error {} {
    srv ErrorCatch
  } finally {
    srv Exit
  }
}

using the following command pipeline:

echo -e "nobody 10 euro\nmanager 1000 dollar\nworker 100 pound" | \
    atool split -d " " @ sort -1 D @ tclsh total.tcl @ \
        atool join -d " : " -0 "%-8s" -1 "%5.2f$"

to create the following result:

nobody   : 13.54$
worker   : 198.96$
manager  : 1000.00$
total    : 1212.50$