main 9.0
Loading...
Searching...
No Matches
The libmqmsgque Project FAQ

Table of Contents

Questions and answers related to libmsgaue usability.

1. How to make different services communicate?

  • every libmqmsgque communication link is a link between one client and one server
  • the link is created and maintained by libmqmsgque as a socket communication using unix-domain-sockets, inet-domain-sockets or pipes
  • client and server are using the same communication library called libmqmsgque

2. What exactly does 'libmqmsgque'?

  • find and start the service (e.g. server)
  • setup and maintain the socket communication interface between client and server
  • choose the right byte-order (little/big endian)
  • verify 'service' api version
  • if an error happen, propagate the error to the client
  • do all the send and recv stuff
  • do (if necessary) the casting of different types
  • provide an event-loop to smoothly integrate multiple tasks
  • after the work was done do a clean shutdown of the communication

3. How does one service do to understand the data another service is sending to it?

  • DEFINITION: understand means knowing the exact definition of the procedure called.
  • client and server using the same protocol as part of libmqmsgque and this protocol include the service called together with the arguments provided in a libmqmsgque package. The type of a argument can be string, binary, list or native. The native type is defined in MkBufferAtomU. The type string is casted into native if the application require this.
  • the protocol layer is totally hidden to the programmer
  • the server provides a service identified as a 4 byte character string
  • the client is calling the service with the arguments included (example calling service ECOI):
    using libmqmsgque code
    MqSendI (ctx, 567);
    MkErrorCheck (MqSendEND_AND_WAIT (ctx, "ECOI", 10));
    #define MqSendSTART(...)
    #define MqSendI(...)
    #define MkErrorCheck(err)
    using ccmqmsgque code
    ctx.SendSTART ();
    ctx.SendI (567);
    ctx.SendEND_AND_WAIT ("ECOI", 10);
    using csmqmsgque code
    ctx.SendSTART ();
    ctx.SendI (567);
    ctx.SendEND_AND_WAIT ("ECOI", 10);
    using tclmqmsgque code
    $ctx SendSTART
    $ctx SendI 567
    $ctx SendEND_AND_WAIT ECOI 10
    
    using javamqmsgque code
    ctx.SendSTART();
    ctx.SendI(567);
    ctx.SendEND_AND_WAIT("ECOI", 10);
    
    using pymqmsgque code
    ctx.SendSTART()
    ctx.SendI(567)
    ctx.SendEND_AND_WAIT("ECOI", 10)
    
    using rbmqmsgque code
    ctx.SendSTART()
    ctx.SendI(567)
    ctx.SendEND_AND_WAIT("ECOI", 10)
    
    using perlmqmsgque code
    $ctx->SendSTART();
    $ctx->SendI(567);
    $ctx->SendEND_AND_WAIT("ECOI", 10);
    

and wait (MqSendEND_AND_WAIT) or does not wait (MqSendEND) for an answer.

  • the server reply the client-service-call with the return data.
  • the number and the type of the call-arguments and the number and the type of the reply-arguments are not agreed by the protocol but the type checking is strict. strict mean that every argument send type have to be read by the same type with the exception that a cast from and to type C is supported. Every type mismatch will result in an error even if the send value will fit into the target type.

4. It is likely that libmqmsgque will similarly need a common basic format?

  • yes, the data-package-format is defined and libmqmsgque is used on the both endpoints of the communication to read this format.

5. What does libmqmsgque not do?

  • no login verification
  • no encoding or decoding of binary data
  • no encryption or decryption of binary data

6. What is the overhead of libmqmsgque?

  • the latest information is available at: performance

7. It's not clear to me how many instances of a server might exist

  • MqContextC LINK API
  • every client-parent-context start its own server
  • every started server is totally independent from the other server might exist
    • advantage: easy load balancing because every server process is able to use an other processor
    • advantage: no namespace pollution by the server
    • advantage: no crash dependency between the client and the server. (If the server crashes the client still continue to work)
    • advantage: using different programming languages between client and server

8. It's not clear where the state for a service resides

  • DEFINITION: The "state" ought to be the totality of everything that can make two identical sequences of libmqmsgque library commands return different results. A typical case could be if one wants to wrap a Tcl shell up as a libmqmsgque service; this then has an internal state (values of variables, commands defined, etc.), and it would have to be part of the total state of the service.
  • INTRO: MqContextC LINK API
  • every client creating a new parent-context is creating a new parent-context on the server.
  • a new parent context on is in fact a new class-instance able to store per-context data.
  • the context-class-instance is defined as: MqContextC

8.a When libmqmsgque was first being mentioned, it was described as a replacement for shared libraries.

  • from the client point of view it is an replacement
  • OLD: I have to link with libXXX.so
  • NEW: I have to link with libmqmsgque.so and using service XXX
  • NEW: no need for libXXX.so, no linking, no searching for the right library for my OS, no errors on linking again a wrong version of library, no need for installing all libraries on my host and last but not least your tcl event-loop will always been working

8.b My expectations were conditioned by how I understand shared libraries operate. It is now more clear that libmqmsgque is an alternative to shared libraries that has characteristics much different from them.

  • I personal call it a network shared library or just a service
  • from a programming point of view writing applications based on services is much more easy as writing applications based on shared libraries
  • reuse service based code is a lot easier as reusing shared library based code
    (Example: libXXX.so need a libperlXXX.so, libtclXXX.so, libphpXXX.so, libjavaXXX.so. libc::XXX.so, infinite long ... interface library but only one libmqmsgque.so interface library)

9. It's not clear how services are shared

  • DEFINITION: shared as think of it means that the same code can be used by multiple threads of execution simultaneously.
  • a service usually has: libSomthing.(a|lib|so|dll) + libmqmsgque.(a|lib|so|dll) + main.c + context.c + ... *.c
  • sharing code means using dynamic linking (so|dll) rather than static linking (a|lib)
  • sharing a running service is available as:
    • only one client connect to one server
    • every client has one parent-context but multiple child-context
    • an advanced event-queue is available to serve multiple requests at the same time

9.a One interpretation is that multiple clients connect to the same server. The server might have a common resource (e.g. a database) that the clients want to share. How does one go ahead with that?

  • INTRO: MqContextC LINK API
  • DEFINITION server:
    • a server is an application which can handle multiple client connections and serve some kind of services, with the help of a server-API, to the client (example: Sybase/Oracle/Db2/... database, Ars, SAP, PeopleSoft, ... infinite long ...)
    • a libmqmsgque - server is a wrapper around the server-API form 1. to disconnect the client from the server-API
  • in libmqmsgque one libmqmsgque - server supports one client but every client supports multiple libmqmsgque - server.
    • libmqmsgque was designed to replace server specific API's (as static or dynamic libraries)
    • libmqmsgque was designed to be very simple and easy to use for programming purpose
    • libmqmsgque was designed to solve the every 'scripting' language maintain its own link between application API's and language API's problem
  • example: Ars (http://en.wikipedia.org/wiki/Action_Request_System) is an application server using a SQL database to serve a work-flow like applications. Ars has its own user management and providing it's own API for login, query, modify ... . the ArsService created by libmqmsgque does not replace the Ars server. a libmqmsgque ArsService is used to replace the Ars specific protocol and the Ars specific API with a common usable API. in this case you can call it a protocol-converter with additional features to pre-process the data.
    OLD:               Database       NEW:       Database                                      *
                           |                         |                                         *
                       ArsServer                 ArsServer                                     *
                       |       |                 |       |           <- Ars-Protocol           *
    Ars-Protocol ->    |       |           ArsService1  ArsService2  <- libmqmsgque-API + Ars-API *
                       |       |                 |       |           <- libmqmsgque-Protocol      *
    Ars-API      -> Client1 Client2           Client1 Client2        <- libmqmsgque-API           *
    
  • OLD: your client depend on Ars - Api
  • NEW: your client depend on libmqmsgque - Api
  • OLD: if you keep in mind that every application (e.g. Ars, SAP, Peoplesoft, Oracle, Sybase, MqSeries, ... infinite long) providing its own API you will recognize that your application(s) will depend on all these API's
  • NEW: it doesn't matter which application you are using, there is only ONE API -> libmqmsgque
  • to use a new, already available, libmqmsgque - service in TCL you only have to write a wrapper in TCL and NOT in C using the, already available, tclmqmsgque API

9.b Does the above mean that the communication between e.g. ArsService and ArsServer is something that libmqmsgque cannot handle, because that communication can be many clients to one server?

  • the connection between the ArsServiceX and the ArsServer is just an traditional software engineering project. libmqmsgque is providing a solution to connect the client to the ArsService not to the ArsServer because the ArsServer is Ars specific and usually hidden to external programmers.
  • the difference between OLD and NEW is that:
    • OLD: ClientX is using the Ars - API direct
    • NEW: ClientX is using the libmqmsgque - API and ArsServiceX is using the libmqmsgque - API plus the Ars - API

9.c what is the difference between Ars - API and libmqmsgque - API?

  • Ars - API: handle the communication between ArsServiceX and ArsServer using the Ars - Protocol (every application-API providing methods like init (ARInitialization), login (ARVerifyUser), send something (ARSetEntry,...) and read something (ARGetEntry,...))
  • Ars - API: is part of the Ars distribution of Remedy and is a closed source library
  • libmqmsgque - API: handle the communication between ClientX and ArsServiceX using the libmqmsgque - Protocol

9.d does this mean I always have to provide an independent process like ArsServiceX to use an already available application?

  • YES and NO
  • YES: if the application is 'closed' source (e.g. commercial) you have no chance to integrate the libmqmsgque - API into the application
  • NO: if the application is 'open' source you can integrate the libmqmsgque - API direct into the application:
                   YES:    'closed'          NO:     'open'                       *
                          Application              Application                    *
    Application-API: ->   |         |              |         |                    *
                       Service1 Service2           |         |   <- libmqmsgque-API *
    libmqmsgque-API:   ->   |         |              |         |                    *
                       Client1   Client2        Client1   Client2                 *
    
  • NO: libmqmsgque is able to use an external created file-descriptor as argument to the "--pipe --socket #" MqLinkCreate configuration option for socket communication

10. Are services supposed to be re-entrant or serially reusable

  • INTRO: MqContextC LINK API
  • DEFINITION: Re-entrant means that the same body of code can be used by multiple threads of control at once, provided that each thread supplies its own data segment for the code to use.
  • Every Context (parent-context and child-context) has its own services and data.
  • DEFINITION: Serially reusable means that the same body of code can be used by multiple threads of control, provided that execution of the code is allowed to complete before another thread can use the shared body of code.
  • The body of a service-function get the according context as first parameter of type MqContextS. This parameter is used to read package data (example: MqReadU) and to answer the service-request (example: MqSendU)

10.a Can an already started server be reused for new requests?

  • INTRO: MqContextC LINK API
  • Every server has exact one parent-context but multiple child-context.
  • Every parent-context is a process or a thread.
  • If a parent-context is closed all child-context are closed too and the server is shut down.
  • If a child-context is closed the server continue to serve the other context.
  • Multiple parent-context are created with the --fork, --thread or --spawn option using tcp and uds communication
    server --tcp --port 7777 --fork
    
  • For pipe communication multiple parent-context are created as default.

11. Do services have to be locked so that they can complete processing complete requests

  • DEFINITION: In this case locked means prevented from execution.
  • Every parent-context is a process or a thread.
  • Every child-context of this parent-context is part of this single process too.
  • One single process can handle only one request per time but is able to listen on other incoming requests if the original request is waiting on an external resource. This waiting is done in the MqSendEND_AND_WAIT and the MqProcessEvent function.
  • libmqmsgque was designed to act as an event based scheduler doing io in a parallel manner.
  • Example: multiple parent-context (CTX1, CTX2, CTX3) : using 'Tcl' code
    # SLEP ... service doing a 'sleep' system-call on the server -> blocking the server
    # client is doing the following 3 jobs
    after 100 "$CTX1 SendEND_AND_WAIT SLEP 10";     # test-case: service sleeps 10 seconds
    after 100 "$CTX2 SendEND_AND_WAIT SLEP 5";      # test-case: service sleeps 5 seconds
    after 100 "$CTX3 SendEND_AND_WAIT SLEP 8";      # test-case: service sleeps 8 seconds
    # wait on finish all 3 jobs
    
    the example needs 10 seconds to finish and not 23
  • Example: single parent-context (CTX) : using 'Tcl' code
    # SLEP ... service doing a 'sleep' system-call on the server -> blocking the server
    # client is doing the following 3 jobs
    after 100 "$CTX SendEND_AND_WAIT SLEP 10";      # test-case: service sleeps 10 seconds
    after 100 "$CTX SendEND_AND_WAIT SLEP 5";       # test-case: service sleeps 5 seconds
    after 100 "$CTX SendEND_AND_WAIT SLEP 8";       # test-case: service sleeps 8 seconds
    # wait on finish all 3 jobs
    
    the example needs 23 seconds to finish and not 10

11.a How does a 'service' detect different 'context' requests''

  • INTRO: MqContextC LINK API
  • The client-parent-context is linked with the server-parent-context.
  • The data-package-format has an data-entry for the context in duty (HdrS::ctxId).

12. How do they signal success or failure

  • INTRO: MqContextC LINK API
  • Every transaction submit a return-code MQ_RETURN_OK or MQ_RETURN_ERROR to the caller.
  • The return-code is part of the package header (HdrS::code) and is processed in the MqSendEND_AND_WAIT function.
  • The function MqSendEND_AND_WAIT will return an error of type MkErrorE if the return-code is MQ_RETURN_ERROR.