Introduction
Interprocess communication (IPC) now is very necessary when developing interactive applications. Such applications include many processes running on the same or different machines.
There are many solutions to resolve this issue, including message queue, shared memory, signaling, semaphores… But to resolve the IPC across machines over the network, we must use the network programming techniques. There are some network protocols that can be used in this case like UDP, TCP, SCTP and TIPC. TIPC has more advantages than other protocols that besides sending messages between processes, it also allows tracking what process joins or leaves the cluster very fast. Let’s discover the details below.
Overview
TIPC is integrated with almost all linux distributions as a system module. Before tipc connection can be established, you must run the tool name tipc-config or tipc (new name). To get tipc-config available, tipcutil must be installed first but with new releases of linux distros like ubuntu, debian… there is a pre-installed tool named tipc (in /usr/sbin) instead.
We usually use this protocol to develop software where there is High Availability and failover.
Some major TIPC concepts
Cluster
A TIPC network consists of individual processing elements or nodes. Many nodes links to each other forming a cluster. Nodes in different clusters cannot communicate with each other using TIPC.
Addressing
The TIPC socket API provides three different address types: Service address: This address type consists of a 32 bit service type identifier and a 32 bit service instance identifier. The type identifier is typically determined and hard coded by the user application programmer, but its value may have to be coordinated with other applications which might be present in the same cluster. The instance identifier is often calculated by the program, based on application specific criteria.
Service range: This address type represents a range of service addresses of the same type and with instances between a lower and an upper range limit. By binding a socket to this address type one can make it represent many instances, something which has proved useful in many cases. This address type is also used as multicast address
Service address: This address is a reference to a specific socket in the cluster. It contains a 32 bit port number and a 32 bit node hash number. The port number is generated by the system when the socket is created, and the node hash is generated from the corresponding node identity as explained earlier. An address of this type can be used for connecting or for sending messages in the same way as service addresses can be used, but is only valid as long as the referenced socket exists.
Messaging
TIPC message transmission can be performed in different ways.
Datagram messaging
Datagram messages are discrete data units between 1 and 66,000 byte of length, transmitted between non-connected sockets of type SOCK_DGRAM or SOCK_RDM. The behaviors are just like UDP
Connection oriented messaging
Connections can be established and used much like we are used to from TCP: the server creates a SOCK_STREAM (but instead of SOCK_STREAM, the sockets can be created as SOCK_SEQPACKET) socket and calls accept(); the client creates a blocking or non-blocking socket of the same type, does connect() on it and start sending or receiving. The address types used can be any of service address or socket address (client side), or service address or service range (server side).
Group messaging
Sending anycast, multicast and broadcast message
Service tracking
This is an important feature of TIPC: it allows to keep track of a node joining or leaving the cluster. To do this, the client program opens a socket with socket type SOCK_SEQPACKET.Then connects to the topology server with addrtype TIPC_SERVICE_ADDR, type TIPC_TOP_SRV and instance TIPC_TOP_SRV. Then send the subscription to that topology server. It should receive a message if a server in the cluster joins or leaves (see the example in the following section.
Example
In the example below, there are 2 programs: server and client. Client subscribes topology event and it’ll receive an event when the server up and down. Both programs accept the first argument as the server type.
File: server.c
#include <sys/types.h>
#include <sys/socket.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <errno.h>
#include <unistd.h>
#include <linux/tipc.h>
int main(int argc, char *argv[])
{
struct sockaddr_tipc server_addr;
int sd;
server_addr.family = AF_TIPC;
server_addr.addrtype = TIPC_SERVICE_RANGE;
server_addr.addr.nameseq.type = arc>1? atoi(argv[1]):1920;
server_addr.addr.nameseq.lower = 1;
server_addr.addr.nameseq.upper = 100;
server_addr.scope = TIPC_ZONE_SCOPE;
/* Make server available */
sd = socket(AF_TIPC, SOCK_RDM, 0);
if (0 != bind(sd, (struct sockaddr *)&server_addr, sizeof(server_addr))) {
printf(“Server: failed to bind port name\n”);
exit(1);
}
printf(“\nServer: port names remain published until server is killed\n”);
/* Wait for user to kill server */
while (1)
sleep(1);
exit(0);
}
File: client.c
#include <sys/types.h>
#include <sys/socket.h>
#include <stdlib.h>
#include <stdio.h>
#include <assert.h>
#include <string.h>
#include <sys/param.h>
#include <sys/poll.h>
#include <netdb.h>
#include <errno.h>
#include <fcntl.h>
#include <sys/ioctl.h>
#include <linux/tipc.h>
static void print_evt(const char *str, struct tipc_event *evt)
{
printf(“%s “, str);
if (evt->event == htonl(TIPC_PUBLISHED))
printf(“published “);
else if (evt->event == htonl(TIPC_WITHDRAWN))
printf(“withdrawn “);
else if (evt->event == htonl(TIPC_SUBSCR_TIMEOUT))
printf(“timeout “);
else {
printf(“unknown event type (%u)\n”, ntohl(evt->event));
return;
}
printf (“{%u,%u,%u} port id <%x:%u>\n”, ntohl(evt->s.seq.type),
ntohl(evt->found_lower), ntohl(evt->found_upper),
ntohl(evt->port.node), ntohl(evt->port.ref));
if (evt->s.seq.type != htonl(0))
printf(“associated with network node %x\n”, ntohl(evt->port.node));
}
int main(int argc, char *argv[])
{
struct sockaddr_tipc topsrv;
struct tipc_subscr subscr;
struct tipc_event event;
int sd;
printf(“TIPC Topology subscriber started\n”);
/* Connect to topology server */
memset(&topsrv, 0, sizeof(topsrv));
topsrv.family = AF_TIPC;
topsrv.addrtype = TIPC_SERVICE_ADDR;
topsrv.addr.name.name.type = TIPC_TOP_SRV;
topsrv.addr.name.name.instance = TIPC_TOP_SRV;
sd = socket (AF_TIPC, SOCK_SEQPACKET, 0);
if (0 > connect(sd, (struct sockaddr *)&topsrv, sizeof(topsrv))) {
perror(“Client: failed to connect to topology server”);
exit(1);
}
printf(“Client: connected to topology server\n”);
/* Subscribe to subscription server name events */
subscr.seq.type = (argc > 1) ? htonl(atoi(argv[1])) : htonl(1920);
subscr.seq.lower = htonl(0);
subscr.seq.upper = htonl(100);
subscr.timeout = (argc > 2) ? htonl(atoi(argv[2])) : htonl(100000000);
subscr.filter = htonl(TIPC_SUB_SERVICE);
subscr.usr_handle[0] = (argc > 3) ? (char)atoi(argv[3]) : (char)2;
if (send(sd, &subscr, sizeof(subscr), 0) != sizeof(subscr)) {
perror(“Client: failed to subscribe to subscription server name”);
exit(1);
}
while (recv(sd, &event, sizeof(event), 0) == sizeof(event)) {
print_evt(“Client: received event for”, &event);
}
perror(“Client: failed to receive event”);
exit(1);
}
How to run the programs?
- Start the tipc module: sudo modprobe tipc
- Enable tipc media ethernet device: tipc bearer enable media eth device eth0
- Start server and client program and observe the output on the console
- Try killing the server and observe the output on the console
Conclusion
TIPC is the protocol that allows you to send/receive messages across machines over the LAN
network. Those machines must connect to each other by the network cable. Furthermore, using
TIPC, we can be aware of a machine in the cluster joining or leaving by connecting to the
topology server and sending the subscription to it. So, it’s usually used in software where there
is high availability or failovers implementation. OpenClovis SAFplus is a middleware like that.
TIPC is integrated with almost all the latest linux distros as a system module, so we can use it
without installing anything.
Other support, please send email to support@openclovis.org.