Friday, July 28, 2017

Orbital Aero Model: TCP Network

I'm designing the Orbital Aero application to support multiple simultaneous types of network connections. The first implemented is a proprietary TCP connection. TCP will make it easier to establish and maintain connections over the Internet. And a proprietary network structures will give me maximum flexibility to replicate the exact same simulation across all connected users.

My class design is as follows:
- The Main Application contains the objects: Orbital Aero Library, Network, and Visual Model Library (among others, but I'm just talking networking for now).
- The Network contains the OANet object (in the future, other network protocol objects).
- The OANet object contains the TCPServer and TCPClient objects and references to the Orbital Aero Library (so it can add/remove bodies) and the Visual Model Library (so it can associate visual models to the newly created entities)..
- The actual networking code is in the TCPServer and TCPClient objects. Function callbacks relay packets back up to OANet so incoming entities can be created and updated.


It's working really well. Currently each entity is updating its position on the network every frame. I don't have any kind of dead reckoning / threshold enabled (yet) which would allow me to cut down the packets but still keep the simulation in sync. That will be a future post.

Here's a screenshot of the TCP network in action:


The left panel is the TCP server creating only the nearby Earth (using a Moon visual model). The center panel is a TCP client creating one of the distant Moons. The right panel is another TCP client creating a second Moon. But all of the bodies are interacting / influencing each other in the simulation.

I'm only supporting one packet type now, an Entity State which shares where the body is in the world. Just that packet is enough for gravity influences to work great. Collisions won't reliably work between the separate simulations yet. I need to create a Collision style packet such that if one simulation detects a collision that it gets updated the same across all the simulations.



Orbital Aero App


I'll be covering just the the new network-specific code below, but it will be easier to follow that way. The TCPServer and TCPClient code can likely be a straight copy-paste into your own application, and then you'll just need to create your own variant of the OANet/Network code. Honestly for most software there is no need to separate the logic in OANet and Network. I plan on supporting multiple network protocols so it made since for me to split them.


Entity.h


Now that the software can support both internally and externally generated entities, I need to keep track of who originated / is managing those entities. I've added a new source variable to my Entity class and timeOfLastUpdate_seconds. The timeOfLastUpdate_seconds is used to timeout entities that have gone stale on the network (likely network dropped out / remote software shut down).


#pragma once

class BodyModel;
class VisualModel;

class Entity
{
public:
Entity(void);
Entity(BodyModel *newBody);
~Entity(void);

BodyModel *bodyObject;

VisualModel *visualModelObject;

bool isVisible;

    enum class SourceEnum
    {
        Undefined = 0,
        Internal = 1,
        OANet = 2,
    };

    SourceEnum source;
    double timeOfLastUpdate_seconds;

private:

};


If the body is internally created, I do:
    newEntity->source = Entity::SourceEnum::Internal;

If the body is externally created from my OANet, I do:
    incomingEntity->source = Entity::SourceEnum::OANet;


OrbitalAeroApp.cpp


I'm including the new Network.h header inside the main app file.


#include ".\Network\Network.h"
.
.

int main(int argc, char* argv[])
{


For testing purposes, I ask the user at program start if this program is standalone, a server, or a client.


    bool isStandAlone = false;
    bool isOANetServer = false;
    bool isOANetClient = false;
    cout << "Network Type:" << endl;
    cout << "  0 Stand Alone" << endl;
    cout << "  1 TCP Server" << endl;
    cout << "  2 TCP Client" << endl;
    int networkInput;
    cin >> networkInput;
    switch (networkInput)
    {
    case 0: // Stand Alone
        isStandAlone = true;
        break;
    case 1: // TCP Server
        isOANetServer = true;
        break;
    case 2: // TCP Client
        isOANetClient = true;
        break;
    }


The server and all clients that connect need to have a unique identification for their own entities so they don't conflict with each other. To do this, I'm asking the user to enter a unique Site ID; all bodies created by that instance of the software will increment off that Site ID to prevent duplicate IDs. Ultimately I'll have the server assign an identification to clients as they connect, but this is quick and easy to start with.


    cout << "Site ID:" << endl;
    int siteID;
    cin >> siteID;


Each time a body is created, the Orbital Aero model pulls a new UInt64 value from a counter to uniquely identify the new body. Then the counter increments by 1 to be ready for the next new body. By initializing the counter to some large initial number based on the user's siteID, I can keep conflicts from happening (in all but very extreme situations). Ultimately I'll have a lower and upper bound for the indices so different networked simulations never step on each other.


    printf("Initializing Orbital Aero...\n");
    OrbitalAeroModel orbitalAero;

    orbitalAero.setIndexCounter(siteID * 100000);


The network is connected based on the options entered earlier. Be sure to have the server running before starting the clients.


    printf("Connecting Network...\n");
    Network network(&orbitalAero, &visualModelLibrary);
    if (isOANetServer)
    {
        network.startOANetServer(4000);
    }
    if (isOANetClient)
    {
        network.connectToOANetServer("127.0.0.1", 4000);
    }
    printf("- %lf\n", Timer::instance()->getElapsedTimeAndResetTimer_seconds(timerReference_seconds));


For testing I only create the Earth if it's a server, and I only create the moon if it's a client.


printf("Adding bodies...\n");
    if (isStandAlone || isOANetServer)
    {
        addBodyEarth(orbitalAero, &visualModelLibrary.planet);
    }

    if (isStandAlone || isOANetClient)
    {
        addBodyMoon(orbitalAero, &visualModelLibrary.planet);
    }


The main program loop (described in detail earlier)...


bool isRunning = true;
while (isRunning)
{
orbitalAero.update(0.2);
.
.


Inside the main loop after all the bodies have updated their positions, I update internally generated entities over the network so other simulations can have the latest positions. I also timeout externally generated entities that haven't been updated in a while (likely the network connection has failed.)

This update frequency (every frame) is very spammy on the network. The long term fix for this is called dead reckoning coupled with threshold breaking. Basically I should send the position of the entity along with its velocities, accelerations, etc. Receiving simulations can take the position I gave it along with the velocities and accelerations and continue flying out that entity without updated network packets. The local simulation keeps track of the true position of the entity and where other simulations think the entity is (based on the velocity/acceleration dead reckoning). When the truth and projected positions exceed a threshold, then the local simulation broadcasts a new packet to resync the entity across the network. Future code to write...


        // Loop through all bodies and update them on the network.
        map <uint64_t, BodyModel*>::iterator iteratorBody2 = orbitalAero.tableBodies.begin();
        while (iteratorBody2 != orbitalAero.tableBodies.end())
        {
            BodyModel *loopBody = iteratorBody2->second;

            // Increment iterator immediately, as the loopBody can be removed here which
            // would invalidate the iterator if it still pointed to a removed body.
            iteratorBody2++;

            if (loopBody->isActive)
            {
                Entity *loopEntity = (Entity*)loopBody->hostData;
                switch (loopEntity->source)
                {
                case Entity::SourceEnum::Internal:
                    network.updateEntityToNetwork(loopEntity);
                    break;
                case Entity::SourceEnum::OANet:
                    if (Timer::instance()->getElapsedTime_seconds(loopEntity->timeOfLastUpdate_seconds) > 1.0)
                    {
                        // Entity hasn't updated in over a second, time out.
                        orbitalAero.removeBody(loopBody->index);
                    }
                    break;
                }
            } // (loopBody.isActive)
        } // (iteratorBody2 != tableBodies.end())

Sleep(20);
}


Network


The Network class contains currently contains just my proprietary OANet. It includes functions so the application can start/stop the OANet server, connect/disconnect to the OANet server, or update an entity on the network.

Network.h



#include ".\OANet\OANet.h"

class Entity;
class OrbitalAeroModel;
class VisualModelLibrary;

class Network
{
private:
    // oaNet networking object.
    OANet oaNet;

public:
    // Network Constructor
    Network(OrbitalAeroModel* newOrbitalAero, VisualModelLibrary* newVisualModelLibrary);

    // Network Destructor
    ~Network();

    // Start the OANet server.
    bool startOANetServer(int port);

    // Stop the OANet server.
    bool stopOANetServer();

    // Connect to an OANet server.
    bool connectToOANetServer(char ipAddress[16], int port);

    // Disconnect from an OANet server.
    bool disconnectFromOANetServer();
    
    // Update an entity to active networks.
    bool updateEntityToNetwork(Entity* entity);
};


Network.cpp



#include "..\Entity.h"
#include ".\OANet\Packets\OANetEntityState.h"
#include "Network.h"

// Network Constructor
Network::Network(OrbitalAeroModel* newOrbitalAero, VisualModelLibrary* newVisualModelLibrary) :
    oaNet(newOrbitalAero, newVisualModelLibrary)
{
    return;
}

// Network Destructor
Network::~Network()
{
    return;
}

// Start the OANet server.
bool Network::startOANetServer(int port)
{
    return oaNet.startServer(port);
}

// Stop the OANet server.
bool Network::stopOANetServer()
{
    return oaNet.stopServer();
}

// Connect to an OANet server.
bool Network::connectToOANetServer(char ipAddress[16], int port)
{
    return oaNet.connectToServer(ipAddress, port);
}

// Disconnect from an OANet server.
bool Network::disconnectFromOANetServer()
{
    return oaNet.disconnectFromServer();
}

// Update an entity to active networks.
bool Network::updateEntityToNetwork(Entity* entity)
{
    if (oaNet.isServerActive() || oaNet.isConnectedToServer())
    {
        oaNet.sendEntityState(entity);
    }
    return true;
}


OANet


OANet is my proprietary network protocol for sharing entity information between simulations. It is based on a TCP Server <--> Client connection. A single Orbital Aero app acts as the server and any number of clients can connect to it.

OANet.h



#include "..\TCP\TCPServer.h"
#include "..\TCP\TCPClient.h"

class Entity;
class OrbitalAeroModel;
class VisualModelLibrary;

class OANet
{
private:
    // TCP Server and Client objects that contain the actual network code.
    TCPServer tcpServer;
    TCPClient tcpClient;

    // Reference to host orbital aero so the network can manipulate network created entities.
    OrbitalAeroModel* orbitalAero;

    // Reference to the visual library so visual models can be associated with incoming entities.
    VisualModelLibrary* visualModelLibrary;

public:
    // OANet Constructor
    OANet(OrbitalAeroModel* newOrbitalAero, VisualModelLibrary* newVisualModelLibrary);

    // OANet Destructor
    ~OANet();

    // Static function used with callbacks for when a packet is received from the network.
    static void callBackProcessIncomingPacket(void* hostNetwork, TCPServer::ConnectionModel* sourceConnection, char packetData[TCPServer::maximumPacketLength_bytes], int packetLength_bytes);
    
    // Function called when a packet is received from the network.
    void processIncomingPacket(TCPServer::ConnectionModel* sourceConnection, char packetData[TCPServer::maximumPacketLength_bytes], int packetLength_bytes);

    // Send an entity over the network.
    bool sendEntityState(Entity* entity);

    // Start a TCP server at a specified port.
    bool startServer(int tcpPort);

    // Stop the TCP server.
    bool stopServer();

    // Return if we're actively hosting a server.
    bool isServerActive();

    // Connect as a client to a server at a specified IP address and port.
    bool connectToServer(char ipAddress[16], int port);

    // Disconnect from a server.
    bool disconnectFromServer();
        
    // Return if we're a client connected to a server.
    bool isConnectedToServer();
};


OANet.cpp



#include "OANet.h"
#include ".\Packets\OANetEntityState.h"
#include "..\..\Entity.h"
#include "..\..\VisualModelLibrary.h"
#include "..\..\..\OrbitalAeroModel\BodyModel.h"
#include "..\..\..\OrbitalAeroModel\OrbitalAeroModel.h"
#include "..\..\..\OrbitalAeroModel\Euler.h"
#include "..\..\..\OrbitalAeroModel\Vector.h"
#include "..\..\..\OrbitalAeroModel\Quaternion.h"
#include "..\..\..\OrbitalAeroModel\ControlSystemModel.h"
#include "..\..\..\OrbitalAeroModel\PartModel.h"
#include "..\..\..\OrbitalAeroModel\BodyModel.h"
#include "..\..\Timer.h"

// OANet Constructor
OANet::OANet(OrbitalAeroModel* newOrbitalAero, VisualModelLibrary* newVisualModelLibrary) :
    // Initialize tcpServer and tcpClient variables.
    // Pass in functions callbacks for when packets are received from the network.
    tcpServer(this, OANet::callBackProcessIncomingPacket),
    tcpClient(this, OANet::callBackProcessIncomingPacket)
{
    // And hold references to the orbital aero and visual model library in this OANet object.
    this->orbitalAero = newOrbitalAero;
    this->visualModelLibrary = newVisualModelLibrary;

    return;
}

// OANet Destructor
OANet::~OANet()
{
    return;
}

// Static function used with callbacks for when a packet is received from the network.
void OANet::callBackProcessIncomingPacket(void* hostNetwork, TCPServer::ConnectionModel* sourceConnection, char packetData[TCPServer::maximumPacketLength_bytes], int packetLength_bytes)
{
    // Packet received from the network!
    // Interpret the hostNetwork pointer as the OA Network, then call its processIncomingPacket function.
    OANet* oaNet = (OANet*)hostNetwork;
    oaNet->processIncomingPacket(sourceConnection, packetData, packetLength_bytes);
    return;
}

// Function called when a packet is received from the network.
void OANet::processIncomingPacket(TCPServer::ConnectionModel* sourceConnection, char packetData[TCPServer::maximumPacketLength_bytes], int packetLength_bytes)
{
    // Packet received from the network.

    // Parse the packet header.
    OANetHeader* packetHeader = (OANetHeader*)packetData;

    // Depending on the packet type, process as appropriate.
    switch (packetHeader->packetType)
    {
    case OANetHeader::PacketTypes::EntityState:
        // Entity State Packet

        if (this->isServerActive())
        {
            // We are a TCP Server, relay incoming EntityState packets to all clients.
            tcpServer.sendPacketToClients(sourceConnection, packetData, packetLength_bytes);
        }

        // Process the packet locally.

        // Interpret the incoming data as an entity state structure.
        OANetEntityState* entityStatePacket = (OANetEntityState*)packetData;

        // Get the BodyModel for the incoming packet.
        BodyModel* incomingBody;
        Entity* incomingEntity;
        if (orbitalAero->hasBody(entityStatePacket->index))
        {
            // Existing body found, get its reference from the orbital aero.
            incomingBody = orbitalAero->getBody(entityStatePacket->index);
            incomingEntity = (Entity*)incomingBody->hostData;
        }
        else
        {
            // Existing body not found, create a new body in the orbital aero.
            orbitalAero->createBody(entityStatePacket->index);
            incomingBody = orbitalAero->getBody(entityStatePacket->index);
            
            // Establish an entity for the body and associate a visual model.
            incomingEntity = new Entity(incomingBody);
            incomingBody->hostData = incomingEntity;
            incomingEntity->visualModelObject = &visualModelLibrary->planet;
            // Set the source for the entity as OANet.
            incomingEntity->source = Entity::SourceEnum::OANet;
        }

        incomingEntity->timeOfLastUpdate_seconds = Timer::instance()->getSystemTime_seconds();

        // Update position & orientation from the packet.
        incomingBody->location_meters = entityStatePacket->location_meters;
        incomingBody->linearVelocity_metersPerSecond = entityStatePacket->linearVelocity_metersPerSecond;
        incomingBody->linearAcceleration_metersPerSecond2 = entityStatePacket->linearAcceleration_metersPerSecond2;
        incomingBody->orientation_quaternions = entityStatePacket->orientation_quaternions;

        // Update physical properties from the packet.
        incomingBody->massBase_kilograms = entityStatePacket->massBase_kilograms;
        incomingBody->radius_meters = entityStatePacket->radius_meters;

        break;
    }

    return;
}

// Start a TCP server at a specified port.
bool OANet::startServer(int tcpPort)
{
    return tcpServer.startServer(tcpPort);
}

// Stop the TCP server.
bool OANet::stopServer()
{
    return tcpServer.stopServer();
}

// Return if we're actively hosting a server.
bool OANet::isServerActive()
{
    return tcpServer.isActive();
}

// Return if we're a client connected to a server.
bool OANet::isConnectedToServer()
{
    return tcpClient.isConnected();
}

// Connect as a client to a server at a specified IP address and port.
bool OANet::connectToServer(char ipAddress[16], int port)
{
    return tcpClient.tcpConnect(ipAddress, port);
}

// Disconnect from a server.
bool OANet::disconnectFromServer()
{
    return tcpClient.tcpDisconnect();

}

// Send an entity to the connected networks.
bool OANet::sendEntityState(Entity* entity)
{
    // Create an entity state packet from the entity object.
    OANetEntityState outgoingEntityStatePacket(entity);

    if (tcpServer.isActive())
    {
        // I am acting as a server, send the entity to all the clients.
        tcpServer.sendPacketToClients(nullptr, (char*)&outgoingEntityStatePacket, sizeof(outgoingEntityStatePacket));
    }
    else if (tcpClient.isConnected())
    {
        // I am a client connected to a server, send only to the server.
        tcpClient.sendPacketToTcp((char*)&outgoingEntityStatePacket, sizeof(outgoingEntityStatePacket));
    }

    return true;
}


OANet Packets


The OANet will eventually support many packets. To be able to parse different packet types, I'm creating a common OANetHeader that all packets will contain. The first item of that header is a single byte that contains the PacketType. The next item is the timeStamp the packet was sent.

OANetHeader.h



#pragma once

#include "stdint.h"

// Header for all OANet Network Messages
struct OANetHeader
{
    // Enumerations of the Packet Types that are sent over OANet
    enum class PacketTypes : uint8_t
    {
        EntityState = 1
    };

    // The packet type of the message.
    PacketTypes packetType;

    // The time stamp the packet was sent.
    double timeStamp_seconds;
};


OANetEntityState.h


The first message I'm supporting is an Entity State. It includes the OANetHeader first (as all my network structures will) followed by the entity's unique index, location, velocity, acceleration, orientation, and other important properties.

#pragma once

#include "stdint.h"
#include "OANetHeader.h"
#include "..\..\..\..\OrbitalAeroModel\Vector.h"
#include "..\..\..\..\OrbitalAeroModel\Quaternion.h"

class Entity;

struct OANetEntityState
{
    OANetHeader header;
    uint64_t index; // Unique identification of this body.
    Vector location_meters;
    Vector linearVelocity_metersPerSecond;
    Vector linearAcceleration_metersPerSecond2;
    Quaternion orientation_quaternions;
    double massBase_kilograms;
    double radius_meters;

    OANetEntityState(Entity* entity);
};


OANetEntityState.cpp


The constructor for the OANetEntityState takes in an Entity pointer and will populate the packet as necessary.

#include <vector>
#include "..\..\..\Entity.h"
#include "..\..\..\..\OrbitalAeroModel\Euler.h"
#include "..\..\..\..\OrbitalAeroModel\Vector.h"
#include "..\..\..\..\OrbitalAeroModel\Quaternion.h"
#include "..\..\..\..\OrbitalAeroModel\ControlSystemModel.h"
#include "..\..\..\..\OrbitalAeroModel\PartModel.h"
#include "..\..\..\..\OrbitalAeroModel\BodyModel.h"

#include "OANetHeader.h"
#include "OANetEntityState.h"

OANetEntityState::OANetEntityState(Entity* entity)
{
    this->header.packetType = OANetHeader::PacketTypes::EntityState;
    this->header.timeStamp_seconds = 0.0;
    this->index = entity->bodyObject->index;
    this->location_meters = entity->bodyObject->location_meters;
    this->linearVelocity_metersPerSecond = entity->bodyObject->linearVelocity_metersPerSecond;;
    this->linearAcceleration_metersPerSecond2 = entity->bodyObject->linearAcceleration_metersPerSecond2;
    this->orientation_quaternions = entity->bodyObject->orientation_quaternions;
    this->massBase_kilograms = entity->bodyObject->massBase_kilograms;
    this->radius_meters = entity->bodyObject->radius_meters;
    return;
}


TCP


Now into the meat of the networking code. I'm actually not going to go into too many details here as it's fairly well commented and covered in a previous blog post. The basic idea is I have a TCPServer class and a TCPClient class.

Creating a server in the TCPServer class causes it to spawn a new thread that listens for connections. When it establishes a new connection, it spools off another thread to handle communication with that new connection. The previous thread listening for connections continues in case someone else wants to connect. If another clients comes in, then another new thread is creating to listen for packets from that second client.

TCPServer.h



#pragma once

#define WIN32_LEAN_AND_MEAN
#include <WinSock2.h>
#include <ws2tcpip.h>
#include <mutex>
#include <vector>

class TCPServer
{
public:

    // Constant denoting the maximum packet size that can be handled.
    static const int maximumPacketLength_bytes = 8192;

    // Class that stores information related to each of the established connections.
    class ConnectionModel
    {
    public:
        SOCKET clientSocket;
        // Add other variables as desired associated with this client connection.

        // Constructor defining the client socket.
        ConnectionModel(SOCKET newClientSocket)
        {
            this->clientSocket = newClientSocket;
        }
    };

    // TCP Server Constructor
    TCPServer(void* newHostNetwork, void(*newCallbackProcessIncomingPacket)(void* hostNetwork, TCPServer::ConnectionModel* sourceConnection, char packetData[TCPServer::maximumPacketLength_bytes], int packetLength_bytes));

    // TCP Server Destructor
    ~TCPServer();

    // Start a TCP server at the given port.
    bool startServer(int tcpPort);

    // Stop the TCP server.
    bool stopServer();

    // Send a packet to all clients except to the client that originated the packet (sourceConnection).
    bool sendPacketToClients(ConnectionModel* sourceConnection, char packetData[maximumPacketLength_bytes], int packetLength_bytes);

    // Return if the server is active and listening for connections.
    bool isActive();

private:

    // Socket that establishes new connections.
    SOCKET listenSocket;

    // Flag if the server is active.
    bool _isActive;

    // Reference to the host network that instantiated the TCP client object.
    // Packets will be relayed to the host network.
    void* hostNetwork;

    // Multithreading lock to protect changes to connections list while iterating through it.
    std::mutex clientConnectionsMutex;

    // List that holds all client connections.
    std::vector<ConnectionModel*> clientConnections;

    // Disconnect all clients. (This does not prevent new clients from connecting later.)
    bool disconnectAllClients();

    // Add a new client to the maintained connections.
    ConnectionModel* addClient(SOCKET clientSocket);

    // Remove a client from the maintained connections.
    bool removeClient(ConnectionModel* clientConnection);

    // Thread function listening for new clients.
    void tcpClientListener();

    // Thread function for listening for packets from clients.
    void tcpPacketListener(ConnectionModel* clientConnection);

    // Variable of the function callback in the host network to call when a packet is received.
    void(*callbackProcessIncomingPacket)(void *hostNetwork, ConnectionModel* sourceConnection, char packetData[maximumPacketLength_bytes], int packetLength_bytes);
};


TCPServer.cpp



#include <process.h>
#include <stdlib.h>
#include <stdio.h>
#include <thread>
#include <iostream>

#pragma comment (lib, "Ws2_32.lib")

#include "TCPServer.h"

//#include <windows.h>

// TCP Server Constructor
TCPServer::TCPServer(void* newHostNetwork, void(*newCallbackProcessIncomingPacket)(void* hostNetwork, TCPServer::ConnectionModel* sourceConnection, char packetData[TCPServer::maximumPacketLength_bytes], int packetLength_bytes))
{
    this->hostNetwork = newHostNetwork;
    this->callbackProcessIncomingPacket = newCallbackProcessIncomingPacket;

    this->_isActive = false;

    // Initialize Winsock.
    WSADATA wsd;
    if (WSAStartup(MAKEWORD(2, 2), &wsd) != 0)
    {
        std::cout << "WSAStartup failed." << std::endl;
    }
    return;
}

// TCP Server Destructor
TCPServer::~TCPServer()
{
    WSACleanup();
    return;
}

// Start a TCP server at the given port.
bool TCPServer::startServer(int tcpPort)
{
    // Bound checking for given TCP port.
    if (tcpPort <= 0 || tcpPort >= 65535)
    {
        std::cout << "Invalid port: " << tcpPort << std::endl;
        return false;
    }

    std::cout << "Listening on port: " << tcpPort << std::endl;

    // Setup address info hints.
    struct addrinfo addressInfoHints;
    memset(&addressInfoHints, 0, sizeof(addressInfoHints));
    addressInfoHints.ai_family = AF_INET;
    addressInfoHints.ai_socktype = SOCK_STREAM;
    addressInfoHints.ai_protocol = IPPROTO_TCP;
    addressInfoHints.ai_flags = AI_PASSIVE;

    // Get address TCP port string.
    char portString[20];
    sprintf(portString, "%d", tcpPort);

    // Resolve the server address and port.
    struct addrinfo* addressInfoResult = NULL;
    int getaddrinfoReturn = getaddrinfo(NULL, portString, &addressInfoHints, &addressInfoResult);
    if (getaddrinfoReturn != 0)
    {
        std::cout << "getaddrinfo failed." << std::endl;
        return false;
    }

    // Create a TCP socket to listen for clients.
    listenSocket = socket(addressInfoResult->ai_family, addressInfoResult->ai_socktype, addressInfoResult->ai_protocol);
    if (listenSocket == INVALID_SOCKET)
    {
        std::cout << "listen socket failed." << std::endl;
        freeaddrinfo(addressInfoResult);
        return false;
    }

    // Bind the TCP socket to the address.
    int bindReturn = bind(listenSocket, addressInfoResult->ai_addr, (int)addressInfoResult->ai_addrlen);
    if (bindReturn == SOCKET_ERROR)
    {
        std::cout << "bind socket failed." << std::endl;
        freeaddrinfo(addressInfoResult);
        closesocket(listenSocket);
        return false;
    }

    freeaddrinfo(addressInfoResult);

    // Initialize / clear the connections list.
    clientConnections.clear();

    // Start the TCP Client Listener thread to listen for connections.
    std::thread{ &TCPServer::tcpClientListener, this }.detach();

    this->_isActive = true;

    return true;
}

// Stop the TCP server.
bool TCPServer::stopServer()
{
    this->_isActive = false;

    // Close the listen socket to stop receiving new connections.
    closesocket(listenSocket);

    // Disconnect all clients.
    disconnectAllClients();

    return true;
}

// Thread function listening for new clients.
void TCPServer::tcpClientListener()
{
    // Listen for new connections until the listenSocket is closed.
    while (1)
    {
        // listen will wait until a connection is requested on the socket. If the socket is closed
        // while waiting, then listen will break from its wait and throw a socket error.
        int listenResult = listen(this->listenSocket, SOMAXCONN);
        if (listenResult != SOCKET_ERROR)
        {
            // Accept a client socket
            SOCKET clientSocket = accept(this->listenSocket, NULL, NULL);
            if (clientSocket != INVALID_SOCKET)
            {
                // Create a connection to the client and begin a listener thread.
                ConnectionModel* clientConnection = addClient(clientSocket);
                std::thread{ &TCPServer::tcpPacketListener, this, clientConnection }.detach();
            }
            else
            {
                std::cout << "TCP accept failed." << std::endl;
                break;
            }
        }
        else
        {
            std::cout << "TCP listen failed." << std::endl;
            break;
        }
    }

    return;
}

// Thread function for listening for packets from clients.
void TCPServer::tcpPacketListener(ConnectionModel* clientConnection)
{
    char receiveBuffer[maximumPacketLength_bytes];

    // Receive until the client connection is closed.
    while (1)
    {
        // recv will wait until a packet is received on the socket. If the socket is closed
        // while waiting, then recv will break from its wait and throw a socket error.
        int packetLength_bytes = recv(clientConnection->clientSocket, receiveBuffer, maximumPacketLength_bytes, 0);
        if (packetLength_bytes > 0)
        {
            // Relay the packet back to the host for processing.
            callbackProcessIncomingPacket(this->hostNetwork, clientConnection, receiveBuffer, packetLength_bytes);
        }
        else if (packetLength_bytes == 0)
        {
            std::cout << "TCP connection closed." << std::endl;
            break;
        }
        else
        {
            std::cout << "TCP socket receive error." << std::endl;
            break;
        }
    };

    // Client connection was closed, remove it from our connection list.
    removeClient(clientConnection);

    return;
}

// Add a new client to the maintained connections.
TCPServer::ConnectionModel* TCPServer::addClient(SOCKET clientSocket)
{
    // Create a new connection for the socket.
    ConnectionModel* clientConnection = new ConnectionModel(clientSocket);

    clientConnectionsMutex.lock();

    // Add the client to the connections list.
    clientConnections.push_back(clientConnection);

    std::cout << "Client added!" << std::endl;
    std::cout << "Active Connections: " << clientConnections.size() << std::endl;

    clientConnectionsMutex.unlock();

    return clientConnection;
}

// Remove a client from the maintained connections.
bool TCPServer::removeClient(ConnectionModel* clientConnection)
{
    clientConnectionsMutex.lock();

    // Shutdown and close the socket.
    shutdown(clientConnection->clientSocket, SD_BOTH);
    closesocket(clientConnection->clientSocket);

    // Remove the client from our connection list.
    auto removedConnection = std::find(clientConnections.begin(), clientConnections.end(), clientConnection);
    if (removedConnection != clientConnections.end())
    {
        clientConnections.erase(removedConnection);
    }

    // Free the connection memory.
    delete clientConnection;

    std::cout << "Client removed!" << std::endl;
    std::cout << "Active Connections: " << clientConnections.size() << std::endl;

    clientConnectionsMutex.unlock();

    return true;
}

// Disconnect all clients. (This does not prevent new clients from connecting later.)
bool TCPServer::disconnectAllClients()
{
    clientConnectionsMutex.lock();

    // Loop through the connections and close each of their connections.
    std::vector<ConnectionModel*>::iterator clientConnectionIterator;
    for (clientConnectionIterator = clientConnections.begin(); clientConnectionIterator != clientConnections.end(); clientConnectionIterator++)
    {
        ConnectionModel* loopClientConnection = *clientConnectionIterator;

        // Closing the socket will cause the listener threads for those sockets to close.
        shutdown(loopClientConnection->clientSocket, SD_BOTH);
        closesocket(loopClientConnection->clientSocket);
    }

    clientConnectionsMutex.unlock();

    return true;
}

// Send a packet to all clients except to the client that originated the packet (sourceConnection).
bool TCPServer::sendPacketToClients(ConnectionModel* sourceConnection, char packetData[maximumPacketLength_bytes], int packetLength_bytes)
{
    clientConnectionsMutex.lock();

    // Loop through the connections and send the packet to each.
    std::vector<ConnectionModel*>::iterator clientConnectionIterator;
    for (clientConnectionIterator = clientConnections.begin(); clientConnectionIterator != clientConnections.end(); clientConnectionIterator++)
    {
        ConnectionModel* loopClientConnection = *clientConnectionIterator;

        // Skip sending the packet if it's to the source.
        if ((sourceConnection != nullptr) && (loopClientConnection->clientSocket == sourceConnection->clientSocket))
        {
            continue;
        }

        // Send the packet on the socket!
        int bytesSent = send(loopClientConnection->clientSocket, packetData, packetLength_bytes, 0);
        if (bytesSent == SOCKET_ERROR)
        {
            std::cout << "Send failed." << std::endl;
        }
    }

    clientConnectionsMutex.unlock();

    return true;
}

// Return if the server is active and listening for connections.
bool TCPServer::isActive()
{
    return this->_isActive;
}



TCPClient.h



#pragma once

#define WIN32_LEAN_AND_MEAN
#include <WinSock2.h>
#include <ws2tcpip.h>

#include "TCPServer.h"

class TCPClient
{
private:
    // Flag if the client has an active connection to a server.
    bool _isConnected;

    // Thread function listening for TCP packets from the server.
    void tcpListener();

    // Socket of the TCP connection to the server.
    SOCKET tcpSocket;

    // Variable of the function callback in the host network to call when a packet is received.
    void(*callbackProcessIncomingPacket)(void *hostNetwork, TCPServer::ConnectionModel* sourceConnection, char packetData[TCPServer::maximumPacketLength_bytes], int packetLength_bytes);

    // Reference to the host network that instantiated the TCP client object.
    // Packets will be relayed to the host network.
    void* hostNetwork;

public:

    // Constant denoting the maximum packet size that can be handled.
    static const int maximumPacketLength_bytes = 8192;

    // Constructor for the TCP Client.
    TCPClient(void* newHostNetwork, void(*newCallbackProcessIncomingPacket)(void* hostNetwork, TCPServer::ConnectionModel* sourceConnection, char packetData[TCPServer::maximumPacketLength_bytes], int packetLength_bytes));

    // Destructor for the TCP Client.
    ~TCPClient();

    // Connect to the TCP server at the specified server address and port.
    bool tcpConnect(char serverAddress[16], int tcpPort);

    // Close the TCP connection with the server.
    bool tcpDisconnect();

    // Send a packet to the TCP server.
    bool sendPacketToTcp(char packetData[maximumPacketLength_bytes], int packetLength_bytes);

    // Return if currently connected to a server.
    bool isConnected();
};


TCPClient.cpp


#include <process.h>
#include <stdio.h>
#include <map>
#include <string>
#include <iostream>
#include <sstream>
#include <time.h>
#include <thread>

#pragma comment (lib, "Ws2_32.lib")

#include "TCPClient.h"

// Constructor for the TCP Client.
TCPClient::TCPClient(void* newHostNetwork, void(*newCallbackProcessIncomingPacket)(void* hostNetwork, TCPServer::ConnectionModel* sourceConnection, char packetData[TCPServer::maximumPacketLength_bytes], int packetLength_bytes))
{
    this->hostNetwork = newHostNetwork;
    this->callbackProcessIncomingPacket = newCallbackProcessIncomingPacket;

    _isConnected = false;

    // Initialize Winsock.
    WSADATA wsd;
    if (WSAStartup(MAKEWORD(2, 2), &wsd) != 0)
    {
        std::cout << "WSAStartup failed." << std::endl;
    }
    return;
}

// Destructor for the TCP Client.
TCPClient::~TCPClient()
{
    // Disconnect the TCP socket.
    tcpDisconnect();

    WSACleanup();
    return;
}

// Connect to the TCP server at the specified server address and port.
bool TCPClient::tcpConnect(char serverAddress[16], int tcpPort)
{
    // Setup address info hints.
    struct addrinfo addressInfoHints;
    memset(&addressInfoHints, 0, sizeof(addressInfoHints));
    addressInfoHints.ai_family = AF_UNSPEC;
    addressInfoHints.ai_socktype = SOCK_STREAM;
    addressInfoHints.ai_protocol = IPPROTO_TCP;

    // Resolve the server address.
    char portString[20];
    sprintf(portString, "%d", tcpPort);
    struct addrinfo* addressInfoResult = NULL;
    int getaddrinfoReturn = getaddrinfo(serverAddress, portString, &addressInfoHints, &addressInfoResult);
    if (getaddrinfoReturn != 0)
    {
        std::cout << "getaddrinfo failed." << std::endl;
        return false;
    }

    // Loop through potential server addresses to attempt connection.
    for (struct addrinfo *ptr = addressInfoResult; ptr != NULL; ptr = ptr->ai_next)
    {
        // Create the TCP socket.
        tcpSocket = socket(ptr->ai_family, ptr->ai_socktype, ptr->ai_protocol);
        if (tcpSocket == INVALID_SOCKET)
        {
            std::cout << "TCP socket failed." << std::endl;
            return false;
        }

        // Connect to the server.
        int connectReturn = connect(tcpSocket, ptr->ai_addr, (int)ptr->ai_addrlen);
        if (connectReturn == SOCKET_ERROR)
        {
            closesocket(tcpSocket);
            tcpSocket = INVALID_SOCKET;
            continue;
        }
        break;
    }

    freeaddrinfo(addressInfoResult);

    if (tcpSocket == INVALID_SOCKET)
    {
        std::cout << "Unable to connect to server!" << std::endl;
        return false;
    }

    _isConnected = true;

    // Start the TCP Listener thread.
    std::thread{ &TCPClient::tcpListener, this }.detach();

    return true;
}

// Close the TCP connection with the server.
bool TCPClient::tcpDisconnect()
{
    // Shutdown and close the socket.
    shutdown(tcpSocket, SD_BOTH);
    closesocket(tcpSocket);

    return true;
}

// Thread function listening for TCP packets from the server.
void TCPClient::tcpListener()
{
    char receiveBuffer[maximumPacketLength_bytes];

    // Receive until the connection is closed.
    while (1)
    {
        // recv will wait until a packet is received on the socket. If the socket is closed
        // while waiting, then recv will break from its wait and throw a socket error.
        int packetLength_bytes = recv(tcpSocket, receiveBuffer, maximumPacketLength_bytes, 0);
        if (packetLength_bytes > 0)
        {
            // Relay the packet back to the host for processing.
            callbackProcessIncomingPacket(this->hostNetwork, nullptr, receiveBuffer, packetLength_bytes);
        }
        else if (packetLength_bytes == 0)
        {
            std::cout << "TCP connection closed." << std::endl;
            break;
        }
        else
        {
            std::cout << "TCP socket receive error." << std::endl;
            break;
        } // receiveSize_bytes
    }

    _isConnected = false;

    return;
}

// Return if currently connected to a server.
bool TCPClient::isConnected()
{
    return _isConnected;
}

// Send a packet to the TCP server.
bool TCPClient::sendPacketToTcp(char packetData[maximumPacketLength_bytes], int packetLength_bytes)
{
    // Send the packet.
    int sendResult = send(tcpSocket, packetData, packetLength_bytes, 0);
    if (sendResult == packetLength_bytes)
    {
        // Bytes successfully sent.
        return true;
    }
    else if (sendResult < 0)
    {
        //std::cout << "TCP send failed at socket level." << std::endl;
        return false;
    }
    else
    {
        //std::cout << "TCP failed to send entire message." << std::endl;
        return false;
    }
}



Copyright (c) 2017 Clinton Kam
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Wednesday, June 28, 2017

TCP / UDP Gateway

I work on UDP networking code near daily for my day job, but I have some use cases for TCP networking in my Orbital Aero among other projects. The Orbital Aero code actually broadcasts UDP states of all its bodies now, and I'll go over its networking soon. However I want the Orbital Aero to have comprehensive networking capabilities, so both TCP and UDP are on its development track. The most fundamental difference between UDP and TCP is whether a connection is maintained or not.


UDP (User Datagram Protocol) is a connectionless protocol. In UDP, messages are sent to a unicast address (one computer), to a broadcast address (all computers on the network), or to a multicast address (all computers subscribed to those messages on the network). The sender has no knowledge if the packet was received by any recipients (unless code is specifically written by the receiver to confirm receipt, but even then that receipt packet may not make it to the sender!). The sender also doesn't even know what recipients are out there (again unless code is specifically written by the receivers to announce themselves, but even then those packets may not make it to the sender!). Although the design of a UDP network may specify a server and clients, that's entirely up to how it was created by the programmer. A large UDP network can just as easily have no single server.


TCP (Transmission Control Protocol) is a connection based protocol. In TCP, a connection is established and maintained between two computers. If a message is sent from one computer to another, the underlying network infrastructure confirms that the message made it through. TCP setups typically have a server and one or many clients that may connect to it.

So if TCP (basically) guarantees delivery and UDP doesn't, why would you choose UDP? It all depends on the use-case. If I was creating a chat program between two people, it would be very important that every message made it through. If a message failed to arrive, I'd want the sending software to know that so it can gracefully handle the situation (try connecting / sending again later, report that the message failed to arrive, try an alternate means of delivery, etc.). But consider simulation software that is broadcasting the positions of hundreds of simulated bodies orbiting each other at real time in space. If the location of a satellite in orbit fails to make it to its destination for any reason (packet dropped, network momentarily interrupted, etc.), there is no need for the sender to re-send that message because the satellite has already moved. And likely the sender has already sent a new location for that satellite that is now more accurate than the lost one anyways. We wouldn't want the overhead of TCP to know an old value, we'd rather just have the latest UDP state.

In the following code I created a simple network gateway for UDP traffic using a TCP server to cross between separate networks. Say network A has a UDP network sharing simulation bodies, network B has a UDP network sharing simulation bodies, and network C has a UDP network sharing simulation bodies. Those networks may be separated down the hallway or across the country, but they all have access to one another, such as via the Internet. I can create a server (at A, B, C, or a completely separate location) to share those UDP messages as though they were all on the same network. Basically a VERY simple VPN (Virtual Private Network).


There are abundant examples online of a TCP server with a single client so no need for a tutorial on that. There are a few examples online of a TCP server with multiple clients, but they were typically way overkill and/or far too much for a teaching aid. I think the following is a fairly straightforward example of a multi-client TCP server. It also includes code for sending and receiving UDP messages so it doubles as a tutorial for that.

Source Code


Below is my explanation of the source code, but you can find the complete files at:
http://kamoly.space/projects/tcp_udp_gateway/

Gateway Server



Headers


This example was written for Windows, but I'll revisit it later to add Linux support. In the mean time, here are the includes you need to compile everything in Windows.


#define WIN32_LEAN_AND_MEAN

#include <process.h>
#include <windows.h>
#include <winsock2.h>
#include <ws2tcpip.h>
#include <stdlib.h>
#include <stdio.h>
#include <vector>
#include <thread>
#include <mutex>
#include <iostream>

#pragma comment (lib, "Ws2_32.lib")

Constants


// Constant denoting the maximum packet size that can be handled.


const int maximumPacketLength_bytes = 8192;

Connection Class


I created a ConnectionModel that can hold any sorts of information I want to keep associated with a connection. For now it's very basic - only keeping track of the socket the client is connected on. However it could be expanded to include a username of who connected, what time they connected, what part of the service they're using, etc.


// Class that stores information related to each of the established connections.
class ConnectionModel
{
public:
    SOCKET clientSocket;
    // Add other variables as desired associated with this client connection.

    // Constructor defining the client socket.
    ConnectionModel(SOCKET newClientSocket)
    {
        this->clientSocket = newClientSocket;
    }
};

Prototypes


I don't have a separate header file to include for the Gateway Server (it's a very small tutorial app), so all the function prototypes are just near the top.


bool sendPacketToClients(ConnectionModel* sourceConnection, char packetData[maximumPacketLength_bytes], int packetLength_bytes);
ConnectionModel* addClient(SOCKET clientSocket);
bool removeClient(ConnectionModel* clientConnection);
void tcpClientListener(SOCKET* listenSocket);
void tcpPacketListener(ConnectionModel* clientConnection);
bool disconnectAllClients();

Global Variables


The Gateway Server has 2 global variables: One is vector (list) that contains pointers to all the client connections. The second is a mutex lock to prevent modifications to the connection list while another part of the server is iterating through it.


// Multithreading lock to protect changes to connections list while iterating through it.
std::mutex clientConnectionsMutex;

// List that holds all client connections.
std::vector<ConnectionModel*> clientConnections;

Main


Now that we're passed the infrastructure setup, we'll begin with the main function! This is a console based application that takes in a single command line argument for the listen port. If no port is given, then it listens on port 4000.


// Main takes a single command line argument for the listen port.
int main(int argc, char *argv[])
{
    // Determine the TCP port to listen for connections.
    // Default to 4000 unless a command line argument is given with another value.
    int tcpPort = 4000;
    if (argc >= 2)
    {
        // Port given, attempt to use it.
        tcpPort = atoi(argv[1]);
    }
    else
    {
        // If no arguments are given, show the format.
        std::cout << "Command line arguments:" << std::endl;
        std::cout << "<TCP Port>" << std::endl;
    }

    // Bound checking for given TCP port.
    if (tcpPort <= 0 || tcpPort >= 65535)
    {
        tcpPort = 4000;
    }
    std::cout << "Listening on port: " << tcpPort << std::endl;


For Windows, we'll need to initialize Winsock.


    // Initialize Winsock.
    WSADATA wsaData;
    int wsaStartupReturn = WSAStartup(MAKEWORD(2, 2), &wsaData);
    if (wsaStartupReturn != 0)
    {
        std::cout << "WSAStartup failed." << std::endl;
        return 1;
    }


Next we'll define an addrinfo with our hints for the TCP server.


    // Setup address info hints.
    struct addrinfo addressInfoHints;
    memset(&addressInfoHints, 0, sizeof(addressInfoHints));
    addressInfoHints.ai_family = AF_INET;
    addressInfoHints.ai_socktype = SOCK_STREAM;
    addressInfoHints.ai_protocol = IPPROTO_TCP;
    addressInfoHints.ai_flags = AI_PASSIVE;

    // Get address TCP port string.
    char portString[20];
    sprintf(portString, "%d", tcpPort);

    // Resolve the server address and port.
    struct addrinfo* addressInfoResult = NULL;
    int getaddrinfoReturn = getaddrinfo(NULL, portString, &addressInfoHints, &addressInfoResult);
    if (getaddrinfoReturn != 0)
    {
        std::cout << "getaddrinfo failed." << std::endl;
        WSACleanup();
        return 1;
    }


The addressInfoResult from getaddrinfo is used to create and bind the listenSocket.


    // Create a TCP socket to listen for clients.
    SOCKET listenSocket = socket(addressInfoResult->ai_family, addressInfoResult->ai_socktype, addressInfoResult->ai_protocol);
    if (listenSocket == INVALID_SOCKET)
    {
        std::cout << "listen socket failed." << std::endl;
        freeaddrinfo(addressInfoResult);
        WSACleanup();
        return 1;
    }

    // Bind the TCP socket to the address.
    int bindReturn = bind(listenSocket, addressInfoResult->ai_addr, (int)addressInfoResult->ai_addrlen);
    if (bindReturn == SOCKET_ERROR)
    {
        std::cout << "bind socket failed." << std::endl;
        freeaddrinfo(addressInfoResult);
        closesocket(listenSocket);
        WSACleanup();
        return 1;
    }

    freeaddrinfo(addressInfoResult);


At this point our server has a listen socket defined and is ready to accept connections. Let's ensure our connections list is clear and kickoff a thread to listen for connections.


    // Initialize / clear the connections list.
    clientConnections.clear();

    // Start the TCP Client Listener thread to listen for connections.
    std::thread{ tcpClientListener, &listenSocket }.detach();


The real work of this gateway is done on the tcpClientListener thread and the connection threads that it spools off. This main thread just needs to sit here and look pretty until the user wants to close the gateway (by pushing a key on the keyboard).


    // All client listening and packet relaying is done on other threads.
    // Continue the program until the user presses a key on the main thread.
    std::cout << "Press any key to quit." << std::endl;
    getchar();


Instead of just waiting on a getchar(), I could make a whole menu with server relevant options for the user. Perhaps an option to list active connections or another to see how much data has been sent. Since the connection management is done on other threads, there is lots of freedom here.

For now, once the user has pressed a key on the server signifying they want to close the app, go ahead and close the listener socket (so we stop receiving new connections) and disconnect all the clients.


    // Close the listen socket to stop receiving new connections.
    closesocket(listenSocket);

    // Disconnect all clients.
    disconnectAllClients();


The listen and client threads have wait locks where they wait for either a new client connection or a new messages from one of their clients in a continuous loop. If their socket is closed, then they get kicked out of that wait lock and are able to cleanly exit their thread. We'll sleep here for 100 milliseconds to let all those threads cleanly exit, but honestly it's way more time than we need to delay.


    // Wait for socket threads to conclude. (Disconnecting above causes them to break out of their socket waits.)
    Sleep(100);


For Windows we'll call the Winsock cleanup, but that's it for our main!


    WSACleanup();

    return 0;
}



TCP Client Listener Thread


The tcpClientListener thread is spooled off from the main function; its duty is to listen for and establish new connections.

// Thread function listening for new clients.
void tcpClientListener(SOCKET* listenSocket)
{


The thread work is done in a while (1) as to allow any number of new clients to connect. As soon as a new connection is established, the loop repeats and listens for another client.


    // Listen for new connections until the listenSocket is closed.
    while (1)
    {


The listen() call will cause the thread to wait until either a new connection is requested or until the socket closes down. The downside is this operation must be done in its own thread (hence why we created the tcpClientListener thread). The upside is the operation does not waste CPU cycles waiting for a connection, and we don't need a hackish Sleep(1) to keep our CPU from spinning.


        // listen will wait until a connection is requested on the socket. If the socket is closed
        // while waiting, then listen will break from its wait and throw a socket error.
        int listenResult = listen(*listenSocket, SOMAXCONN);
        if (listenResult != SOCKET_ERROR)
        {


A new client is attempting to connect! accept() the connection.


            // Accept a client socket
            SOCKET clientSocket = accept(*listenSocket, NULL, NULL);
            if (clientSocket != INVALID_SOCKET)
            {


Now that we have the new client, we need a way to listen for data from that client. addClient() is described later on, but it essentially creates a ConnectionModel and returns a pointer to that data. For now the ConnectionModel just contains the socket the client is on.

A new tcpPacketListener thread is created with the clientConnection passed in as an argument so it knows what to listen from. Calling .detach() has the new thread execute independently so we can get back to work listening for new clients.


                // Create a connection to the client and begin a listener thread.
                ConnectionModel* clientConnection = addClient(clientSocket);
                std::thread{ tcpPacketListener, clientConnection }.detach();
            }
            else
            {


If we fell into here, we were unable to accept the connection so break out of the while (1) loop. This will cause us to stop receiving any new connections. You may not wait to break out here and instead let the thread continue to attempt to make other connections.


                std::cout << "TCP accept failed." << std::endl;
                break;
            }
        }
        else
        {


If we fell into here, we were unable to listen for a new connection so break out of the while (1) loop. This will cause us to stop receiving any new connections. This error is caused when the listenSocket is closed, and it is how we end this thread at shutdown.


            std::cout << "TCP listen failed." << std::endl;
            break;
        }
    }

    return;
}



TCP Packet Listener Thread


The tcpPacketListener thread is spooled off from the tcpClientListener thread; its duty is to listen for and packets from a single connection.

// Thread function for listening for packets from clients to relay.
void tcpPacketListener(ConnectionModel* clientConnection)
{
    char receiveBuffer[maximumPacketLength_bytes];


The thread work is done in a while (1) as to continuously receive new packets. As soon as a new packet is received, the loop repeats and listens for another packet.


    // Receive until the client connection is closed.
    while (1)
    {


The recv() call will cause the thread to wait until either a new packet arrives or until the socket closes down. The downside is this operation must be done in its own thread (hence why we created the tcpPacketListener thread). The upside is the operation does not waste CPU cycles waiting for a packet, and we don't need a hackish Sleep(1) to keep our CPU from spinning.


        // recv will wait until a packet is received on the socket. If the socket is closed
        // while waiting, then recv will break from its wait and throw a socket error.
        int packetLength_bytes = recv(clientConnection->clientSocket, receiveBuffer, maximumPacketLength_bytes, 0);
        if (packetLength_bytes > 0)
        {


New packet received! In this TCP/UDP gateway example, I'm relaying the packets to other clients that have connected to the server. You may want to do the same or something completely different. It's entirely possible your clients never needs packets relayed to each other and just communicate with the server.


            std::cout << "Bytes received: " << packetLength_bytes << std::endl;

            // Relay the packet back to the other clients.
            sendPacketToClients(clientConnection, receiveBuffer, packetLength_bytes);
        }
        else if (packetLength_bytes == 0)
        {


If the number of bytes received is 0, then the connection cleanly closed. We should break out of the while (1) loop which will remove the client and end the thread.


            std::cout << "TCP connection closed." << std::endl;
            break;
        }
        else
        {


If the number of bytes received is less than 0, then the connection abruptly closed. We should break out of the while (1) loop which will remove the client and end the thread.


            std::cout << "TCP socket receive error." << std::endl;
            break;
        }
    };


We are now outside the while (1) loop! Either the connection cleanly closed, the connection had an error, or the socket was closed (we are shutting down). In all cases, remove the client from our clients list and let the thread close out.


    // Client connection was closed, remove it from our connection list.
    removeClient(clientConnection);

    return;
}



Client Management Functions



Client connections are maintained in the clientConnections std::vector. The following functions just add and remove from that vector.

When a new connection is established, the clientSocket established in the tcpClientListener thread is passed to addClient. Here it creates a new ConnectionModel to store parameters of the client and then adds them to the clientConnections std::vector. A Mutex lock prevents the clientConnections from being modified while we're iterating through it elsewhere.

// Add a new client to the maintained connections.
ConnectionModel* addClient(SOCKET clientSocket)
{
    // Create a new connection for the socket.
    ConnectionModel* clientConnection = new ConnectionModel(clientSocket);

    clientConnectionsMutex.lock();

    // Add the client to the connections list.
    clientConnections.push_back(clientConnection);

    std::cout << "Client added!" << std::endl;
    std::cout << "Active Connections: " << clientConnections.size() << std::endl;

    clientConnectionsMutex.unlock();

    return clientConnection;
}


When a client connection is closed (or told to close), the closing clientConnection is passed to removeClient. Here it ensures the socket is closed and the client is removed from the clientConnections std::vector. A Mutex lock prevents the clientConnections form being modified while we're iterating through it elsewhere.


// Remove a client from the maintained connections.
bool removeClient(ConnectionModel* clientConnection)
{
    clientConnectionsMutex.lock();

    // Shutdown and close the socket.
    shutdown(clientConnection->clientSocket, SD_BOTH);
    closesocket(clientConnection->clientSocket);

    // Remove the client from our connection list.
    auto removedConnection = std::find(clientConnections.begin(), clientConnections.end(), clientConnection);
    if (removedConnection != clientConnections.end())
    {
        clientConnections.erase(removedConnection);
    }

    // Free the connection memory.
    delete clientConnection;

    std::cout << "Client removed!" << std::endl;
    std::cout << "Active Connections: " << clientConnections.size() << std::endl;

    clientConnectionsMutex.unlock();

    return true;
}


And finally as a cleanup function for when the application is closing, we have a disconnectAllClients. It iterates through our clientConnections std::vector to close every client sockets.

Each client has its own thread that listens for packets, and the threads waits on a recv() for new packets. By closing the socket, the recv() will stop waiting and throw an error. The thread knows when it receives a recv() error to break out of the loop, remove the clientConnection from our std::vector, and end the thread.


// Disconnect all clients. (This does not prevent new clients from connecting later.)
bool disconnectAllClients()
{
    clientConnectionsMutex.lock();

    // Loop through the connections and close each of their connections.
    std::vector<ConnectionModel*>::iterator clientConnectionIterator;
    for (clientConnectionIterator = clientConnections.begin(); clientConnectionIterator != clientConnections.end(); clientConnectionIterator++)
    {
        ConnectionModel* loopClientConnection = *clientConnectionIterator;

        // Closing the socket will cause the listener threads for those sockets to close.
        shutdown(loopClientConnection->clientSocket, SD_BOTH);
        closesocket(loopClientConnection->clientSocket);
    }

    clientConnectionsMutex.unlock();

    return true;
}



TCP Sending



The last function we have sends packets back to the clients. In this example, it is relaying a packet receive from one client (sourceConnection) to all other clients. Your use-case for a TCP server may be completely different, and your send function look nothing like this. However, it is a good example of how to loop through clientConnections and send data to them.

// Relay a packet to all clients except to the client that originated the packet (sourceConnection).
bool sendPacketToClients(ConnectionModel* sourceConnection, char packetData[maximumPacketLength_bytes], int packetLength_bytes)
{
    clientConnectionsMutex.lock();

    // Loop through the connections and send the packet to each.
    std::vector<ConnectionModel*>::iterator clientConnectionIterator;
    for (clientConnectionIterator = clientConnections.begin(); clientConnectionIterator != clientConnections.end(); clientConnectionIterator++)
    {
        ConnectionModel* loopClientConnection = *clientConnectionIterator;

        // Skip sending the packet if it's to the source.
        if (loopClientConnection->clientSocket == sourceConnection->clientSocket)
        {
            std::cout << "Skipped ID: " << loopClientConnection->clientSocket << std::endl;
            continue;
        }
        std::cout << "Sent to: " << loopClientConnection->clientSocket << std::endl;

        // Send the packet on the socket!
        int bytesSent = send(loopClientConnection->clientSocket, packetData, packetLength_bytes, 0);
        if (bytesSent == SOCKET_ERROR)
        {
            std::cout << "Send failed." << std::endl;
        }
    }

    clientConnectionsMutex.unlock();

    return true;
}




TCP Client



Again this example was written for Windows, but I'll eventually revisit to add Linux support. Overall the following will be helpful with how to connect and communicate with a TCP server. The UDP code is specific for my use-case (relaying UDP packets), so likely your own software won't have any UDP code. You could use the following as an example on how to do UDP communication and in that case you'll likely ignore the TCP stuff.

I'm using separate sockets for UDP sending and receiving. Depending on your network design, that may not be necessary. I'm using separate sockets so I can identify the source of the UDP messages so I don't receive & rebroadcast messages I just sent.

Headers


#define WIN32_LEAN_AND_MEAN

#include <process.h>
#include <windows.h>
#include <winsock2.h>
#include <ws2tcpip.h>
#include <stdio.h>
#include <map>
#include <string>
#include <iostream>
#include <sstream>
#include <time.h>
#include <thread>

#pragma comment (lib, "Ws2_32.lib")


Constants


// Constant denoting the maximum packet size that can be handled.
const int maximumPacketLength_bytes = 8192;


Prototypes


Again I don't have a separate header file for the client code. The function prototypes are just at the top.


// Function Prototypes
bool udpConnect(char udpLocalIpAddress[16], char udpSendIpAddress[16], int udpPort);
void udpListener();
bool udpDisconnect();
bool tcpConnect(char serverAddress[16], int tcpPort);
bool tcpDisconnect();
void tcpListener();
bool sendPacketToUdp(char packetData[maximumPacketLength_bytes], int packetLength_bytes);
bool sendPacketToTcp(char packetData[maximumPacketLength_bytes], int packetLength_bytes);
bool sendInitSelfPacket();


Global Variables


There are 3 global variables to keep track of the network sockets (one for TCP, one for receiving UDP, and one for sending UDP).


// UDP and TCP Sockets
SOCKET udpReceiveSocket; // Socket to receive UDP messages on.
SOCKET udpSendSocket; // Socket to send UDP messages on.
SOCKET tcpSocket; // Socket of the TCP connection to the server.


And there are 3 socket addresses (one for the address to listen for UDP, one for the address to send UDP, and one for our own UDP address).


// UDP Socket Addresses
SOCKADDR_IN udpSocketAddressIncoming; // Address for incoming packets.
SOCKADDR_IN udpSocketAddressBroadcast; // Address for broadcasting packets.
SOCKADDR_IN udpSocketAddressSelf; // Address to identify packets that originated from ourself.


To identify our sending UDP address, we'll be sending an initial packet with a random value. These 2 global variables are to keep track of if we've found our own address and the value to check in that first packet.


// Variables to keep track of initialize the self address.
int initSelfValue = 0; // Random value sent to identify our own address.
bool initSelfComplete = false; // Flag to know if we've identified our own address.


Main


On to the actual functions! The client takes in far more arguments than the server. Here we need the TCP IP Address, the TCP Port, the UDP Local Address (that we're binding to), the UDP Send Address, and the UDP Port.


// Main takes several command line arguments for the TCP server address and the local addresses for UDP traffic.
int main(int argc, char *argv[])
{
    // If insufficient arguments are given, show the format.
    if (argc < 6)
{
        std::cout << "Command line arguments:" << std::endl;
        std::cout << "<TCP Address> <TCP Port> <UDP Local Address> <UDP Send Address> <UDP Port>" << std::endl;
return 0;
}

    char serverAddress[16];
strcpy(serverAddress, argv[1]);

    int tcpPort = atoi(argv[2]);

    char udpLocalIpAddress[16];
strcpy(udpLocalIpAddress, argv[3]);

    char udpSendIpAddress[16];
strcpy(udpSendIpAddress, argv[4]);

    int udpPort = atoi(argv[5]);


Since we're on Windows, initialize Winsock.


 // Initialize Winsock.
 WSADATA wsd;
if (WSAStartup(MAKEWORD(2, 2), &wsd) != 0)
{
        std::cout << "WSAStartup failed." << std::endl;
return false;
}


Call udpConnect with the arguments passed in to bind our UDP socket. This will also start the UDP listener thread.


 // Connect to sending and receiving UDP sockets.
if (!udpConnect(udpLocalIpAddress, udpSendIpAddress, udpPort))
{
        std::cout << "UDP connect failed." << std::endl;
WSACleanup();
return 0;
}


Then call tcpConnect with the arguments passed in to connect to the remote TCP server. This will also start the TCP listener thread.


 // Connect to the TCP server.
if (!tcpConnect(serverAddress, tcpPort))
{
        std::cout << "TCP connect failed." << std::endl;
WSACleanup();
return 0;
}


Now that the network sockets are ready to go, we can send an self-identification packet. We send it to our UDP broadcast address where it will get received via the UDP listener thread. That way we can identify packets that originated from ourselves.


    // Send a single packet over UDP so we can receive it and know how to filter our own messages.
    sendInitSelfPacket();


Similar to the server, the real work of this application is done on the UDP and TCP listener threads that were spooled off in udpConnect and tcpConnect respectively. All we need to do on this main thread is have some kind of stop until the user wants to close the application.

To keep things simple, I'm just using a getchar() and when the user presses a key the app will close down.


    // All packet communication is done on other threads.
    // Continue the program until the user presses a key on the main thread.
    std::cout << "Press any key to quit." << std::endl;
getchar();


At this point, the user had signaled he or she wants the program to close. Disconnect both the UDP and TCP sockets. By doing this, we'll throw errors in our socket receive functions which will kick us out of our thread while loops which will allow those threads to cleanly close.


// Disconnect both the UDP and TCP sockets.
udpDisconnect();
tcpDisconnect();

    // Wait for threads to conclude. (Disconnecting above will cause them to break out of their socket waits.)
    Sleep(100);


On Windows, so cleanup Winsock on exit.


WSACleanup();

return 0;
}


UDP Socket


The udpConnect function binds a UDP socket based on the command line arguments given.


// Create sending and receiving UDP sockets, and create a thread to listen for UDP packets.
bool udpConnect(char udpLocalIpAddress[16], char udpSendIpAddress[16], int udpPort)
{
    std::cout << "Connecting to socket..." << std::endl;


Two sockets are created. The first is the socket that will receive packets.


    // Create the UDP receive socket.
    udpReceiveSocket = socket(AF_INET, SOCK_DGRAM, 0); // IPPROTO_UDP
    if (udpReceiveSocket == INVALID_SOCKET)
{
        std::cout << "receive socket failed." << std::endl;
return false;
}



The second is the socket that will send packets.


    // Create the UDP send socket.
    udpSendSocket = socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP);
    if (udpSendSocket == INVALID_SOCKET)
    {
        std::cout << "send socket failed." << std::endl;
        return false;
    }


We want to allow reuse for both sockets - basically allow other applications to bind to the same port to also listen for UDP messages. I've never encountered a situation where I didn't want to allow socket reuse. I have encountered other software that prevented reuse and I hated it. So as a general rule, please don't prevent other software from reading off the same port as you are!


    // Enable reuse for the UDP receive socket.
bool receiveSocketReuse = 1;
    if (setsockopt(udpReceiveSocket, SOL_SOCKET, SO_REUSEADDR, (char *)&receiveSocketReuse, sizeof(receiveSocketReuse)) == SOCKET_ERROR)
{
        std::cout << "receive setsockopt reuse failed." << std::endl;
return false;
}

    // Enable reuse for the UDP send socket.
    bool sendSocketReuse = 1;
    if (setsockopt(udpSendSocket, SOL_SOCKET, SO_REUSEADDR, (char *)&sendSocketReuse, sizeof(sendSocketReuse)) == SOCKET_ERROR)
    {
        std::cout << "send setsockopt reuse failed." << std::endl;
        return false;
    }


When receiving packets on a very congested network, I have seen messages inexplicably get dropped. Bumping up the receive buffer size solved the problem. I have not encountered any downsides to doing this.


    // Increase the receive buffer size. This can help when many packets are being transmitted simultaneously.
    int receiveBufferSize = 65535;
    if (setsockopt(udpReceiveSocket, SOL_SOCKET, SO_RCVBUF, (char *)&receiveBufferSize, sizeof(receiveBufferSize)) == SOCKET_ERROR)
    {
        std::cout << "receive setsockopt failed buffer size." << std::endl;
    }


Next configure the socket addresses.


    // Configure the UDP broadcast address.
udpSocketAddressBroadcast.sin_family = AF_INET;
udpSocketAddressBroadcast.sin_port = htons(udpPort);
udpSocketAddressBroadcast.sin_addr.S_un.S_addr = inet_addr(udpSendIpAddress);

    // Configure the UDP local address.
    SOCKADDR_IN udpSocketAddressLocal;
    udpSocketAddressLocal.sin_family = AF_INET;
udpSocketAddressLocal.sin_port = htons(udpPort);
udpSocketAddressLocal.sin_addr.S_un.S_addr = inet_addr(udpLocalIpAddress); // htonl(INADDR_ANY);


And bind the UDP receive socket to the local address.

The UDP local address could be set to INADDR_ANY as opposed to a specific IP. If the computer happened to have multiple network connections (multiple IP addresses) and it was bound to INADDR_ANY, then it could receive packets from any of those IP addresses with a single socket. There are pros and cons to doing this. Perhaps you want to monitor all traffic regardless of its network source. Perhaps instead you want to specifically only monitor traffic from a single network source. Both use cases are common and easy to configure just by changing udpSocketAddressLocal.sin_addr.S_un.S_addr between: inet_addr(udpLocalIpAddress) and htonl(INADDRY_ANY).


    // Bind the UDP receiving socket to the local address.
    if (bind(udpReceiveSocket, (SOCKADDR *)&udpSocketAddressLocal, sizeof(udpSocketAddressLocal)) == SOCKET_ERROR)
{
        std::cout << "bind() failed." << std::endl;
return false;
}


Next initialize a variable that will keep track of the source address of incoming messages and start the thread.


    // Prepare the UDP incoming address.
udpSocketAddressIncoming.sin_family = AF_INET;
udpSocketAddressIncoming.sin_port = htons(udpPort);

    // Start the UDP Listener thread.
    std::thread{ udpListener }.detach();

return true;
}


udpDisconnect just has to close the send and receive sockets. The UDP listener thread will automatically cleanly exit when the udpReceiveSocket is closed.


// Close the UDP sending and receiving sockets.
bool udpDisconnect()
{
// Close the socket.
 closesocket(udpReceiveSocket);
 closesocket(udpSendSocket);

return true;
}


UDP Listener Thread


The bottom of udpConnect creates a thread using the udpListener function to listen for incoming packets. The function remains in a continuous while (1) loop to process those incoming packets until the socket is closed.


// Thread function listening for UDP packets.
void udpListener()
{


Define a receiveBuffer that will hold the incoming data.


char receiveBuffer[maximumPacketLength_bytes];
int dwSenderSize = sizeof(udpSocketAddressIncoming);


Begin the loop!


    // Listen for packets until the UDP socket is closed.
    while (1)
    {


Similar to the TCP recv() function described in the server code above, the UDP recvfrom() will cause the thread to wait until either a new packet arrives or until the socket closes down. The downside is this operation must be done in its own thread (hence why we created the udpListener thread). The upside is the operation does not waste CPU cycles waiting for a packet, and we don't need a hackish Sleep(1) to keep our CPU from spinning.


        // recvfrom will wait until a packet is received on the socket. If the socket is closed
        // while waiting, then recvfrom will break from its wait and throw a socket error.
        int packetLength_bytes = recvfrom(udpReceiveSocket, &receiveBuffer[0], maximumPacketLength_bytes, 0, (SOCKADDR *)&udpSocketAddressIncoming, &dwSenderSize);


At this point, either recvfrom successfully received an incoming UDP packet or there was an error with the socket.

If there was an error with the socket, bail out of the thread now. This is also caused when the application is closing - we close the receiving socket which in turn kicks us out of the recvfrom when falls into this socket error code which allows this thread to exit nicely..


        if (packetLength_bytes < 0)
        {
// Socket error.
            std::cout << "UDP socket receive error." << std::endl;
            break;
        }


At startup, we send an initial UDP packet out to broadcast that loops back and gets received by our gateway. That is for identifying our sending address. If we haven't yet received that init packet, check to see if the incoming packet is the correct size (just 4 bytes for a single Int variable). If the incoming packet is 4 bytes and its value matches the random value we had sent in the packet, then this packet likely originated from us. Record the source address as our udpSocketAddressSelf and set the flag that initSelfComplete is done.


        else if (!initSelfComplete)
        {
            // We have not yet recieved the initialization packet, is this it?
            if (packetLength_bytes == 4)
            {
                // This packet is the correct size for the initialization, does its value match what we sent?
                int* packetValue = (int*)receiveBuffer;
                if (*packetValue == initSelfValue)
                {
                    // It does! This is our initialization packet; we now know our own broadcasting address.
                    // With udpSocketAddressSelf, we can ignore packets that we originated.
                    udpSocketAddressSelf = udpSocketAddressIncoming;
                    initSelfComplete = true;
                    std::cout << "Received initialization packet!" << std::endl;
                }
            }
        }
        else if (udpSocketAddressSelf.sin_port == udpSocketAddressIncoming.sin_port &&
                 udpSocketAddressSelf.sin_addr.S_un.S_addr == udpSocketAddressIncoming.sin_addr.S_un.S_addr &&
                 udpSocketAddressSelf.sin_family == udpSocketAddressIncoming.sin_family)
        {
            // We originated this packet, ignore it.
            std::cout << "Skipped." << std::endl;
        }


If we received a good packet and initSelfComplete has completed, then we can process the incoming packet! For this gateway example, I'm relaying the packets to the TCP server (so it can rebroadcast it out to all of its clients). For your use-case, you'll likely do something very different here.


else if (packetLength_bytes >= 0)
{
// Packet received, relay it to the TCP socket to be shared to other clients.
            std::cout << "UDP bytes received: " << packetLength_bytes << std::endl;
sendPacketToTcp(receiveBuffer, packetLength_bytes);
}

return;
}


TCP Socket


At startup we connect to the given TCP server.


// Connect to the TCP server.
bool tcpConnect(char serverAddress[16], int tcpPort)
{


Define an addrinfo with our hints for the TCP server.


    // Setup address info hints.
    struct addrinfo addressInfoHints;
    memset(&addressInfoHints, 0, sizeof(addressInfoHints));
    addressInfoHints.ai_family = AF_UNSPEC;
    addressInfoHints.ai_socktype = SOCK_STREAM;
    addressInfoHints.ai_protocol = IPPROTO_TCP;


With the addressInfoHints, resolve the server IP address and port from what was given in the command line arguments.


// Resolve the server address.
char portString[20];
sprintf(portString, "%d", tcpPort);
    struct addrinfo* addressInfoResult = NULL;
    int getaddrinfoReturn = getaddrinfo(serverAddress, portString, &addressInfoHints, &addressInfoResult);
    if (getaddrinfoReturn != 0)
{
        std::cout << "getaddrinfo failed." << std::endl;
return false;
}


Next up, create the TCP socket.


// Loop through potential server addresses to attempt connection.
    for (struct addrinfo *ptr = addressInfoResult; ptr != NULL; ptr = ptr->ai_next)
{
        // Create the TCP socket.
tcpSocket = socket(ptr->ai_family, ptr->ai_socktype, ptr->ai_protocol);
if (tcpSocket == INVALID_SOCKET)
{
            std::cout << "TCP socket failed." << std::endl;
return false;
}


And connect to the server!


// Connect to the server.
        int connectReturn = connect(tcpSocket, ptr->ai_addr, (int)ptr->ai_addrlen);
        if (connectReturn == SOCKET_ERROR)
{
closesocket(tcpSocket);
tcpSocket = INVALID_SOCKET;
continue;
}
break;
}

    freeaddrinfo(addressInfoResult);

if (tcpSocket == INVALID_SOCKET)
{
        std::cout << "Unable to connect to server!" << std::endl;
return false;
}


Now that our server connection is established, start a TCP listener thread that will wait for and process incoming TCP packets.


    // Start the TCP Listener thread.
    std::thread{ tcpListener }.detach();

return true;
}


tcpDisconnect just has to close the TCP socket. The TCP listener thread will automatically cleanly exit when the tcpReceiveSocket is closed.


// Close the TCP connection with the server.
bool tcpDisconnect()

    // Shutdown and close the socket.
    shutdown(tcpSocket, SD_BOTH);
closesocket(tcpSocket);

return true;
}


TCP Listener Thread


The bottom of tcpConnect creates a thread using the tcpListener function to listen for incoming packets. The function remains in a continuous while (1) loop  to process those incoming packets until the socket is closed.


// Thread function listening for TCP packets.
void tcpListener()
{
char receiveBuffer[maximumPacketLength_bytes];

    // Receive until the connection is closed.
    while (1)
{


The recv() call will cause the thread to wait until either a new packet arrives or until the socket closes down. The downside is this operation must be done in its own thread (hence why we created the tcpListener thread). The upside is the operation does not waste CPU cycles waiting for a packet, and we don't need a hackish Sleep(1) to keep our CPU from spinning.


        // recv will wait until a packet is received on the socket. If the socket is closed
        // while waiting, then recv will break from its wait and throw a socket error.
        int packetLength_bytes = recv(tcpSocket, receiveBuffer, maximumPacketLength_bytes, 0);


If we get a valid packet, relay it back to our local UDP network! You will likely do something different for your own use-case with the incoming packets.


if (packetLength_bytes > 0)
{
            std::cout << "TCP bytes received: " << packetLength_bytes << std::endl;

            // Relay the packet to the local UDP network.
            sendPacketToUdp(receiveBuffer, packetLength_bytes);
}


If the number of bytes is 0, then the connection cleanly closed. We should break out of the while (1) loop and end the thread.


else if (packetLength_bytes == 0)
{
            std::cout << "TCP connection closed." << std::endl;
            break;
}


If the number of bytes received is less than 0, then the connection abruptly closed (possibly by our shutdown process closing the thread). We should break out of the while (1) loop which will end the thread.


else
{
            std::cout << "TCP socket receive error." << std::endl;
            break;
} // receiveSize_bytes
}

return;
}


Packet Sending


Sending packets for both TCP and UDP is very straightforward. Both functions just take in an array of the packet data and the number of bytes from that array to send.

The TCP version of sending...


// Send the packet received from the UDP address to the TCP server.
bool sendPacketToTcp(char packetData[maximumPacketLength_bytes], int packetLength_bytes)
{
    // Send the packet.
int sendResult = send(tcpSocket, packetData, packetLength_bytes, 0);
if (sendResult == packetLength_bytes)
{
        std::cout << "TCP bytes sent: " << packetLength_bytes << std::endl;
return true;
}
else if (sendResult < 0)
{
        std::cout << "TCP send failed at socket level." << std::endl;
return false;
}
else
{
        std::cout << "TCP failed to send entire message." << std::endl;
return false;
}
}


The UDP version of sending...


// Send the packet received from the TCP server to the UDP address.
bool sendPacketToUdp(char packetData[maximumPacketLength_bytes], int packetLength_bytes)
{
    // Send the packet.
    int sendResult = sendto(udpSendSocket, packetData, packetLength_bytes, 0, (const struct sockaddr *)&udpSocketAddressBroadcast, sizeof(struct sockaddr));
if (sendResult == packetLength_bytes)
{
        std::cout << "UDP bytes sent: " << packetLength_bytes << std::endl;
return true;
}
else if (sendResult < 0)
{
        std::cout << "UDP send failed at socket level." << std::endl;
return false;
}
else
{
        std::cout << "UDP failed to send entire message." << std::endl;
return false;
}
}


Above I mentioned that we send an init packet at startup that we can use to self-identify our own UDP sending address. The following function takes a random integer and sends it as the init packet. This is basically an example of how to send a single integer over the network. When I move this into the orbital aero code, I'll have examples of how to send entire structures over a network.


// Send an initilization packet to identify our own sending address.
bool sendInitSelfPacket()
{
    std::cout << "Sending initialization packet to find own address." << std::endl;

    // Pick a random value to be put in the packet that we'll compare against upon receipt.
    srand((unsigned int)time(NULL));
    initSelfValue = rand();

    // Create a packet and stuff the 4 byte random value into it.
    char packetData[maximumPacketLength_bytes];
    memcpy(packetData, &initSelfValue, 4);

    // Send the packet.
    return sendPacketToUdp(packetData, 4);
}






The code within this post is released into the public domain. You're free to use it however you wish, but it is provided "as-is" without warranty of any kind. In no event shall the author be liable for any claims or damages in connection with this software.