Udp on ZRJ | 学习笔记

修改网卡缓冲以提高吞吐量

Fri, 28 Feb 2014 16:36:55 +0800

在单机并发上万的时候，发现 sendto 函数比较容易出现返回错误 EAGAIN，对应的错误消息为 Resource temporarily unavailable，这种情况下认为是应用层写入过快，导致网卡缓冲满了

使用这个命令来查看缓冲区现有大小

# ethtool -g eth1
Ring parameters for eth1:
Pre-set maximums:
RX:		4096
RX Mini:	0
RX Jumbo:	0
TX:		4096
Current hardware settings:
RX:		256
RX Mini:	0
RX Jumbo:	0
TX:		256

使用这个命令来修改缓冲区大小

# ethtool -G eth1 rx 2048
# ethtool -G eth1 tx 2048

修改之后再查看一下

# ethtool -g eth1
Ring parameters for eth1:
Pre-set maximums:
RX:		4096
RX Mini:	0
RX Jumbo:	0
TX:		4096
Current hardware settings:
RX:		2048
RX Mini:	0
RX Jumbo:	0
TX:		2048

但是发现这样修改之后，没有起作用，sendto 函数的失败率还是很高，继续看，发现 socket 族函数本身有一个 setsockopt 的函数，在代码里面可以设置，写法如下

int iOriginalSendBufferSize = 0;
socklen_t iIntLen = sizeof(int);
getsockopt(iSocket, SOL_SOCKET, SO_SNDBUF, &iOriginalSendBufferSize, &iIntLen);
LOG_INFO("before set, send buffer is %d", iOriginalSendBufferSize);

int nSendBuf=32*1024;
setsockopt(iSocket,SOL_SOCKET,SO_SNDBUF,(const  char*)&nSendBuf,sizeof(int));  

int iFinalSendBufferSize = 0;
getsockopt(iSocket, SOL_SOCKET, SO_SNDBUF, &iFinalSendBufferSize, &iIntLen);
LOG_INFO("after set, send buffer is %d", iFinalSendBufferSize);

这里可能会觉得多此一举，设完还要看，有啥好看的，但是，看输出

before set, send buffer is 8388608
after set, send buffer is 65536

神奇吧，设的明明是 32k，出来居然 64k 了，买一送一啊这是

查资料看到这里，http://blog.csdn.net/c359719435/article/details/8815499

Author：阿冬哥

Created：2013-4-17

Blog：http://blog.csdn.net/c359719435/

Copyright 2013 阿冬哥 http://blog.csdn.net/c359719435/

使用以及转载请注明出处

1 设置socket tcp缓冲区大小的疑惑疑惑1：通过setsockopt设置SO_SNDBUF、SO_RCVBUF这连个默认缓冲区的值，再用getsockopt获取设置的值，发现返回值是设置值的两倍。为什么？通过网上查找，看到linux的内核代码/usr/src/linux-2.6.13.2/net/core/sock.c，找到sock_setsockopt这个函数的这段代码：

case SO_SNDBUF: /* Don’t error on this BSD doesn’t and if you think about it this is right. Otherwise apps have to play ‘guess the biggest size’ games. RCVBUF/SNDBUF are treated in BSD as hints */ if (val > sysctl_wmem_max)//val是我们想设置的缓冲区大小的值 val = sysctl_wmem_max;//大于最大值，则val值设置成最大值 sk->sk_userlocks |= SOCK_SNDBUF_LOCK; if ((val * 2) < SOCK_MIN_SNDBUF)//val的两倍小于最小值，则设置成最小值 sk->sk_sndbuf = SOCK_MIN_SNDBUF; else sk->sk_sndbuf = val * 2;//val的两倍大于最小值，则设置成val值的两倍 /* * Wake up sending tasks if we * upped the value. */ sk->sk_write_space(sk); break; case SO_RCVBUF: /* Don’t error on this BSD doesn’t and if you think about it this is right. Otherwise apps have to play ‘guess the biggest size’ games. RCVBUF/SNDBUF are treated in BSD as hints */ if (val > sysctl_rmem_max) val = sysctl_rmem_max; sk->sk_userlocks |= SOCK_RCVBUF_LOCK; /* FIXME: is this lower bound the right one? */ if ((val * 2) < SOCK_MIN_RCVBUF) sk->sk_rcvbuf = SOCK_MIN_RCVBUF; else sk->sk_rcvbuf = val * 2; break; 从上述代码可以看出：（1）当设置的值val > 最大值sysctl_wmem_max，则设置为最大值的2倍：2*sysctl_wmem_max；（2）当设置的值的两倍val*2 > 最小值，则设置成最小值：SOCK_MIN_SNDBUF；（3）当设置的值val < 最大值sysctl_wmem_max，且 val*2 > SOCK_MIN_SNDBUF，则设置成2*val。

查看linux 手册： SO_RCVBUF： Sets or gets the maximum socket receive buffer in bytes. The kernel doubles this value (to allow space for bookkeeping overhead) when it is set using setsockopt(2), and this doubled value is returned by getsockopt(2). The default value is set by the /proc/sys/net/core/rmem_default file, and the maximum allowed value is set by the /proc/sys/net/core/rmem_max file. The minimum (doubled) value for this option is 256. 查看我的主机Linux 2.6.6 ：/proc/sys/net/core/rmem_max： 4194304 //4M 查看/proc/sys/net/core/wmem_max： 8388608 //8M 所以，能设置的接收缓冲区的最大值是8M，发送缓冲区的最大值是16M。

疑惑2：为什么要有2倍这样的一个内核设置呢？我的理解是，用户在设置这个值的时候，可能只考虑到数据的大小，没有考虑数据封包的字节开销。所以将这个值设置成两倍。

注：overhead，在计算机网络的帧结构中，除了有用数据以外，还有很多控制信息，这些控制信息用来保证通信的完成。这些控制信息被称作系统开销。 2 tcp缓冲区大小的默认值建立一个socket，通过getsockopt获取缓冲区的值如下：发送缓冲区大小：SNDBufSize = 16384 接收缓冲区大小：RCVBufSize = 87380

疑惑3：linux手册中，接收缓冲区的默认值保存在/proc/sys/net/core/rmem_default，发送缓冲区保存在/proc/sys/net/core/wmem_default。 [root@cfs_netstorage core]# cat /proc/sys/net/core/rmem_default 1048576 [root@cfs_netstorage core]# cat /proc/sys/net/core/wmem_default 512488

可知，接收缓冲区的默认值是：1048576，1M。发送缓冲区的默认值是：512488，512K。为什么建立一个socket时得到的默认值是87380、16384？？？

进一步查阅资料发现， linux下socket缓冲区大小的默认值在/proc虚拟文件系统中有配置。分别在一下两个文件中： /proc/sys/net/ipv4/tcp_wmem [root@cfs_netstorage core]# cat /proc/sys/net/ipv4/tcp_wmem 4096 16384 131072 //第一个表示最小值，第二个表示默认值，第三个表示最大值。 /proc/sys/net/ipv4/tcp_rmem [root@cfs_netstorage core]# cat /proc/sys/net/ipv4/tcp_rmem 4096 87380 174760

由此可见，新建socket，选取的默认值都是从这两个文件中读取的。可以通过更改这两个文件中的值进行调优，但是最可靠的方法还是在程序中调用setsockopt进行设置。通过setsockopt的设置，能设置的接收缓冲区的最大值是8M，发送缓冲区的最大值是16M（Linux 2.6.6中）。

原来还真的是买一送一

于是在这个地方，把 SNDBUF 这个选项改成 8m 或者更大，sendto 的错误就没有了

-——————–

2014-2-28 17:31:44 update 原来对于 tcp 和 udp，这个缓冲区默认大小还不不一样

#include <stdio.h>
#include <assert.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <string.h>
#include <unistd.h>
int tcpSrv(short iListenPort)
{
    int iListenFd = socket(AF_INET, SOCK_STREAM, 0);
    assert(iListenFd > 0);
    
    int iOriginalSendBufferSize = 0;
    socklen_t iIntLen = sizeof(iOriginalSendBufferSize);
    getsockopt(iListenFd, SOL_SOCKET, SO_SNDBUF, &iOriginalSendBufferSize, &iIntLen);
    printf("default send buffer size %d\n", iOriginalSendBufferSize);
    
    struct sockaddr_in stServerAddr;
    stServerAddr.sin_family = AF_INET;
    stServerAddr.sin_addr.s_addr = htonl(INADDR_ANY);
    stServerAddr.sin_port = htons(iListenPort);
    assert(bind(iListenFd, (struct sockaddr*)&stServerAddr, sizeof(stServerAddr)) == 0);
    assert(listen(iListenFd, 1024) == 0);
    printf("listening...\n");
    while (1) {
        int iAcceptFd = accept(iListenFd, (struct sockaddr*)NULL, NULL);
        assert(iAcceptFd > 0);
        char* pStrResponse = "HTTP/1.1 200 OK\r\nConnection: close\r\n\r\n";
        int iResponseLen = strlen(pStrResponse);
        int iSendByte = send(iAcceptFd, pStrResponse, iResponseLen, 0);
        assert(iSendByte == iResponseLen);
        close(iAcceptFd);
    }
    close(iListenFd);
    return 0;
}


int udpSrv(short iListenPort)
{
    int iListenFd = socket(AF_INET, SOCK_DGRAM, 0);
    assert(iListenFd > 0);
    
    int iOriginalSendBufferSize = 0;
    socklen_t iIntLen = sizeof(iOriginalSendBufferSize);
    getsockopt(iListenFd, SOL_SOCKET, SO_SNDBUF, &iOriginalSendBufferSize, &iIntLen);
    printf("default send buffer size %d\n", iOriginalSendBufferSize);
    
    struct sockaddr_in stServerAddr;
    stServerAddr.sin_family = AF_INET;
    stServerAddr.sin_addr.s_addr = htonl(INADDR_ANY);
    stServerAddr.sin_port = htons(iListenPort);
    assert(bind(iListenFd, (struct sockaddr*)&stServerAddr, sizeof(stServerAddr)) == 0);
    printf("listening...\n");
    while (1) {
        char buffer[BUFSIZ] = {0};
        struct sockaddr_in stClientAddr;
        socklen_t stClientAddrLen = sizeof(stClientAddr);
        int iReceivedLen = recvfrom(iListenFd, buffer, sizeof(buffer), 0, (struct sockaddr*)&stClientAddr, &stClientAddrLen);
        int iSendByte = sendto(iListenFd, &buffer, iReceivedLen, 0, (struct sockaddr*)&stClientAddr, stClientAddrLen);
    }
    close(iListenFd);
    return 0;
}


int main()
{
    udpSrv(8000);
    return 0;
}

tcp 的

default send buffer size 16384
listening...

udp 的

default send buffer size 8388608
listening...

历史评论

wanax (2014-03-19 16:43:48):

PDU与SDU的考虑吧，猜的…

ZRJ (2014-03-19 20:12:50):

我能说我看了你回复才知道这两个名词吗。。。

基于队列的 UDP 多线程收发 demo

Tue, 10 Dec 2013 15:46:01 +0800

来看这个代码

#include <stdio.h>
#include <assert.h>
#include <sys/socket.h>
#include <sys/types.h>
#include <netinet/in.h>
#include <fcntl.h>
#include <string.h>
#include <sys/epoll.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <queue>
#include <pthread.h>

#define _PRINTF_(format, ...) printf("[%s:%20s@%20s,%04d] " format "\n", "DEBUG",__FUNCTION__, __FILE__, __LINE__, ##__VA_ARGS__ );

static const int iBufferSize = 2048;
int iUdpServerFd;
static const unsigned short uiUdpServerPort = 6000;
static const int iUdpProcessThreadNumber = 2;

struct InQueueEvent
{
    struct sockaddr_in stClientAddr;
    socklen_t stClienAddrLen;
    char szRecvBuffer[iBufferSize];
    int iRecvLen;
};

struct OutQueueEvent
{
    struct sockaddr_in stDestAddr;
    socklen_t stDestAddrLen;
    char szSendBuffer[iBufferSize];
    int iSendContentLen;
};

std::queue<struct InQueueEvent> stInQueue;
pthread_mutex_t stInQueueMutex;
std::queue<struct OutQueueEvent> stOutQueue;
pthread_mutex_t stOutQueueMutex;

void* UdpProcessThread(void*)
{
    _PRINTF_("udp process thread %ld start", pthread_self());
    while (1)
    {
        if (stInQueue.empty())
        {
            continue;
        }
        pthread_mutex_lock(&stInQueueMutex);
        struct InQueueEvent stInQueueEvent = stInQueue.front();
        stInQueue.pop();
        pthread_mutex_unlock(&stInQueueMutex);
        _PRINTF_("thread %ld processing the request %s", pthread_self(), stInQueueEvent.szRecvBuffer);
        struct OutQueueEvent stOutQueueEvent;
        stOutQueueEvent.stDestAddr = stInQueueEvent.stClientAddr;
        stOutQueueEvent.stDestAddrLen = stInQueueEvent.stClienAddrLen;
        memcpy(stOutQueueEvent.szSendBuffer, stInQueueEvent.szRecvBuffer, stInQueueEvent.iRecvLen);
        stOutQueueEvent.iSendContentLen = stInQueueEvent.iRecvLen;
        pthread_mutex_lock(&stOutQueueMutex);
        stOutQueue.push(stOutQueueEvent);
        pthread_mutex_unlock(&stOutQueueMutex);        
    }
}

void* UdpRecvThread(void*)
{
    _PRINTF_("udp recv thread start");
    while(1)
    {
        struct InQueueEvent stInQueueEvent;
        memset(&stInQueueEvent, 0, sizeof(stInQueueEvent));
        stInQueueEvent.stClienAddrLen = sizeof(stInQueueEvent.stClientAddr);
        stInQueueEvent.iRecvLen = recvfrom(iUdpServerFd, &(stInQueueEvent.szRecvBuffer), sizeof(stInQueueEvent.szRecvBuffer), 0, (struct sockaddr*)&(stInQueueEvent.stClientAddr), &(stInQueueEvent.stClienAddrLen));
        _PRINTF_("recv %d bytes from ip %s port %d message %s", stInQueueEvent.iRecvLen, inet_ntoa(stInQueueEvent.stClientAddr.sin_addr), ntohs(stInQueueEvent.stClientAddr.sin_port), stInQueueEvent.szRecvBuffer);
        pthread_mutex_lock(&stInQueueMutex);
        stInQueue.push(stInQueueEvent);
        pthread_mutex_unlock(&stInQueueMutex);
    }
}

void* UdpSendThread(void*)
{
    _PRINTF_("udp send thread start");
    while(1)
    {
        if (stOutQueue.empty())
        {
            continue;
        }
        pthread_mutex_lock(&stOutQueueMutex);
        struct OutQueueEvent stOutQueueEvent = stOutQueue.front();
        stOutQueue.pop();
        pthread_mutex_unlock(&stOutQueueMutex);
        int iSendLen = sendto(iUdpServerFd, &(stOutQueueEvent.szSendBuffer), stOutQueueEvent.iSendContentLen, 0, (struct sockaddr*)&(stOutQueueEvent.stDestAddr), stOutQueueEvent.stDestAddrLen);
        _PRINTF_("send %d bytes to ip %s port %d message %s", iSendLen, inet_ntoa(stOutQueueEvent.stDestAddr.sin_addr), ntohs(stOutQueueEvent.stDestAddr.sin_port), stOutQueueEvent.szSendBuffer);
    }
}

int CreateUdpServer(unsigned short uiUdpServerPort)
{
    int iListenFd = socket(AF_INET, SOCK_DGRAM, 0);
    assert(iListenFd > 0);
    struct sockaddr_in stListenAddr;
    memset(&stListenAddr, 0, sizeof(stListenAddr));
    stListenAddr.sin_family = AF_INET;
    stListenAddr.sin_addr.s_addr = htonl(INADDR_ANY);
    stListenAddr.sin_port = htons(uiUdpServerPort);
    int iReuseFlag = 1;
    assert(setsockopt(iListenFd, SOL_SOCKET, SO_REUSEADDR, &iReuseFlag, sizeof(int)) == 0);
    assert(bind(iListenFd, (struct sockaddr*)&stListenAddr, sizeof(stListenAddr)) == 0);  
    return iListenFd;
}

int main()
{
    pthread_mutex_init(&stInQueueMutex, NULL);
    pthread_mutex_init(&stOutQueueMutex, NULL);
    iUdpServerFd = CreateUdpServer(uiUdpServerPort);
    pthread_t tid;
    assert(pthread_create(&tid, NULL, UdpRecvThread, NULL) == 0);
    assert(pthread_create(&tid, NULL, UdpSendThread, NULL) == 0);
    for (int i = 0; i < iUdpProcessThreadNumber; i++)
    {
        assert(pthread_create(&tid, NULL, UdpProcessThread, NULL) == 0);
    }    
    while (1)
    {
        sleep(10);
    }
    return 0;
}

用 python 写个客户端测试一下

#!/usr/bin/env python

import socket

HOST = 'xx.xx.xx.xx'
PORT = 6000
BUFSIZ = 1024
ADDR = (HOST, PORT)

udpCliSock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)

while True:
    data = raw_input('> ')
    if not data:
        break
    udpCliSock.sendto(data, ADDR)
    data, ADDR = udpCliSock.recvfrom(BUFSIZ)
    if not data:
        break
    print ADDR, data

udpCliSock.close()

跑起来似乎没有问题

[DEBUG:       UdpRecvThread@        EpollUdp.cpp,0069] udp recv thread start
[DEBUG:       UdpSendThread@        EpollUdp.cpp,0085] udp send thread start
[DEBUG:    UdpProcessThread@        EpollUdp.cpp,0044] udp process thread 1107310912 start
[DEBUG:    UdpProcessThread@        EpollUdp.cpp,0044] udp process thread 1098918208 start
[DEBUG:       UdpRecvThread@        EpollUdp.cpp,0076] recv 1 bytes from ip 10.31.18.40 port 62272 message 1
[DEBUG:    UdpProcessThread@        EpollUdp.cpp,0055] thread 1107310912 processing the request 1
[DEBUG:       UdpSendThread@        EpollUdp.cpp,0097] send 1 bytes to ip 10.31.18.40 port 62272 message 1
[DEBUG:       UdpRecvThread@        EpollUdp.cpp,0076] recv 1 bytes from ip 10.31.18.40 port 62272 message 2
[DEBUG:    UdpProcessThread@        EpollUdp.cpp,0055] thread 1107310912 processing the request 2
[DEBUG:       UdpSendThread@        EpollUdp.cpp,0097] send 1 bytes to ip 10.31.18.40 port 62272 message 2

但是多发几次，就 core 了，这是为什么呢，gdb 调试发现 core 在 memcpy 上面，这个也不太能理解，不过，把代码尝试的改成这样就可以了

#include <stdio.h>
#include <assert.h>
#include <sys/socket.h>
#include <sys/types.h>
#include <netinet/in.h>
#include <fcntl.h>
#include <string.h>
#include <sys/epoll.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <queue>
#include <pthread.h>

#define _PRINTF_(format, ...) printf("[%s:%20s@%20s,%04d] " format "\n", "DEBUG",__FUNCTION__, __FILE__, __LINE__, ##__VA_ARGS__ );

static const int iBufferSize = 2048;
int iUdpServerFd;
static const unsigned short uiUdpServerPort = 6000;
static const int iUdpProcessThreadNumber = 2;

struct InQueueEvent
{
    struct sockaddr_in stClientAddr;
    socklen_t stClienAddrLen;
    char szRecvBuffer[iBufferSize];
    int iRecvLen;
};

struct OutQueueEvent
{
    struct sockaddr_in stDestAddr;
    socklen_t stDestAddrLen;
    char szSendBuffer[iBufferSize];
    int iSendContentLen;
};

std::queue<struct InQueueEvent> stInQueue;
pthread_mutex_t stInQueueMutex;
std::queue<struct OutQueueEvent> stOutQueue;
pthread_mutex_t stOutQueueMutex;

void* UdpProcessThread(void*)
{
    _PRINTF_("udp process thread %ld start", pthread_self());
    while (1)
    {
        pthread_mutex_lock(&stInQueueMutex);
        if (stInQueue.empty())
        {
            pthread_mutex_unlock(&stInQueueMutex);
            continue;
        }
        struct InQueueEvent stInQueueEvent = stInQueue.front();
        stInQueue.pop();
        pthread_mutex_unlock(&stInQueueMutex);
        _PRINTF_("thread %ld processing the request %s", pthread_self(), stInQueueEvent.szRecvBuffer);
        struct OutQueueEvent stOutQueueEvent;
        memset(&stOutQueueEvent, 0, sizeof(stOutQueueEvent));
        stOutQueueEvent.stDestAddr = stInQueueEvent.stClientAddr;
        stOutQueueEvent.stDestAddrLen = stInQueueEvent.stClienAddrLen;
        memcpy(stOutQueueEvent.szSendBuffer, stInQueueEvent.szRecvBuffer, stInQueueEvent.iRecvLen);
        stOutQueueEvent.iSendContentLen = stInQueueEvent.iRecvLen;
        pthread_mutex_lock(&stOutQueueMutex);
        stOutQueue.push(stOutQueueEvent);
        pthread_mutex_unlock(&stOutQueueMutex);        
    }
}

void* UdpRecvThread(void*)
{
    _PRINTF_("udp recv thread start");
    while(1)
    {
        struct InQueueEvent stInQueueEvent;
        memset(&stInQueueEvent, 0, sizeof(stInQueueEvent));
        stInQueueEvent.stClienAddrLen = sizeof(stInQueueEvent.stClientAddr);
        stInQueueEvent.iRecvLen = recvfrom(iUdpServerFd, &(stInQueueEvent.szRecvBuffer), sizeof(stInQueueEvent.szRecvBuffer), 0, (struct sockaddr*)&(stInQueueEvent.stClientAddr), &(stInQueueEvent.stClienAddrLen));
        _PRINTF_("recv %d bytes from ip %s port %d message %s", stInQueueEvent.iRecvLen, inet_ntoa(stInQueueEvent.stClientAddr.sin_addr), ntohs(stInQueueEvent.stClientAddr.sin_port), stInQueueEvent.szRecvBuffer);
        pthread_mutex_lock(&stInQueueMutex);
        stInQueue.push(stInQueueEvent);
        pthread_mutex_unlock(&stInQueueMutex);
    }
}

void* UdpSendThread(void*)
{
    _PRINTF_("udp send thread start");
    while(1)
    {
        pthread_mutex_lock(&stOutQueueMutex);
        if (stOutQueue.empty())
        {
            pthread_mutex_unlock(&stOutQueueMutex);
            continue;
        }
        struct OutQueueEvent stOutQueueEvent = stOutQueue.front();
        stOutQueue.pop();
        pthread_mutex_unlock(&stOutQueueMutex);
        int iSendLen = sendto(iUdpServerFd, &(stOutQueueEvent.szSendBuffer), stOutQueueEvent.iSendContentLen, 0, (struct sockaddr*)&(stOutQueueEvent.stDestAddr), stOutQueueEvent.stDestAddrLen);
        _PRINTF_("send %d bytes to ip %s port %d message %s", iSendLen, inet_ntoa(stOutQueueEvent.stDestAddr.sin_addr), ntohs(stOutQueueEvent.stDestAddr.sin_port), stOutQueueEvent.szSendBuffer);
    }
}

int CreateUdpServer(unsigned short uiUdpServerPort)
{
    int iListenFd = socket(AF_INET, SOCK_DGRAM, 0);
    assert(iListenFd > 0);
    struct sockaddr_in stListenAddr;
    memset(&stListenAddr, 0, sizeof(stListenAddr));
    stListenAddr.sin_family = AF_INET;
    stListenAddr.sin_addr.s_addr = htonl(INADDR_ANY);
    stListenAddr.sin_port = htons(uiUdpServerPort);
    int iReuseFlag = 1;
    assert(setsockopt(iListenFd, SOL_SOCKET, SO_REUSEADDR, &iReuseFlag, sizeof(int)) == 0);
    assert(bind(iListenFd, (struct sockaddr*)&stListenAddr, sizeof(stListenAddr)) == 0);  
    return iListenFd;
}

int main()
{
    pthread_mutex_init(&stInQueueMutex, NULL);
    pthread_mutex_init(&stOutQueueMutex, NULL);
    iUdpServerFd = CreateUdpServer(uiUdpServerPort);
    pthread_t tid;
    assert(pthread_create(&tid, NULL, UdpRecvThread, NULL) == 0);
    assert(pthread_create(&tid, NULL, UdpSendThread, NULL) == 0);
    for (int i = 0; i < iUdpProcessThreadNumber; i++)
    {
        assert(pthread_create(&tid, NULL, UdpProcessThread, NULL) == 0);
    }    
    while (1)
    {
        sleep(10);
    }
    return 0;
}

注意高亮的部分，在检查队列是否为空的时候，也要包含在锁的区域里面，这样就可以了

-———————————————

2013-12-10 16:37:06 update 使用这个脚本，可以多线程压测服务

#!/usr/bin/env python

import socket
import random
import threading

HOST = 'xx.xx.xx.xx'
PORT = 6000
BUFSIZ = 1024
ADDR = (HOST, PORT)

class UdpTestThread(threading.Thread):
    def run(self):
        udpCliSock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
        for i in range(1000):
            data = str(random.random())
            udpCliSock.sendto(data, ADDR)
            data = udpCliSock.recvfrom(BUFSIZ)
            #print data
        udpCliSock.close()

for i in range(50):
    UdpTestThread().start()

-—————————

2013-12-10 17:21:05 update 发现上面的例子，压测结果很差，于是写了个极简的，作为比对

#include <sys/socket.h>
#include <arpa/inet.h>

int main() {
    int fd = socket(AF_INET, SOCK_DGRAM, 0);
    struct sockaddr_in addr;
    addr.sin_family = AF_INET;
    addr.sin_port = htons(6000);
    addr.sin_addr.s_addr = htonl(INADDR_ANY);
    bind(fd, (struct sockaddr*)&addr, sizeof(addr));
    while (1) {
        char buffer[2048] = {0};
        struct sockaddr cli;
        socklen_t cli_len = sizeof(cli);
        int recv_len = recvfrom(fd, buffer, sizeof(buffer), 0, &cli, &cli_len);
        sendto(fd, buffer, recv_len, 0, &cli, cli_len);
    }
    return 0;
}

发现差不多，看来还要找原因

历史评论

wanax (2014-03-19 16:52:29):

这几天挖出来个GLib的，自带线程池和宏定义好了的锁，感觉再用C就顺手多了

ZRJ (2014-03-19 20:15:41):

其实用 libevent 基本就够了，不用造轮子

wanax (2014-03-20 09:49:23):

那玩意儿就是个轮子还附带各种数据结构与常用工具类

ZRJ (2014-03-20 19:58:08):

反正我现在是觉得能有稳定开源的就用呗

UDP 包的边界

Mon, 22 Apr 2013 21:04:40 +0800

首先明确一个问题，如果定义了一个数据结构，大小是，比方说 32 个字节，然后 UDP 客户端连续向服务端发了两个包。现在假设这两个包都已经到达了服务器，那么服务端调用 recvfrom 来接收数据，并且缓冲区开得远大于 64，例如，开了 1024 个字节，那么，服务端的 recvfrom 函数是会一次收到两个数据包呢，还是只能收到一个。

答案是只能收到一个。

来看代码：

struct.h

#ifndef STRUCT_H
#define STRUCT_H

typedef struct _UDP_MSG {
    int add1;
    int add2;
    int sum;
    char str1[16];
    char str2[16];
    char cat[32];
} UDP_MSG;

#endif /* STRUCT_H */

服务器的代码：

#include <sys/types.h>
#include <sys/socket.h>
#include <string.h>
#include <unistd.h>
#include <netinet/in.h>
#include <stdio.h>
#include <stdlib.h>

#include "struct.h"

#define MAX_LINE 1024
#define SERV_PORT 8080

int udp_serv();

int main() {
    return udp_serv();
}

int udp_serv() {
    int sockfd = socket(AF_INET, SOCK_DGRAM, 0);
    if (sockfd == -1) {
        perror("socket");
        return -1;
    }
    
    struct sockaddr_in serv_addr;
    memset(&serv_addr, 0, sizeof(serv_addr));
    serv_addr.sin_family = AF_INET;
    serv_addr.sin_addr.s_addr = htonl(INADDR_ANY);
    serv_addr.sin_port = htons(SERV_PORT);

    if (bind(sockfd, (struct sockaddr*)&serv_addr, sizeof(serv_addr)) == -1) {
        perror("bind");
        return -1;
    }

    while (1) {
        printf("sleeping\n");
        sleep(10);
        printf("akwaked\n");
        char buf[BUFSIZ];
        struct sockaddr_in cli_addr;
        memset(&cli_addr, 0, sizeof(cli_addr));
        socklen_t cli_addr_len = sizeof(cli_addr);
        int recvn = recvfrom(sockfd, buf, sizeof(buf), 0, (struct sockaddr*)&cli_addr, &cli_addr_len);
        printf("recv %d bytes\n", recvn);
        UDP_MSG msg;
        memset(&msg, 0, sizeof(msg));
        printf("UDP_MSG size is %d\n", sizeof(msg));
        memcpy(&msg, buf, sizeof(msg));
        msg.sum = msg.add1 + msg.add2;
        strcpy(msg.cat, msg.str1);
        strcat(msg.cat, msg.str2);
        printf("msg.add1 is: %d\n", msg.add1);
        printf("msg.add2 is: %d\n", msg.add2);
        printf("msg.sum is: %d\n", msg.sum);
        printf("msg.str1 is: %s\n", msg.str1);
        printf("msg.str2 is: %s\n", msg.str2);
        printf("msg.cat is: %s\n", msg.cat);
        sendto(sockfd, &msg, sizeof(msg), 0, (struct sockaddr*)&cli_addr, cli_addr_len);
    }

    return 0;
}

客户端的：

#include <sys/types.h>
#include <sys/socket.h>
#include <string.h>
#include <netinet/in.h>
#include <stdio.h>
#include <stdlib.h>
#include <arpa/inet.h>
#include <unistd.h>

#include "struct.h"

#define MAX_LINE 1024
#define SERV_PORT 8080

int udp_cli(const char* serv_ip);

int main(int argc, char* argv[]) {
    if (argc < 2) {
        printf("Usage: %s serv_ip\n", argv[0]);
        return 1;
    }
    return udp_cli(argv[1]);
}

int udp_cli(const char* serv_ip) {
    int sockfd = socket(AF_INET, SOCK_DGRAM, 0);
    if (sockfd == -1) {
        perror("socket");
        return -1;
    }
    struct sockaddr_in serv_addr;
    memset(&serv_addr, 0, sizeof(serv_addr));
    serv_addr.sin_family = AF_INET;
    serv_addr.sin_port = htons(SERV_PORT);
    if (inet_pton(AF_INET, serv_ip, &serv_addr.sin_addr) <= 0) {
        perror("inet_pton");
        return -1;
    }
    while (1) {
        UDP_MSG msg;
        memset(&msg, 0, sizeof(msg));
        msg.add1 = 1;
        msg.add2 = 2;
        scanf("%s%s", msg.str1, msg.str2);
        if (sendto(sockfd, &msg, sizeof(msg), 0,
            (struct sockaddr*)&serv_addr, sizeof(serv_addr)) == -1) {
            perror("write");
            return -1;
        } else {
            printf("send success\n");
        }
        if (sendto(sockfd, &msg, sizeof(msg), 0,
            (struct sockaddr*)&serv_addr, sizeof(serv_addr)) == -1) {
            perror("write2");
            return -1;
        } else {
            printf("send2 success\n");
        }
        socklen_t serv_addr_len = sizeof(serv_addr);
        if (recvfrom(sockfd, &msg, sizeof(msg), 0,
            (struct sockaddr*)&serv_addr, &serv_addr_len) == -1) {
            perror("read");
            return -1;
        }
        printf("msg.add1 is: %d\n", msg.add1);
        printf("msg.add2 is: %d\n", msg.add2);
        printf("msg.sum is: %d\n", msg.sum);
        printf("msg.str1 is: %s\n", msg.str1);
        printf("msg.str2 is: %s\n", msg.str2);
        printf("msg.cat is: %s\n", msg.cat);
        break;
    }
    return 0;
}

运行起来后，服务端的输出如下：

sleeping
akwaked
recv 76 bytes
UDP_MSG size is 76
msg.add1 is: 1
msg.add2 is: 2
msg.sum is: 3
msg.str1 is: 123
msg.str2 is: 456
msg.cat is: 123456
sleeping
akwaked
recv 76 bytes
UDP_MSG size is 76
msg.add1 is: 1
msg.add2 is: 2
msg.sum is: 3
msg.str1 is: 123
msg.str2 is: 456
msg.cat is: 123456
sleeping

客户端如下：

123
456
send success
send2 success
msg.add1 is: 1
msg.add2 is: 2
msg.sum is: 3
msg.str1 is: 123
msg.str2 is: 456
msg.cat is: 123456

这里涉及到一个边界的问题。 TCP 是流式的数据传输，消息没有边界，需要应用层自己去定义消息边界，而 UDP 是数据报传输，所以协议保证了一次只能接收一个数据报。

详细的解释看到这里，http://hi.baidu.com/chongerfeia/item/c38f91e075d742226dabb8a4

2010-08-11 14:56 有关TCP和UDP 粘包消息保护边界在socket网络程序中，TCP和UDP分别是面向连接和非面向连接的。因此TCP的socket编程，收发两端（客户端和服务器端）都要有一一成对的socket，因此，发送端为了将多个发往接收端的包，更有效的发到对方，使用了优化方法（Nagle算法），将多次间隔较小且数据量小的数据，合并成一个大的数据块，然后进行封包。这样，接收端，就难于分辨出来了，必须提供科学的拆包机制。对于UDP，不会使用块的合并优化算法，这样，实际上目前认为，是由于UDP支持的是一对多的模式，所以接收端的skbuff(套接字缓冲区）采用了链式结构来记录每一个到达的UDP包，在每个UDP包中就有了消息头（消息来源地址，端口等信息），这样，对于接收端来说，就容易进行区分处理了

保护消息边界和流那么什么是保护消息边界和流呢?

保护消息边界，就是指传输协议把数据当作一条独立的消息在网上传输,接收端只能接收独立的消息.也就是说存在保护消息边界,接收端一次只能接收发送端发出的一个数据包. 而面向流则是指无保护消息保护边界的,如果发送端连续发送数据, 接收端有可能在一次接收动作中,会接收两个或者更多的数据包.

我们举个例子来说,例如,我们连续发送三个数据包,大小分别是2k, 4k , 8k,这三个数据包,都已经到达了接收端的网络堆栈中,如果使用UDP协议,不管我们使用多大的接收缓冲区去接收数据,我们必须有三次接收动作,才能够把所有的数据包接收完.而使用TCP协议,我们只要把接收的缓冲区大小设置在14k以上,我们就能够一次把所有的数据包接收下来.只需要有一次接收动作.

这就是因为UDP协议的保护消息边界使得每一个消息都是独立的.而流传输,却把数据当作一串数据流,他不认为数据是一个一个的消息.

所以有很多人在使用tcp协议通讯的时候,并不清楚tcp是基于流的传输,当连续发送数据的时候,他们时常会认识tcp会丢包.其实不然, 因为当他们使用的缓冲区足够大时,他们有可能会一次接收到两个甚至更多的数据包,而很多人往往会忽视这一点,只解析检查了第一个数据包,而已经接收的其他数据包却被忽略了.所以大家如果要作这类的网络编程的时候,必须要注意这一点.

结论：根据以上所说，可以这样理解，TCP为了保证可靠传输，尽量减少额外开销（每次发包都要验证），因此采用了流式传输，面向流的传输，相对于面向消息的传输，可以减少发送包的数量。从而减少了额外开销。但是，对于数据传输频繁的程序来讲，使用TCP可能会容易粘包。当然，对接收端的程序来讲，如果机器负荷很重，也会在接收缓冲里粘包。这样，就需要接收端额外拆包，增加了工作量。因此，这个特别适合的是数据要求可靠传输，但是不需要太频繁传输的场合（两次操作间隔100ms，具体是由TCP等待发送间隔决定的，取决于内核中的socket的写法）

而UDP，由于面向的是消息传输，它把所有接收到的消息都挂接到缓冲区的接受队列中，因此，它对于数据的提取分离就更加方便，但是，它没有粘包机制，因此，当发送数据量较小的时候，就会发生数据包有效载荷较小的情况，也会增加多次发送的系统发送开销（系统调用，写硬件等）和接收开销。因此，应该最好设置一个比较合适的数据包的包长，来进行UDP数据的发送。（UDP最大载荷为1472，因此最好能每次传输接近这个数的数据量，这特别适合于视频，音频等大块数据的发送，同时，通过减少握手来保证流媒体的实时性）

历史评论

wanax (2013-09-23 13:40:08):

哈哈特么这是unix网络编程的课后题啊

ZRJ (2013-09-23 14:10:10):

老子不是科班出身的。。

wanax (2013-09-23 15:34:31):

没这也不是教材别紧张

UDP Socket - itlisten (2017-09-13 23:16:57):

[…] UDP包的边界 […]

Linux TCP UDP 混合

Mon, 08 Apr 2013 16:07:14 +0800

基本的思路是这样的：在服务器端，有两个机器，一个对外开启 TCP 监听，然后把监听到的请求内容送到后面的另外一台，或者多台轮询机器上，内网之间使用 UDP，然后等待业务逻辑机器处理完成，这个地方可以做成异步的，然后再返回到用户。

即是这样： client <-TCP-> server_front <-UDP-> server_back

下面来看代码，首先是 server_front 的 demo

#include <stdio.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <string.h>
#include <unistd.h>
#include <sys/types.h>
#include <arpa/inet.h>

#include "struct.h"

int main() {
    int tcpfd = socket(AF_INET, SOCK_STREAM, 0);
    if (tcpfd == -1) {
        perror("tcpfd socket");
        return -1;
    }
    struct sockaddr_in serv_addr;
    memset(&serv_addr, 0, sizeof(serv_addr));
    serv_addr.sin_family = AF_INET;
    serv_addr.sin_port = htons(38080);
    serv_addr.sin_addr.s_addr = htonl(INADDR_ANY);
    if (bind(tcpfd, (struct sockaddr*)&serv_addr, sizeof(serv_addr)) == -1) {
        perror("tcpfd bind");
        return -1;
    }
    if (listen(tcpfd, 10) == -1) {
        perror("tcpfd listen");
        return -1;
    }
    
    // udp
    int udpfd = socket(AF_INET, SOCK_DGRAM, 0);
    if (udpfd == -1) {
        perror("udpfd socket");
        return -1;
    }
    struct sockaddr_in udp_serv_addr;
    memset(&udp_serv_addr, 0, sizeof(udp_serv_addr));
    udp_serv_addr.sin_family = AF_INET;
    udp_serv_addr.sin_port = htons(8080);
    if (inet_pton(AF_INET, "10.1.164.18", &udp_serv_addr.sin_addr) <= 0) {
        perror("udpfd inet pton");
        return -1;
    }
    
    while (1) {
        int tcp_conn_fd = accept(tcpfd, NULL, NULL);
        if (tcp_conn_fd == -1) {
            perror("tcpfd accept");
            return -1;
        }
        char buff[2048];
        memset(buff, 0, sizeof(buff));
        int recvn = recv(tcp_conn_fd, buff, sizeof(buff), 0);
        if (recvn == -1) {
            perror("tcpfd recv");
            return -1;
        }
        printf("recv: [%s]\n", buff);
        //char html_resp[] = "HTTP/1.1 200 OK\r\n\r\ntcp udp mix";
        UDP_MSG udp_msg;
        memset(&udp_msg, 0, sizeof(udp_msg));
        strcpy(udp_msg.str1, "HTTP/1.1 200 OK\r\n\r\n");
        strcpy(udp_msg.str2, "send to udp for cat");
        if (sendto(udpfd, &udp_msg, sizeof(udp_msg), 0, (struct sockaddr*)&udp_serv_addr, sizeof(udp_serv_addr)) == -1) {
            perror("udp send to");
            return -1;
        }
        if (recvfrom(udpfd, &udp_msg, sizeof(udp_msg), 0, NULL, NULL) == -1) {
            perror("udp recv from");
            return -1;
        }
        if (send(tcp_conn_fd, udp_msg.cat, strlen(udp_msg.cat), 0) == -1) {
            perror("tcp send");
            return -1;
        }
        close(tcp_conn_fd);
    }
    close(tcpfd);
    return 0;
}

然后是内部的报文格式 struct.h

#ifndef STRUCT_H
#define STRUCT_H

typedef struct _UDP_MSG {
    int add1;
    int add2;
    int sum;
    char str1[128];
    char str2[128];
    char cat[256];
} UDP_MSG;

#endif /* STRUCT_H */

然后是后端的业务逻辑服务器

#include <sys/types.h>
#include <sys/socket.h>
#include <string.h>
#include <netinet/in.h>
#include <stdio.h>
#include <stdlib.h>

#include "struct.h"

#define MAX_LINE 1024
#define SERV_PORT 8080

int udp_serv();

int main() {
    return udp_serv();
}

int udp_serv() {
    int sockfd = socket(AF_INET, SOCK_DGRAM, 0);
    if (sockfd == -1) {
        perror("socket");
        return -1;
    }
    
    struct sockaddr_in serv_addr;
    memset(&serv_addr, 0, sizeof(serv_addr));
    serv_addr.sin_family = AF_INET;
    serv_addr.sin_addr.s_addr = htonl(INADDR_ANY);
    serv_addr.sin_port = htons(SERV_PORT);

    if (bind(sockfd, (struct sockaddr*)&serv_addr, sizeof(serv_addr)) == -1) {
        perror("bind");
        return -1;
    }

    while (1) {
        struct sockaddr_in cli_addr;
        memset(&cli_addr, 0, sizeof(cli_addr));
        UDP_MSG msg;
        memset(&msg, 0, sizeof(msg));
        socklen_t cli_addr_len = sizeof(cli_addr);
        recvfrom(sockfd, &msg, sizeof(msg), 0, (struct sockaddr*)&cli_addr, &cli_addr_len);
        msg.sum = msg.add1 + msg.add2;
        strcpy(msg.cat, msg.str1);
        strcat(msg.cat, msg.str2);
        printf("msg.add1 is: %d\n", msg.add1);
        printf("msg.add2 is: %d\n", msg.add2);
        printf("msg.sum is: %d\n", msg.sum);
        printf("msg.str1 is: %s\n", msg.str1);
        printf("msg.str2 is: %s\n", msg.str2);
        printf("msg.cat is: %s\n", msg.cat);
        sendto(sockfd, &msg, sizeof(msg), 0, (struct sockaddr*)&cli_addr, cli_addr_len);
    }

    return 0;
}

编译一下，就可以测试了

$ wget http://localhost:38080/123 && nl 123
--16:05:03--  http://localhost:38080/123
           => `123.2'
Resolving localhost... 127.0.0.1
Connecting to localhost|127.0.0.1|:38080... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified

    [<=> ] 19            --.--K/s             

16:05:03 (3.62 MB/s) - `123.2' saved [19]

     1	send to udp for cat

期间，server_front 的输出应该是这样的

recv: [GET /123 HTTP/1.0
User-Agent: Wget/1.10.2
Accept: */*
Host: localhost:38080
Connection: Keep-Alive

]

后台业务逻辑服务器的输出应该是这样的

msg.add1 is: 0
msg.add2 is: 0
msg.sum is: 0
msg.str1 is: HTTP/1.1 200 OK


msg.str2 is: send to udp for cat
msg.cat is: HTTP/1.1 200 OK

send to udp for cat

Linux UDP 收发实例

Sun, 07 Apr 2013 18:33:05 +0800

首先我们需要一个 struct.h

#ifndef STRUCT_H
#define STRUCT_H

typedef struct _UDP_MSG {
    int add1;
    int add2;
    int sum;
    char str1[128];
    char str2[128];
    char cat[256];
} UDP_MSG;

#endif /* STRUCT_H */

然后是服务端的

#include <sys/types.h>
#include <sys/socket.h>
#include <string.h>
#include <netinet/in.h>
#include <stdio.h>
#include <stdlib.h>

#include "struct.h"

#define MAX_LINE 1024
#define SERV_PORT 8080

int udp_serv();

int main() {
    return udp_serv();
}

int udp_serv() {
    int sockfd = socket(AF_INET, SOCK_DGRAM, 0);
    if (sockfd == -1) {
        perror("socket");
        return -1;
    }
    
    struct sockaddr_in serv_addr;
    memset(&serv_addr, 0, sizeof(serv_addr));
    serv_addr.sin_family = AF_INET;
    serv_addr.sin_addr.s_addr = htonl(INADDR_ANY);
    serv_addr.sin_port = htons(SERV_PORT);

    if (bind(sockfd, (struct sockaddr*)&serv_addr, sizeof(serv_addr)) == -1) {
        perror("bind");
        return -1;
    }

    while (1) {
        struct sockaddr_in cli_addr;
        memset(&cli_addr, 0, sizeof(cli_addr));
        UDP_MSG msg;
        memset(&msg, 0, sizeof(msg));
        socklen_t cli_addr_len = sizeof(cli_addr);
        recvfrom(sockfd, &msg, sizeof(msg), 0, (struct sockaddr*)&cli_addr, &cli_addr_len);
        msg.sum = msg.add1 + msg.add2;
        strcpy(msg.cat, msg.str1);
        strcat(msg.cat, msg.str2);
        printf("msg.add1 is: %d\n", msg.add1);
        printf("msg.add2 is: %d\n", msg.add2);
        printf("msg.sum is: %d\n", msg.sum);
        printf("msg.str1 is: %s\n", msg.str1);
        printf("msg.str2 is: %s\n", msg.str2);
        printf("msg.cat is: %s\n", msg.cat);
        sendto(sockfd, &msg, sizeof(msg), 0, (struct sockaddr*)&cli_addr, cli_addr_len);
    }

    return 0;
}

然后是客户端的

#include <sys/types.h>
#include <sys/socket.h>
#include <string.h>
#include <netinet/in.h>
#include <stdio.h>
#include <stdlib.h>
#include <arpa/inet.h>
#include <unistd.h>

#include "struct.h"

#define MAX_LINE 1024
#define SERV_PORT 8080

int udp_cli(const char* serv_ip);

int main(int argc, char* argv[]) {
    if (argc < 2) {
        printf("Usage: %s serv_ip\n", argv[0]);
        return 1;
    }
    return udp_cli(argv[1]);
}

int udp_cli(const char* serv_ip) {
    int sockfd = socket(AF_INET, SOCK_DGRAM, 0);
    if (sockfd == -1) {
        perror("socket");
        return -1;
    }
    struct sockaddr_in serv_addr;
    memset(&serv_addr, 0, sizeof(serv_addr));
    serv_addr.sin_family = AF_INET;
    serv_addr.sin_port = htons(SERV_PORT);
    if (inet_pton(AF_INET, serv_ip, &serv_addr.sin_addr) <= 0) {
        perror("inet_pton");
        return -1;
    }
    while (1) {
        UDP_MSG msg;
        memset(&msg, 0, sizeof(msg));
        msg.add1 = 1;
        msg.add2 = 2;
        sprintf(msg.str1, "hello ");
        sprintf(msg.str2, "world");
        if (sendto(sockfd, &msg, sizeof(msg), 0,
            (struct sockaddr*)&serv_addr, sizeof(serv_addr)) == -1) {
            perror("write");
            return -1;
        }
        socklen_t serv_addr_len = sizeof(serv_addr);
        if (recvfrom(sockfd, &msg, sizeof(msg), 0,
            (struct sockaddr*)&serv_addr, &serv_addr_len) == -1) {
            perror("read");
            return -1;
        }
        printf("msg.add1 is: %d\n", msg.add1);
        printf("msg.add2 is: %d\n", msg.add2);
        printf("msg.sum is: %d\n", msg.sum);
        printf("msg.str1 is: %s\n", msg.str1);
        printf("msg.str2 is: %s\n", msg.str2);
        printf("msg.cat is: %s\n", msg.cat);
        break;
    }
    return 0;
}

客户端另外一种，像这样的写法也行。。

#include <sys/types.h>
#include <sys/socket.h>
#include <string.h>
#include <netinet/in.h>
#include <stdio.h>
#include <stdlib.h>
#include <arpa/inet.h>
#include <unistd.h>

#include "struct.h"

#define MAX_LINE 1024
#define SERV_PORT 8080

int udp_cli(const char* serv_ip);

int main(int argc, char* argv[]) {
    if (argc < 2) {
        printf("Usage: %s serv_ip\n", argv[0]);
        return 1;
    }
    return udp_cli(argv[1]);
}

int udp_cli(const char* serv_ip) {
    int sockfd = socket(AF_INET, SOCK_DGRAM, 0);
    if (sockfd == -1) {
        perror("socket");
        return -1;
    }
    struct sockaddr_in serv_addr;
    memset(&serv_addr, 0, sizeof(serv_addr));
    serv_addr.sin_family = AF_INET;
    serv_addr.sin_port = htons(SERV_PORT);
    if (inet_pton(AF_INET, serv_ip, &serv_addr.sin_addr) <= 0) {
        perror("inet_pton");
        return -1;
    }
    if (connect(sockfd, (struct sockaddr*)&serv_addr, sizeof(serv_addr)) == -1) {
        perror("connect");
        return -1;
    }
    while (1) {
        UDP_MSG msg;
        memset(&msg, 0, sizeof(msg));
        msg.add1 = 1;
        msg.add2 = 2;
        sprintf(msg.str1, "hello ");
        sprintf(msg.str2, "world");
        if (write(sockfd, &msg, sizeof(msg)) == -1) {
            perror("write");
            return -1;
        }
        if (read(sockfd, &msg, sizeof(msg)) == -1) {
            perror("read");
            return -1;
        }
        printf("msg.add1 is: %d\n", msg.add1);
        printf("msg.add2 is: %d\n", msg.add2);
        printf("msg.sum is: %d\n", msg.sum);
        printf("msg.str1 is: %s\n", msg.str1);
        printf("msg.str2 is: %s\n", msg.str2);
        printf("msg.cat is: %s\n", msg.cat);
        break;
    }
    return 0;
}

然后编译

g++ -Wall -g serv.cpp -o serv.exe
g++ -Wall -g cli.cpp -o cli.exe

就可以运行了

历史评论

ZRJ (2013-04-07 18:33:34):

补充来源：http://www.chinaitpower.com/A200508/2005-08-02/182672.html

PHP 发 UDP 包时出错

Fri, 24 Aug 2012 23:08:53 +0800

今天金泉给了一个问题，说线上的腾讯 SDK PHP 版本的跑起来报错，看了一下，调用的代码类似这样

<?php
require("inc/OpenApiV3.php");

// 应用基本信息
$appid = xxxxxx;
$appkey = 'xxxxxxx';

// OpenAPI的服务器IP
// 最新的API服务器地址请参考wiki文档: http://wiki.open.qq.com/wiki/API3.0%E6%96%87%E6%A1%A3
$server_name = '119.147.19.43';

// 用户的OpenID/OpenKey
$openid = 'xxxxxxx';
$openkey = 'xxxxxxxx';

// 所要访问的平台, pf的其他取值参考wiki文档: http://wiki.open.qq.com/wiki/API3.0%E6%96%87%E6%A1%A3
$pf = 'qzone';


$sdk = new OpenApiV3($appid, $appkey);
$sdk->setServerName($server_name);

$ret = get_user_info($sdk, $openid, $openkey, $pf);
print_r("===========================n");
print_r($ret);

/**
 * 获取好友资料
 *
 * @param object $sdk OpenApiV3 Object
 * @param string $openid openid
 * @param string $openkey openkey
 * @param string $pf 平台
 * @return array 好友资料数组
 */
function get_user_info(&$sdk, $openid, $openkey, $pf)
{
	$params = array(
		'openid' => $openid,
		'openkey' => $openkey,
		'pf' => $pf,
	);

	$script_name = '/v3/user/get_info';

	return $sdk->api($script_name, $params);
}

// end of script

跑起来之后，报错是

Warning: socket_sendto() [function.socket-sendto]: unable to write to socket [1]: Operation not permitted in /home/naniwco1/public_html/draw/inc/lib/SnsStat.php on line 40

找到那个 SnsStat 文件，是一个用于统计的类，代码是这样的

<?php

/**
 * 统计上报接口调用情况
 *
 * @version 3.0.2
 * @author open.qq.com
 * @copyright © 2011, Tencent Corporation. All rights reserved.
 * @ History:
 *               3.0.2 | sparkeli | 2012-03-06 15:33:04 | initialize statistic fuction which can report API's access time and number to background server

 */

class SnsStat
{
	/**
	 * 执行一个 统计上报
	 *
	 * @param string $stat_url 统计上报的URL
	 * @param float $start_time 统计开始时间
	 * @param array $params 统计参数数组
	 * @return
	 */
	static public function statReport($stat_url, $start_time, $params)
	{
		$end_time = self::getTime();
		$params['time'] = round($end_time - $start_time, 4);
		$params['timestamp'] = time();
		$params['collect_point'] = 'sdk-php-v3';
		$stat_str = json_encode($params);
		//发送上报信息
		$host_ip = gethostbyname($stat_url);
		if ($host_ip != $stat_url)
		{
			$sock = socket_create(AF_INET, SOCK_DGRAM, 0);
			if (false === $sock)
			{
				return;
			}
			socket_sendto($sock, $stat_str, strlen($stat_str), 0, $host_ip, 19888);
			socket_close($sock);
		}
	}

	static public function getTime()
	{
		list($usec, $sec) = explode(" ", microtime());
		return ((float)$usec + (float)$sec);
	}
}

// end of script

把代码迁移到本地环境跑，在已经开了 socket PHP 扩展的前提下，是很顺利的，没有报错，于是考虑环境问题，google 一阵搜，看到这里，http://www.bluehostforum.com/showthread.php?23649-PHP-socket_send()-Not-permitted

I took a quick look. Your script is attempting to connect out on port 27025 which is blocked unless it has been whitelisted to be allowed through the firewall. Contact support and they’ll be happy to open the port for you.

On an unrelated note, there is NEVER a reason to provide world-writable permissions to a file or directory on Bluehost’s servers. All script execution is done by your user which means that a file need only have user permissions (7xx for execution and 6xx for writing). World-writable permissions simply open your account to compromise.

说是防火墙问题，有一定的可能，但是好像没有办法验证，于是只能注释掉了。。

历史评论

home business risks (2013-04-25 06:18:55):

Hi, yeah this post is truly nice and I have learned lot of things from it regarding blogging. thanks.

overdrive effects pedal reviews (2014-09-25 13:27:51):

Hello to all, since I am genuinely eager of reading this website’s post to be updated regularly. It consists of nice stuff.