多线程 on ZRJ | 学习笔记

c++11 thread

Tue, 24 Apr 2018 10:01:32 +0800

关于 c++11 的线程库，我们有如下几个问题：

可否不传参数
可否传递多个参数
可否传递 const 参数
可否传递引用参数
可否传递 const 引用参数
std::thread 结构可否放入容器
std::thread 可否在类内部使用

针对以上问题，我们先来看一个标准的 c++11 线程库要怎么使用：

#include <stdio.h>
#include <thread>

void f(int a) {
    printf("%d\n", a);
}

int main() {
    printf("hello world\n");
    std::thread t(f, 1);
    t.join();
    return 0;
}

>g++ test_thread.cpp -std=c++11 -pthread && ./a.out
hello world
1

然后试了一下不传参数，是可以的；多个参数，也是可以的；const 参数也是可以的

当试到传引用的时候，需要这么写才可以

#include <stdio.h>
#include <thread>
#include <string>

void f(int a, std::string& b) {
    printf("%d\n%s\n", a, b.c_str());
    b = "456";
}

int main() {
    printf("hello world\n");
    std::string str = "123";
    std::thread t(f, 1, std::ref(str));
    t.join();
    printf("%s\n", str.c_str());
    return 0;
}

>g++ test_thread.cpp -std=c++11 -pthread && ./a.out
hello world
1
123
456

然后 const 引用也是可以的

容器中也可以放，但是取出的时候只能取出引用，而不能取出值

#include <stdio.h>
#include <thread>
#include <string>
#include <vector>

void f(int a, const std::string& b) {
    printf("%d\n%s\n", a, b.c_str());
}

int main() {
    printf("hello world\n");
    std::string str = "123";
    std::vector<std::thread> vecT;
    vecT.push_back(std::thread(f, 1, std::ref(str)));
    std::thread& t = vecT[0];
    t.join();
    printf("%s\n", str.c_str());
    return 0;
}

>g++ test_thread.cpp -std=c++11 -pthread && ./a.out
hello world
1
123
123

最后，在类中使用的话，像这么写的话

#include <stdio.h>
#include <thread>
#include <string>
#include <vector>

class TestClass {
public:
    void f(int a, const std::string& b) {
        printf("%d\n%s\n", a, b.c_str());
    }

    void r() {
        printf("hello world\n");
        std::string str = "123";
        std::vector<std::thread> vecT;
        vecT.push_back(std::thread(f, 1, std::ref(str)));
        std::thread& t = vecT[0];
        t.join();
        printf("%s\n", str.c_str());
    }
};

int main() {
    TestClass testClass;
    testClass.r();
    return 0;
}

是会报错的

>g++ test_thread.cpp -std=c++11 -pthread && ./a.out
test_thread.cpp: In member function ‘void TestClass::r()’:
test_thread.cpp:16:55: error: no matching function for call to ‘std::thread::thread(<unresolved overloaded function type>, int, std::reference_wrapper<std::basic_string<char> >)’
         vecT.push_back(std::thread(f, 1, std::ref(str)));
                                                       ^
test_thread.cpp:16:55: note: candidates are:
In file included from test_thread.cpp:2:0:
/usr/local/include/c++/4.8.2/thread:133:7: note: std::thread::thread(_Callable&&, _Args&& ...) [with _Callable = void (TestClass::*)(int, const std::basic_string<char>&); _Args = {int, std::reference_wrapper<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >}]
       thread(_Callable&& __f, _Args&&... __args)
       ^
/usr/local/include/c++/4.8.2/thread:133:7: note:   no known conversion for argument 1 from ‘<unresolved overloaded function type>’ to ‘void (TestClass::*&&)(int, const std::basic_string<char>&)’
/usr/local/include/c++/4.8.2/thread:128:5: note: std::thread::thread(std::thread&&)
     thread(thread&& __t) noexcept
     ^
/usr/local/include/c++/4.8.2/thread:128:5: note:   candidate expects 1 argument, 3 provided
/usr/local/include/c++/4.8.2/thread:122:5: note: std::thread::thread()
     thread() noexcept = default;
     ^
/usr/local/include/c++/4.8.2/thread:122:5: note:   candidate expects 0 arguments, 3 provided

搜了一下，看到 stackoverflow 上有人说， https://stackoverflow.com/questions/13888453/stdthread-unresolved-overloaded-function-type-error?utm_medium=organic&utm_source=google_rich_qa&utm_campaign=google_rich_qa

The problem is that a member function can’t be called without an object. Provide a pointer to this so that the current object is used:

thread t(&ThreadClass::myThread, this, 10); You could use an instance of any ThreadClass object, but in your case, it seems this is the right thing to do.

NOTE: Remember you need a reference to the created thread so that you can do a join() later on.

所以其实是类的成员函数中，隐含了第一个参数是一个 this 指针的原因

#include <stdio.h>
#include <thread>
#include <string>
#include <vector>

class TestClass {
public:
    void f(int a, const std::string& b) {
        printf("%d\n%s\n", a, b.c_str());
    }

    void r() {
        printf("hello world\n");
        std::string str = "123";
        std::vector<std::thread> vecT;
        vecT.push_back(std::thread(&TestClass::f, this, 1, std::ref(str)));
        std::thread& t = vecT[0];
        t.join();
        printf("%s\n", str.c_str());
    }
};

int main() {
    TestClass testClass;
    testClass.r();
    return 0;
}

改写成这样，就可以了

>g++ test_thread.cpp -std=c++11 -pthread && ./a.out
hello world
1
123
123

那很自然的，会接着一个问题，就是这个 f 能否是一个私有函数呢

试了一下，会发现，答案是可以的

单例，多线程的一些验证

Wed, 07 Jun 2017 19:34:18 +0800

作为一个基础知识，java 下的一种单例实现是这样的

package me.zrj.test.test20170607;

public class Singleton {
    private Singleton() {}
    
    private static Singleton instance;
    
    public static Singleton getInstance() {
        if (instance == null) {
            instance = new Singleton();
        }
        return instance;
    }
}

这种单例的写法的明显问题就是线程安全，说都知道这么说，那么怎么来复现这个线程安全问题呢

一个最直接的写法就是这样

package me.zrj.test.test20170607;

public class MyThread implements Runnable {
    
    @Override
    public void run() {        
        System.out.println(Singleton.getInstance());
    }
    
}

package me.zrj.test.test20170607;

public class App 
{
    public static void main( String[] args ) throws InterruptedException
    {
        System.out.println( "Hello World!" );        
        
        new Thread(new MyThread()).start();
        new Thread(new MyThread()).start();        

    }
    
}

但是这个复现的验证代码本身也是能看出来是有问题的，就是 new Thread 的先后 start 并不能保证他们在“同一时间”开始跑，而如果他们先后开始跑的时刻相差比较大，那么就无法复现那个单例的实现的 bug 了

但是。。。实际上验证的时候发现，这种写法几乎每次都能复现单例实现的 bug，由此可见，相比于 new 一个对象的耗时来说，new 一个线程的耗时还是更短的

那如果我们把这个验证的代码写成这样子呢

package me.zrj.test.test20170607;

public class App 
{
    public static void main( String[] args ) throws InterruptedException
    {
        System.out.println( "Hello World!" );        
        
        new Thread(new MyThread()).start();
        Thread.sleep(100);
        new Thread(new MyThread()).start();        

    }
    
}

这样的话，就可以看到两次拿回来的单例都是指向同一个地址了

那么，如果在这种情况下，我们还想能够复现那个单例实现的 bug，应该怎么来做呢，或者说，怎么来实现一个 countdownlatch 呢

按照理解去写，可以写成这样

package me.zrj.test.test20170607;

public class MyThread implements Runnable {
    
    synchronized void print() {
        try {
            wait();
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        System.out.println(Thread.currentThread().getName() + "\t" + Singleton.getInstance());
    }
    
    synchronized void go() {
        notifyAll();
    }
    
    @Override
    public void run() {       
        print();        
    }
    
}

package me.zrj.test.test20170607;

import java.util.Date;

public class App 
{
    public static void main( String[] args ) throws InterruptedException
    {
        System.out.println( "Hello World!" );        
        
        MyThread myThread = new MyThread();
        
        new Thread(myThread).start();
        Thread.sleep(100);
        new Thread(myThread).start();

        myThread.go();
        
        while (true) {
            Thread.sleep(1000);
            System.out.println(new Date());
        }
    }
    
}

但是发现这种写法会出现一个问题，就是虽然 notifyAll() 了，但是依然只有一个线程打印出来东西

看到这里， https://www.zhihu.com/question/37601861 说 notifyAll() 只是让所有线程去抢锁，但是其实还是只有一个线程能跑 synchronized 中的代码

所以真正的 countdownlatch 应该是这么来写

package me.zrj.test.test20170607;

import java.util.concurrent.CountDownLatch;

public class MyThread implements Runnable {
    
    CountDownLatch countDownLatch;
    
    public MyThread(CountDownLatch countDownLatch) {
        this.countDownLatch = countDownLatch;
    }
    
    synchronized void print() {
        try {
            countDownLatch.await();
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        System.out.println(Thread.currentThread().getName() + " " + Singleton.getInstance());
    }
    
    @Override
    public void run() {       
        print();        
    }
    
}

package me.zrj.test.test20170607;

import java.util.Date;
import java.util.concurrent.CountDownLatch;

public class App 
{
    public static void main( String[] args ) throws InterruptedException
    {
        System.out.println( "Hello World!" );        
        
        CountDownLatch countDownLatch = new CountDownLatch(1);
        
        MyThread myThread = new MyThread(countDownLatch);
        
        new Thread(myThread).start();
        Thread.sleep(100);
        new Thread(myThread).start();

        countDownLatch.countDown();
        
        while (true) {
            Thread.sleep(1000);
            System.out.println(new Date());
        }
    }
    
}

分享一个多线程的日志库

Thu, 29 Jan 2015 13:12:43 +0800

思路部分参考了陈硕的 muduo，主要就是多线程写的安全性，以及一些日志分级，自动分割等常用功能，线上几十近百台机器一两年的运行没有发现什么问题

/*
 * encoding: gbk
 * created: 2013-09-05 10:33:59
 * 
 * 日志工具
 * 工具在初始化的时候从堆上分配一片内存，当作二维 char 数组来用
 * 每个写入线程在写入前对数组的一维下标原子加 1，作为本次写入
 * 的位置，写入的位置偏移 1 个字节，第 1 个字节用作标志位，写
 * 入完成后标志位置 1
 * 磁盘文件落地线程循环的检查是否有未读取的日志行，并将新的
 * 内存中日志写入磁盘
 * 
 * 使用方法：
 * #include "logger.h"
 * assert(Logger::Init() == 0);
 * LOG_INFO("sample log message %d", 123);
 */

#ifndef LOGGER_H
#define LOGGER_H

#include <errno.h>
#include <string>
#include <pthread.h>
#include <sys/syscall.h>
#include <libgen.h>
#include <stdlib.h>

#ifdef DEBUG
#define LOG_DEBUG(format, ...) do { if (Logger::m_pBeforeLogHook==NULL || (Logger::m_pBeforeLogHook!=NULL && Logger::m_pBeforeLogHook(__FILE__, __FUNCTION__)!=0)) { Logger::m_pLog(Logger::LOG_LEVEL_DEBUG, "[%s][%d][DEBUG][%25s@%25s,%04d]  " format "\n", Logger::m_szLogTimestampStr[Logger::m_iLogTimestampIndex], syscall(__NR_gettid), __FUNCTION__, __FILE__, __LINE__, ##__VA_ARGS__); } } while (0)
#define LOG_INFO(format, ...)  do { if (Logger::m_pBeforeLogHook==NULL || (Logger::m_pBeforeLogHook!=NULL && Logger::m_pBeforeLogHook(__FILE__, __FUNCTION__)!=0)) { Logger::m_pLog(Logger::LOG_LEVEL_INFO,  "[%s][%d] [INFO][%25s@%25s,%04d]  " format "\n", Logger::m_szLogTimestampStr[Logger::m_iLogTimestampIndex], syscall(__NR_gettid), __FUNCTION__, __FILE__, __LINE__, ##__VA_ARGS__); } } while (0)
#define LOG_WARN(format, ...)  do { if (Logger::m_pBeforeLogHook==NULL || (Logger::m_pBeforeLogHook!=NULL && Logger::m_pBeforeLogHook(__FILE__, __FUNCTION__)!=0)) { Logger::m_pLog(Logger::LOG_LEVEL_WARN,  "\033[1;33m" "[%s][%d] [WARN][%25s@%25s,%04d]  " format "\033[0m" "\n", Logger::m_szLogTimestampStr[Logger::m_iLogTimestampIndex], syscall(__NR_gettid), __FUNCTION__, __FILE__, __LINE__, ##__VA_ARGS__); } } while (0)
#define LOG_ERR(format, ...)   do { if (Logger::m_pBeforeLogHook==NULL || (Logger::m_pBeforeLogHook!=NULL && Logger::m_pBeforeLogHook(__FILE__, __FUNCTION__)!=0)) { Logger::m_pLog(Logger::LOG_LEVEL_ERROR, "\033[1;31m" "[%s][%d][ERROR][%25s@%25s,%04d]  " format "\033[0m" "\n", Logger::m_szLogTimestampStr[Logger::m_iLogTimestampIndex], syscall(__NR_gettid), __FUNCTION__, __FILE__, __LINE__, ##__VA_ARGS__); } } while (0)
#else
#define LOG_DEBUG(format, ...) Logger::m_pLog(Logger::LOG_LEVEL_DEBUG, "[%s][DEBUG][%25s@%25s,%04d]  " format "\n", Logger::m_szLogTimestampStr[Logger::m_iLogTimestampIndex], __FUNCTION__, __FILE__, __LINE__, ##__VA_ARGS__)
#define LOG_INFO(format, ...)  Logger::m_pLog(Logger::LOG_LEVEL_INFO,  "[%s] [INFO][%25s@%25s,%04d]  " format "\n", Logger::m_szLogTimestampStr[Logger::m_iLogTimestampIndex],  __FUNCTION__, __FILE__, __LINE__, ##__VA_ARGS__)
#define LOG_WARN(format, ...)  Logger::m_pLog(Logger::LOG_LEVEL_WARN,  "[%s] [WARN][%25s@%25s,%04d]  " format "\n", Logger::m_szLogTimestampStr[Logger::m_iLogTimestampIndex],  __FUNCTION__, __FILE__, __LINE__, ##__VA_ARGS__)
#define LOG_ERR(format, ...)   Logger::m_pLog(Logger::LOG_LEVEL_ERROR, "[%s][ERROR][%25s@%25s,%04d]  " format "\n", Logger::m_szLogTimestampStr[Logger::m_iLogTimestampIndex], __FUNCTION__, __FILE__, __LINE__, ##__VA_ARGS__)
#endif

class Logger
{
public:
    enum eLogLevel {LOG_LEVEL_DEBUG = 10, LOG_LEVEL_INFO = 20, LOG_LEVEL_WARN = 30, LOG_LEVEL_ERROR = 40, LOG_LEVEL_OFF = 50};    
    //************************************
    // Method:    Init
    // FullName:  Logger::Init
    // Access:    public static 
    // Returns:   int 0 表示初始化成功，-1 表示失败
    // Parameter: logLevel level 日志过滤等级，默认 INFO，低于该指定等级的日志将不会写入磁盘文件，也不会写入内存
    // Parameter: const char * pLogFileDir 日志文件存放的目录，默认当前目录
    // Parameter: const char * pLogFileBaseName 日志文件的文件名，不需要包括扩展名，默认当前进程名，当文件名置为 stdout 时，即时输出到屏幕
    // Parameter: uint32_t uiLogFileRollingSize 日志文件的滚动条件，文件尺寸（字节），默认 10 MB
    // Parameter: uint32_t uiMaxCharPerLine 每行最大字符数
    // Parameter: uint32_t uiLogBufferLine 缓冲区行数
    //************************************
    static int Init(eLogLevel level = LOG_LEVEL_INFO, const char* pLogFileDir = "./", const char* pLogFileBaseName = basename(getenv("_")), uint32_t uiLogFileRollingSize = 10*1024*1024, uint32_t uiMaxCharPerLine = 16384, uint32_t uiLogBufferLine = 4096);
    static void (*m_pLog)(eLogLevel level, const char* format, ...);
    static void LogToMem(eLogLevel level, const char* format, ...);
    static void LogToStdout(eLogLevel level, const char* format, ...);
    static int (*m_pBeforeLogHook)(const char* szFilename, const char* szFunctionName);
    static char m_szLogTimestampStr[2][32];
    static time_t m_stLogTimestamp;
    static int m_iLogTimestampIndex;
    // 每行最大字符数，每行的第一个字节用作标志位，为 1 表示该行写入完成可供读取，读前检查是否为 1 读取后置为 0
    static uint32_t m_uiMaxCharPerLine;
    // 最大行数，必须保证 m_uiLogBufferLines 是 2 的整数次幂
    static uint32_t m_uiLogBufferLines;
private:
    Logger();                               // 不允许实例化
    static pthread_mutex_t m_stInitLock;    // 防止多个线程同时调用 Init 函数
    static bool m_bInited;                  // 标识日志库是否初始化完毕
    static eLogLevel m_eLevel;              // 标识日志库的记录等级，大于等于这个等级的，才会写入内存，然后被写入文件
    static char *m_pLogBufferMem;           // 指向日志内存的首字节
    static uint32_t m_uiWriteLineCount;     // 日志内存写入行数累加器
};

// 磁盘文件写入线程
void* LogDumper(void *args);
struct LogDumperArgs
{
    char* pLogBufferMem;
    std::string strLogFileDir;
    std::string strLogFileName;
    uint32_t uiLogFileMaxSize; 
};

// 日志时间字符串更新线程，每 1 秒更新一次时间字符串，交替更新两个字符串数组
void* LogTimeStrGenerator(void*);

#endif

#include <string.h>
#include <string>
#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#include <libgen.h>
#include <unistd.h>
#include <stdint.h>
#include <pthread.h>
#include <time.h>
#include <stdarg.h>
#include <ifaddrs.h>
#include <net/if.h>
#include <arpa/inet.h>
#include "logger.h"

int Logger::Init(eLogLevel level, const char* pLogFileDir, const char* pLogFileBaseName, uint32_t uiLogFileRollingSize, uint32_t uiMaxCharPerLine, uint32_t uiLogBufferLines)
{
    pthread_mutex_lock(&m_stInitLock);
    if (m_bInited || level == LOG_LEVEL_OFF)
    {
        pthread_mutex_unlock(&m_stInitLock);
        return 0;
    }

    m_eLevel = level;
    m_uiMaxCharPerLine = uiMaxCharPerLine;
    if (m_uiMaxCharPerLine <= 32)
    {
        fprintf(stderr, "max char per line config error, should greater than 32, now using default value 16384\n");
        m_uiMaxCharPerLine = 16384;
    }
    m_uiLogBufferLines = uiLogBufferLines;
    if (m_uiLogBufferLines & (m_uiLogBufferLines-1) != 0)
    {
        fprintf(stderr, "log buffer line config error, must be power of 2, like 2048  4096, now using default value 4096\n");
        m_uiLogBufferLines = 4096;
    }

    memset(m_szLogTimestampStr, 0, sizeof(m_szLogTimestampStr));
    time_t stRawtime = time(NULL);
    struct tm* stTimeinfo = localtime(&stRawtime);
    strftime(Logger::m_szLogTimestampStr[0], sizeof(Logger::m_szLogTimestampStr[0])-1, "%Y-%m-%d %H:%M:%S", stTimeinfo);
    m_iLogTimestampIndex = 0;

    pthread_t tid;
    if (pthread_create(&tid, NULL, LogTimeStrGenerator, NULL) != 0)
    {
        fprintf(stderr, "create log timestamp thread failed: %s\n", strerror(errno));
        pthread_mutex_unlock(&m_stInitLock);
        return -1;
    }

    if (strcmp("stdout", pLogFileBaseName) == 0)
    {
        m_pLog = &(Logger::LogToStdout);
        m_bInited = true;
        pthread_mutex_unlock(&m_stInitLock);
        return 0;
    }
    else
    {
        m_pLog = &(Logger::LogToMem);
    }    

    int iLogBufferMemSize = m_uiMaxCharPerLine * m_uiLogBufferLines;
    m_pLogBufferMem = (char *)malloc(iLogBufferMemSize);
    if (m_pLogBufferMem == NULL)
    {
        fprintf(stderr, "alloc heap memory failed: %s\n", strerror(errno));
        pthread_mutex_unlock(&m_stInitLock);
        return -1;
    }
    m_uiWriteLineCount = 0;
    memset(m_pLogBufferMem, 0, iLogBufferMemSize);
    
    struct LogDumperArgs* pLogDumperArgs = new struct LogDumperArgs;
    pLogDumperArgs->pLogBufferMem = m_pLogBufferMem;
    pLogDumperArgs->strLogFileDir.assign(pLogFileDir);
    pLogDumperArgs->strLogFileName.assign(pLogFileBaseName);
    pLogDumperArgs->uiLogFileMaxSize = uiLogFileRollingSize;
    if (pthread_create(&tid, NULL, LogDumper, (void *)pLogDumperArgs) != 0)
    {
        fprintf(stderr, "create log dump thread failed: %s\n", strerror(errno));
        pthread_mutex_unlock(&m_stInitLock);
        return -1;
    }

    m_bInited = true;
    pthread_mutex_unlock(&m_stInitLock);
    return 0;
}

void Logger::LogToMem(eLogLevel level, const char* format, ...)
{
    if (m_bInited && level >= m_eLevel)
    {
        va_list args;
        va_start(args, format);            
        char *pWritePtr = m_pLogBufferMem+((__sync_add_and_fetch(&m_uiWriteLineCount, 1) % m_uiLogBufferLines) * m_uiMaxCharPerLine);
        vsnprintf(pWritePtr+1, m_uiMaxCharPerLine-2, format, args); 
        *(pWritePtr) = 1;
        va_end(args);
    }        
}

void Logger::LogToStdout(eLogLevel level, const char* format, ...)
{    
    if (m_bInited && level >= m_eLevel)
    {
        va_list args;
        va_start(args, format);            
        vprintf(format, args);
        va_end(args);
    }        
}

pthread_mutex_t Logger::m_stInitLock = PTHREAD_MUTEX_INITIALIZER;
bool Logger::m_bInited = false;
Logger::eLogLevel Logger::m_eLevel = Logger::LOG_LEVEL_INFO;
char* Logger::m_pLogBufferMem = NULL;
uint32_t Logger::m_uiWriteLineCount = 0;
int Logger::m_iLogTimestampIndex = 0;
char Logger::m_szLogTimestampStr[2][32] = {{0}, {0}};
time_t Logger::m_stLogTimestamp = 0;
uint32_t Logger::m_uiMaxCharPerLine = 16384;
uint32_t Logger::m_uiLogBufferLines = 4096;
void (*Logger::m_pLog)(Logger::eLogLevel level, const char* format, ...) = &(Logger::LogToMem);
int (*Logger::m_pBeforeLogHook)(const char*, const char*) = NULL;

void* LogTimeStrGenerator(void*)
{    
    while (1)
    {
        Logger::m_stLogTimestamp = time(NULL);
        struct tm* stTimeinfo = localtime(&Logger::m_stLogTimestamp);
        strftime(Logger::m_szLogTimestampStr[(Logger::m_iLogTimestampIndex+1)%2], sizeof(Logger::m_szLogTimestampStr[0])-1, "%Y-%m-%d %H:%M:%S", stTimeinfo);
        Logger::m_iLogTimestampIndex = (Logger::m_iLogTimestampIndex + 1) % 2;
        sleep(1);
    }
    return NULL;
}

void* LogDumper(void *args)
{    
    struct LogDumperArgs* pLogDumperArgs = (struct LogDumperArgs*)args;
    char* pLogBufferMem = pLogDumperArgs->pLogBufferMem;
    const std::string strLogFileDir = pLogDumperArgs->strLogFileDir;
    const std::string strLogFileName = pLogDumperArgs->strLogFileName;
    const uint32_t uiLogFileMaxSize = pLogDumperArgs->uiLogFileMaxSize;
    delete pLogDumperArgs;
    pLogDumperArgs = NULL;
    args = NULL;

    uint32_t iReadLineCount = 1;

    // 获取 IP 地址，用于文件名        
    char* pSzIpAddr = "0.0.0.0";
    char szIpAddrBuffer[32] = {0};
    struct ifaddrs* pIfAddrs;
    if (getifaddrs(&pIfAddrs) == 0)
    {
        for (struct ifaddrs* pIfAddr = pIfAddrs; pIfAddr; pIfAddr = pIfAddr->ifa_next)
        {
            if (pIfAddr->ifa_addr == NULL || !(pIfAddr->ifa_flags & IFF_UP) || strncmp(pIfAddr->ifa_name, "eth1", strlen("eth1")) != 0)
            {
                continue;
            }
            if (inet_ntop(pIfAddr->ifa_addr->sa_family, &((struct sockaddr_in *)pIfAddr->ifa_addr)->sin_addr, szIpAddrBuffer, sizeof(szIpAddrBuffer)))
            {   
                pSzIpAddr = szIpAddrBuffer;
                break;                
            }
        }
        freeifaddrs(pIfAddrs);
    }
    // 获取进程 ID，用于文件名
    pid_t pid = getpid();

    // 检查目录是否存在，不存在则自动创建
    char szMkdirpCmd[512] = {0};
    snprintf(szMkdirpCmd, sizeof(szMkdirpCmd), "mkdir -p %s", strLogFileDir.c_str());
    system(szMkdirpCmd);

    // 开始写入文件
    while (1)
    {
        // 获取当前时间，用于文件名
        time_t stRawtime = time(NULL);
        struct tm* stTimeinfo = localtime(&stRawtime);
        char pTimeBuffer[80] = {0}, pLogFileFullName[512] = {0}, pLogFileLinkName[512] = {0};
        strftime(pTimeBuffer, sizeof(pTimeBuffer)-1, "%Y%m%d%H%M%S", stTimeinfo);
        snprintf(pLogFileFullName, sizeof(pLogFileFullName)-1, "%s/%s_%s_%s_%d.log", strLogFileDir.c_str(), strLogFileName.c_str(), pTimeBuffer, pSzIpAddr, pid);
        snprintf(pLogFileLinkName, sizeof(pLogFileLinkName)-1, "%s.log", strLogFileName.c_str());
        FILE* fp = fopen(pLogFileFullName, "a+");
        if (fp == NULL)
        {
            fprintf(stderr, "open log file %s failed: %s\n", pLogFileFullName, strerror(errno));
            return NULL;
        }
        // 建立软链接
        unlink(pLogFileLinkName);
        symlink(pLogFileFullName, pLogFileLinkName);

        uint32_t uiWriteByteCount = 0;
        time_t stLastFlushTimestamp = 0;
        while (1)
        {        
            char* pReadPtr = pLogBufferMem+((iReadLineCount % Logger::m_uiLogBufferLines) * Logger::m_uiMaxCharPerLine);
            if (*pReadPtr == 1)
            {
                uiWriteByteCount += fprintf(fp, "%s", pReadPtr + 1);
                *pReadPtr = 0;
                iReadLineCount++;
                if (uiWriteByteCount > uiLogFileMaxSize)
                {
                    break;
                }
            } 
            else
            {
                if (Logger::m_stLogTimestamp - stLastFlushTimestamp >= 3)
                {
                    stLastFlushTimestamp = Logger::m_stLogTimestamp;
                    fflush(fp);
                }
                usleep(5000);
            }    
        }        

        fclose(fp);
    }
      
    return NULL;
}

历史评论

动态链接库中的全局变量 | ZRJ (2015-03-26 14:56:49):

[…] 想给自己之前的日志库，加一个染色的功能 […]

基于队列的 UDP 多线程收发 demo

Tue, 10 Dec 2013 15:46:01 +0800

来看这个代码

#include <stdio.h>
#include <assert.h>
#include <sys/socket.h>
#include <sys/types.h>
#include <netinet/in.h>
#include <fcntl.h>
#include <string.h>
#include <sys/epoll.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <queue>
#include <pthread.h>

#define _PRINTF_(format, ...) printf("[%s:%20s@%20s,%04d] " format "\n", "DEBUG",__FUNCTION__, __FILE__, __LINE__, ##__VA_ARGS__ );

static const int iBufferSize = 2048;
int iUdpServerFd;
static const unsigned short uiUdpServerPort = 6000;
static const int iUdpProcessThreadNumber = 2;

struct InQueueEvent
{
    struct sockaddr_in stClientAddr;
    socklen_t stClienAddrLen;
    char szRecvBuffer[iBufferSize];
    int iRecvLen;
};

struct OutQueueEvent
{
    struct sockaddr_in stDestAddr;
    socklen_t stDestAddrLen;
    char szSendBuffer[iBufferSize];
    int iSendContentLen;
};

std::queue<struct InQueueEvent> stInQueue;
pthread_mutex_t stInQueueMutex;
std::queue<struct OutQueueEvent> stOutQueue;
pthread_mutex_t stOutQueueMutex;

void* UdpProcessThread(void*)
{
    _PRINTF_("udp process thread %ld start", pthread_self());
    while (1)
    {
        if (stInQueue.empty())
        {
            continue;
        }
        pthread_mutex_lock(&stInQueueMutex);
        struct InQueueEvent stInQueueEvent = stInQueue.front();
        stInQueue.pop();
        pthread_mutex_unlock(&stInQueueMutex);
        _PRINTF_("thread %ld processing the request %s", pthread_self(), stInQueueEvent.szRecvBuffer);
        struct OutQueueEvent stOutQueueEvent;
        stOutQueueEvent.stDestAddr = stInQueueEvent.stClientAddr;
        stOutQueueEvent.stDestAddrLen = stInQueueEvent.stClienAddrLen;
        memcpy(stOutQueueEvent.szSendBuffer, stInQueueEvent.szRecvBuffer, stInQueueEvent.iRecvLen);
        stOutQueueEvent.iSendContentLen = stInQueueEvent.iRecvLen;
        pthread_mutex_lock(&stOutQueueMutex);
        stOutQueue.push(stOutQueueEvent);
        pthread_mutex_unlock(&stOutQueueMutex);        
    }
}

void* UdpRecvThread(void*)
{
    _PRINTF_("udp recv thread start");
    while(1)
    {
        struct InQueueEvent stInQueueEvent;
        memset(&stInQueueEvent, 0, sizeof(stInQueueEvent));
        stInQueueEvent.stClienAddrLen = sizeof(stInQueueEvent.stClientAddr);
        stInQueueEvent.iRecvLen = recvfrom(iUdpServerFd, &(stInQueueEvent.szRecvBuffer), sizeof(stInQueueEvent.szRecvBuffer), 0, (struct sockaddr*)&(stInQueueEvent.stClientAddr), &(stInQueueEvent.stClienAddrLen));
        _PRINTF_("recv %d bytes from ip %s port %d message %s", stInQueueEvent.iRecvLen, inet_ntoa(stInQueueEvent.stClientAddr.sin_addr), ntohs(stInQueueEvent.stClientAddr.sin_port), stInQueueEvent.szRecvBuffer);
        pthread_mutex_lock(&stInQueueMutex);
        stInQueue.push(stInQueueEvent);
        pthread_mutex_unlock(&stInQueueMutex);
    }
}

void* UdpSendThread(void*)
{
    _PRINTF_("udp send thread start");
    while(1)
    {
        if (stOutQueue.empty())
        {
            continue;
        }
        pthread_mutex_lock(&stOutQueueMutex);
        struct OutQueueEvent stOutQueueEvent = stOutQueue.front();
        stOutQueue.pop();
        pthread_mutex_unlock(&stOutQueueMutex);
        int iSendLen = sendto(iUdpServerFd, &(stOutQueueEvent.szSendBuffer), stOutQueueEvent.iSendContentLen, 0, (struct sockaddr*)&(stOutQueueEvent.stDestAddr), stOutQueueEvent.stDestAddrLen);
        _PRINTF_("send %d bytes to ip %s port %d message %s", iSendLen, inet_ntoa(stOutQueueEvent.stDestAddr.sin_addr), ntohs(stOutQueueEvent.stDestAddr.sin_port), stOutQueueEvent.szSendBuffer);
    }
}

int CreateUdpServer(unsigned short uiUdpServerPort)
{
    int iListenFd = socket(AF_INET, SOCK_DGRAM, 0);
    assert(iListenFd > 0);
    struct sockaddr_in stListenAddr;
    memset(&stListenAddr, 0, sizeof(stListenAddr));
    stListenAddr.sin_family = AF_INET;
    stListenAddr.sin_addr.s_addr = htonl(INADDR_ANY);
    stListenAddr.sin_port = htons(uiUdpServerPort);
    int iReuseFlag = 1;
    assert(setsockopt(iListenFd, SOL_SOCKET, SO_REUSEADDR, &iReuseFlag, sizeof(int)) == 0);
    assert(bind(iListenFd, (struct sockaddr*)&stListenAddr, sizeof(stListenAddr)) == 0);  
    return iListenFd;
}

int main()
{
    pthread_mutex_init(&stInQueueMutex, NULL);
    pthread_mutex_init(&stOutQueueMutex, NULL);
    iUdpServerFd = CreateUdpServer(uiUdpServerPort);
    pthread_t tid;
    assert(pthread_create(&tid, NULL, UdpRecvThread, NULL) == 0);
    assert(pthread_create(&tid, NULL, UdpSendThread, NULL) == 0);
    for (int i = 0; i < iUdpProcessThreadNumber; i++)
    {
        assert(pthread_create(&tid, NULL, UdpProcessThread, NULL) == 0);
    }    
    while (1)
    {
        sleep(10);
    }
    return 0;
}

用 python 写个客户端测试一下

#!/usr/bin/env python

import socket

HOST = 'xx.xx.xx.xx'
PORT = 6000
BUFSIZ = 1024
ADDR = (HOST, PORT)

udpCliSock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)

while True:
    data = raw_input('> ')
    if not data:
        break
    udpCliSock.sendto(data, ADDR)
    data, ADDR = udpCliSock.recvfrom(BUFSIZ)
    if not data:
        break
    print ADDR, data

udpCliSock.close()

跑起来似乎没有问题

[DEBUG:       UdpRecvThread@        EpollUdp.cpp,0069] udp recv thread start
[DEBUG:       UdpSendThread@        EpollUdp.cpp,0085] udp send thread start
[DEBUG:    UdpProcessThread@        EpollUdp.cpp,0044] udp process thread 1107310912 start
[DEBUG:    UdpProcessThread@        EpollUdp.cpp,0044] udp process thread 1098918208 start
[DEBUG:       UdpRecvThread@        EpollUdp.cpp,0076] recv 1 bytes from ip 10.31.18.40 port 62272 message 1
[DEBUG:    UdpProcessThread@        EpollUdp.cpp,0055] thread 1107310912 processing the request 1
[DEBUG:       UdpSendThread@        EpollUdp.cpp,0097] send 1 bytes to ip 10.31.18.40 port 62272 message 1
[DEBUG:       UdpRecvThread@        EpollUdp.cpp,0076] recv 1 bytes from ip 10.31.18.40 port 62272 message 2
[DEBUG:    UdpProcessThread@        EpollUdp.cpp,0055] thread 1107310912 processing the request 2
[DEBUG:       UdpSendThread@        EpollUdp.cpp,0097] send 1 bytes to ip 10.31.18.40 port 62272 message 2

但是多发几次，就 core 了，这是为什么呢，gdb 调试发现 core 在 memcpy 上面，这个也不太能理解，不过，把代码尝试的改成这样就可以了

#include <stdio.h>
#include <assert.h>
#include <sys/socket.h>
#include <sys/types.h>
#include <netinet/in.h>
#include <fcntl.h>
#include <string.h>
#include <sys/epoll.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <queue>
#include <pthread.h>

#define _PRINTF_(format, ...) printf("[%s:%20s@%20s,%04d] " format "\n", "DEBUG",__FUNCTION__, __FILE__, __LINE__, ##__VA_ARGS__ );

static const int iBufferSize = 2048;
int iUdpServerFd;
static const unsigned short uiUdpServerPort = 6000;
static const int iUdpProcessThreadNumber = 2;

struct InQueueEvent
{
    struct sockaddr_in stClientAddr;
    socklen_t stClienAddrLen;
    char szRecvBuffer[iBufferSize];
    int iRecvLen;
};

struct OutQueueEvent
{
    struct sockaddr_in stDestAddr;
    socklen_t stDestAddrLen;
    char szSendBuffer[iBufferSize];
    int iSendContentLen;
};

std::queue<struct InQueueEvent> stInQueue;
pthread_mutex_t stInQueueMutex;
std::queue<struct OutQueueEvent> stOutQueue;
pthread_mutex_t stOutQueueMutex;

void* UdpProcessThread(void*)
{
    _PRINTF_("udp process thread %ld start", pthread_self());
    while (1)
    {
        pthread_mutex_lock(&stInQueueMutex);
        if (stInQueue.empty())
        {
            pthread_mutex_unlock(&stInQueueMutex);
            continue;
        }
        struct InQueueEvent stInQueueEvent = stInQueue.front();
        stInQueue.pop();
        pthread_mutex_unlock(&stInQueueMutex);
        _PRINTF_("thread %ld processing the request %s", pthread_self(), stInQueueEvent.szRecvBuffer);
        struct OutQueueEvent stOutQueueEvent;
        memset(&stOutQueueEvent, 0, sizeof(stOutQueueEvent));
        stOutQueueEvent.stDestAddr = stInQueueEvent.stClientAddr;
        stOutQueueEvent.stDestAddrLen = stInQueueEvent.stClienAddrLen;
        memcpy(stOutQueueEvent.szSendBuffer, stInQueueEvent.szRecvBuffer, stInQueueEvent.iRecvLen);
        stOutQueueEvent.iSendContentLen = stInQueueEvent.iRecvLen;
        pthread_mutex_lock(&stOutQueueMutex);
        stOutQueue.push(stOutQueueEvent);
        pthread_mutex_unlock(&stOutQueueMutex);        
    }
}

void* UdpRecvThread(void*)
{
    _PRINTF_("udp recv thread start");
    while(1)
    {
        struct InQueueEvent stInQueueEvent;
        memset(&stInQueueEvent, 0, sizeof(stInQueueEvent));
        stInQueueEvent.stClienAddrLen = sizeof(stInQueueEvent.stClientAddr);
        stInQueueEvent.iRecvLen = recvfrom(iUdpServerFd, &(stInQueueEvent.szRecvBuffer), sizeof(stInQueueEvent.szRecvBuffer), 0, (struct sockaddr*)&(stInQueueEvent.stClientAddr), &(stInQueueEvent.stClienAddrLen));
        _PRINTF_("recv %d bytes from ip %s port %d message %s", stInQueueEvent.iRecvLen, inet_ntoa(stInQueueEvent.stClientAddr.sin_addr), ntohs(stInQueueEvent.stClientAddr.sin_port), stInQueueEvent.szRecvBuffer);
        pthread_mutex_lock(&stInQueueMutex);
        stInQueue.push(stInQueueEvent);
        pthread_mutex_unlock(&stInQueueMutex);
    }
}

void* UdpSendThread(void*)
{
    _PRINTF_("udp send thread start");
    while(1)
    {
        pthread_mutex_lock(&stOutQueueMutex);
        if (stOutQueue.empty())
        {
            pthread_mutex_unlock(&stOutQueueMutex);
            continue;
        }
        struct OutQueueEvent stOutQueueEvent = stOutQueue.front();
        stOutQueue.pop();
        pthread_mutex_unlock(&stOutQueueMutex);
        int iSendLen = sendto(iUdpServerFd, &(stOutQueueEvent.szSendBuffer), stOutQueueEvent.iSendContentLen, 0, (struct sockaddr*)&(stOutQueueEvent.stDestAddr), stOutQueueEvent.stDestAddrLen);
        _PRINTF_("send %d bytes to ip %s port %d message %s", iSendLen, inet_ntoa(stOutQueueEvent.stDestAddr.sin_addr), ntohs(stOutQueueEvent.stDestAddr.sin_port), stOutQueueEvent.szSendBuffer);
    }
}

int CreateUdpServer(unsigned short uiUdpServerPort)
{
    int iListenFd = socket(AF_INET, SOCK_DGRAM, 0);
    assert(iListenFd > 0);
    struct sockaddr_in stListenAddr;
    memset(&stListenAddr, 0, sizeof(stListenAddr));
    stListenAddr.sin_family = AF_INET;
    stListenAddr.sin_addr.s_addr = htonl(INADDR_ANY);
    stListenAddr.sin_port = htons(uiUdpServerPort);
    int iReuseFlag = 1;
    assert(setsockopt(iListenFd, SOL_SOCKET, SO_REUSEADDR, &iReuseFlag, sizeof(int)) == 0);
    assert(bind(iListenFd, (struct sockaddr*)&stListenAddr, sizeof(stListenAddr)) == 0);  
    return iListenFd;
}

int main()
{
    pthread_mutex_init(&stInQueueMutex, NULL);
    pthread_mutex_init(&stOutQueueMutex, NULL);
    iUdpServerFd = CreateUdpServer(uiUdpServerPort);
    pthread_t tid;
    assert(pthread_create(&tid, NULL, UdpRecvThread, NULL) == 0);
    assert(pthread_create(&tid, NULL, UdpSendThread, NULL) == 0);
    for (int i = 0; i < iUdpProcessThreadNumber; i++)
    {
        assert(pthread_create(&tid, NULL, UdpProcessThread, NULL) == 0);
    }    
    while (1)
    {
        sleep(10);
    }
    return 0;
}

注意高亮的部分，在检查队列是否为空的时候，也要包含在锁的区域里面，这样就可以了

-———————————————

2013-12-10 16:37:06 update 使用这个脚本，可以多线程压测服务

#!/usr/bin/env python

import socket
import random
import threading

HOST = 'xx.xx.xx.xx'
PORT = 6000
BUFSIZ = 1024
ADDR = (HOST, PORT)

class UdpTestThread(threading.Thread):
    def run(self):
        udpCliSock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
        for i in range(1000):
            data = str(random.random())
            udpCliSock.sendto(data, ADDR)
            data = udpCliSock.recvfrom(BUFSIZ)
            #print data
        udpCliSock.close()

for i in range(50):
    UdpTestThread().start()

-—————————

2013-12-10 17:21:05 update 发现上面的例子，压测结果很差，于是写了个极简的，作为比对

#include <sys/socket.h>
#include <arpa/inet.h>

int main() {
    int fd = socket(AF_INET, SOCK_DGRAM, 0);
    struct sockaddr_in addr;
    addr.sin_family = AF_INET;
    addr.sin_port = htons(6000);
    addr.sin_addr.s_addr = htonl(INADDR_ANY);
    bind(fd, (struct sockaddr*)&addr, sizeof(addr));
    while (1) {
        char buffer[2048] = {0};
        struct sockaddr cli;
        socklen_t cli_len = sizeof(cli);
        int recv_len = recvfrom(fd, buffer, sizeof(buffer), 0, &cli, &cli_len);
        sendto(fd, buffer, recv_len, 0, &cli, cli_len);
    }
    return 0;
}

发现差不多，看来还要找原因

历史评论

wanax (2014-03-19 16:52:29):

这几天挖出来个GLib的，自带线程池和宏定义好了的锁，感觉再用C就顺手多了

ZRJ (2014-03-19 20:15:41):

其实用 libevent 基本就够了，不用造轮子

wanax (2014-03-20 09:49:23):

那玩意儿就是个轮子还附带各种数据结构与常用工具类

ZRJ (2014-03-20 19:58:08):

反正我现在是觉得能有稳定开源的就用呗

写一个日志模块

Thu, 05 Sep 2013 20:48:27 +0800

今天在写一个日志模块，遇到一些问题，学到了一些东西，记一下。

一开始想到，cpp 这么成熟的社区，日志这么基本的模块，应该现成的有很多吧，随便找一个来用就可以了，上网搜了一圈，发现真是一团混乱啊，看到这里，http://blog.csdn.net/junchaox/article/details/7346514

待选为 glog、log4cplus、log4cpp、log4cxx

目前准备使用glog，使用方便，性能也不错，待进一步试验，如果有不能满足的功能就转用 log4cplus，功能很全面，不过稍复杂些。

其它两个都是三年前就没更新，没好感，暂不准备使用。

1.log4cplus 最新版本：1.1.0 2012-03-11

下载地址：http://sourceforge.net/projects/log4cplus/files/log4cplus-stable/1.1.0

功能全面，使用稍复杂。

代码示例：

#include <log4cplus/layout.h>  
  
#include <log4cplus/configurator.h>  
  
#include <iomanip>  
  
SharedAppenderPtr pFileAppender(new FileAppender("testlog.log"));  
  
Logger pTestLogger = Logger::getInstance("LoggerName");  
  
pTestLogger.addAppender(pFileAppender);   
  
sprintf(a,"%d",i);  
  
LOG4CPLUS_WARN(pTestLogger, "This is a <Warn> log message..." << a );

2.log4cxx 最新版本： 0.10.0 2008-04-03

下载地址：http://logging.apache.org/log4cxx/download.html 编译很麻烦新的版本0.10.0需要使用Apache的产品Ant来编译，而Ant又需要JDK。。。

怕麻烦，没有下载测试。

3. glog 最新版本： 0.3.2 2012-1-12

下载地址：http://code.google.com/p/google-glog/downloads/list

使用方便，性能也不错。

Google Glog 是一个C++语言的应用级日志记录框架，提供了 C++ 风格的流操作和各种助手宏。

Google glog是一个基于程序级记录日志信息的c++库，编程使用方式与c++的stream操作类似

代码示例：

#include <glog/logging.h>  
  
google::InitGoogleLogging(argv[0]);  
  
google::SetLogDestination(google::INFO,"./myInfo_");  
  
LOG(INFO) << "This is a <Warn> log message..." << i;

Log4cpp 最新版1.0.x 2007-09-03

下载地址： http://sourceforge.net/projects/log4cpp/files

感觉跟 log4cplus类似，结构稍简单些，不过代码也不少写。

#include <log4cpp/Category.hh>  
#include <log4cpp/PropertyConfigurator.hh>  
#include <log4cpp/NDC.hh>  
#include <log4cpp/FileAppender.hh>  
#include <log4cpp/BasicLayout.hh>  
log4cpp::Layout* layout = new log4cpp::BasicLayout();  
log4cpp::Appender* appender = new log4cpp::FileAppender("FileAppender", "./test_log4cpp1.log");  
appender->setLayout(layout);  
log4cpp::Category& warn_log = log4cpp::Category::getInstance("mywarn");  
warn_log.setAdditivity(false);  
warn_log.setAppender(appender);  
warn_log.setPriority(log4cpp::Priority::WARN);  
warn_log.critStream() << "This is a <Warn> log message..." << i;

看到上面的那些东西，真是令人望而生畏，我这才理解为什么之前看到的几个开源项目都是自己实现了一个日志类，好吧，cpp 程序员就是喜欢重复造轮子，术语叫什么来着，not invented here

好吧，既然求开源而不得，那只能自己动手丰衣足食了，总的来说，我们希望这个日志模块满足以下几个特点：

易用，最好是只要 include 一个头文件，然后就可以用了，退一步，我们可以接受在使用之前调用一次 init 函数，不需要另外编译依赖库，更不要安装。
轻量，对业务的能耗尽量减少到最小。
实现简洁。用尽量少的代码实现。
线程安全。
输出层级可控。
异步落地磁盘，而且按照文件大小或者时间自动滚动。

而最终的效果，使用上，我们希望是这样的用法。

init_log();
LOG_DEBUG("test log %d", 123);
LOG_ERR("test err %s", "456");

这个地方，之所以用 c 的转义，而不用 cpp 的流式，一个是考虑到效率，一个是考虑到现有的代码兼容。

首先上来遇到的第一个问题，我们需要打印行号，文件名，函数名等等，这些怎么得到呢，想到用宏，__LINE__，__FILE__，等等，但是，宏展开之后，需要调用我们自己的函数吧，这些可变参数怎么从我们的函数传到 printf 呢？

在这个地方纠结了很久，一开始去找各种 va_list 的传递方法，发现这个几乎是不可能的，于是后面又找了 vsnprintf 这种的函数，才发现有这么一类可以直接接收 va_list 的函数存在。

然后就是日志类的初始化问题，一开始，想到用类的静态函数来做，但是又发现，这样的话，也是有问题的，于是后面用单例来做。

然后就是异步的问题，这个是想到先把日志存到内存，然后用线程异步写入磁盘文件。但是遇到问题是，内存要怎么布局呢，如果我们把内存连续的用起来，那么，多个线程同时写的时候，怎么解决字符串交错的问题呢，如果用锁，那么性能必然大大下降。

另外的一个问题是，我们申请一片定长内存，然后把它当作循环队列来用，但是，在快到结尾的时候，例如，假如还剩下 50 个字符的空间，但是有一条日志过来了，而这个日志要占用 100 个字符，那怎么处理剩下的 50 个呢，按照循环队列的意义，应该转到头部继续写的，但是，到时候怎么打印出来呢。如果不截断的话，有什么办法解决呢。而且更麻烦的问题是，在一个字符串被格式化完成之前，我们是不知道这个字符串会有多长的。

对于这个问题，有一个办法是，我们留下一部分内存空间，当作保留区，例如，我们申请 15M，但是只有 14M 是拿来当循环队列的，那么，当指针超过 14 的时候，下一条就回来开始的地方了，这样的话，就要限定，刚好在 14 M 边缘的那条，单行不能超过 1MB，这个限制，应该还是可以接受的。

但是依然没有解决多线程的并发写入问题。

这个问题，或许用一个二维的字符型数组来实现，然后用下标的原子操作来避免多线程混叠，但是，这样的话，我们就需要按照最长的可能单行来分配，那么，就必然造成大量的空间浪费，特别是在最长的单行比大多数的单行都长很多倍的情况下。

待续。

历史评论

wanax (2013-09-23 15:36:44):

iphone开发下面有个开源日志还可以打印图片的….客户端输出还不是控制台碉堡了

ZRJ (2013-09-23 18:49:18):

堂堂 c++ 一大帮人在用，居然连个日志标准都没有。。

不同服务器端模型在并发请求下的表现差异

Mon, 25 Mar 2013 11:22:04 +0800

一般来说，服务器的模型有下面几种：

单进程单线程
多进程（一个请求对应一个进程，Apache 之类貌似就是这个）
多线程（一个请求对应一个线程，这个挺少见到的）
select / poll
epoll

它们的差别可以看这个，http://www.cnblogs.com/sharra/archive/2010/12/30/1921287.html

因为需要了解底层设备访问的原理，所以惯用高层应用语言的我，需要了解一下Linux的设备访问机制，尤其是处理一组非阻塞IO的原理方法，标准的术语好像是叫多路复用。以下文章部分句子有引用之处，恕没有一一指出出处。对于接触过Linux内核或设备驱动开发的读者，一定清楚poll和select系统调用，以及从2.5版本引入的epoll机制（epoll机制包含三个系统调用）。网上关于它们的文章，有说用法的，甚为详细，更有分析源代码的，又比较深入，且枝节颇多。经过几篇文章的阅读，我把觉得比较核心的东西写下来吧。我的用意是尽可能以简单的概念，比对他们三者的异同。几经查找我才确定下来，poll和select应该被归类为这样的系统调用，它们可以阻塞地同时探测一组支持非阻塞的IO设备，是否有事件发生（如可读，可写，有高优先级的错误输出，出现错误等等），直至某一个设备触发了事件或者超过了指定的等待时间——也就是它们的职责不是做IO，而是帮助调用者寻找当前就绪的设备。同类型的产品是Windows的IOCP，它也是处理多路复用，只是把IO和探测封装在了一起了。准备的知识有两点：1、fd；2、op->poll。在Linux里面，设备都被抽象为文件，一系列的设备文件就有自己独立的虚拟文件系统，所以，设备在系统调用参数中的表示就是file description。fd其实就是一个整数（特别地，标准输入，输出，错误输出分别对应的fd是0，1，2）。与内核打交道的时候，传递整数的fd可以在自己的文件系统中作进一步的检查是否合法，如果只是返回指针就不能这样操作了，毕竟指针是无差别无意义的。通过fd访问file，通过file可以访问其fileOperator，这里面我们要关心的一个fileOp就是poll。因为系统调用poll和select，就是依靠这个文件操作poll实现的。poll文件操作有两个参数，一个是文件本身，一个可以看做是当设备尚未就绪时调用的回调函数，这个函数是把设备自己特有的等待队列传给内核，让内核把当前的进程挂载到其中（因为当设备就绪时，设备就应该去唤醒在自己特有等待队列中的所有节点，这样当前进程就获取了完成的信号了）。poll文件操作返回的必须是一组标准的掩码，其中的各个位指示当前的不同的就绪状态（全0为没有任何事件触发）。再谈谈早期多路复用的版本poll和select。本质而言，poll和select的共同点就是，对全部指定设备做一次poll，当然这往往都是还没有就绪的，那就会通过回调函数把当前进程注册到设备的等待队列，如果所有设备返回的掩码都没有显示任何的事件触发，就去掉回调函数的函数指针，进入有限时的睡眠状态，再恢复和不断做poll，再作有限时的睡眠，直到其中一个设备有事件触发为止。只要有事件触发，系统调用返回，回到用户态，用户就可以对相关的fd作进一步的读或者写操作了。当然，这个时候还不是所有的设备都就绪的喔，那就得不断地poll或者select了，而做一次这样的系统调用都得轮询所有的设备，次数是设备数*（睡眠次数-1），也就是时间复杂度是O（n），还得做几次O（n）呢。可见，对于现在普遍的服务器程序，需要同时并发监听数千个连接，并且连接需要重复使用的情况，poll和select就存在这样的性能瓶颈。另外，数千个设备fd在每次调用时，都需要将其从用户空间复制到内核空间，这里的开销不可忽略。 poll和select放在一起，是因为其机制一致，而参数和数据结构就略有不同。select一次性传入三组作用于不同信道的设备fd，分别是输入，输出和错误异常。各组的fd期待各组所特有的，由代码指定的一组事件，如输入信道期待输入就绪，输入挂起和错误等事件。然后，select就挑选调用者关心的fd做poll文件操作，检测返回的掩码，看看是否有fd所属信道感兴趣的事件，比如看看这个属于输出信道的fd有没有输出就绪等一系列的事件发生，一样地，如果有一个fd发生感兴趣事件就返回调用了。select，为了同时处理三组使用不同的事件判断规则的fd，采用了位图的方式表示，一组一个位图，位长度是当中最大的fd值，上限是1024，三组就是3072，而且这还只是传入的位图，还有一样大小的传出的位图。当fd数越来越多时，所需的存储开销比较大。既然，一组fd处理起来比较粗放，那就各个fd自己准备好了。poll()系统调用是System V的多元I/O解决方案。它有三个参数，第一个是pollfd结构的数组指针，也就是指向一组fd及其相关信息的指针，因为这个结构包含的除了fd，还有期待的事件掩码和返回的事件掩码，实质上就是将select的中的fd，传入和传出参数归到一个结构之下，也不再把fd分为三组，也不再硬性规定fd感兴趣的事件，这由调用者自己设定。这样，不使用位图来组织数据，也就不需要位图的全部遍历了。按照一般队列地遍历，每个fd做poll文件操作，检查返回的掩码是否有期待的事件，以及做是否有挂起和错误的必要性检查，如果有事件触发，就可以返回调用了。回到poll和select的共同点，面对高并发多连接的应用情境，它们显现出原来没有考虑到的不足，虽然poll比起select又有所改进了。除了上述的关于每次调用都需要做一次从用户空间到内核空间的拷贝，还有这样的问题，就是当处于这样的应用情境时，poll和select会不得不多次操作，并且每次操作都很有可能需要多次进入睡眠状态，也就是多次全部轮询fd，我们应该怎么处理一些会出现重复而无意义的操作。这些重复而无意义的操作有：1、从用户到内核空间拷贝，既然长期监视这几个fd，甚至连期待的事件也不会改变，那拷贝无疑就是重复而无意义的，我们可以让内核长期保存所有需要监视的fd甚至期待事件，或者可以再需要时对部分期待事件进行修改；2、将当前线程轮流加入到每个fd对应设备的等待队列，这样做无非是哪一个设备就绪时能够通知进程退出调用，聪明的开发者想到，那就找个“代理”的回调函数，代替当前进程加入fd的等待队列好了（这也是我后来才总结出来，Linux的等待队列，实质上是回调函数队列吧，也可以使用宏来将当前进程“加入”等待队列，其实就是将唤醒当前进程的回调函数加入队列）。这样，像poll系统调用一样，做poll文件操作发现尚未就绪时，它就调用传入的一个回调函数，这是epoll指定的回调函数，它不再像以前的poll系统调用指定的回调函数那样，而是就将那个“代理”的回调函数加入设备的等待队列就好了，这个代理的回调函数就自己乖乖地等待设备就绪时将它唤醒，然后它就把这个设备fd放到一个指定的地方，同时唤醒可能在等待的进程，到这个指定的地方取fd就好了。我们把1和2结合起来就可以这样做了，只拷贝一次fd，一旦确定了fd就可以做poll文件操作，如果有事件当然好啦，马上就把fd放到指定的地方，而通常都是没有的，那就给这个fd的等待队列加一个回调函数，有事件就自动把fd放到指定的地方，当前进程不需要再一个个poll和睡眠等待了。 epoll机制就是这样改进的了。诚然，fd少的时候，当前进程一个个地等问题不大，可是现在和尚多了，方丈就不好管了。以前设备事件触发时，只负责唤醒当前进程就好了，而当前进程也只能傻傻地在poll里面等待或者循环，再来一次poll，也不知道这个由设备提供的poll性能如何，能不能检查出当前进程已经在等待了就立即返回，当然，我也不明白为什么做了一遍的poll之后，去掉回调函数指针了，还得再做，不是说好了会去唤醒进程的吗？现在就让事件触发回调函数多做一步。本来设备还没就绪就调用一个回调函数了，现在再在这个回调函数里面做一个注册另一个回调函数的操作，目的就是使得设备事件触发多走一步，不仅仅是唤醒当前进程，还要把自己的fd放到指定的地方。就像收本子的班长，以前得一个个学生地去问有没有本子，如果没有，它还得等待一段时间而后又继续问，现在好了，只走一次，如果没有本子，班长就告诉大家去那里交本子，当班长想起要取本子，就去那里看看或者等待一定时间后离开，有本子到了就叫醒他，然后取走。这个道理很简单，就是老师和班干们常说的，大家多做一点工作，我的工作就轻松很多了，尤其是需要管理的东西越来越多时。这种机制或者说模式，我想在Java的FutureTask里面应该也会用到的，一堆在线程池里面跑着的线程（当然这是任务，不是线程，接口是Callable，不是Runnable.run，是Callable.call，它是可以返回结果的），谁先做好就应该先处理呀，可是难道得一个个问吗？干脆就谁好了，谁就按照既定的操作暴露自己，这样FutureTask的get方法就可以马上知道当前最先完成的线程了，就可以取此线程返回结果了。 epoll由三个系统调用组成，分别是epoll_create，epoll_ctl和epoll_wait。epoll_create用于创建和初始化一些内部使用的数据结构；epoll_ctl用于添加，删除或者修改指定的fd及其期待的事件，epoll_wait就是用于等待任何先前指定的fd事件。关于epoll内部的数据结构，我就不能详细了解了。

下面就是分别实现一个 demo，然后用 ab 去压力测试，每回都是 10W 个请求，根据不同的并发量测试

单进程单线程

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>

#define BUFFER_SIZE 4096

int tcpSrv(int listen_port);

int main(int argc, char** argv) {
    tcpSrv(8080);
    return 0;
}

int tcpSrv(int listen_port) {
    int listenfd, connfd;
    struct sockaddr_in servaddr;
    char buff[BUFFER_SIZE+1];
    int n;

    if ((listenfd = socket(AF_INET, SOCK_STREAM, 0)) == -1) {
        printf("create socket error: %s(errno: %d)\n", strerror(errno), errno);
        return -1;
    }

    memset(&servaddr, 0, sizeof(servaddr));
    servaddr.sin_family = AF_INET;
    servaddr.sin_addr.s_addr = htonl(INADDR_ANY);
    servaddr.sin_port = htons(listen_port);

    if (bind(listenfd, (struct sockaddr *)&servaddr, sizeof(servaddr)) == -1) {
        printf("bind socket error: %s(errno: %d)\n", strerror(errno), errno);
        close(listenfd);
        return -1;
    }

    if (listen(listenfd, 2048) == -1) {
        printf("listen socket error: %s(errno: %d)\n", strerror(errno), errno);
        close(listenfd);
        return -1;
    }

    printf("-----waiting for client's request-----\n");
    while (1) {
        if ((connfd = accept(listenfd, (struct sockaddr *)NULL, NULL)) == -1) {
            printf("accept socket error: %s(errno: %d)\n", strerror(errno), errno);
            close(listenfd);
            return -1;
        }
        memset(&buff, 0, sizeof(buff));
        recv(connfd, buff, BUFFER_SIZE, 0);
        //printf("recv msg:\n%s\n", buff);
        char resp[] = "HTTP/1.0 200 OK\r\n\r\n<html>hello</html>";
        send(connfd, resp, strlen(resp), 0);
        close(connfd);
    }
    
    close(listenfd);
    return 0;
}

一开始觉得这种简单粗暴的方式肯定不行，结果发现还别说，这东西的处理能力还是可以的，看测试结果

单并发

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 1 http://localhost:8080/
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            8080

Document Path:          /
Document Length:        18 bytes

Concurrency Level:      1
Time taken for tests:   7.210 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      3700000 bytes
HTML transferred:       1800000 bytes
Requests per second:    13869.50 [#/sec] (mean)
Time per request:       0.072 [ms] (mean)
Time per request:       0.072 [ms] (mean, across all concurrent requests)
Transfer rate:          501.14 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       4
Processing:     0    0   0.0      0       4
Waiting:        0    0   0.0      0       4
Total:          0    0   0.0      0       4

Percentage of the requests served within a certain time (ms)
  50%      0
  66%      0
  75%      0
  80%      0
  90%      0
  95%      0
  98%      0
  99%      0
 100%      4 (longest request)
linux:~/ab/httpd-2.4.3/support$

10 个并发

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 10 http://localhost:8080/
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            8080

Document Path:          /
Document Length:        18 bytes

Concurrency Level:      10
Time taken for tests:   5.498 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      3700000 bytes
HTML transferred:       1800000 bytes
Requests per second:    18186.97 [#/sec] (mean)
Time per request:       0.550 [ms] (mean)
Time per request:       0.055 [ms] (mean, across all concurrent requests)
Transfer rate:          657.15 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       6
Processing:     0    0   0.1      0       6
Waiting:        0    0   0.1      0       6
Total:          0    1   0.1      1       6

Percentage of the requests served within a certain time (ms)
  50%      1
  66%      1
  75%      1
  80%      1
  90%      1
  95%      1
  98%      1
  99%      1
 100%      6 (longest request)
linux:~/ab/httpd-2.4.3/support$

100 个并发

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 100 http://localhost:8080/
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            8080

Document Path:          /
Document Length:        18 bytes

Concurrency Level:      100
Time taken for tests:   5.816 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      3700000 bytes
HTML transferred:       1800000 bytes
Requests per second:    17194.97 [#/sec] (mean)
Time per request:       5.816 [ms] (mean)
Time per request:       0.058 [ms] (mean, across all concurrent requests)
Transfer rate:          621.30 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    1   0.6      1      11
Processing:     1    5   0.8      5      15
Waiting:        0    2   0.6      2      12
Total:          2    6   0.6      6      15

Percentage of the requests served within a certain time (ms)
  50%      6
  66%      6
  75%      6
  80%      6
  90%      6
  95%      6
  98%      7
  99%      7
 100%     15 (longest request)
linux:~/ab/httpd-2.4.3/support$

150 个并发的时候就不行了

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 150 http://localhost:8080/
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
apr_pollset_poll: The timeout specified has expired (70007)
Total of 99956 requests completed
linux:~/ab/httpd-2.4.3/support$

接下来是多进程模型

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>
#include <unistd.h>
#include <signal.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>

#define BUFFER_SIZE 4096

int tcpSrv(int listen_port);

int main(int argc, char** argv) {
    tcpSrv(8080);
    return 0;
}

int tcpSrv(int listen_port) {
    int listenfd, connfd;
    struct sockaddr_in servaddr;
    char buff[BUFFER_SIZE+1];
    int n;

    if ((listenfd = socket(AF_INET, SOCK_STREAM, 0)) == -1) {
        printf("create socket error: %s(errno: %d)\n", strerror(errno), errno);
        return -1;
    }

    memset(&servaddr, 0, sizeof(servaddr));
    servaddr.sin_family = AF_INET;
    servaddr.sin_addr.s_addr = htonl(INADDR_ANY);
    servaddr.sin_port = htons(listen_port);

    if (bind(listenfd, (struct sockaddr *)&servaddr, sizeof(servaddr)) == -1) {
        printf("bind socket error: %s(errno: %d)\n", strerror(errno), errno);
        close(listenfd);
        return -1;
    }

    if (listen(listenfd, 2048) == -1) {
        printf("listen socket error: %s(errno: %d)\n", strerror(errno), errno);
        close(listenfd);
        return -1;
    }

    printf("-----waiting for client's request-----\n");
    while (1) {
        if ((connfd = accept(listenfd, (struct sockaddr *)NULL, NULL)) == -1) {
            printf("accept socket error: %s(errno: %d)\n", strerror(errno), errno);
            close(listenfd);
            return -1;
        }
        signal(SIGCHLD, SIG_IGN);
        pid_t pid = fork();
        if (pid < 0) {
            printf("fork error: %s(errno: %d)\n", strerror(errno), errno);
            close(listenfd);
            return -1;
        } else if (pid == 0) {
            //printf("in child process\n");
            memset(&buff, 0, sizeof(buff));
            recv(connfd, buff, BUFFER_SIZE, 0);
            //printf("recv msg:\n%s\n", buff);
            char resp[] = "HTTP/1.0 200 OK\r\n\r\n<html>hello</html>";
            send(connfd, resp, strlen(resp), 0);
            close(connfd);
            exit(0);
        } else if (pid > 0) {
            //printf("in parent process\n");
            close(connfd);
            continue;
        }
    }
    
    close(listenfd);
    return 0;
}

单并发

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 1 http://localhost:8080/
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            8080

Document Path:          /
Document Length:        18 bytes

Concurrency Level:      1
Time taken for tests:   41.408 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      3700000 bytes
HTML transferred:       1800000 bytes
Requests per second:    2415.02 [#/sec] (mean)
Time per request:       0.414 [ms] (mean)
Time per request:       0.414 [ms] (mean, across all concurrent requests)
Transfer rate:          87.26 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:     0    0   0.1      0       9
Waiting:        0    0   0.1      0       9
Total:          0    0   0.1      0       9

Percentage of the requests served within a certain time (ms)
  50%      0
  66%      0
  75%      0
  80%      0
  90%      0
  95%      0
  98%      0
  99%      0
 100%      9 (longest request)
linux:~/ab/httpd-2.4.3/support$

10 个并发

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 10 http://localhost:8080/
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            8080

Document Path:          /
Document Length:        18 bytes

Concurrency Level:      10
Time taken for tests:   39.517 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      3700000 bytes
HTML transferred:       1800000 bytes
Requests per second:    2530.53 [#/sec] (mean)
Time per request:       3.952 [ms] (mean)
Time per request:       0.395 [ms] (mean, across all concurrent requests)
Transfer rate:          91.44 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       3
Processing:     0    4   0.4      4      64
Waiting:        0    4   0.4      4      64
Total:          1    4   0.4      4      64

Percentage of the requests served within a certain time (ms)
  50%      4
  66%      4
  75%      4
  80%      4
  90%      4
  95%      4
  98%      4
  99%      4
 100%     64 (longest request)
linux:~/ab/httpd-2.4.3/support$

100 个并发

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 100 http://localhost:8080/
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            8080

Document Path:          /
Document Length:        18 bytes

Concurrency Level:      100
Time taken for tests:   39.980 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      3700000 bytes
HTML transferred:       1800000 bytes
Requests per second:    2501.28 [#/sec] (mean)
Time per request:       39.980 [ms] (mean)
Time per request:       0.400 [ms] (mean, across all concurrent requests)
Transfer rate:          90.38 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.3      0      17
Processing:     0   40   1.7     40     120
Waiting:        0   40   1.7     40     120
Total:          3   40   1.6     40     120

Percentage of the requests served within a certain time (ms)
  50%     40
  66%     40
  75%     41
  80%     41
  90%     41
  95%     42
  98%     43
  99%     44
 100%    120 (longest request)
linux:~/ab/httpd-2.4.3/support$

500 个并发

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 500 http://localhost:8080/
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            8080

Document Path:          /
Document Length:        18 bytes

Concurrency Level:      500
Time taken for tests:   63.222 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      3700000 bytes
HTML transferred:       1800000 bytes
Requests per second:    1581.72 [#/sec] (mean)
Time per request:       316.112 [ms] (mean)
Time per request:       0.632 [ms] (mean, across all concurrent requests)
Transfer rate:          57.15 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0   38 520.0      0   21004
Processing:     3  208 2942.9     51   60202
Waiting:        0  208 2942.9     51   60202
Total:         24  247 3153.2     51   63206

Percentage of the requests served within a certain time (ms)
  50%     51
  66%     52
  75%     52
  80%     52
  90%     53
  95%     54
  98%     55
  99%     61
 100%  63206 (longest request)
linux:~/ab/httpd-2.4.3/support$

1000 个并发

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 1000 http://localhost:8080/
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            8080

Document Path:          /
Document Length:        18 bytes

Concurrency Level:      1000
Time taken for tests:   71.625 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      3700000 bytes
HTML transferred:       1800000 bytes
Requests per second:    1396.16 [#/sec] (mean)
Time per request:       716.252 [ms] (mean)
Time per request:       0.716 [ms] (mean, across all concurrent requests)
Transfer rate:          50.45 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0   68 628.8      0   21002
Processing:     1  442 4622.4     51   68587
Waiting:        0  442 4622.3     51   68587
Total:         17  510 4934.8     51   71588

Percentage of the requests served within a certain time (ms)
  50%     51
  66%     52
  75%     52
  80%     52
  90%     54
  95%     55
  98%     59
  99%   4107
 100%  71588 (longest request)
linux:~/ab/httpd-2.4.3/support$

再来是多线程

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>
#include <unistd.h>
#include <pthread.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>

#define BUFFER_SIZE 4096

int tcpSrv(int listen_port);
void* process_thread(void* sockfd);

int main(int argc, char** argv) {
    tcpSrv(8080);
    return 0;
}

void* process_thread(void* sockfd) {
    //printf("thread created\n");
    int connfd = *((int *)sockfd);
    char buff[BUFFER_SIZE+1];
    memset(&buff, 0, sizeof(buff));
    recv(connfd, buff, BUFFER_SIZE, 0);
    //printf("recv msg:\n%s\n", buff);
    char resp[] = "HTTP/1.0 200 OK\r\n\r\n<html>hello</html>";
    send(connfd, resp, strlen(resp), 0);
    close(connfd);
    //printf("thread about to exit\n");
    pthread_exit(NULL);
}

int tcpSrv(int listen_port) {
    int listenfd, connfd;
    struct sockaddr_in servaddr;

    if ((listenfd = socket(AF_INET, SOCK_STREAM, 0)) == -1) {
        printf("create socket error: %s(errno: %d)\n", strerror(errno), errno);
        return -1;
    }

    memset(&servaddr, 0, sizeof(servaddr));
    servaddr.sin_family = AF_INET;
    servaddr.sin_addr.s_addr = htonl(INADDR_ANY);
    servaddr.sin_port = htons(listen_port);

    if (bind(listenfd, (struct sockaddr *)&servaddr, sizeof(servaddr)) == -1) {
        printf("bind socket error: %s(errno: %d)\n", strerror(errno), errno);
        close(listenfd);
        return -1;
    }

    if (listen(listenfd, 2048) == -1) {
        printf("listen socket error: %s(errno: %d)\n", strerror(errno), errno);
        close(listenfd);
        return -1;
    }

    printf("-----waiting for client's request-----\n");
    while (1) {
        if ((connfd = accept(listenfd, (struct sockaddr *)NULL, NULL)) == -1) {
            printf("accept socket error: %s(errno: %d)\n", strerror(errno), errno);
            close(listenfd);
            return -1;
        }
        //printf("main thread about to create thread\n");
        pthread_t tid;
        pthread_attr_t opt;
        pthread_attr_init(&opt);
        pthread_attr_setdetachstate(&opt, PTHREAD_CREATE_DETACHED);
        if (pthread_create(&tid, &opt, process_thread, &connfd) != 0) {
            printf("error line: %d\t", __LINE__);
            perror(__FILE__);
            return -1;
        }
        pthread_attr_destroy(&opt);
        //printf("main thread finish creating thread\n");
    }
    
    close(listenfd);
    return 0;
}

单并发

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 1 http://localhost:8080/123
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            8080

Document Path:          /123
Document Length:        18 bytes

Concurrency Level:      1
Time taken for tests:   7.629 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      3700000 bytes
HTML transferred:       1800000 bytes
Requests per second:    13108.37 [#/sec] (mean)
Time per request:       0.076 [ms] (mean)
Time per request:       0.076 [ms] (mean, across all concurrent requests)
Transfer rate:          473.64 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       4
Processing:     0    0   0.1      0      24
Waiting:        0    0   0.1      0      23
Total:          0    0   0.1      0      24

Percentage of the requests served within a certain time (ms)
  50%      0
  66%      0
  75%      0
  80%      0
  90%      0
  95%      0
  98%      0
  99%      0
 100%     24 (longest request)
linux:~/ab/httpd-2.4.3/support$

10 个并发

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 10 http://localhost:8080/123
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
apr_pollset_poll: The timeout specified has expired (70007)
Total of 99991 requests completed
linux:~/ab/httpd-2.4.3/support$

至于 select 和 poll 就没有写了，直接写 epoll 模型

#include <iostream>
#include <sys/socket.h>
#include <sys/epoll.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <fcntl.h>
#include <unistd.h>
#include <stdio.h>
#include <errno.h>

using namespace std;

#define MAX_LINE 1024
#define OPEN_MAX 100
#define LISTENQ 1024
#define SERV_PORT 8080
#define INFTIM 1000
#define MAX_EVENTS 1024
#define EP_TIMEOUT 500

int tcpSrv(int listen_port);
int set_non_block(int fd);

int main(int argc, char* argv[]) {
    tcpSrv(SERV_PORT);
    return 0;
}

int set_non_block(int fd) {
    int fd_fl = fcntl(fd, F_GETFL); 
    fd_fl |= O_NONBLOCK;
    if (fcntl(fd, F_SETFL, fd_fl) < 0) {
        printf("%d\n:", __LINE__);perror(__FILE__);
        return -1;
    }
    return 0;
}

int tcpSrv(int listen_port) {
    int listenfd = socket(AF_INET, SOCK_STREAM, 0);
    if (-1 == listenfd) {
        printf("%d\n:", __LINE__);perror(__FILE__);
        return -1;
    }
    set_non_block(listenfd);

    struct epoll_event ev;
    ev.data.fd = listenfd;
    ev.events = EPOLLIN|EPOLLET;
    int epfd = epoll_create(4096);
    epoll_ctl(epfd, EPOLL_CTL_ADD, listenfd, &ev);

    struct sockaddr_in serveraddr;
    bzero(&serveraddr, sizeof(serveraddr));
    serveraddr.sin_family = AF_INET;
    char* local_addr = "127.0.0.1";
    inet_aton(local_addr, &(serveraddr.sin_addr));
    serveraddr.sin_port = htons(listen_port);

    if (bind(listenfd, (sockaddr*)&serveraddr, sizeof(serveraddr)) == -1) {
        printf("%d\n:", __LINE__);perror(__FILE__);
        close(listenfd);
        return -1;
    }
    
    if (listen(listenfd, LISTENQ) == -1) {
        printf("%d\n:", __LINE__);perror(__FILE__);
        close(listenfd);
        return -1;
    }

    struct epoll_event events[MAX_EVENTS];
    //struct sockaddr_in clientaddr;
    //socklen_t clilen;
    while (1) {
        int nfds = epoll_wait(epfd, events, MAX_EVENTS, EP_TIMEOUT);
        //printf("events count:%d\n", nfds);
        for (int i=0; i<nfds; i++) {
            if (events[i].data.fd == listenfd) {
                while (1) {
                    int connfd = accept(listenfd, NULL, NULL);
                    if (connfd < 0) {
                        //printf("%d\n:", __LINE__);perror(__FILE__);
                        break;
                    }
                    //char *str_client_addr = inet_ntoa(clientaddr.sin_addr);
                    //cout<<"accept a connection from: "<<str_client_addr<<endl;
                    ev.data.fd = connfd;
                    ev.events = EPOLLIN | EPOLLET;
                    epoll_ctl(epfd, EPOLL_CTL_ADD, connfd, &ev);
                }
            } else if (events[i].events & EPOLLIN) {
                //cout<<"EPOLLIN"<<endl;
                int sockfd = events[i].data.fd;
                if (sockfd < 0) {
                    printf("%d\n:", __LINE__);perror(__FILE__);
                    continue;
                }
                char buffer[MAX_LINE+1];
                memset(buffer, 0, sizeof(buffer));
                read(sockfd, buffer, MAX_LINE);
                //printf("%s\n", buffer);
                set_non_block(sockfd);
                char html_resp[] = "HTTP/1.1 200 OK\r\n\r\n<html>epoll</html>";
                write(sockfd, html_resp, strlen(html_resp));
                epoll_ctl(epfd, EPOLL_CTL_DEL, sockfd, NULL);
                close(sockfd);
            }
        }
    }
    
    return 0;
}

单并发

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 1 http://localhost:8080/123
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            8080

Document Path:          /123
Document Length:        18 bytes

Concurrency Level:      1
Time taken for tests:   7.474 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      3700000 bytes
HTML transferred:       1800000 bytes
Requests per second:    13380.33 [#/sec] (mean)
Time per request:       0.075 [ms] (mean)
Time per request:       0.075 [ms] (mean, across all concurrent requests)
Transfer rate:          483.47 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       4
Processing:     0    0   0.0      0       5
Waiting:        0    0   0.0      0       5
Total:          0    0   0.0      0       5

Percentage of the requests served within a certain time (ms)
  50%      0
  66%      0
  75%      0
  80%      0
  90%      0
  95%      0
  98%      0
  99%      0
 100%      5 (longest request)

10 个并发

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 10 http://localhost:8080/123
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            8080

Document Path:          /123
Document Length:        18 bytes

Concurrency Level:      10
Time taken for tests:   5.991 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      3700000 bytes
HTML transferred:       1800000 bytes
Requests per second:    16691.90 [#/sec] (mean)
Time per request:       0.599 [ms] (mean)
Time per request:       0.060 [ms] (mean, across all concurrent requests)
Transfer rate:          603.13 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       6
Processing:     0    0   0.1      0       7
Waiting:        0    0   0.1      0       6
Total:          0    1   0.2      1       7

Percentage of the requests served within a certain time (ms)
  50%      1
  66%      1
  75%      1
  80%      1
  90%      1
  95%      1
  98%      1
  99%      1
 100%      7 (longest request)

100 个并发

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 100 http://localhost:8080/123
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            8080

Document Path:          /123
Document Length:        18 bytes

Concurrency Level:      100
Time taken for tests:   6.233 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      3700000 bytes
HTML transferred:       1800000 bytes
Requests per second:    16043.99 [#/sec] (mean)
Time per request:       6.233 [ms] (mean)
Time per request:       0.062 [ms] (mean, across all concurrent requests)
Transfer rate:          579.71 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    1   1.0      1       9
Processing:     1    5   0.9      5      12
Waiting:        1    2   0.9      2      10
Total:          2    6   1.4      6      15

Percentage of the requests served within a certain time (ms)
  50%      6
  66%      7
  75%      7
  80%      7
  90%      8
  95%      8
  98%     10
  99%     10
 100%     15 (longest request)

1000 个并发

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 1000 http://localhost:8080/123
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            8080

Document Path:          /123
Document Length:        18 bytes

Concurrency Level:      1000
Time taken for tests:   7.099 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      3700000 bytes
HTML transferred:       1800000 bytes
Requests per second:    14087.17 [#/sec] (mean)
Time per request:       70.987 [ms] (mean)
Time per request:       0.071 [ms] (mean, across all concurrent requests)
Transfer rate:          509.01 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        1   26 115.1     19    3054
Processing:    16   44   9.7     43      76
Waiting:        1   18   8.0     18      54
Total:         20   70 115.7     65    3103

Percentage of the requests served within a certain time (ms)
  50%     65
  66%     73
  75%     79
  80%     82
  90%     89
  95%     95
  98%    101
  99%    106
 100%   3103 (longest request)
linux:~/ab/httpd-2.4.3/support$

可以看到 epoll 确实是很猛的，这里有一个地方需要注意，我调了两天，就是在 epoll 的 EPOLLIN 事件发生在 listenfd 上的时候，这个时候需要去 accept，这个应该都没有什么问题，但是我一开始只是 accept 了一个，而实际上，这个时候，是可能有多个连接在等待 accept 的，于是就导致了并发能力很不行，正确的做法应该是一直 accept，直到没有东西可以 accept 了，这样才能有并发能力。

在 HTTP 协议下分段下载文件

Thu, 04 Oct 2012 22:15:10 +0800

最近想要搞一个 linux 下的多线程下载工具，就像 wget 那样可以，只不过是多线程的，其中遇到一个问题，就是 http 如何分段下载，查找资料最后看到这里，http://www.rosoo.net/a/201006/9570.html

假如一个文件有1000个字节，那么其范围就是0-999，则： Range: bytes=500- 表示读取该文件的500-999字节，共500字节。 Range: bytes=500-599 表示读取该文件的500-599字节，共100字节。 Range还有其它几种写法，但上面这两种是最常用的，对于断点续传也足矣了。如果HTTP请求中包含Range字段，那么服务器会返回206（Partial Content），同时HTTP头中也会有一个相应的Content-Range字段，类似下面的格式： Content-Range: bytes 500-999/1000 Content-Range字段说明服务器返回了文件的某个范围及文件的总长度。这时Content-Length字段就不是整个文件的大小了，而是对应文件这个范围的字节数，这一点一定要注意。

他原文是用来做断点续传的，不过用来做多线程下载也是一样的道理

Linux C 多线程初步

Tue, 04 Sep 2012 22:07:23 +0800

晚上想起来要敲一个 C 的多线程，于是搜了一下，找到这篇，http://zhuwenlong.blog.51cto.com/blog/209020/40339

#include <stdio.h>
#include <pthread.h>
#include <string.h>
#include <stdlib.h>
#include <sys/time.h>

#define MAX 10

pthread_t thread[2];
pthread_mutex_t mut;

int number=0, i;

void *thread1() {
    printf("thread1n");
    for (i=0; i<MAX; i++) {
        printf("thread1, number = %dn", number);
        pthread_mutex_lock(&mut);
        number++;
        pthread_mutex_unlock(&mut);
        sleep(2);
    }
    printf("thread1, is main processing waiting for me?n");
    pthread_exit(NULL);
}

void *thread2() {
    printf("thread2n");
    for (i=0; i<MAX; i++) {
        printf("thread2, number = %dn", number);
        pthread_mutex_lock(&mut);
        number++;
        pthread_mutex_unlock(&mut);
        sleep(3);
    }
    printf("thread2, is main processing waiting for me?n");
    pthread_exit(NULL);
}

void thread_create() {
    int temp;
    memset(&thread, 0, sizeof(thread));
    if ((temp = pthread_create(&thread[0], NULL, thread1, NULL)) != 0) {
        printf("create threadh 1 failedn");
    } else {
        printf("successfully create thread 1n");
    }

    if ((temp = pthread_create(&thread[1], NULL, thread2, NULL)) != 0) {
        printf("create threadh 2 failedn");
    } else {
        printf("successfully create thread 2n");
    }
}

void thread_wait() {
    if (thread[0] != 0) {
        pthread_join(thread[0], NULL);
        printf("thread 1 endn");
    }
    if (thread[1] != 0) {
        pthread_join(thread[1], NULL);
        printf("thread 2 endn");
    }
}

int main() {
    printf("hello worldn");
    pthread_mutex_init(&mut, NULL);
    printf("this is the main processn");
    thread_create();
    printf("main process is waiting for pthread to finishn");
    thread_wait();
    return 0;
}

需要注意的是，编译的时候需要手工链接进程库

gcc -pthread thread.c

后续的解释，直接引用了。。

引文:

线程相关操作

一 pthread_t

pthread_t在头文件/usr/include/bits/pthreadtypes.h中定义：

typedef unsigned long int pthread_t;

它是一个线程的标识符。

二 pthread_create

函数pthread_create用来创建一个线程，它的原型为：

extern int pthread_create __P ((pthread_t *__thread, __const pthread_attr_t *__attr,

void *(*__start_routine) (void *), void *__arg));

第一个参数为指向线程标识符的指针，第二个参数用来设置线程属性，第三个参数是线程运行函数的起始地址，最后一个参数是运行函数的参数。这里，我们的函数thread不需要参数，所以最后一个参数设为空指针。第二个参数我们也设为空指针，这样将生成默认属性的线程。对线程属性的设定和修改我们将在下一节阐述。当创建线程成功时，函数返回0，若不为0则说明创建线程失败，常见的错误返回代码为EAGAIN和EINVAL。前者表示系统限制创建新的线程，例如线程数目过多了；后者表示第二个参数代表的线程属性值非法。创建线程成功后，新创建的线程则运行参数三和参数四确定的函数，原来的线程则继续运行下一行代码。

三 pthread_join pthread_exit

函数pthread_join用来等待一个线程的结束。函数原型为：

extern int pthread_join __P ((pthread_t __th, void **__thread_return));

第一个参数为被等待的线程标识符，第二个参数为一个用户定义的指针，它可以用来存储被等待线程的返回值。这个函数是一个线程阻塞的函数，调用它的函数将一直等待到被等待的线程结束为止，当函数返回时，被等待线程的资源被收回。一个线程的结束有两种途径，一种是象我们上面的例子一样，函数结束了，调用它的线程也就结束了；另一种方式是通过函数pthread_exit来实现。它的函数原型为：

extern void pthread_exit __P ((void *__retval)) __attribute__ ((__noreturn__));

唯一的参数是函数的返回代码，只要pthread_join中的第二个参数thread_return不是NULL，这个值将被传递给 thread_return。最后要说明的是，一个线程不能被多个线程等待，否则第一个接收到信号的线程成功返回，其余调用pthread_join的线程则返回错误代码ESRCH。

在这一节里，我们编写了一个最简单的线程，并掌握了最常用的三个函数pthread_create，pthread_join和pthread_exit。下面，我们来了解线程的一些常用属性以及如何设置这些属性。

互斥锁相关

互斥锁用来保证一段时间内只有一个线程在执行一段代码。

一 pthread_mutex_init

函数pthread_mutex_init用来生成一个互斥锁。NULL参数表明使用默认属性。如果需要声明特定属性的互斥锁，须调用函数 pthread_mutexattr_init。函数pthread_mutexattr_setpshared和函数 pthread_mutexattr_settype用来设置互斥锁属性。前一个函数设置属性pshared，它有两个取值， PTHREAD_PROCESS_PRIVATE和PTHREAD_PROCESS_SHARED。前者用来不同进程中的线程同步，后者用于同步本进程的不同线程。在上面的例子中，我们使用的是默认属性PTHREAD_PROCESS_ PRIVATE。后者用来设置互斥锁类型，可选的类型有PTHREAD_MUTEX_NORMAL、PTHREAD_MUTEX_ERRORCHECK、 PTHREAD_MUTEX_RECURSIVE和PTHREAD _MUTEX_DEFAULT。它们分别定义了不同的上所、解锁机制，一般情况下，选用最后一个默认属性。

二 pthread_mutex_lock pthread_mutex_unlock pthread_delay_np

pthread_mutex_lock声明开始用互斥锁上锁，此后的代码直至调用pthread_mutex_unlock为止，均被上锁，即同一时间只能被一个线程调用执行。当一个线程执行到pthread_mutex_lock处时，如果该锁此时被另一个线程使用，那此线程被阻塞，即程序将等待到另一个线程释放此互斥锁。

注意：

1 需要说明的是，上面的两处sleep不光是为了演示的需要，也是为了让线程睡眠一段时间，让线程释放互斥锁，等待另一个线程使用此锁。下面的参考资料1里头说明了该问题。但是在linux下好像没有pthread_delay_np那个函数（我试了一下，提示没有定义该函数的引用），所以我用了sleep来代替，不过参考资料2中给出另一种方法，好像是通过pthread_cond_timedwait来代替，里头给出了一种实现的办法。

2 请千万要注意里头的注释comment1-5，那是我花了几个小时才找出的问题所在。

如果没有comment1和comment4,comment5,将导致在pthread_join的时候出现段错误，另外，上面的comment2和comment3是根源所在，所以千万要记得写全代码。因为上面的线程可能没有创建成功，导致下面不可能等到那个线程结束，而在用pthread_join的时候出现段错误（访问了未知的内存区）。另外，在使用memset的时候，需要包含string.h头文件哦

==================================================

刚刚需要用的时候，发现上面的代码不能跑，神奇，我当时是怎么出结果的，下面的代码可以跑

#include <stdio.h>
#include <pthread.h>
#include <string.h>
#include <stdlib.h>
#include <sys/time.h>
#include <unistd.h>

#define MAX 10

pthread_t thread[2];
pthread_mutex_t mut;

int number=0, i;

void *thread1(void *) {
    printf("thread1\n");
    for (i=0; i<MAX; i++) {
        printf("thread1, number = %d\n", number);
        pthread_mutex_lock(&mut);
        number++;
        pthread_mutex_unlock(&mut);
        sleep(2);
    }
    printf("thread1, is main processing waiting for me?\n");
    pthread_exit(NULL);
}

void *thread2(void *) {
    printf("thread2\n");
    for (i=0; i<MAX; i++) {
        printf("thread2, number = %d\n", number);
        pthread_mutex_lock(&mut);
        number++;
        pthread_mutex_unlock(&mut);
        sleep(3);
    }
    printf("thread2, is main processing waiting for me?\n");
    pthread_exit(NULL);
}

void thread_create() {
    int temp;
    memset(&thread, 0, sizeof(thread));
    if ((temp = pthread_create(&thread[0], NULL, thread1, NULL)) != 0) {
        printf("create threadh 1 failed\n");
    } else {
        printf("successfully create thread 1\n");
    }

    if ((temp = pthread_create(&thread[1], NULL, thread2, NULL)) != 0) {
        printf("create threadh 2 failed\n");
    } else {
        printf("successfully create thread 2\n");
    }
}

void thread_wait() {
    if (thread[0] != 0) {
        pthread_join(thread[0], NULL);
        printf("thread 1 end\n");
    }
    if (thread[1] != 0) {
        pthread_join(thread[1], NULL);
        printf("thread 2 end\n");
    }
}

int main() {
    printf("hello world\n");
    pthread_mutex_init(&mut, NULL);
    printf("this is the main process\n");
    thread_create();
    printf("main process is waiting for pthread to finish\n");
    thread_wait();
    return 0;
}

C# .NET 中停止一个线程

Wed, 29 Aug 2012 16:22:21 +0800

在下载一些文件的时候，网络 IO 可能导致超时，于是想要杀掉那个线程，但是却发现这个不太好做，查了一下，这里说，http://stackoverflow.com/questions/1327102/how-to-kill-a-thread-instantly-in-c

The reason it’s hard to just kill a thread is because the language designers want to avoid the following problem: your thread takes a lock, and then you kill it before it can release it. Now anyone who needs that lock will get stuck.

What you have to do is use some global variable to tell the thread to stop. You have to manually, in your thread code, check that global variable and return if you see it indicates you should stop.

原来是这样。。

C# 状态栏更新不及时的问题

Wed, 29 Aug 2012 10:27:58 +0800

根据这里，http://social.msdn.microsoft.com/Forums/zh-CN/visualcshartzhchs/thread/a9bdac04-c536-4fdb-97b7-96e7732ee140/，根本问题是线程忙于跑循环，所以没有能及时来绘制 UI，所以解决方法自然就是多线程绘制 UI，他给了几个链接，不过这个不是目前最亟待解决的，所以先放一放

因为循环跑太快了，UI来不急更新画面，UI线程忙着处理循环内的事，虽然可以使用DoEvents让UI更新，但这并不是正确的方式，若你的专案里有多个大量工作要执行，再强大的硬件，资源也会被耗尽

http://www.dotblogs.com.tw/yc421206/archive/2008/10/28/5795.aspx

循环内饰属于大量的工作，又必须更新UI线程，这时你应该学会用多线程，由其他线程去更新UI线程，UI更新的速度越快表示用掉的资源越多

http://www.dotblogs.com.tw/yc421206/archive/2009/02/13/7141.aspx

http://www.dotblogs.com.tw/yc421206/archive/2009/02/15/7174.aspx

Python 多线程初学遇到的输出混叠问题

Fri, 03 Aug 2012 21:45:44 +0800

照着书上敲了一个多线程的例子：

#!/usr/bin/env python

import thread
from time import sleep, ctime

loops = [4, 2]

def loop(nloop, nsec, lock):
    print 'start loop', nloop, 'at:', ctime()
    sleep(nsec)
    print 'loop', nloop, 'done at:', ctime()
    lock.release()

def main():
    print 'starting at:', ctime()
    locks = []
    nloops = range(len(loops))

    for i in nloops:
        lock = thread.allocate_lock()
        lock.acquire()
        locks.append(lock)

    for i in nloops:
        thread.start_new_thread(loop,
                    (i, loops[i], locks[i]))

    for i in nloops:
        while locks[i].locked():
            pass

    print 'all done at:', ctime()

if __name__ == '__main__':
    main()

这里解释一下 21 行，accquire 方法的调用表示把“把锁锁上”，下面的 start 循环就是把准备好的线程一个个启动了，再下面那个的那个 pass 循环，是在“等待所有的锁解开”，这里需要注意的是，他是从第一个锁开始“等待”的，也就是说，就算后面的锁都解了，但是第一个还锁着，他也不知道，也不管的，简单想一下就明白这样处理是没问题的，因为总要等到所有的锁都解开才能跑后续的代码嘛

但是问题出现了，在输出的时候，得到这样的输出

starting at: Fri Aug 03 21:38:55 2012
start loopstart loop  01  at:at:  Fri Aug 03 21:38:55 2012Fri Aug 03 21:38:55 2012

loop 1 done at: Fri Aug 03 21:38:57 2012
loop 0 done at: Fri Aug 03 21:39:03 2012
all done at: Fri Aug 03 21:39:03 2012

正如所见，两个线程开始的那个打印语句冲突了，在同一行输出，这应该是一个多线程对输出设备写操作上的问题，怎么解决呢，自己刚学多线程，完全没有思路，于是上 stackoverflow 去问，看到别人这个问题的答案 http://stackoverflow.com/questions/1501651/log-output-of-multiprocessing-process ，

The easiest way might be to just override sys.stdout.
Slightly modifying an example from the multiprocessing
manual:

from multiprocessing import Process
import os
import sys

def info(title):
    print title
    print 'module name:', __name__
    print 'parent process:', os.getppid()
    print 'process id:', os.getpid()

def f(name):
    sys.stdout = open(str(os.getpid()) + ".out", "w")
    info('function f')
    print 'hello', name

if __name__ == '__main__':
    p = Process(target=f, args=('bob',))
    p.start()
    q = Process(target=f, args=('fred',))
    q.start()
    p.join()
    q.join()
And running it:

$ ls
m.py
$ python m.py
$ ls
27493.out  27494.out  m.py
$ cat 27493.out
function f
module name: __main__
parent process: 27492
process id: 27493
hello bob
$ cat 27494.out
function f
module name: __main__
parent process: 27492
process id: 27494
hello fred

看起来，他是重载了输出文件，这个比较高深，暂时看不懂，而且暂时也没有深入的涉及到 python 多线程的应用，所以这个问题暂时放着吧，等以后需要的时候回头来看

历史评论

Gecko (2013-08-16 12:33:17):

解决python多线程输出重叠问题，可以从输出入手，一般输出采用的是print函数，Print函数是可以被线程打断，如果不被打断的话就不会出现重叠的情况。可以采用sys.stdout.write来解决。

ZRJ (2013-08-20 10:35:28):

嗯，这个问题后来就放下没管了，下次要用的时候可以试试这个，学习了，thanks!