Http on ZRJ | 学习笔记

不同服务器端模型在并发请求下的表现差异

Mon, 25 Mar 2013 11:22:04 +0800

一般来说，服务器的模型有下面几种：

单进程单线程
多进程（一个请求对应一个进程，Apache 之类貌似就是这个）
多线程（一个请求对应一个线程，这个挺少见到的）
select / poll
epoll

它们的差别可以看这个，http://www.cnblogs.com/sharra/archive/2010/12/30/1921287.html

因为需要了解底层设备访问的原理，所以惯用高层应用语言的我，需要了解一下Linux的设备访问机制，尤其是处理一组非阻塞IO的原理方法，标准的术语好像是叫多路复用。以下文章部分句子有引用之处，恕没有一一指出出处。对于接触过Linux内核或设备驱动开发的读者，一定清楚poll和select系统调用，以及从2.5版本引入的epoll机制（epoll机制包含三个系统调用）。网上关于它们的文章，有说用法的，甚为详细，更有分析源代码的，又比较深入，且枝节颇多。经过几篇文章的阅读，我把觉得比较核心的东西写下来吧。我的用意是尽可能以简单的概念，比对他们三者的异同。几经查找我才确定下来，poll和select应该被归类为这样的系统调用，它们可以阻塞地同时探测一组支持非阻塞的IO设备，是否有事件发生（如可读，可写，有高优先级的错误输出，出现错误等等），直至某一个设备触发了事件或者超过了指定的等待时间——也就是它们的职责不是做IO，而是帮助调用者寻找当前就绪的设备。同类型的产品是Windows的IOCP，它也是处理多路复用，只是把IO和探测封装在了一起了。准备的知识有两点：1、fd；2、op->poll。在Linux里面，设备都被抽象为文件，一系列的设备文件就有自己独立的虚拟文件系统，所以，设备在系统调用参数中的表示就是file description。fd其实就是一个整数（特别地，标准输入，输出，错误输出分别对应的fd是0，1，2）。与内核打交道的时候，传递整数的fd可以在自己的文件系统中作进一步的检查是否合法，如果只是返回指针就不能这样操作了，毕竟指针是无差别无意义的。通过fd访问file，通过file可以访问其fileOperator，这里面我们要关心的一个fileOp就是poll。因为系统调用poll和select，就是依靠这个文件操作poll实现的。poll文件操作有两个参数，一个是文件本身，一个可以看做是当设备尚未就绪时调用的回调函数，这个函数是把设备自己特有的等待队列传给内核，让内核把当前的进程挂载到其中（因为当设备就绪时，设备就应该去唤醒在自己特有等待队列中的所有节点，这样当前进程就获取了完成的信号了）。poll文件操作返回的必须是一组标准的掩码，其中的各个位指示当前的不同的就绪状态（全0为没有任何事件触发）。再谈谈早期多路复用的版本poll和select。本质而言，poll和select的共同点就是，对全部指定设备做一次poll，当然这往往都是还没有就绪的，那就会通过回调函数把当前进程注册到设备的等待队列，如果所有设备返回的掩码都没有显示任何的事件触发，就去掉回调函数的函数指针，进入有限时的睡眠状态，再恢复和不断做poll，再作有限时的睡眠，直到其中一个设备有事件触发为止。只要有事件触发，系统调用返回，回到用户态，用户就可以对相关的fd作进一步的读或者写操作了。当然，这个时候还不是所有的设备都就绪的喔，那就得不断地poll或者select了，而做一次这样的系统调用都得轮询所有的设备，次数是设备数*（睡眠次数-1），也就是时间复杂度是O（n），还得做几次O（n）呢。可见，对于现在普遍的服务器程序，需要同时并发监听数千个连接，并且连接需要重复使用的情况，poll和select就存在这样的性能瓶颈。另外，数千个设备fd在每次调用时，都需要将其从用户空间复制到内核空间，这里的开销不可忽略。 poll和select放在一起，是因为其机制一致，而参数和数据结构就略有不同。select一次性传入三组作用于不同信道的设备fd，分别是输入，输出和错误异常。各组的fd期待各组所特有的，由代码指定的一组事件，如输入信道期待输入就绪，输入挂起和错误等事件。然后，select就挑选调用者关心的fd做poll文件操作，检测返回的掩码，看看是否有fd所属信道感兴趣的事件，比如看看这个属于输出信道的fd有没有输出就绪等一系列的事件发生，一样地，如果有一个fd发生感兴趣事件就返回调用了。select，为了同时处理三组使用不同的事件判断规则的fd，采用了位图的方式表示，一组一个位图，位长度是当中最大的fd值，上限是1024，三组就是3072，而且这还只是传入的位图，还有一样大小的传出的位图。当fd数越来越多时，所需的存储开销比较大。既然，一组fd处理起来比较粗放，那就各个fd自己准备好了。poll()系统调用是System V的多元I/O解决方案。它有三个参数，第一个是pollfd结构的数组指针，也就是指向一组fd及其相关信息的指针，因为这个结构包含的除了fd，还有期待的事件掩码和返回的事件掩码，实质上就是将select的中的fd，传入和传出参数归到一个结构之下，也不再把fd分为三组，也不再硬性规定fd感兴趣的事件，这由调用者自己设定。这样，不使用位图来组织数据，也就不需要位图的全部遍历了。按照一般队列地遍历，每个fd做poll文件操作，检查返回的掩码是否有期待的事件，以及做是否有挂起和错误的必要性检查，如果有事件触发，就可以返回调用了。回到poll和select的共同点，面对高并发多连接的应用情境，它们显现出原来没有考虑到的不足，虽然poll比起select又有所改进了。除了上述的关于每次调用都需要做一次从用户空间到内核空间的拷贝，还有这样的问题，就是当处于这样的应用情境时，poll和select会不得不多次操作，并且每次操作都很有可能需要多次进入睡眠状态，也就是多次全部轮询fd，我们应该怎么处理一些会出现重复而无意义的操作。这些重复而无意义的操作有：1、从用户到内核空间拷贝，既然长期监视这几个fd，甚至连期待的事件也不会改变，那拷贝无疑就是重复而无意义的，我们可以让内核长期保存所有需要监视的fd甚至期待事件，或者可以再需要时对部分期待事件进行修改；2、将当前线程轮流加入到每个fd对应设备的等待队列，这样做无非是哪一个设备就绪时能够通知进程退出调用，聪明的开发者想到，那就找个“代理”的回调函数，代替当前进程加入fd的等待队列好了（这也是我后来才总结出来，Linux的等待队列，实质上是回调函数队列吧，也可以使用宏来将当前进程“加入”等待队列，其实就是将唤醒当前进程的回调函数加入队列）。这样，像poll系统调用一样，做poll文件操作发现尚未就绪时，它就调用传入的一个回调函数，这是epoll指定的回调函数，它不再像以前的poll系统调用指定的回调函数那样，而是就将那个“代理”的回调函数加入设备的等待队列就好了，这个代理的回调函数就自己乖乖地等待设备就绪时将它唤醒，然后它就把这个设备fd放到一个指定的地方，同时唤醒可能在等待的进程，到这个指定的地方取fd就好了。我们把1和2结合起来就可以这样做了，只拷贝一次fd，一旦确定了fd就可以做poll文件操作，如果有事件当然好啦，马上就把fd放到指定的地方，而通常都是没有的，那就给这个fd的等待队列加一个回调函数，有事件就自动把fd放到指定的地方，当前进程不需要再一个个poll和睡眠等待了。 epoll机制就是这样改进的了。诚然，fd少的时候，当前进程一个个地等问题不大，可是现在和尚多了，方丈就不好管了。以前设备事件触发时，只负责唤醒当前进程就好了，而当前进程也只能傻傻地在poll里面等待或者循环，再来一次poll，也不知道这个由设备提供的poll性能如何，能不能检查出当前进程已经在等待了就立即返回，当然，我也不明白为什么做了一遍的poll之后，去掉回调函数指针了，还得再做，不是说好了会去唤醒进程的吗？现在就让事件触发回调函数多做一步。本来设备还没就绪就调用一个回调函数了，现在再在这个回调函数里面做一个注册另一个回调函数的操作，目的就是使得设备事件触发多走一步，不仅仅是唤醒当前进程，还要把自己的fd放到指定的地方。就像收本子的班长，以前得一个个学生地去问有没有本子，如果没有，它还得等待一段时间而后又继续问，现在好了，只走一次，如果没有本子，班长就告诉大家去那里交本子，当班长想起要取本子，就去那里看看或者等待一定时间后离开，有本子到了就叫醒他，然后取走。这个道理很简单，就是老师和班干们常说的，大家多做一点工作，我的工作就轻松很多了，尤其是需要管理的东西越来越多时。这种机制或者说模式，我想在Java的FutureTask里面应该也会用到的，一堆在线程池里面跑着的线程（当然这是任务，不是线程，接口是Callable，不是Runnable.run，是Callable.call，它是可以返回结果的），谁先做好就应该先处理呀，可是难道得一个个问吗？干脆就谁好了，谁就按照既定的操作暴露自己，这样FutureTask的get方法就可以马上知道当前最先完成的线程了，就可以取此线程返回结果了。 epoll由三个系统调用组成，分别是epoll_create，epoll_ctl和epoll_wait。epoll_create用于创建和初始化一些内部使用的数据结构；epoll_ctl用于添加，删除或者修改指定的fd及其期待的事件，epoll_wait就是用于等待任何先前指定的fd事件。关于epoll内部的数据结构，我就不能详细了解了。

下面就是分别实现一个 demo，然后用 ab 去压力测试，每回都是 10W 个请求，根据不同的并发量测试

单进程单线程

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>

#define BUFFER_SIZE 4096

int tcpSrv(int listen_port);

int main(int argc, char** argv) {
    tcpSrv(8080);
    return 0;
}

int tcpSrv(int listen_port) {
    int listenfd, connfd;
    struct sockaddr_in servaddr;
    char buff[BUFFER_SIZE+1];
    int n;

    if ((listenfd = socket(AF_INET, SOCK_STREAM, 0)) == -1) {
        printf("create socket error: %s(errno: %d)\n", strerror(errno), errno);
        return -1;
    }

    memset(&servaddr, 0, sizeof(servaddr));
    servaddr.sin_family = AF_INET;
    servaddr.sin_addr.s_addr = htonl(INADDR_ANY);
    servaddr.sin_port = htons(listen_port);

    if (bind(listenfd, (struct sockaddr *)&servaddr, sizeof(servaddr)) == -1) {
        printf("bind socket error: %s(errno: %d)\n", strerror(errno), errno);
        close(listenfd);
        return -1;
    }

    if (listen(listenfd, 2048) == -1) {
        printf("listen socket error: %s(errno: %d)\n", strerror(errno), errno);
        close(listenfd);
        return -1;
    }

    printf("-----waiting for client's request-----\n");
    while (1) {
        if ((connfd = accept(listenfd, (struct sockaddr *)NULL, NULL)) == -1) {
            printf("accept socket error: %s(errno: %d)\n", strerror(errno), errno);
            close(listenfd);
            return -1;
        }
        memset(&buff, 0, sizeof(buff));
        recv(connfd, buff, BUFFER_SIZE, 0);
        //printf("recv msg:\n%s\n", buff);
        char resp[] = "HTTP/1.0 200 OK\r\n\r\n<html>hello</html>";
        send(connfd, resp, strlen(resp), 0);
        close(connfd);
    }
    
    close(listenfd);
    return 0;
}

一开始觉得这种简单粗暴的方式肯定不行，结果发现还别说，这东西的处理能力还是可以的，看测试结果

单并发

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 1 http://localhost:8080/
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            8080

Document Path:          /
Document Length:        18 bytes

Concurrency Level:      1
Time taken for tests:   7.210 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      3700000 bytes
HTML transferred:       1800000 bytes
Requests per second:    13869.50 [#/sec] (mean)
Time per request:       0.072 [ms] (mean)
Time per request:       0.072 [ms] (mean, across all concurrent requests)
Transfer rate:          501.14 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       4
Processing:     0    0   0.0      0       4
Waiting:        0    0   0.0      0       4
Total:          0    0   0.0      0       4

Percentage of the requests served within a certain time (ms)
  50%      0
  66%      0
  75%      0
  80%      0
  90%      0
  95%      0
  98%      0
  99%      0
 100%      4 (longest request)
linux:~/ab/httpd-2.4.3/support$

10 个并发

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 10 http://localhost:8080/
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            8080

Document Path:          /
Document Length:        18 bytes

Concurrency Level:      10
Time taken for tests:   5.498 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      3700000 bytes
HTML transferred:       1800000 bytes
Requests per second:    18186.97 [#/sec] (mean)
Time per request:       0.550 [ms] (mean)
Time per request:       0.055 [ms] (mean, across all concurrent requests)
Transfer rate:          657.15 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       6
Processing:     0    0   0.1      0       6
Waiting:        0    0   0.1      0       6
Total:          0    1   0.1      1       6

Percentage of the requests served within a certain time (ms)
  50%      1
  66%      1
  75%      1
  80%      1
  90%      1
  95%      1
  98%      1
  99%      1
 100%      6 (longest request)
linux:~/ab/httpd-2.4.3/support$

100 个并发

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 100 http://localhost:8080/
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            8080

Document Path:          /
Document Length:        18 bytes

Concurrency Level:      100
Time taken for tests:   5.816 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      3700000 bytes
HTML transferred:       1800000 bytes
Requests per second:    17194.97 [#/sec] (mean)
Time per request:       5.816 [ms] (mean)
Time per request:       0.058 [ms] (mean, across all concurrent requests)
Transfer rate:          621.30 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    1   0.6      1      11
Processing:     1    5   0.8      5      15
Waiting:        0    2   0.6      2      12
Total:          2    6   0.6      6      15

Percentage of the requests served within a certain time (ms)
  50%      6
  66%      6
  75%      6
  80%      6
  90%      6
  95%      6
  98%      7
  99%      7
 100%     15 (longest request)
linux:~/ab/httpd-2.4.3/support$

150 个并发的时候就不行了

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 150 http://localhost:8080/
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
apr_pollset_poll: The timeout specified has expired (70007)
Total of 99956 requests completed
linux:~/ab/httpd-2.4.3/support$

接下来是多进程模型

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>
#include <unistd.h>
#include <signal.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>

#define BUFFER_SIZE 4096

int tcpSrv(int listen_port);

int main(int argc, char** argv) {
    tcpSrv(8080);
    return 0;
}

int tcpSrv(int listen_port) {
    int listenfd, connfd;
    struct sockaddr_in servaddr;
    char buff[BUFFER_SIZE+1];
    int n;

    if ((listenfd = socket(AF_INET, SOCK_STREAM, 0)) == -1) {
        printf("create socket error: %s(errno: %d)\n", strerror(errno), errno);
        return -1;
    }

    memset(&servaddr, 0, sizeof(servaddr));
    servaddr.sin_family = AF_INET;
    servaddr.sin_addr.s_addr = htonl(INADDR_ANY);
    servaddr.sin_port = htons(listen_port);

    if (bind(listenfd, (struct sockaddr *)&servaddr, sizeof(servaddr)) == -1) {
        printf("bind socket error: %s(errno: %d)\n", strerror(errno), errno);
        close(listenfd);
        return -1;
    }

    if (listen(listenfd, 2048) == -1) {
        printf("listen socket error: %s(errno: %d)\n", strerror(errno), errno);
        close(listenfd);
        return -1;
    }

    printf("-----waiting for client's request-----\n");
    while (1) {
        if ((connfd = accept(listenfd, (struct sockaddr *)NULL, NULL)) == -1) {
            printf("accept socket error: %s(errno: %d)\n", strerror(errno), errno);
            close(listenfd);
            return -1;
        }
        signal(SIGCHLD, SIG_IGN);
        pid_t pid = fork();
        if (pid < 0) {
            printf("fork error: %s(errno: %d)\n", strerror(errno), errno);
            close(listenfd);
            return -1;
        } else if (pid == 0) {
            //printf("in child process\n");
            memset(&buff, 0, sizeof(buff));
            recv(connfd, buff, BUFFER_SIZE, 0);
            //printf("recv msg:\n%s\n", buff);
            char resp[] = "HTTP/1.0 200 OK\r\n\r\n<html>hello</html>";
            send(connfd, resp, strlen(resp), 0);
            close(connfd);
            exit(0);
        } else if (pid > 0) {
            //printf("in parent process\n");
            close(connfd);
            continue;
        }
    }
    
    close(listenfd);
    return 0;
}

单并发

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 1 http://localhost:8080/
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            8080

Document Path:          /
Document Length:        18 bytes

Concurrency Level:      1
Time taken for tests:   41.408 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      3700000 bytes
HTML transferred:       1800000 bytes
Requests per second:    2415.02 [#/sec] (mean)
Time per request:       0.414 [ms] (mean)
Time per request:       0.414 [ms] (mean, across all concurrent requests)
Transfer rate:          87.26 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:     0    0   0.1      0       9
Waiting:        0    0   0.1      0       9
Total:          0    0   0.1      0       9

Percentage of the requests served within a certain time (ms)
  50%      0
  66%      0
  75%      0
  80%      0
  90%      0
  95%      0
  98%      0
  99%      0
 100%      9 (longest request)
linux:~/ab/httpd-2.4.3/support$

10 个并发

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 10 http://localhost:8080/
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            8080

Document Path:          /
Document Length:        18 bytes

Concurrency Level:      10
Time taken for tests:   39.517 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      3700000 bytes
HTML transferred:       1800000 bytes
Requests per second:    2530.53 [#/sec] (mean)
Time per request:       3.952 [ms] (mean)
Time per request:       0.395 [ms] (mean, across all concurrent requests)
Transfer rate:          91.44 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       3
Processing:     0    4   0.4      4      64
Waiting:        0    4   0.4      4      64
Total:          1    4   0.4      4      64

Percentage of the requests served within a certain time (ms)
  50%      4
  66%      4
  75%      4
  80%      4
  90%      4
  95%      4
  98%      4
  99%      4
 100%     64 (longest request)
linux:~/ab/httpd-2.4.3/support$

100 个并发

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 100 http://localhost:8080/
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            8080

Document Path:          /
Document Length:        18 bytes

Concurrency Level:      100
Time taken for tests:   39.980 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      3700000 bytes
HTML transferred:       1800000 bytes
Requests per second:    2501.28 [#/sec] (mean)
Time per request:       39.980 [ms] (mean)
Time per request:       0.400 [ms] (mean, across all concurrent requests)
Transfer rate:          90.38 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.3      0      17
Processing:     0   40   1.7     40     120
Waiting:        0   40   1.7     40     120
Total:          3   40   1.6     40     120

Percentage of the requests served within a certain time (ms)
  50%     40
  66%     40
  75%     41
  80%     41
  90%     41
  95%     42
  98%     43
  99%     44
 100%    120 (longest request)
linux:~/ab/httpd-2.4.3/support$

500 个并发

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 500 http://localhost:8080/
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            8080

Document Path:          /
Document Length:        18 bytes

Concurrency Level:      500
Time taken for tests:   63.222 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      3700000 bytes
HTML transferred:       1800000 bytes
Requests per second:    1581.72 [#/sec] (mean)
Time per request:       316.112 [ms] (mean)
Time per request:       0.632 [ms] (mean, across all concurrent requests)
Transfer rate:          57.15 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0   38 520.0      0   21004
Processing:     3  208 2942.9     51   60202
Waiting:        0  208 2942.9     51   60202
Total:         24  247 3153.2     51   63206

Percentage of the requests served within a certain time (ms)
  50%     51
  66%     52
  75%     52
  80%     52
  90%     53
  95%     54
  98%     55
  99%     61
 100%  63206 (longest request)
linux:~/ab/httpd-2.4.3/support$

1000 个并发

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 1000 http://localhost:8080/
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            8080

Document Path:          /
Document Length:        18 bytes

Concurrency Level:      1000
Time taken for tests:   71.625 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      3700000 bytes
HTML transferred:       1800000 bytes
Requests per second:    1396.16 [#/sec] (mean)
Time per request:       716.252 [ms] (mean)
Time per request:       0.716 [ms] (mean, across all concurrent requests)
Transfer rate:          50.45 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0   68 628.8      0   21002
Processing:     1  442 4622.4     51   68587
Waiting:        0  442 4622.3     51   68587
Total:         17  510 4934.8     51   71588

Percentage of the requests served within a certain time (ms)
  50%     51
  66%     52
  75%     52
  80%     52
  90%     54
  95%     55
  98%     59
  99%   4107
 100%  71588 (longest request)
linux:~/ab/httpd-2.4.3/support$

再来是多线程

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>
#include <unistd.h>
#include <pthread.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>

#define BUFFER_SIZE 4096

int tcpSrv(int listen_port);
void* process_thread(void* sockfd);

int main(int argc, char** argv) {
    tcpSrv(8080);
    return 0;
}

void* process_thread(void* sockfd) {
    //printf("thread created\n");
    int connfd = *((int *)sockfd);
    char buff[BUFFER_SIZE+1];
    memset(&buff, 0, sizeof(buff));
    recv(connfd, buff, BUFFER_SIZE, 0);
    //printf("recv msg:\n%s\n", buff);
    char resp[] = "HTTP/1.0 200 OK\r\n\r\n<html>hello</html>";
    send(connfd, resp, strlen(resp), 0);
    close(connfd);
    //printf("thread about to exit\n");
    pthread_exit(NULL);
}

int tcpSrv(int listen_port) {
    int listenfd, connfd;
    struct sockaddr_in servaddr;

    if ((listenfd = socket(AF_INET, SOCK_STREAM, 0)) == -1) {
        printf("create socket error: %s(errno: %d)\n", strerror(errno), errno);
        return -1;
    }

    memset(&servaddr, 0, sizeof(servaddr));
    servaddr.sin_family = AF_INET;
    servaddr.sin_addr.s_addr = htonl(INADDR_ANY);
    servaddr.sin_port = htons(listen_port);

    if (bind(listenfd, (struct sockaddr *)&servaddr, sizeof(servaddr)) == -1) {
        printf("bind socket error: %s(errno: %d)\n", strerror(errno), errno);
        close(listenfd);
        return -1;
    }

    if (listen(listenfd, 2048) == -1) {
        printf("listen socket error: %s(errno: %d)\n", strerror(errno), errno);
        close(listenfd);
        return -1;
    }

    printf("-----waiting for client's request-----\n");
    while (1) {
        if ((connfd = accept(listenfd, (struct sockaddr *)NULL, NULL)) == -1) {
            printf("accept socket error: %s(errno: %d)\n", strerror(errno), errno);
            close(listenfd);
            return -1;
        }
        //printf("main thread about to create thread\n");
        pthread_t tid;
        pthread_attr_t opt;
        pthread_attr_init(&opt);
        pthread_attr_setdetachstate(&opt, PTHREAD_CREATE_DETACHED);
        if (pthread_create(&tid, &opt, process_thread, &connfd) != 0) {
            printf("error line: %d\t", __LINE__);
            perror(__FILE__);
            return -1;
        }
        pthread_attr_destroy(&opt);
        //printf("main thread finish creating thread\n");
    }
    
    close(listenfd);
    return 0;
}

单并发

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 1 http://localhost:8080/123
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            8080

Document Path:          /123
Document Length:        18 bytes

Concurrency Level:      1
Time taken for tests:   7.629 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      3700000 bytes
HTML transferred:       1800000 bytes
Requests per second:    13108.37 [#/sec] (mean)
Time per request:       0.076 [ms] (mean)
Time per request:       0.076 [ms] (mean, across all concurrent requests)
Transfer rate:          473.64 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       4
Processing:     0    0   0.1      0      24
Waiting:        0    0   0.1      0      23
Total:          0    0   0.1      0      24

Percentage of the requests served within a certain time (ms)
  50%      0
  66%      0
  75%      0
  80%      0
  90%      0
  95%      0
  98%      0
  99%      0
 100%     24 (longest request)
linux:~/ab/httpd-2.4.3/support$

10 个并发

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 10 http://localhost:8080/123
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
apr_pollset_poll: The timeout specified has expired (70007)
Total of 99991 requests completed
linux:~/ab/httpd-2.4.3/support$

至于 select 和 poll 就没有写了，直接写 epoll 模型

#include <iostream>
#include <sys/socket.h>
#include <sys/epoll.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <fcntl.h>
#include <unistd.h>
#include <stdio.h>
#include <errno.h>

using namespace std;

#define MAX_LINE 1024
#define OPEN_MAX 100
#define LISTENQ 1024
#define SERV_PORT 8080
#define INFTIM 1000
#define MAX_EVENTS 1024
#define EP_TIMEOUT 500

int tcpSrv(int listen_port);
int set_non_block(int fd);

int main(int argc, char* argv[]) {
    tcpSrv(SERV_PORT);
    return 0;
}

int set_non_block(int fd) {
    int fd_fl = fcntl(fd, F_GETFL); 
    fd_fl |= O_NONBLOCK;
    if (fcntl(fd, F_SETFL, fd_fl) < 0) {
        printf("%d\n:", __LINE__);perror(__FILE__);
        return -1;
    }
    return 0;
}

int tcpSrv(int listen_port) {
    int listenfd = socket(AF_INET, SOCK_STREAM, 0);
    if (-1 == listenfd) {
        printf("%d\n:", __LINE__);perror(__FILE__);
        return -1;
    }
    set_non_block(listenfd);

    struct epoll_event ev;
    ev.data.fd = listenfd;
    ev.events = EPOLLIN|EPOLLET;
    int epfd = epoll_create(4096);
    epoll_ctl(epfd, EPOLL_CTL_ADD, listenfd, &ev);

    struct sockaddr_in serveraddr;
    bzero(&serveraddr, sizeof(serveraddr));
    serveraddr.sin_family = AF_INET;
    char* local_addr = "127.0.0.1";
    inet_aton(local_addr, &(serveraddr.sin_addr));
    serveraddr.sin_port = htons(listen_port);

    if (bind(listenfd, (sockaddr*)&serveraddr, sizeof(serveraddr)) == -1) {
        printf("%d\n:", __LINE__);perror(__FILE__);
        close(listenfd);
        return -1;
    }
    
    if (listen(listenfd, LISTENQ) == -1) {
        printf("%d\n:", __LINE__);perror(__FILE__);
        close(listenfd);
        return -1;
    }

    struct epoll_event events[MAX_EVENTS];
    //struct sockaddr_in clientaddr;
    //socklen_t clilen;
    while (1) {
        int nfds = epoll_wait(epfd, events, MAX_EVENTS, EP_TIMEOUT);
        //printf("events count:%d\n", nfds);
        for (int i=0; i<nfds; i++) {
            if (events[i].data.fd == listenfd) {
                while (1) {
                    int connfd = accept(listenfd, NULL, NULL);
                    if (connfd < 0) {
                        //printf("%d\n:", __LINE__);perror(__FILE__);
                        break;
                    }
                    //char *str_client_addr = inet_ntoa(clientaddr.sin_addr);
                    //cout<<"accept a connection from: "<<str_client_addr<<endl;
                    ev.data.fd = connfd;
                    ev.events = EPOLLIN | EPOLLET;
                    epoll_ctl(epfd, EPOLL_CTL_ADD, connfd, &ev);
                }
            } else if (events[i].events & EPOLLIN) {
                //cout<<"EPOLLIN"<<endl;
                int sockfd = events[i].data.fd;
                if (sockfd < 0) {
                    printf("%d\n:", __LINE__);perror(__FILE__);
                    continue;
                }
                char buffer[MAX_LINE+1];
                memset(buffer, 0, sizeof(buffer));
                read(sockfd, buffer, MAX_LINE);
                //printf("%s\n", buffer);
                set_non_block(sockfd);
                char html_resp[] = "HTTP/1.1 200 OK\r\n\r\n<html>epoll</html>";
                write(sockfd, html_resp, strlen(html_resp));
                epoll_ctl(epfd, EPOLL_CTL_DEL, sockfd, NULL);
                close(sockfd);
            }
        }
    }
    
    return 0;
}

单并发

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 1 http://localhost:8080/123
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            8080

Document Path:          /123
Document Length:        18 bytes

Concurrency Level:      1
Time taken for tests:   7.474 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      3700000 bytes
HTML transferred:       1800000 bytes
Requests per second:    13380.33 [#/sec] (mean)
Time per request:       0.075 [ms] (mean)
Time per request:       0.075 [ms] (mean, across all concurrent requests)
Transfer rate:          483.47 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       4
Processing:     0    0   0.0      0       5
Waiting:        0    0   0.0      0       5
Total:          0    0   0.0      0       5

Percentage of the requests served within a certain time (ms)
  50%      0
  66%      0
  75%      0
  80%      0
  90%      0
  95%      0
  98%      0
  99%      0
 100%      5 (longest request)

10 个并发

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 10 http://localhost:8080/123
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            8080

Document Path:          /123
Document Length:        18 bytes

Concurrency Level:      10
Time taken for tests:   5.991 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      3700000 bytes
HTML transferred:       1800000 bytes
Requests per second:    16691.90 [#/sec] (mean)
Time per request:       0.599 [ms] (mean)
Time per request:       0.060 [ms] (mean, across all concurrent requests)
Transfer rate:          603.13 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       6
Processing:     0    0   0.1      0       7
Waiting:        0    0   0.1      0       6
Total:          0    1   0.2      1       7

Percentage of the requests served within a certain time (ms)
  50%      1
  66%      1
  75%      1
  80%      1
  90%      1
  95%      1
  98%      1
  99%      1
 100%      7 (longest request)

100 个并发

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 100 http://localhost:8080/123
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            8080

Document Path:          /123
Document Length:        18 bytes

Concurrency Level:      100
Time taken for tests:   6.233 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      3700000 bytes
HTML transferred:       1800000 bytes
Requests per second:    16043.99 [#/sec] (mean)
Time per request:       6.233 [ms] (mean)
Time per request:       0.062 [ms] (mean, across all concurrent requests)
Transfer rate:          579.71 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    1   1.0      1       9
Processing:     1    5   0.9      5      12
Waiting:        1    2   0.9      2      10
Total:          2    6   1.4      6      15

Percentage of the requests served within a certain time (ms)
  50%      6
  66%      7
  75%      7
  80%      7
  90%      8
  95%      8
  98%     10
  99%     10
 100%     15 (longest request)

1000 个并发

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 1000 http://localhost:8080/123
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            8080

Document Path:          /123
Document Length:        18 bytes

Concurrency Level:      1000
Time taken for tests:   7.099 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      3700000 bytes
HTML transferred:       1800000 bytes
Requests per second:    14087.17 [#/sec] (mean)
Time per request:       70.987 [ms] (mean)
Time per request:       0.071 [ms] (mean, across all concurrent requests)
Transfer rate:          509.01 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        1   26 115.1     19    3054
Processing:    16   44   9.7     43      76
Waiting:        1   18   8.0     18      54
Total:         20   70 115.7     65    3103

Percentage of the requests served within a certain time (ms)
  50%     65
  66%     73
  75%     79
  80%     82
  90%     89
  95%     95
  98%    101
  99%    106
 100%   3103 (longest request)
linux:~/ab/httpd-2.4.3/support$

可以看到 epoll 确实是很猛的，这里有一个地方需要注意，我调了两天，就是在 epoll 的 EPOLLIN 事件发生在 listenfd 上的时候，这个时候需要去 accept，这个应该都没有什么问题，但是我一开始只是 accept 了一个，而实际上，这个时候，是可能有多个连接在等待 accept 的，于是就导致了并发能力很不行，正确的做法应该是一直 accept，直到没有东西可以 accept 了，这样才能有并发能力。

python http head 工具

Thu, 18 Oct 2012 21:00:21 +0800

群上闲聊，说道自己封装了一个 python 的 http head 工具

#!/usr/bin/env python

from socket import *

while True:
    HOST = raw_input('> ')
    if not HOST:
        break
    tcpCliSock = socket(AF_INET, SOCK_STREAM)
    PORT = 80
    BUFSIZ = 1024
    ADDR = (HOST, PORT)
    tcpCliSock.connect(ADDR)
    data = "HEAD / HTTP/1.1rnHost: %srnrn" % HOST
    tcpCliSock.send(data)
    data = tcpCliSock.recv(BUFSIZ)
    if not data:
        break
    print data
tcpCliSock.close()

输入输出

C:UsersZRJDesktop>python head
> 360.cn
HTTP/1.1 200 OK
Date: Thu, 18 Oct 2012 12:56:35 GMT
Server: Apache
P3P: CP="CAO DSP COR CUR ADM DEV TAI PSA PSD IVAi IVDi CONi TELo OTPi OUR DELi S
AMi OTRi UNRi PUBi IND PHY ONL UNI PUR FIN COM NAV INT DEM CNT STA POL HEA PRE G
OV"
Set-Cookie: B=ID=809401350564995:V=2:S=236f7e7338; path=/; expires=Sun, 17-Jun-2
038 23:59:59 GMT; domain=.360.cn
Set-Cookie: UT=T=1350564995:F=MzYwLmNu:S=1; path=/; domain=.360.cn
Vary: Accept-Encoding
Connection: close
Content-Type: text/html

在 HTTP 协议下分段下载文件

Thu, 04 Oct 2012 22:15:10 +0800

最近想要搞一个 linux 下的多线程下载工具，就像 wget 那样可以，只不过是多线程的，其中遇到一个问题，就是 http 如何分段下载，查找资料最后看到这里，http://www.rosoo.net/a/201006/9570.html

假如一个文件有1000个字节，那么其范围就是0-999，则： Range: bytes=500- 表示读取该文件的500-999字节，共500字节。 Range: bytes=500-599 表示读取该文件的500-599字节，共100字节。 Range还有其它几种写法，但上面这两种是最常用的，对于断点续传也足矣了。如果HTTP请求中包含Range字段，那么服务器会返回206（Partial Content），同时HTTP头中也会有一个相应的Content-Range字段，类似下面的格式： Content-Range: bytes 500-999/1000 Content-Range字段说明服务器返回了文件的某个范围及文件的总长度。这时Content-Length字段就不是整个文件的大小了，而是对应文件这个范围的字节数，这一点一定要注意。

他原文是用来做断点续传的，不过用来做多线程下载也是一样的道理

PHP 发 UDP 包时出错

Fri, 24 Aug 2012 23:08:53 +0800

今天金泉给了一个问题，说线上的腾讯 SDK PHP 版本的跑起来报错，看了一下，调用的代码类似这样

<?php
require("inc/OpenApiV3.php");

// 应用基本信息
$appid = xxxxxx;
$appkey = 'xxxxxxx';

// OpenAPI的服务器IP
// 最新的API服务器地址请参考wiki文档: http://wiki.open.qq.com/wiki/API3.0%E6%96%87%E6%A1%A3
$server_name = '119.147.19.43';

// 用户的OpenID/OpenKey
$openid = 'xxxxxxx';
$openkey = 'xxxxxxxx';

// 所要访问的平台, pf的其他取值参考wiki文档: http://wiki.open.qq.com/wiki/API3.0%E6%96%87%E6%A1%A3
$pf = 'qzone';


$sdk = new OpenApiV3($appid, $appkey);
$sdk->setServerName($server_name);

$ret = get_user_info($sdk, $openid, $openkey, $pf);
print_r("===========================n");
print_r($ret);

/**
 * 获取好友资料
 *
 * @param object $sdk OpenApiV3 Object
 * @param string $openid openid
 * @param string $openkey openkey
 * @param string $pf 平台
 * @return array 好友资料数组
 */
function get_user_info(&$sdk, $openid, $openkey, $pf)
{
	$params = array(
		'openid' => $openid,
		'openkey' => $openkey,
		'pf' => $pf,
	);

	$script_name = '/v3/user/get_info';

	return $sdk->api($script_name, $params);
}

// end of script

跑起来之后，报错是

Warning: socket_sendto() [function.socket-sendto]: unable to write to socket [1]: Operation not permitted in /home/naniwco1/public_html/draw/inc/lib/SnsStat.php on line 40

找到那个 SnsStat 文件，是一个用于统计的类，代码是这样的

<?php

/**
 * 统计上报接口调用情况
 *
 * @version 3.0.2
 * @author open.qq.com
 * @copyright © 2011, Tencent Corporation. All rights reserved.
 * @ History:
 *               3.0.2 | sparkeli | 2012-03-06 15:33:04 | initialize statistic fuction which can report API's access time and number to background server

 */

class SnsStat
{
	/**
	 * 执行一个 统计上报
	 *
	 * @param string $stat_url 统计上报的URL
	 * @param float $start_time 统计开始时间
	 * @param array $params 统计参数数组
	 * @return
	 */
	static public function statReport($stat_url, $start_time, $params)
	{
		$end_time = self::getTime();
		$params['time'] = round($end_time - $start_time, 4);
		$params['timestamp'] = time();
		$params['collect_point'] = 'sdk-php-v3';
		$stat_str = json_encode($params);
		//发送上报信息
		$host_ip = gethostbyname($stat_url);
		if ($host_ip != $stat_url)
		{
			$sock = socket_create(AF_INET, SOCK_DGRAM, 0);
			if (false === $sock)
			{
				return;
			}
			socket_sendto($sock, $stat_str, strlen($stat_str), 0, $host_ip, 19888);
			socket_close($sock);
		}
	}

	static public function getTime()
	{
		list($usec, $sec) = explode(" ", microtime());
		return ((float)$usec + (float)$sec);
	}
}

// end of script

把代码迁移到本地环境跑，在已经开了 socket PHP 扩展的前提下，是很顺利的，没有报错，于是考虑环境问题，google 一阵搜，看到这里，http://www.bluehostforum.com/showthread.php?23649-PHP-socket_send()-Not-permitted

I took a quick look. Your script is attempting to connect out on port 27025 which is blocked unless it has been whitelisted to be allowed through the firewall. Contact support and they’ll be happy to open the port for you.

On an unrelated note, there is NEVER a reason to provide world-writable permissions to a file or directory on Bluehost’s servers. All script execution is done by your user which means that a file need only have user permissions (7xx for execution and 6xx for writing). World-writable permissions simply open your account to compromise.

说是防火墙问题，有一定的可能，但是好像没有办法验证，于是只能注释掉了。。

历史评论

home business risks (2013-04-25 06:18:55):

Hi, yeah this post is truly nice and I have learned lot of things from it regarding blogging. thanks.

overdrive effects pedal reviews (2014-09-25 13:27:51):

Hello to all, since I am genuinely eager of reading this website’s post to be updated regularly. It consists of nice stuff.

Apache ab 压力测试

Sat, 18 Aug 2012 17:34:51 +0800

apache 的 ab 真的是非常容易用，直接的说，就是下面这条命令就够了

C:Program FilesApache Software FoundationApache2.2bin>ab -n 100 -c 10 http://localhost/index2.html

这里我 index 的 html 文件就是 apache 的 it works 文件，在我的机子上，测试结果的吞吐率是 2000 多，应该算是比较低的一个数字了，对于这么简单的文本来说，正式的服务器，应该有 5000 左右，不过我的平台是 Windows 嘛。。

C:Program FilesApache Software FoundationApache2.2bin>ab -n 100 -c 10 http:/
/localhost/index2.html
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient).....done

Server Software:        Apache/2.2.22
Server Hostname:        localhost
Server Port:            80

Document Path:          /index2.html
Document Length:        44 bytes

Concurrency Level:      10
Time taken for tests:   0.049 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Total transferred:      32000 bytes
HTML transferred:       4400 bytes
Requests per second:    2040.69 [#/sec] (mean)
Time per request:       4.900 [ms] (mean)
Time per request:       0.490 [ms] (mean, across all concurrent requests)
Transfer rate:          637.72 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.4      0       1
Processing:     2    4   2.5      3      20
Waiting:        1    4   2.5      3      19
Total:          2    5   2.5      4      20

Percentage of the requests served within a certain time (ms)
  50%      4
  66%      4
  75%      5
  80%      6
  90%      8
  95%      8
  98%      9
  99%     20
 100%     20 (longest request)

C:Program FilesApache Software FoundationApache2.2bin>

对于一个打印 phpinfo 的 php 页面来说，成绩就更加惨不忍睹了，只有 90 多

C:Program FilesApache Software FoundationApache2.2bin>ab -n 100 -c 10 http:/
/localhost/index2.php
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient).....done

Server Software:        Apache/2.2.22
Server Hostname:        localhost
Server Port:            80

Document Path:          /index2.php
Document Length:        67839 bytes

Concurrency Level:      10
Time taken for tests:   1.015 seconds
Complete requests:      100
Failed requests:        45
   (Connect: 0, Receive: 0, Length: 45, Exceptions: 0)
Write errors:           0
Total transferred:      6801290 bytes
HTML transferred:       6783990 bytes
Requests per second:    98.57 [#/sec] (mean)
Time per request:       101.454 [ms] (mean)
Time per request:       10.145 [ms] (mean, across all concurrent requests)
Transfer rate:          6546.67 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    1   1.2      0       8
Processing:     3   50 153.5     15    1012
Waiting:        3   49 153.6     13    1011
Total:          3   51 153.4     15    1012

Percentage of the requests served within a certain time (ms)
  50%     15
  66%     18
  75%     21
  80%     25
  90%     88
  95%    348
  98%   1012
  99%   1012
 100%   1012 (longest request)

C:Program FilesApache Software FoundationApache2.2bin>

那么，如果我们把 it work 用 php echo 出来呢，就是说，送进 php 解释器里面转一圈，来看看，依然只有 90 多

C:Program FilesApache Software FoundationApache2.2bin>ab -n 100 -c 10 http:/
/localhost/index3.php
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient).....done

Server Software:        Apache/2.2.22
Server Hostname:        localhost
Server Port:            80

Document Path:          /index3.php
Document Length:        8 bytes

Concurrency Level:      10
Time taken for tests:   1.049 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Total transferred:      18100 bytes
HTML transferred:       800 bytes
Requests per second:    95.32 [#/sec] (mean)
Time per request:       104.906 [ms] (mean)
Time per request:       10.491 [ms] (mean, across all concurrent requests)
Transfer rate:          16.85 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.5      0       1
Processing:     2   18  99.6      8    1004
Waiting:        2   18  99.7      8    1004
Total:          2   19  99.6      9    1004

Percentage of the requests served within a certain time (ms)
  50%      9
  66%      9
  75%     10
  80%     11
  90%     13
  95%     14
  98%     31
  99%   1004
 100%   1004 (longest request)

C:Program FilesApache Software FoundationApache2.2bin>

再来看看 php 访问数据库的时候的成绩，访问数据我们就用一条 select 语句，php 代码如下

<?php
mysql_connect("localhost", "root", "root") or die(mysql_error());
mysql_select_db("mailbox");
mysql_query("set names gbk");
$result = mysql_query("select * from mail where id = 118593");
while($row = mysql_fetch_array($result)) {
	var_dump($row);
}

算是非常中规中矩的读取了，结果如下，吞吐率只有 80 多

C:Program FilesApache Software FoundationApache2.2bin>ab -n 100 -c 10 http:/
/localhost/index4.php
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient).....done

Server Software:        Apache/2.2.22
Server Hostname:        localhost
Server Port:            80

Document Path:          /index4.php
Document Length:        6108 bytes

Concurrency Level:      10
Time taken for tests:   1.157 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Total transferred:      628100 bytes
HTML transferred:       610800 bytes
Requests per second:    86.43 [#/sec] (mean)
Time per request:       115.707 [ms] (mean)
Time per request:       11.571 [ms] (mean, across all concurrent requests)
Transfer rate:          530.12 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.4      0       1
Processing:     6  100 301.8     10    1154
Waiting:        6   99 301.8     10    1154
Total:          6  100 301.7     11    1154

Percentage of the requests served within a certain time (ms)
  50%     11
  66%     16
  75%     16
  80%     16
  90%     22
  95%   1145
  98%   1153
  99%   1154
 100%   1154 (longest request)

C:Program FilesApache Software FoundationApache2.2bin>

那么，如果我们用 apache 把 php 的结果缓存起来呢？这个想法非常赞，可惜我这里配的是 apache fastcgi php 的模式，简单 google 了一下并没有发现相关的文章，倒是 nginx fastcgi php 的缓存有很多的讨论，也是，人家本来就是做反向代理的嘛，不过，根据高性能 web 那本书的记载，应该是可以有 5 倍左右的提升的，还是很垂涎的，呵呵。强烈推荐这本高性能 web 的书，http://book.douban.com/subject/3924175/

最后插播一下，对于一台 debian 的虚拟机，成绩其实没有想象中的飞跃，这台虚拟机开的内存是 256，一样请求一个静态文件，结果如下，顺便一提，虚拟机里面跑着的是 nginx，不是 apache

C:Program FilesApache Software FoundationApache2.2bin>ab -n 1000 -c 10 http:
//192.168.128.133:8080/index.html
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 192.168.128.133 (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests

Server Software:        Apache/2.2.20
Server Hostname:        192.168.128.133
Server Port:            8080

Document Path:          /index.html
Document Length:        12 bytes

Concurrency Level:      10
Time taken for tests:   0.632 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      297000 bytes
HTML transferred:       12000 bytes
Requests per second:    1582.19 [#/sec] (mean)
Time per request:       6.320 [ms] (mean)
Time per request:       0.632 [ms] (mean, across all concurrent requests)
Transfer rate:          458.90 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    1   0.5      1       2
Processing:     1    6   1.1      6      19
Waiting:        0    4   1.5      4      17
Total:          2    6   1.1      6      20

Percentage of the requests served within a certain time (ms)
  50%      6
  66%      6
  75%      6
  80%      7
  90%      7
  95%      8
  98%      9
  99%     10
 100%     20 (longest request)

C:Program FilesApache Software FoundationApache2.2bin>

Apache 控制台输出

Wed, 15 Aug 2012 16:19:15 +0800

在 wamp 套装中，apache 的错误日志一般是放在这样的路径下，D:Documentscodewamplogs ，这个错误日志在大多数时候是可以帮助排查错误的，但是，如果在改写 httpd.conf 文件过程中出错了话，那么在 wamp 中启动 apache 是直接就启动不起来的，错误日志也没有，这个时候可以通过控制台，直接定位到 apache 的 httpd 程序，在那里，可以看到明确的错误提示， httpd 的程序一般在 D:DocumentscodewampbinapacheApache2.2.17bin 类似的目录下面

一种特殊情况下的优酷广告过滤

Thu, 26 Jul 2012 17:34:14 +0800

最近在看优酷上的电视剧，烦了那个广告，于是想到过滤，一般来说，过滤优酷的广告，网上都有现成的方法，思路是改 hosts 干扰域名解析，这是屏蔽广告的常见思路，而且比较通用，另外，对付优酷的特殊情况，他会使用 flash 判断广告的播放情况，如果没有播放成功，那么会给一个提示，所以需要修改 flash 的一些相关文件，具体的是修改

C:UsersZRJAppDataRoamingMacromediaFlash Player#SharedObjectsBGTENWSR

下面的一个 static.youku.com 文件夹，删掉然后用一个 com 文件占用位置，让其无法写入，于是这个问题也得以解决。

但是我这里的情况稍稍不同，由于处在校内网，所以出入口速度比较慢，刚好知道有一个直连 Internet 的代理服务器，于是对于一些大的流媒体文件，我一般走这个代理去下载，速度还是比较快的，代理的切换方法的话，我是用 chrome 下的 Proxy SwitchySharp，还比较好用，可以自定义规则来控制代理，这个东西一般是用来做本地 goagent 配合着用的，对于被墙的域名走本地搭起的 goagent 代理，对于普通域名走直接连接的通道。

同时他还有一个手动切换的选项，我一般在遇到流媒体的时候会手动切换到那台代理，问题就出在这里：当我们把优酷负责广告的域名通过 hosts 干扰了解析之后，正常情况下是可以奏效的，但是，如果这个时候走代理，那么本地修改的 hosts 就鞭长莫及了。我觉得这是因为他的处理流程导致的：首先当 chrome 有一个 HTTP 请求的时候，被 SwitchySharp 拦截，这个时候，如果 SwitchySharp 在匹配列表中没有发现可以匹配的规则的话，那么就放行，然后 chrome 继续处理请求的流程，这里才发起解析域名的步骤，于是 hosts 才有用，相反，如果 SwitchySharp 能匹配到规则的话，就直接把请求转发到代理去了，于是本地的 hosts 根本就不会被访问，悲剧就这么发生了。

本地 hosts 修改没用的话，那么怎么办呢，沿用这个思路，很快的可以想到，既然如此，我们可以起一个根本不存在的代理，就用本地 127.0.0.1 的地址，来接受那些广告域名，然后配置 SwitchySharp ，让他把所有的广告请求都转到这个不存在的代理，然后正常的视频就从可用的代理那里请求数据。

可惜后面跟着还有问题，问题在于，对于真正的视频文件，优酷并没有使用一个域名来提供内容，而是使用一个类似

http://f.youku.com/player/getFlvPath/sid/134329032214611466521_00/st/flv/fileid/0300020A004FDF4C44FD8A032DBBC7F1B1B40B-AC36-FF2F-113A-10996097DEBD?start=1&K=c1a370c934fd64c2241114b0&hd=0&myp=0&ts=406&ymovie=1

的地址来回应一个 302 到一个 IP 命名的服务器上来提供，估计是 CDN 吧，这样的话，想要使用一个通配的 URL 规则来匹配这些视频就没用了，因为就算那个 f.youku.com 域名下的请求是通过代理得到的，也帮不了什么忙，那只是一个 302 而已，回想之前为什么没有这样的问题呢，因为当时我是把全局的请求都通过可用代理去取内容的，所以根本都不需要关心这些细节。

于是最后就只能配置成：广告域名的请求走一个本地的空白代理，其他的所有请求走可用代理，然后在看完视频后，再把那条规则暂时屏蔽掉。

浏览器对编码的确定

Mon, 16 Jul 2012 21:50:30 +0800

我们知道，在一次 HTTP 请求中，浏览器向 HTTP 服务器发起 HTTP request，然后服务器返回 response header，到目前为止，字符的编码都是 ANCII 的，所以都还好，不需要考虑什么解码问题，但是服务器紧跟着传回来的 response body 本质上就是一个字节流了，于是自然而然的就面临一个解码的问题，怎么处理这个问题是我一直以来疑惑已久的。

对于这个问题，如果服务器比较规范，那么应该在 response header 中指明 charset ，这种情况当然是最好的，浏览器直接就可以按照给定的 charset 去解码正文，但是如果没有给出呢？很自然的会想到，HTML 正文里面，是有一个 meta 标签的，在那个 meta 标签中，我们可以指明 charset ，但是，我一直想不通的问题就在这里了：既然正文本身的编码方式就是未知的，而指明编码方式的 meta 又位于正文中，所以这不是一个自我循环的无解问题吗？

在这之前，我记得我曾经在网络上看到一些文章说，这种时候，对于 meta 标签的位置就很重要了，一定要写在 title 之前，给出的原因是说，title 中可能含有非 ANCII 字符，所以如果 meta 在 title 之前的话，就可以让浏览器顺利把 meta 找到并看懂，而如果 meta 在 title 之后，那么 title 中的多字节文本就可能影响了 meta 的被发现。这种轮调，甚至知道今天，（刚刚我搜了一下），还十分流行，见这里，http://touya.iteye.com/blog/265949，还有这里，http://www.jb51.net/article/16138.htm，不可否认，他们的这个思路确实有一定的道理，甚至有人还做了十分详尽的对比测试，例如这里，http://epie.blogbus.com/logs/23927589.html，但是，我依然不理解，我的疑惑是，如果说把 meta 放在 title 之前可以解决问题，那么应该是建立在那个编码是 ANCII 兼容的基础上的，也就是说，需要是 ANCII 的超集，例如变长的 UTF-8 ，如果遇到定长的非兼容编码，不是一样不行吗，例如 16 位定长的 Unicode ，虽然我也自己动手试过，例如这段代码：

<html>
	<head>
		<meta http-equiv="Content-Type" content="text/html; charset=utf-16">
		<title>中文</title>
	</head>
	<body>
		<p>中文</p>
	</body>
</html>

把这段代码以 16 位定长 Unicode 保存再浏览，虽然是可以正常解码，但是按照我的理解，应该是不行的啊。

直到今天，才看到一篇博文，介绍的是 Gecko 的解码方案，终于解决了这个疑惑，原来，渲染引擎是会对字节流做一定程度的智能判断的，再在这个基础上，决定解码页面使用的解码方式，博文来自百度搜索研发部，很不错的一个博客：

http://stblog.baidu-tech.com/?p=1909

这篇文章讲的十分清楚，我也就不再多言了，强烈建议仔细读一读，最后附上两篇遇到的讲字符编码比较好的文章：

这个，中文编码杂谈，不知道为什么没了，要看 google cache，还有这个，http://www.fmddlmyy.cn/text16.html

历史评论

从点击到呈现 — 详解一次HTTP请求（4） | ZRJ (2012-10-11 22:11:58):

[…] 当浏览得到一个正确的 200 响应之后，接下来面临的一个问题就是多国语言的编码解析了，响应头是一个 ascii 的标准字符集的文本，这个还好办，但是响应的正文本质上就是一个字节流，对于这一坨字节流，浏览器要怎么去处理呢，首先浏览器会去看响应头里面指定的 encoding 域，如果有了这个东西，那么就按照指定的 encoding 去解析字符，如果没有的话，那么浏览器会使用一些比较智能的方式，去猜测和判断这一坨字节流应该使用什么字符集去解码。相关的笔记可以看这里，浏览器对编码的确定 […]