网络 on ZRJ | 学习笔记

linux ss

Wed, 18 Nov 2015 16:27:22 +0800

ss 是一个类似 netstat 的网络连接查看工具，man page 是这么说的

ss - another utility to investigate sockets

而 netstat 的 man page 中也有说到

This program is obsolete. Replacement for netstat is ss.

常用命令有

-s 参数，打印 summary

# ss -s   
Total: 1596 (kernel 1890)
TCP:   7587 (estab 718, closed 6800, orphaned 0, synrecv 0, timewait 6779/0), ports 7376

Transport Total     IP        IPv6
*         1890      -         -        
RAW       0         0         0        
UDP       420       420       0        
TCP       787       787       0        
INET      1207      1207      0        
FRAG      0         0         0

-p 参数，打印 process name

# ss -p |head  
State      Recv-Q Send-Q      Local Address:Port          Peer Address:Port   
ESTAB      0      0           10.198.17.218:20869         10.133.33.10:9092     users:(("xxx",32492,37))
ESTAB      0      0           10.198.17.218:32516        10.231.139.30:8886     users:(("xxx",19706,17))
ESTAB      0      0           10.198.17.218:50831       10.208.146.112:31415    users:(("xx",1559,40))
ESTAB      0      0           10.198.17.218:26957       10.177.141.160:8886     users:(("xxx",1056,27))
ESTAB      0      0           10.198.17.218:56118        10.238.18.176:56328    users:(("xx",7491,19))
ESTAB      0      0           10.198.17.218:29310       10.177.130.230:6900     users:(("xxx",19715,14))
ESTAB      0      0           10.198.17.218:31249         10.209.0.153:8886     users:(("xxx",1052,29))
ESTAB      0      0           10.198.17.218:21980        10.241.90.225:50000    users:(("xxx",9211,39))
ESTAB      0      0           10.198.17.218:2452        10.177.154.166:10010    users:(("xxx",7875,24))

Python inet_ntoa 小脚本

Fri, 07 Mar 2014 09:19:39 +0800

用的时候发现这东西居然还要自己写

#!/usr/bin/env python

import socket
import struct
import sys

def Int2Ip(ip):
    return socket.inet_ntoa(struct.pack('!I', ip))

if __name__ == "__main__":
    if len(sys.argv) > 1:
        print Int2Ip(int(sys.argv[1]))

对于多行的文件，可以这么来批量处理

cat ip.txt | xargs -i  ./Int2Ip.py {}

-——————————-

2014-4-6 15:36:42 update

http://hi.baidu.com/wg_wang/item/338dbb3913ce5a617c034bfc

在工作中需要将ip地址从整数（如168442432）到以点号分隔的字符串类型（如10.10.58.64）之间的转换，或者反过来从字符串到整数的转换。这个转换可以使用的方式很多，例如，linux下的C语言编程，调用inet_aton等函数就可以方便的实现（参考man inet_aton）。不过，脚本语言可以提供更加快速的方式实现，在python中，如下：

首先： import socket import struct

字符串到整数的转换： socket.ntohl(struct.unpack(‘i’,socket.inet_aton(“10.10.58.64”))[0])

更新版，结果都为正整数，前面这个当结果ip地址为128.0.0.0及其以上时，结果会表示为负数。 struct.unpack(‘I’,struct.pack(‘i’,(socket.ntohl(struct.unpack(‘i’, socket.inet_aton(“201.119.123.249”))[0]))))[0]

整数到字符串的转换： socket.inet_ntoa(struct.pack(‘i’,socket.htonl(980942848 )))

各个函数的功能可以参考python的文档，例如： »> help(struct.pack) pack(fmt, *args) Return string containing values v1, v2, … packed according to fmt. See struct.__doc__ for more on format strings.

更详细的可以参考python documentation，有联机版的，也有脱机版的，在windows安装python2.5时还附带chm的文档。

20091117更新如果在c/c++程序中用inet_aton和inet_ntoa进行转换的，int类型和字符串类型的对应关系，在python中还原的话，则无须进行网络主机字节序（host byte order）到网络字节序（network byte order）之间的转换，或者反过来主机字节序到网络字节序的转换。这样，转换的语句如下： socket.inet_ntoa(struct.pack(‘I’, 1077545482)) struct.unpack(‘I’,socket.inet_aton(“10.10.58.64”))[0]

对应的c程序分别为： 18 #include 19 #include 20 #include 21 #include 22 #include 23 #include 24 25 int main(int argc, char ** argv) 26 { 27 if ( argc != 2 ) 28 { 29 fprintf(stderr, “error argument\nusage: %s int_ip\n”, argv[0]); 30 return -1; 31 } 32 33 struct in_addr in; 34 long a=atol(argv[1]); 35 in.s_addr=(unsigned int)(a); 36 fprintf(stdout, “%u <–> %s\n”, in.s_addr, inet_ntoa(in)); 37 return 0; 38 }

18 #include 19 #include 20 #include 21 22 int main(int argc, char *argv[]) 23 { 24 struct in_addr addr; 25 if (argc != 2) 26 { 27 fprintf(stderr, “argument wrong\nusage: %s \n”, argv[0]); 28 return -1; 29 } 30 if (inet_aton(argv[1], &addr) == 0) { 31 fprintf(stderr, “inet_aton”); 32 } 33 34 fprintf(stdout, “%s <–> %u\n”, argv[1], addr.s_addr); 35 return 0; 36 }

修改网卡缓冲以提高吞吐量

Fri, 28 Feb 2014 16:36:55 +0800

在单机并发上万的时候，发现 sendto 函数比较容易出现返回错误 EAGAIN，对应的错误消息为 Resource temporarily unavailable，这种情况下认为是应用层写入过快，导致网卡缓冲满了

使用这个命令来查看缓冲区现有大小

# ethtool -g eth1
Ring parameters for eth1:
Pre-set maximums:
RX:		4096
RX Mini:	0
RX Jumbo:	0
TX:		4096
Current hardware settings:
RX:		256
RX Mini:	0
RX Jumbo:	0
TX:		256

使用这个命令来修改缓冲区大小

# ethtool -G eth1 rx 2048
# ethtool -G eth1 tx 2048

修改之后再查看一下

# ethtool -g eth1
Ring parameters for eth1:
Pre-set maximums:
RX:		4096
RX Mini:	0
RX Jumbo:	0
TX:		4096
Current hardware settings:
RX:		2048
RX Mini:	0
RX Jumbo:	0
TX:		2048

但是发现这样修改之后，没有起作用，sendto 函数的失败率还是很高，继续看，发现 socket 族函数本身有一个 setsockopt 的函数，在代码里面可以设置，写法如下

int iOriginalSendBufferSize = 0;
socklen_t iIntLen = sizeof(int);
getsockopt(iSocket, SOL_SOCKET, SO_SNDBUF, &iOriginalSendBufferSize, &iIntLen);
LOG_INFO("before set, send buffer is %d", iOriginalSendBufferSize);

int nSendBuf=32*1024;
setsockopt(iSocket,SOL_SOCKET,SO_SNDBUF,(const  char*)&nSendBuf,sizeof(int));  

int iFinalSendBufferSize = 0;
getsockopt(iSocket, SOL_SOCKET, SO_SNDBUF, &iFinalSendBufferSize, &iIntLen);
LOG_INFO("after set, send buffer is %d", iFinalSendBufferSize);

这里可能会觉得多此一举，设完还要看，有啥好看的，但是，看输出

before set, send buffer is 8388608
after set, send buffer is 65536

神奇吧，设的明明是 32k，出来居然 64k 了，买一送一啊这是

查资料看到这里，http://blog.csdn.net/c359719435/article/details/8815499

Author：阿冬哥

Created：2013-4-17

Blog：http://blog.csdn.net/c359719435/

Copyright 2013 阿冬哥 http://blog.csdn.net/c359719435/

使用以及转载请注明出处

1 设置socket tcp缓冲区大小的疑惑疑惑1：通过setsockopt设置SO_SNDBUF、SO_RCVBUF这连个默认缓冲区的值，再用getsockopt获取设置的值，发现返回值是设置值的两倍。为什么？通过网上查找，看到linux的内核代码/usr/src/linux-2.6.13.2/net/core/sock.c，找到sock_setsockopt这个函数的这段代码：

case SO_SNDBUF: /* Don’t error on this BSD doesn’t and if you think about it this is right. Otherwise apps have to play ‘guess the biggest size’ games. RCVBUF/SNDBUF are treated in BSD as hints */ if (val > sysctl_wmem_max)//val是我们想设置的缓冲区大小的值 val = sysctl_wmem_max;//大于最大值，则val值设置成最大值 sk->sk_userlocks |= SOCK_SNDBUF_LOCK; if ((val * 2) < SOCK_MIN_SNDBUF)//val的两倍小于最小值，则设置成最小值 sk->sk_sndbuf = SOCK_MIN_SNDBUF; else sk->sk_sndbuf = val * 2;//val的两倍大于最小值，则设置成val值的两倍 /* * Wake up sending tasks if we * upped the value. */ sk->sk_write_space(sk); break; case SO_RCVBUF: /* Don’t error on this BSD doesn’t and if you think about it this is right. Otherwise apps have to play ‘guess the biggest size’ games. RCVBUF/SNDBUF are treated in BSD as hints */ if (val > sysctl_rmem_max) val = sysctl_rmem_max; sk->sk_userlocks |= SOCK_RCVBUF_LOCK; /* FIXME: is this lower bound the right one? */ if ((val * 2) < SOCK_MIN_RCVBUF) sk->sk_rcvbuf = SOCK_MIN_RCVBUF; else sk->sk_rcvbuf = val * 2; break; 从上述代码可以看出：（1）当设置的值val > 最大值sysctl_wmem_max，则设置为最大值的2倍：2*sysctl_wmem_max；（2）当设置的值的两倍val*2 > 最小值，则设置成最小值：SOCK_MIN_SNDBUF；（3）当设置的值val < 最大值sysctl_wmem_max，且 val*2 > SOCK_MIN_SNDBUF，则设置成2*val。

查看linux 手册： SO_RCVBUF： Sets or gets the maximum socket receive buffer in bytes. The kernel doubles this value (to allow space for bookkeeping overhead) when it is set using setsockopt(2), and this doubled value is returned by getsockopt(2). The default value is set by the /proc/sys/net/core/rmem_default file, and the maximum allowed value is set by the /proc/sys/net/core/rmem_max file. The minimum (doubled) value for this option is 256. 查看我的主机Linux 2.6.6 ：/proc/sys/net/core/rmem_max： 4194304 //4M 查看/proc/sys/net/core/wmem_max： 8388608 //8M 所以，能设置的接收缓冲区的最大值是8M，发送缓冲区的最大值是16M。

疑惑2：为什么要有2倍这样的一个内核设置呢？我的理解是，用户在设置这个值的时候，可能只考虑到数据的大小，没有考虑数据封包的字节开销。所以将这个值设置成两倍。

注：overhead，在计算机网络的帧结构中，除了有用数据以外，还有很多控制信息，这些控制信息用来保证通信的完成。这些控制信息被称作系统开销。 2 tcp缓冲区大小的默认值建立一个socket，通过getsockopt获取缓冲区的值如下：发送缓冲区大小：SNDBufSize = 16384 接收缓冲区大小：RCVBufSize = 87380

疑惑3：linux手册中，接收缓冲区的默认值保存在/proc/sys/net/core/rmem_default，发送缓冲区保存在/proc/sys/net/core/wmem_default。 [root@cfs_netstorage core]# cat /proc/sys/net/core/rmem_default 1048576 [root@cfs_netstorage core]# cat /proc/sys/net/core/wmem_default 512488

可知，接收缓冲区的默认值是：1048576，1M。发送缓冲区的默认值是：512488，512K。为什么建立一个socket时得到的默认值是87380、16384？？？

进一步查阅资料发现， linux下socket缓冲区大小的默认值在/proc虚拟文件系统中有配置。分别在一下两个文件中： /proc/sys/net/ipv4/tcp_wmem [root@cfs_netstorage core]# cat /proc/sys/net/ipv4/tcp_wmem 4096 16384 131072 //第一个表示最小值，第二个表示默认值，第三个表示最大值。 /proc/sys/net/ipv4/tcp_rmem [root@cfs_netstorage core]# cat /proc/sys/net/ipv4/tcp_rmem 4096 87380 174760

由此可见，新建socket，选取的默认值都是从这两个文件中读取的。可以通过更改这两个文件中的值进行调优，但是最可靠的方法还是在程序中调用setsockopt进行设置。通过setsockopt的设置，能设置的接收缓冲区的最大值是8M，发送缓冲区的最大值是16M（Linux 2.6.6中）。

原来还真的是买一送一

于是在这个地方，把 SNDBUF 这个选项改成 8m 或者更大，sendto 的错误就没有了

-——————–

2014-2-28 17:31:44 update 原来对于 tcp 和 udp，这个缓冲区默认大小还不不一样

#include <stdio.h>
#include <assert.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <string.h>
#include <unistd.h>
int tcpSrv(short iListenPort)
{
    int iListenFd = socket(AF_INET, SOCK_STREAM, 0);
    assert(iListenFd > 0);
    
    int iOriginalSendBufferSize = 0;
    socklen_t iIntLen = sizeof(iOriginalSendBufferSize);
    getsockopt(iListenFd, SOL_SOCKET, SO_SNDBUF, &iOriginalSendBufferSize, &iIntLen);
    printf("default send buffer size %d\n", iOriginalSendBufferSize);
    
    struct sockaddr_in stServerAddr;
    stServerAddr.sin_family = AF_INET;
    stServerAddr.sin_addr.s_addr = htonl(INADDR_ANY);
    stServerAddr.sin_port = htons(iListenPort);
    assert(bind(iListenFd, (struct sockaddr*)&stServerAddr, sizeof(stServerAddr)) == 0);
    assert(listen(iListenFd, 1024) == 0);
    printf("listening...\n");
    while (1) {
        int iAcceptFd = accept(iListenFd, (struct sockaddr*)NULL, NULL);
        assert(iAcceptFd > 0);
        char* pStrResponse = "HTTP/1.1 200 OK\r\nConnection: close\r\n\r\n";
        int iResponseLen = strlen(pStrResponse);
        int iSendByte = send(iAcceptFd, pStrResponse, iResponseLen, 0);
        assert(iSendByte == iResponseLen);
        close(iAcceptFd);
    }
    close(iListenFd);
    return 0;
}


int udpSrv(short iListenPort)
{
    int iListenFd = socket(AF_INET, SOCK_DGRAM, 0);
    assert(iListenFd > 0);
    
    int iOriginalSendBufferSize = 0;
    socklen_t iIntLen = sizeof(iOriginalSendBufferSize);
    getsockopt(iListenFd, SOL_SOCKET, SO_SNDBUF, &iOriginalSendBufferSize, &iIntLen);
    printf("default send buffer size %d\n", iOriginalSendBufferSize);
    
    struct sockaddr_in stServerAddr;
    stServerAddr.sin_family = AF_INET;
    stServerAddr.sin_addr.s_addr = htonl(INADDR_ANY);
    stServerAddr.sin_port = htons(iListenPort);
    assert(bind(iListenFd, (struct sockaddr*)&stServerAddr, sizeof(stServerAddr)) == 0);
    printf("listening...\n");
    while (1) {
        char buffer[BUFSIZ] = {0};
        struct sockaddr_in stClientAddr;
        socklen_t stClientAddrLen = sizeof(stClientAddr);
        int iReceivedLen = recvfrom(iListenFd, buffer, sizeof(buffer), 0, (struct sockaddr*)&stClientAddr, &stClientAddrLen);
        int iSendByte = sendto(iListenFd, &buffer, iReceivedLen, 0, (struct sockaddr*)&stClientAddr, stClientAddrLen);
    }
    close(iListenFd);
    return 0;
}


int main()
{
    udpSrv(8000);
    return 0;
}

tcp 的

default send buffer size 16384
listening...

udp 的

default send buffer size 8388608
listening...

历史评论

wanax (2014-03-19 16:43:48):

PDU与SDU的考虑吧，猜的…

ZRJ (2014-03-19 20:12:50):

我能说我看了你回复才知道这两个名词吗。。。

netstat 查看连接的时候打印进程名

Fri, 27 Dec 2013 17:35:38 +0800

这个本来是个很基本的需求啊，搜了一下，stackover flow 上面有讨论，在这里，http://stackoverflow.com/questions/4247932/how-to-parse-netstat-command-in-order-to-get-process-name-and-pid-from-it/20798071#20798071

Try

ps -p $(lsof -ti tcp:80) o comm=,pid= or

netstat -tlnp | awk ‘/:80 */ {split($NF,a,"/"); print a[2],a[1]}’

好麻烦，而且跑不了，后来发现加一个 -p 参数就可以了。。

netstat -p

不过需要 root 权限

[转]一份高质量的 Epoll 服务端代码

Mon, 25 Mar 2013 14:10:37 +0800

来自https://banu.com/blog/2/how-to-use-epoll-a-complete-example-in-c/epoll-example.c，由http://www.oschina.net/translate/how-to-use-epoll-a-complete-example-in-c?from=20130303提及

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netdb.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/epoll.h>
#include <errno.h>

#define MAXEVENTS 64

static int
make_socket_non_blocking (int sfd)
{
  int flags, s;

  flags = fcntl (sfd, F_GETFL, 0);
  if (flags == -1)
    {
      perror ("fcntl");
      return -1;
    }

  flags |= O_NONBLOCK;
  s = fcntl (sfd, F_SETFL, flags);
  if (s == -1)
    {
      perror ("fcntl");
      return -1;
    }

  return 0;
}

static int
create_and_bind (char *port)
{
  struct addrinfo hints;
  struct addrinfo *result, *rp;
  int s, sfd;

  memset (&hints, 0, sizeof (struct addrinfo));
  hints.ai_family = AF_UNSPEC;     /* Return IPv4 and IPv6 choices */
  hints.ai_socktype = SOCK_STREAM; /* We want a TCP socket */
  hints.ai_flags = AI_PASSIVE;     /* All interfaces */

  s = getaddrinfo (NULL, port, &hints, &result);
  if (s != 0)
    {
      fprintf (stderr, "getaddrinfo: %s\n", gai_strerror (s));
      return -1;
    }

  for (rp = result; rp != NULL; rp = rp->ai_next)
    {
      sfd = socket (rp->ai_family, rp->ai_socktype, rp->ai_protocol);
      if (sfd == -1)
        continue;

      s = bind (sfd, rp->ai_addr, rp->ai_addrlen);
      if (s == 0)
        {
          /* We managed to bind successfully! */
          break;
        }

      close (sfd);
    }

  if (rp == NULL)
    {
      fprintf (stderr, "Could not bind\n");
      return -1;
    }

  freeaddrinfo (result);

  return sfd;
}

int
main (int argc, char *argv[])
{
  int sfd, s;
  int efd;
  struct epoll_event event;
  struct epoll_event *events;

  if (argc != 2)
    {
      fprintf (stderr, "Usage: %s [port]\n", argv[0]);
      exit (EXIT_FAILURE);
    }

  sfd = create_and_bind (argv[1]);
  if (sfd == -1)
    abort ();

  s = make_socket_non_blocking (sfd);
  if (s == -1)
    abort ();

  s = listen (sfd, SOMAXCONN);
  if (s == -1)
    {
      perror ("listen");
      abort ();
    }

  efd = epoll_create1 (0);
  if (efd == -1)
    {
      perror ("epoll_create");
      abort ();
    }

  event.data.fd = sfd;
  event.events = EPOLLIN | EPOLLET;
  s = epoll_ctl (efd, EPOLL_CTL_ADD, sfd, &event);
  if (s == -1)
    {
      perror ("epoll_ctl");
      abort ();
    }

  /* Buffer where events are returned */
  events = calloc (MAXEVENTS, sizeof event);

  /* The event loop */
  while (1)
    {
      int n, i;

      n = epoll_wait (efd, events, MAXEVENTS, -1);
      for (i = 0; i < n; i++)
	{
	  if ((events[i].events & EPOLLERR) ||
              (events[i].events & EPOLLHUP) ||
              (!(events[i].events & EPOLLIN)))
	    {
              /* An error has occured on this fd, or the socket is not
                 ready for reading (why were we notified then?) */
	      fprintf (stderr, "epoll error\n");
	      close (events[i].data.fd);
	      continue;
	    }

	  else if (sfd == events[i].data.fd)
	    {
              /* We have a notification on the listening socket, which
                 means one or more incoming connections. */
              while (1)
                {
                  struct sockaddr in_addr;
                  socklen_t in_len;
                  int infd;
                  char hbuf[NI_MAXHOST], sbuf[NI_MAXSERV];

                  in_len = sizeof in_addr;
                  infd = accept (sfd, &in_addr, &in_len);
                  if (infd == -1)
                    {
                      if ((errno == EAGAIN) ||
                          (errno == EWOULDBLOCK))
                        {
                          /* We have processed all incoming
                             connections. */
                          break;
                        }
                      else
                        {
                          perror ("accept");
                          break;
                        }
                    }

                  s = getnameinfo (&in_addr, in_len,
                                   hbuf, sizeof hbuf,
                                   sbuf, sizeof sbuf,
                                   NI_NUMERICHOST | NI_NUMERICSERV);
                  if (s == 0)
                    {
                      printf("Accepted connection on descriptor %d "
                             "(host=%s, port=%s)\n", infd, hbuf, sbuf);
                    }

                  /* Make the incoming socket non-blocking and add it to the
                     list of fds to monitor. */
                  s = make_socket_non_blocking (infd);
                  if (s == -1)
                    abort ();

                  event.data.fd = infd;
                  event.events = EPOLLIN | EPOLLET;
                  s = epoll_ctl (efd, EPOLL_CTL_ADD, infd, &event);
                  if (s == -1)
                    {
                      perror ("epoll_ctl");
                      abort ();
                    }
                }
              continue;
            }
          else
            {
              /* We have data on the fd waiting to be read. Read and
                 display it. We must read whatever data is available
                 completely, as we are running in edge-triggered mode
                 and won't get a notification again for the same
                 data. */
              int done = 0;

              while (1)
                {
                  ssize_t count;
                  char buf[512];

                  count = read (events[i].data.fd, buf, sizeof buf);
                  if (count == -1)
                    {
                      /* If errno == EAGAIN, that means we have read all
                         data. So go back to the main loop. */
                      if (errno != EAGAIN)
                        {
                          perror ("read");
                          done = 1;
                        }
                      break;
                    }
                  else if (count == 0)
                    {
                      /* End of file. The remote has closed the
                         connection. */
                      done = 1;
                      break;
                    }

                  /* Write the buffer to standard output */
                  s = write (1, buf, count);
                  if (s == -1)
                    {
                      perror ("write");
                      abort ();
                    }
                }

              if (done)
                {
                  printf ("Closed connection on descriptor %d\n",
                          events[i].data.fd);

                  /* Closing the descriptor will make epoll remove it
                     from the set of descriptors which are monitored. */
                  close (events[i].data.fd);
                }
            }
        }
    }

  free (events);

  close (sfd);

  return EXIT_SUCCESS;
}

不同服务器端模型在并发请求下的表现差异

Mon, 25 Mar 2013 11:22:04 +0800

一般来说，服务器的模型有下面几种：

单进程单线程
多进程（一个请求对应一个进程，Apache 之类貌似就是这个）
多线程（一个请求对应一个线程，这个挺少见到的）
select / poll
epoll

它们的差别可以看这个，http://www.cnblogs.com/sharra/archive/2010/12/30/1921287.html

因为需要了解底层设备访问的原理，所以惯用高层应用语言的我，需要了解一下Linux的设备访问机制，尤其是处理一组非阻塞IO的原理方法，标准的术语好像是叫多路复用。以下文章部分句子有引用之处，恕没有一一指出出处。对于接触过Linux内核或设备驱动开发的读者，一定清楚poll和select系统调用，以及从2.5版本引入的epoll机制（epoll机制包含三个系统调用）。网上关于它们的文章，有说用法的，甚为详细，更有分析源代码的，又比较深入，且枝节颇多。经过几篇文章的阅读，我把觉得比较核心的东西写下来吧。我的用意是尽可能以简单的概念，比对他们三者的异同。几经查找我才确定下来，poll和select应该被归类为这样的系统调用，它们可以阻塞地同时探测一组支持非阻塞的IO设备，是否有事件发生（如可读，可写，有高优先级的错误输出，出现错误等等），直至某一个设备触发了事件或者超过了指定的等待时间——也就是它们的职责不是做IO，而是帮助调用者寻找当前就绪的设备。同类型的产品是Windows的IOCP，它也是处理多路复用，只是把IO和探测封装在了一起了。准备的知识有两点：1、fd；2、op->poll。在Linux里面，设备都被抽象为文件，一系列的设备文件就有自己独立的虚拟文件系统，所以，设备在系统调用参数中的表示就是file description。fd其实就是一个整数（特别地，标准输入，输出，错误输出分别对应的fd是0，1，2）。与内核打交道的时候，传递整数的fd可以在自己的文件系统中作进一步的检查是否合法，如果只是返回指针就不能这样操作了，毕竟指针是无差别无意义的。通过fd访问file，通过file可以访问其fileOperator，这里面我们要关心的一个fileOp就是poll。因为系统调用poll和select，就是依靠这个文件操作poll实现的。poll文件操作有两个参数，一个是文件本身，一个可以看做是当设备尚未就绪时调用的回调函数，这个函数是把设备自己特有的等待队列传给内核，让内核把当前的进程挂载到其中（因为当设备就绪时，设备就应该去唤醒在自己特有等待队列中的所有节点，这样当前进程就获取了完成的信号了）。poll文件操作返回的必须是一组标准的掩码，其中的各个位指示当前的不同的就绪状态（全0为没有任何事件触发）。再谈谈早期多路复用的版本poll和select。本质而言，poll和select的共同点就是，对全部指定设备做一次poll，当然这往往都是还没有就绪的，那就会通过回调函数把当前进程注册到设备的等待队列，如果所有设备返回的掩码都没有显示任何的事件触发，就去掉回调函数的函数指针，进入有限时的睡眠状态，再恢复和不断做poll，再作有限时的睡眠，直到其中一个设备有事件触发为止。只要有事件触发，系统调用返回，回到用户态，用户就可以对相关的fd作进一步的读或者写操作了。当然，这个时候还不是所有的设备都就绪的喔，那就得不断地poll或者select了，而做一次这样的系统调用都得轮询所有的设备，次数是设备数*（睡眠次数-1），也就是时间复杂度是O（n），还得做几次O（n）呢。可见，对于现在普遍的服务器程序，需要同时并发监听数千个连接，并且连接需要重复使用的情况，poll和select就存在这样的性能瓶颈。另外，数千个设备fd在每次调用时，都需要将其从用户空间复制到内核空间，这里的开销不可忽略。 poll和select放在一起，是因为其机制一致，而参数和数据结构就略有不同。select一次性传入三组作用于不同信道的设备fd，分别是输入，输出和错误异常。各组的fd期待各组所特有的，由代码指定的一组事件，如输入信道期待输入就绪，输入挂起和错误等事件。然后，select就挑选调用者关心的fd做poll文件操作，检测返回的掩码，看看是否有fd所属信道感兴趣的事件，比如看看这个属于输出信道的fd有没有输出就绪等一系列的事件发生，一样地，如果有一个fd发生感兴趣事件就返回调用了。select，为了同时处理三组使用不同的事件判断规则的fd，采用了位图的方式表示，一组一个位图，位长度是当中最大的fd值，上限是1024，三组就是3072，而且这还只是传入的位图，还有一样大小的传出的位图。当fd数越来越多时，所需的存储开销比较大。既然，一组fd处理起来比较粗放，那就各个fd自己准备好了。poll()系统调用是System V的多元I/O解决方案。它有三个参数，第一个是pollfd结构的数组指针，也就是指向一组fd及其相关信息的指针，因为这个结构包含的除了fd，还有期待的事件掩码和返回的事件掩码，实质上就是将select的中的fd，传入和传出参数归到一个结构之下，也不再把fd分为三组，也不再硬性规定fd感兴趣的事件，这由调用者自己设定。这样，不使用位图来组织数据，也就不需要位图的全部遍历了。按照一般队列地遍历，每个fd做poll文件操作，检查返回的掩码是否有期待的事件，以及做是否有挂起和错误的必要性检查，如果有事件触发，就可以返回调用了。回到poll和select的共同点，面对高并发多连接的应用情境，它们显现出原来没有考虑到的不足，虽然poll比起select又有所改进了。除了上述的关于每次调用都需要做一次从用户空间到内核空间的拷贝，还有这样的问题，就是当处于这样的应用情境时，poll和select会不得不多次操作，并且每次操作都很有可能需要多次进入睡眠状态，也就是多次全部轮询fd，我们应该怎么处理一些会出现重复而无意义的操作。这些重复而无意义的操作有：1、从用户到内核空间拷贝，既然长期监视这几个fd，甚至连期待的事件也不会改变，那拷贝无疑就是重复而无意义的，我们可以让内核长期保存所有需要监视的fd甚至期待事件，或者可以再需要时对部分期待事件进行修改；2、将当前线程轮流加入到每个fd对应设备的等待队列，这样做无非是哪一个设备就绪时能够通知进程退出调用，聪明的开发者想到，那就找个“代理”的回调函数，代替当前进程加入fd的等待队列好了（这也是我后来才总结出来，Linux的等待队列，实质上是回调函数队列吧，也可以使用宏来将当前进程“加入”等待队列，其实就是将唤醒当前进程的回调函数加入队列）。这样，像poll系统调用一样，做poll文件操作发现尚未就绪时，它就调用传入的一个回调函数，这是epoll指定的回调函数，它不再像以前的poll系统调用指定的回调函数那样，而是就将那个“代理”的回调函数加入设备的等待队列就好了，这个代理的回调函数就自己乖乖地等待设备就绪时将它唤醒，然后它就把这个设备fd放到一个指定的地方，同时唤醒可能在等待的进程，到这个指定的地方取fd就好了。我们把1和2结合起来就可以这样做了，只拷贝一次fd，一旦确定了fd就可以做poll文件操作，如果有事件当然好啦，马上就把fd放到指定的地方，而通常都是没有的，那就给这个fd的等待队列加一个回调函数，有事件就自动把fd放到指定的地方，当前进程不需要再一个个poll和睡眠等待了。 epoll机制就是这样改进的了。诚然，fd少的时候，当前进程一个个地等问题不大，可是现在和尚多了，方丈就不好管了。以前设备事件触发时，只负责唤醒当前进程就好了，而当前进程也只能傻傻地在poll里面等待或者循环，再来一次poll，也不知道这个由设备提供的poll性能如何，能不能检查出当前进程已经在等待了就立即返回，当然，我也不明白为什么做了一遍的poll之后，去掉回调函数指针了，还得再做，不是说好了会去唤醒进程的吗？现在就让事件触发回调函数多做一步。本来设备还没就绪就调用一个回调函数了，现在再在这个回调函数里面做一个注册另一个回调函数的操作，目的就是使得设备事件触发多走一步，不仅仅是唤醒当前进程，还要把自己的fd放到指定的地方。就像收本子的班长，以前得一个个学生地去问有没有本子，如果没有，它还得等待一段时间而后又继续问，现在好了，只走一次，如果没有本子，班长就告诉大家去那里交本子，当班长想起要取本子，就去那里看看或者等待一定时间后离开，有本子到了就叫醒他，然后取走。这个道理很简单，就是老师和班干们常说的，大家多做一点工作，我的工作就轻松很多了，尤其是需要管理的东西越来越多时。这种机制或者说模式，我想在Java的FutureTask里面应该也会用到的，一堆在线程池里面跑着的线程（当然这是任务，不是线程，接口是Callable，不是Runnable.run，是Callable.call，它是可以返回结果的），谁先做好就应该先处理呀，可是难道得一个个问吗？干脆就谁好了，谁就按照既定的操作暴露自己，这样FutureTask的get方法就可以马上知道当前最先完成的线程了，就可以取此线程返回结果了。 epoll由三个系统调用组成，分别是epoll_create，epoll_ctl和epoll_wait。epoll_create用于创建和初始化一些内部使用的数据结构；epoll_ctl用于添加，删除或者修改指定的fd及其期待的事件，epoll_wait就是用于等待任何先前指定的fd事件。关于epoll内部的数据结构，我就不能详细了解了。

下面就是分别实现一个 demo，然后用 ab 去压力测试，每回都是 10W 个请求，根据不同的并发量测试

单进程单线程

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>

#define BUFFER_SIZE 4096

int tcpSrv(int listen_port);

int main(int argc, char** argv) {
    tcpSrv(8080);
    return 0;
}

int tcpSrv(int listen_port) {
    int listenfd, connfd;
    struct sockaddr_in servaddr;
    char buff[BUFFER_SIZE+1];
    int n;

    if ((listenfd = socket(AF_INET, SOCK_STREAM, 0)) == -1) {
        printf("create socket error: %s(errno: %d)\n", strerror(errno), errno);
        return -1;
    }

    memset(&servaddr, 0, sizeof(servaddr));
    servaddr.sin_family = AF_INET;
    servaddr.sin_addr.s_addr = htonl(INADDR_ANY);
    servaddr.sin_port = htons(listen_port);

    if (bind(listenfd, (struct sockaddr *)&servaddr, sizeof(servaddr)) == -1) {
        printf("bind socket error: %s(errno: %d)\n", strerror(errno), errno);
        close(listenfd);
        return -1;
    }

    if (listen(listenfd, 2048) == -1) {
        printf("listen socket error: %s(errno: %d)\n", strerror(errno), errno);
        close(listenfd);
        return -1;
    }

    printf("-----waiting for client's request-----\n");
    while (1) {
        if ((connfd = accept(listenfd, (struct sockaddr *)NULL, NULL)) == -1) {
            printf("accept socket error: %s(errno: %d)\n", strerror(errno), errno);
            close(listenfd);
            return -1;
        }
        memset(&buff, 0, sizeof(buff));
        recv(connfd, buff, BUFFER_SIZE, 0);
        //printf("recv msg:\n%s\n", buff);
        char resp[] = "HTTP/1.0 200 OK\r\n\r\n<html>hello</html>";
        send(connfd, resp, strlen(resp), 0);
        close(connfd);
    }
    
    close(listenfd);
    return 0;
}

一开始觉得这种简单粗暴的方式肯定不行，结果发现还别说，这东西的处理能力还是可以的，看测试结果

单并发

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 1 http://localhost:8080/
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            8080

Document Path:          /
Document Length:        18 bytes

Concurrency Level:      1
Time taken for tests:   7.210 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      3700000 bytes
HTML transferred:       1800000 bytes
Requests per second:    13869.50 [#/sec] (mean)
Time per request:       0.072 [ms] (mean)
Time per request:       0.072 [ms] (mean, across all concurrent requests)
Transfer rate:          501.14 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       4
Processing:     0    0   0.0      0       4
Waiting:        0    0   0.0      0       4
Total:          0    0   0.0      0       4

Percentage of the requests served within a certain time (ms)
  50%      0
  66%      0
  75%      0
  80%      0
  90%      0
  95%      0
  98%      0
  99%      0
 100%      4 (longest request)
linux:~/ab/httpd-2.4.3/support$

10 个并发

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 10 http://localhost:8080/
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            8080

Document Path:          /
Document Length:        18 bytes

Concurrency Level:      10
Time taken for tests:   5.498 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      3700000 bytes
HTML transferred:       1800000 bytes
Requests per second:    18186.97 [#/sec] (mean)
Time per request:       0.550 [ms] (mean)
Time per request:       0.055 [ms] (mean, across all concurrent requests)
Transfer rate:          657.15 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       6
Processing:     0    0   0.1      0       6
Waiting:        0    0   0.1      0       6
Total:          0    1   0.1      1       6

Percentage of the requests served within a certain time (ms)
  50%      1
  66%      1
  75%      1
  80%      1
  90%      1
  95%      1
  98%      1
  99%      1
 100%      6 (longest request)
linux:~/ab/httpd-2.4.3/support$

100 个并发

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 100 http://localhost:8080/
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            8080

Document Path:          /
Document Length:        18 bytes

Concurrency Level:      100
Time taken for tests:   5.816 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      3700000 bytes
HTML transferred:       1800000 bytes
Requests per second:    17194.97 [#/sec] (mean)
Time per request:       5.816 [ms] (mean)
Time per request:       0.058 [ms] (mean, across all concurrent requests)
Transfer rate:          621.30 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    1   0.6      1      11
Processing:     1    5   0.8      5      15
Waiting:        0    2   0.6      2      12
Total:          2    6   0.6      6      15

Percentage of the requests served within a certain time (ms)
  50%      6
  66%      6
  75%      6
  80%      6
  90%      6
  95%      6
  98%      7
  99%      7
 100%     15 (longest request)
linux:~/ab/httpd-2.4.3/support$

150 个并发的时候就不行了

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 150 http://localhost:8080/
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
apr_pollset_poll: The timeout specified has expired (70007)
Total of 99956 requests completed
linux:~/ab/httpd-2.4.3/support$

接下来是多进程模型

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>
#include <unistd.h>
#include <signal.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>

#define BUFFER_SIZE 4096

int tcpSrv(int listen_port);

int main(int argc, char** argv) {
    tcpSrv(8080);
    return 0;
}

int tcpSrv(int listen_port) {
    int listenfd, connfd;
    struct sockaddr_in servaddr;
    char buff[BUFFER_SIZE+1];
    int n;

    if ((listenfd = socket(AF_INET, SOCK_STREAM, 0)) == -1) {
        printf("create socket error: %s(errno: %d)\n", strerror(errno), errno);
        return -1;
    }

    memset(&servaddr, 0, sizeof(servaddr));
    servaddr.sin_family = AF_INET;
    servaddr.sin_addr.s_addr = htonl(INADDR_ANY);
    servaddr.sin_port = htons(listen_port);

    if (bind(listenfd, (struct sockaddr *)&servaddr, sizeof(servaddr)) == -1) {
        printf("bind socket error: %s(errno: %d)\n", strerror(errno), errno);
        close(listenfd);
        return -1;
    }

    if (listen(listenfd, 2048) == -1) {
        printf("listen socket error: %s(errno: %d)\n", strerror(errno), errno);
        close(listenfd);
        return -1;
    }

    printf("-----waiting for client's request-----\n");
    while (1) {
        if ((connfd = accept(listenfd, (struct sockaddr *)NULL, NULL)) == -1) {
            printf("accept socket error: %s(errno: %d)\n", strerror(errno), errno);
            close(listenfd);
            return -1;
        }
        signal(SIGCHLD, SIG_IGN);
        pid_t pid = fork();
        if (pid < 0) {
            printf("fork error: %s(errno: %d)\n", strerror(errno), errno);
            close(listenfd);
            return -1;
        } else if (pid == 0) {
            //printf("in child process\n");
            memset(&buff, 0, sizeof(buff));
            recv(connfd, buff, BUFFER_SIZE, 0);
            //printf("recv msg:\n%s\n", buff);
            char resp[] = "HTTP/1.0 200 OK\r\n\r\n<html>hello</html>";
            send(connfd, resp, strlen(resp), 0);
            close(connfd);
            exit(0);
        } else if (pid > 0) {
            //printf("in parent process\n");
            close(connfd);
            continue;
        }
    }
    
    close(listenfd);
    return 0;
}

单并发

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 1 http://localhost:8080/
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            8080

Document Path:          /
Document Length:        18 bytes

Concurrency Level:      1
Time taken for tests:   41.408 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      3700000 bytes
HTML transferred:       1800000 bytes
Requests per second:    2415.02 [#/sec] (mean)
Time per request:       0.414 [ms] (mean)
Time per request:       0.414 [ms] (mean, across all concurrent requests)
Transfer rate:          87.26 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:     0    0   0.1      0       9
Waiting:        0    0   0.1      0       9
Total:          0    0   0.1      0       9

Percentage of the requests served within a certain time (ms)
  50%      0
  66%      0
  75%      0
  80%      0
  90%      0
  95%      0
  98%      0
  99%      0
 100%      9 (longest request)
linux:~/ab/httpd-2.4.3/support$

10 个并发

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 10 http://localhost:8080/
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            8080

Document Path:          /
Document Length:        18 bytes

Concurrency Level:      10
Time taken for tests:   39.517 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      3700000 bytes
HTML transferred:       1800000 bytes
Requests per second:    2530.53 [#/sec] (mean)
Time per request:       3.952 [ms] (mean)
Time per request:       0.395 [ms] (mean, across all concurrent requests)
Transfer rate:          91.44 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       3
Processing:     0    4   0.4      4      64
Waiting:        0    4   0.4      4      64
Total:          1    4   0.4      4      64

Percentage of the requests served within a certain time (ms)
  50%      4
  66%      4
  75%      4
  80%      4
  90%      4
  95%      4
  98%      4
  99%      4
 100%     64 (longest request)
linux:~/ab/httpd-2.4.3/support$

100 个并发

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 100 http://localhost:8080/
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            8080

Document Path:          /
Document Length:        18 bytes

Concurrency Level:      100
Time taken for tests:   39.980 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      3700000 bytes
HTML transferred:       1800000 bytes
Requests per second:    2501.28 [#/sec] (mean)
Time per request:       39.980 [ms] (mean)
Time per request:       0.400 [ms] (mean, across all concurrent requests)
Transfer rate:          90.38 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.3      0      17
Processing:     0   40   1.7     40     120
Waiting:        0   40   1.7     40     120
Total:          3   40   1.6     40     120

Percentage of the requests served within a certain time (ms)
  50%     40
  66%     40
  75%     41
  80%     41
  90%     41
  95%     42
  98%     43
  99%     44
 100%    120 (longest request)
linux:~/ab/httpd-2.4.3/support$

500 个并发

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 500 http://localhost:8080/
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            8080

Document Path:          /
Document Length:        18 bytes

Concurrency Level:      500
Time taken for tests:   63.222 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      3700000 bytes
HTML transferred:       1800000 bytes
Requests per second:    1581.72 [#/sec] (mean)
Time per request:       316.112 [ms] (mean)
Time per request:       0.632 [ms] (mean, across all concurrent requests)
Transfer rate:          57.15 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0   38 520.0      0   21004
Processing:     3  208 2942.9     51   60202
Waiting:        0  208 2942.9     51   60202
Total:         24  247 3153.2     51   63206

Percentage of the requests served within a certain time (ms)
  50%     51
  66%     52
  75%     52
  80%     52
  90%     53
  95%     54
  98%     55
  99%     61
 100%  63206 (longest request)
linux:~/ab/httpd-2.4.3/support$

1000 个并发

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 1000 http://localhost:8080/
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            8080

Document Path:          /
Document Length:        18 bytes

Concurrency Level:      1000
Time taken for tests:   71.625 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      3700000 bytes
HTML transferred:       1800000 bytes
Requests per second:    1396.16 [#/sec] (mean)
Time per request:       716.252 [ms] (mean)
Time per request:       0.716 [ms] (mean, across all concurrent requests)
Transfer rate:          50.45 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0   68 628.8      0   21002
Processing:     1  442 4622.4     51   68587
Waiting:        0  442 4622.3     51   68587
Total:         17  510 4934.8     51   71588

Percentage of the requests served within a certain time (ms)
  50%     51
  66%     52
  75%     52
  80%     52
  90%     54
  95%     55
  98%     59
  99%   4107
 100%  71588 (longest request)
linux:~/ab/httpd-2.4.3/support$

再来是多线程

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>
#include <unistd.h>
#include <pthread.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>

#define BUFFER_SIZE 4096

int tcpSrv(int listen_port);
void* process_thread(void* sockfd);

int main(int argc, char** argv) {
    tcpSrv(8080);
    return 0;
}

void* process_thread(void* sockfd) {
    //printf("thread created\n");
    int connfd = *((int *)sockfd);
    char buff[BUFFER_SIZE+1];
    memset(&buff, 0, sizeof(buff));
    recv(connfd, buff, BUFFER_SIZE, 0);
    //printf("recv msg:\n%s\n", buff);
    char resp[] = "HTTP/1.0 200 OK\r\n\r\n<html>hello</html>";
    send(connfd, resp, strlen(resp), 0);
    close(connfd);
    //printf("thread about to exit\n");
    pthread_exit(NULL);
}

int tcpSrv(int listen_port) {
    int listenfd, connfd;
    struct sockaddr_in servaddr;

    if ((listenfd = socket(AF_INET, SOCK_STREAM, 0)) == -1) {
        printf("create socket error: %s(errno: %d)\n", strerror(errno), errno);
        return -1;
    }

    memset(&servaddr, 0, sizeof(servaddr));
    servaddr.sin_family = AF_INET;
    servaddr.sin_addr.s_addr = htonl(INADDR_ANY);
    servaddr.sin_port = htons(listen_port);

    if (bind(listenfd, (struct sockaddr *)&servaddr, sizeof(servaddr)) == -1) {
        printf("bind socket error: %s(errno: %d)\n", strerror(errno), errno);
        close(listenfd);
        return -1;
    }

    if (listen(listenfd, 2048) == -1) {
        printf("listen socket error: %s(errno: %d)\n", strerror(errno), errno);
        close(listenfd);
        return -1;
    }

    printf("-----waiting for client's request-----\n");
    while (1) {
        if ((connfd = accept(listenfd, (struct sockaddr *)NULL, NULL)) == -1) {
            printf("accept socket error: %s(errno: %d)\n", strerror(errno), errno);
            close(listenfd);
            return -1;
        }
        //printf("main thread about to create thread\n");
        pthread_t tid;
        pthread_attr_t opt;
        pthread_attr_init(&opt);
        pthread_attr_setdetachstate(&opt, PTHREAD_CREATE_DETACHED);
        if (pthread_create(&tid, &opt, process_thread, &connfd) != 0) {
            printf("error line: %d\t", __LINE__);
            perror(__FILE__);
            return -1;
        }
        pthread_attr_destroy(&opt);
        //printf("main thread finish creating thread\n");
    }
    
    close(listenfd);
    return 0;
}

单并发

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 1 http://localhost:8080/123
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            8080

Document Path:          /123
Document Length:        18 bytes

Concurrency Level:      1
Time taken for tests:   7.629 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      3700000 bytes
HTML transferred:       1800000 bytes
Requests per second:    13108.37 [#/sec] (mean)
Time per request:       0.076 [ms] (mean)
Time per request:       0.076 [ms] (mean, across all concurrent requests)
Transfer rate:          473.64 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       4
Processing:     0    0   0.1      0      24
Waiting:        0    0   0.1      0      23
Total:          0    0   0.1      0      24

Percentage of the requests served within a certain time (ms)
  50%      0
  66%      0
  75%      0
  80%      0
  90%      0
  95%      0
  98%      0
  99%      0
 100%     24 (longest request)
linux:~/ab/httpd-2.4.3/support$

10 个并发

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 10 http://localhost:8080/123
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
apr_pollset_poll: The timeout specified has expired (70007)
Total of 99991 requests completed
linux:~/ab/httpd-2.4.3/support$

至于 select 和 poll 就没有写了，直接写 epoll 模型

#include <iostream>
#include <sys/socket.h>
#include <sys/epoll.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <fcntl.h>
#include <unistd.h>
#include <stdio.h>
#include <errno.h>

using namespace std;

#define MAX_LINE 1024
#define OPEN_MAX 100
#define LISTENQ 1024
#define SERV_PORT 8080
#define INFTIM 1000
#define MAX_EVENTS 1024
#define EP_TIMEOUT 500

int tcpSrv(int listen_port);
int set_non_block(int fd);

int main(int argc, char* argv[]) {
    tcpSrv(SERV_PORT);
    return 0;
}

int set_non_block(int fd) {
    int fd_fl = fcntl(fd, F_GETFL); 
    fd_fl |= O_NONBLOCK;
    if (fcntl(fd, F_SETFL, fd_fl) < 0) {
        printf("%d\n:", __LINE__);perror(__FILE__);
        return -1;
    }
    return 0;
}

int tcpSrv(int listen_port) {
    int listenfd = socket(AF_INET, SOCK_STREAM, 0);
    if (-1 == listenfd) {
        printf("%d\n:", __LINE__);perror(__FILE__);
        return -1;
    }
    set_non_block(listenfd);

    struct epoll_event ev;
    ev.data.fd = listenfd;
    ev.events = EPOLLIN|EPOLLET;
    int epfd = epoll_create(4096);
    epoll_ctl(epfd, EPOLL_CTL_ADD, listenfd, &ev);

    struct sockaddr_in serveraddr;
    bzero(&serveraddr, sizeof(serveraddr));
    serveraddr.sin_family = AF_INET;
    char* local_addr = "127.0.0.1";
    inet_aton(local_addr, &(serveraddr.sin_addr));
    serveraddr.sin_port = htons(listen_port);

    if (bind(listenfd, (sockaddr*)&serveraddr, sizeof(serveraddr)) == -1) {
        printf("%d\n:", __LINE__);perror(__FILE__);
        close(listenfd);
        return -1;
    }
    
    if (listen(listenfd, LISTENQ) == -1) {
        printf("%d\n:", __LINE__);perror(__FILE__);
        close(listenfd);
        return -1;
    }

    struct epoll_event events[MAX_EVENTS];
    //struct sockaddr_in clientaddr;
    //socklen_t clilen;
    while (1) {
        int nfds = epoll_wait(epfd, events, MAX_EVENTS, EP_TIMEOUT);
        //printf("events count:%d\n", nfds);
        for (int i=0; i<nfds; i++) {
            if (events[i].data.fd == listenfd) {
                while (1) {
                    int connfd = accept(listenfd, NULL, NULL);
                    if (connfd < 0) {
                        //printf("%d\n:", __LINE__);perror(__FILE__);
                        break;
                    }
                    //char *str_client_addr = inet_ntoa(clientaddr.sin_addr);
                    //cout<<"accept a connection from: "<<str_client_addr<<endl;
                    ev.data.fd = connfd;
                    ev.events = EPOLLIN | EPOLLET;
                    epoll_ctl(epfd, EPOLL_CTL_ADD, connfd, &ev);
                }
            } else if (events[i].events & EPOLLIN) {
                //cout<<"EPOLLIN"<<endl;
                int sockfd = events[i].data.fd;
                if (sockfd < 0) {
                    printf("%d\n:", __LINE__);perror(__FILE__);
                    continue;
                }
                char buffer[MAX_LINE+1];
                memset(buffer, 0, sizeof(buffer));
                read(sockfd, buffer, MAX_LINE);
                //printf("%s\n", buffer);
                set_non_block(sockfd);
                char html_resp[] = "HTTP/1.1 200 OK\r\n\r\n<html>epoll</html>";
                write(sockfd, html_resp, strlen(html_resp));
                epoll_ctl(epfd, EPOLL_CTL_DEL, sockfd, NULL);
                close(sockfd);
            }
        }
    }
    
    return 0;
}

单并发

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 1 http://localhost:8080/123
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            8080

Document Path:          /123
Document Length:        18 bytes

Concurrency Level:      1
Time taken for tests:   7.474 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      3700000 bytes
HTML transferred:       1800000 bytes
Requests per second:    13380.33 [#/sec] (mean)
Time per request:       0.075 [ms] (mean)
Time per request:       0.075 [ms] (mean, across all concurrent requests)
Transfer rate:          483.47 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       4
Processing:     0    0   0.0      0       5
Waiting:        0    0   0.0      0       5
Total:          0    0   0.0      0       5

Percentage of the requests served within a certain time (ms)
  50%      0
  66%      0
  75%      0
  80%      0
  90%      0
  95%      0
  98%      0
  99%      0
 100%      5 (longest request)

10 个并发

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 10 http://localhost:8080/123
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            8080

Document Path:          /123
Document Length:        18 bytes

Concurrency Level:      10
Time taken for tests:   5.991 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      3700000 bytes
HTML transferred:       1800000 bytes
Requests per second:    16691.90 [#/sec] (mean)
Time per request:       0.599 [ms] (mean)
Time per request:       0.060 [ms] (mean, across all concurrent requests)
Transfer rate:          603.13 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       6
Processing:     0    0   0.1      0       7
Waiting:        0    0   0.1      0       6
Total:          0    1   0.2      1       7

Percentage of the requests served within a certain time (ms)
  50%      1
  66%      1
  75%      1
  80%      1
  90%      1
  95%      1
  98%      1
  99%      1
 100%      7 (longest request)

100 个并发

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 100 http://localhost:8080/123
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            8080

Document Path:          /123
Document Length:        18 bytes

Concurrency Level:      100
Time taken for tests:   6.233 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      3700000 bytes
HTML transferred:       1800000 bytes
Requests per second:    16043.99 [#/sec] (mean)
Time per request:       6.233 [ms] (mean)
Time per request:       0.062 [ms] (mean, across all concurrent requests)
Transfer rate:          579.71 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    1   1.0      1       9
Processing:     1    5   0.9      5      12
Waiting:        1    2   0.9      2      10
Total:          2    6   1.4      6      15

Percentage of the requests served within a certain time (ms)
  50%      6
  66%      7
  75%      7
  80%      7
  90%      8
  95%      8
  98%     10
  99%     10
 100%     15 (longest request)

1000 个并发

linux:~/ab/httpd-2.4.3/support$ ./ab -n 100000 -c 1000 http://localhost:8080/123
This is ApacheBench, Version 2.3 <$Revision: 1373084 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            8080

Document Path:          /123
Document Length:        18 bytes

Concurrency Level:      1000
Time taken for tests:   7.099 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      3700000 bytes
HTML transferred:       1800000 bytes
Requests per second:    14087.17 [#/sec] (mean)
Time per request:       70.987 [ms] (mean)
Time per request:       0.071 [ms] (mean, across all concurrent requests)
Transfer rate:          509.01 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        1   26 115.1     19    3054
Processing:    16   44   9.7     43      76
Waiting:        1   18   8.0     18      54
Total:         20   70 115.7     65    3103

Percentage of the requests served within a certain time (ms)
  50%     65
  66%     73
  75%     79
  80%     82
  90%     89
  95%     95
  98%    101
  99%    106
 100%   3103 (longest request)
linux:~/ab/httpd-2.4.3/support$

可以看到 epoll 确实是很猛的，这里有一个地方需要注意，我调了两天，就是在 epoll 的 EPOLLIN 事件发生在 listenfd 上的时候，这个时候需要去 accept，这个应该都没有什么问题，但是我一开始只是 accept 了一个，而实际上，这个时候，是可能有多个连接在等待 accept 的，于是就导致了并发能力很不行，正确的做法应该是一直 accept，直到没有东西可以 accept 了，这样才能有并发能力。

PHP 发 UDP 包时出错

Fri, 24 Aug 2012 23:08:53 +0800

今天金泉给了一个问题，说线上的腾讯 SDK PHP 版本的跑起来报错，看了一下，调用的代码类似这样

<?php
require("inc/OpenApiV3.php");

// 应用基本信息
$appid = xxxxxx;
$appkey = 'xxxxxxx';

// OpenAPI的服务器IP
// 最新的API服务器地址请参考wiki文档: http://wiki.open.qq.com/wiki/API3.0%E6%96%87%E6%A1%A3
$server_name = '119.147.19.43';

// 用户的OpenID/OpenKey
$openid = 'xxxxxxx';
$openkey = 'xxxxxxxx';

// 所要访问的平台, pf的其他取值参考wiki文档: http://wiki.open.qq.com/wiki/API3.0%E6%96%87%E6%A1%A3
$pf = 'qzone';


$sdk = new OpenApiV3($appid, $appkey);
$sdk->setServerName($server_name);

$ret = get_user_info($sdk, $openid, $openkey, $pf);
print_r("===========================n");
print_r($ret);

/**
 * 获取好友资料
 *
 * @param object $sdk OpenApiV3 Object
 * @param string $openid openid
 * @param string $openkey openkey
 * @param string $pf 平台
 * @return array 好友资料数组
 */
function get_user_info(&$sdk, $openid, $openkey, $pf)
{
	$params = array(
		'openid' => $openid,
		'openkey' => $openkey,
		'pf' => $pf,
	);

	$script_name = '/v3/user/get_info';

	return $sdk->api($script_name, $params);
}

// end of script

跑起来之后，报错是

Warning: socket_sendto() [function.socket-sendto]: unable to write to socket [1]: Operation not permitted in /home/naniwco1/public_html/draw/inc/lib/SnsStat.php on line 40

找到那个 SnsStat 文件，是一个用于统计的类，代码是这样的

<?php

/**
 * 统计上报接口调用情况
 *
 * @version 3.0.2
 * @author open.qq.com
 * @copyright © 2011, Tencent Corporation. All rights reserved.
 * @ History:
 *               3.0.2 | sparkeli | 2012-03-06 15:33:04 | initialize statistic fuction which can report API's access time and number to background server

 */

class SnsStat
{
	/**
	 * 执行一个 统计上报
	 *
	 * @param string $stat_url 统计上报的URL
	 * @param float $start_time 统计开始时间
	 * @param array $params 统计参数数组
	 * @return
	 */
	static public function statReport($stat_url, $start_time, $params)
	{
		$end_time = self::getTime();
		$params['time'] = round($end_time - $start_time, 4);
		$params['timestamp'] = time();
		$params['collect_point'] = 'sdk-php-v3';
		$stat_str = json_encode($params);
		//发送上报信息
		$host_ip = gethostbyname($stat_url);
		if ($host_ip != $stat_url)
		{
			$sock = socket_create(AF_INET, SOCK_DGRAM, 0);
			if (false === $sock)
			{
				return;
			}
			socket_sendto($sock, $stat_str, strlen($stat_str), 0, $host_ip, 19888);
			socket_close($sock);
		}
	}

	static public function getTime()
	{
		list($usec, $sec) = explode(" ", microtime());
		return ((float)$usec + (float)$sec);
	}
}

// end of script

把代码迁移到本地环境跑，在已经开了 socket PHP 扩展的前提下，是很顺利的，没有报错，于是考虑环境问题，google 一阵搜，看到这里，http://www.bluehostforum.com/showthread.php?23649-PHP-socket_send()-Not-permitted

I took a quick look. Your script is attempting to connect out on port 27025 which is blocked unless it has been whitelisted to be allowed through the firewall. Contact support and they’ll be happy to open the port for you.

On an unrelated note, there is NEVER a reason to provide world-writable permissions to a file or directory on Bluehost’s servers. All script execution is done by your user which means that a file need only have user permissions (7xx for execution and 6xx for writing). World-writable permissions simply open your account to compromise.

说是防火墙问题，有一定的可能，但是好像没有办法验证，于是只能注释掉了。。

历史评论

home business risks (2013-04-25 06:18:55):

Hi, yeah this post is truly nice and I have learned lot of things from it regarding blogging. thanks.

overdrive effects pedal reviews (2014-09-25 13:27:51):

Hello to all, since I am genuinely eager of reading this website’s post to be updated regularly. It consists of nice stuff.

Python 网络编程初步

Thu, 02 Aug 2012 22:33:04 +0800

晚上跟着书上教程写了一些 socket 收发的代码，记下来当作备忘吧：

原生 socket 的 TCP 收发：

#!/usr/bin/env python

from socket import *
from time import ctime

HOST = ''
PORT = 21567
BUFSIZ = 1024
ADDR = (HOST, PORT)

tcpSerSock = socket(AF_INET, SOCK_STREAM)
tcpSerSock.bind(ADDR)
tcpSerSock.listen(5)

while True:
    print 'waiting for connection...'
    tcpCliSock, addr = tcpSerSock.accept()
    print '...connected from: ', addr

    while True:
        data = tcpCliSock.recv(BUFSIZ)
        if not data:
            break
        print 'received: ' + data
        tcpCliSock.send('[%s] %s ' % (ctime(), data))
    tcpCliSock.close()
tcpSerSock.close()

其他都好懂，解释一下 13 行，文档是这么说的：

listen(backlog)

Enable a server to accept connections. The backlog argument must be at least 0 (if it is lower, it is set to 0); it specifies the number of unaccepted connections that the system will allow before refusing new connections.

也就是缓冲区大小，能暂存的连接数的意思了。

还有 17 行，返回了一个元组，文档的解释：

accept() -> (socket object, address info)

Wait for an incoming connection. Return a new socket representing the connection, and the address of the client. For IP sockets, the address info is a pair (hostaddr, port).

下面是客户端的代码：

#!/usr/bin/env python

from socket import *

HOST = 'localhost'
PORT = 21567
BUFSIZ = 1024
ADDR = (HOST, PORT)

tcpCliSock = socket(AF_INET, SOCK_STREAM)
tcpCliSock.connect(ADDR)

while True:
    data = raw_input('> ')
    if not data:
        break
    tcpCliSock.send(data)
    data = tcpCliSock.recv(BUFSIZ)
    if not data:
        break
    print data
tcpCliSock.close()

原生 socket 的 UDP 收发

#!/usr/bin/env python

from socket import *
from time import ctime

HOST = ''
PORT = 21567
BUFSIZ = 1024
ADDR = (HOST, PORT)

udpSerSock = socket(AF_INET, SOCK_DGRAM)
udpSerSock.bind(ADDR)

while True:
    print 'waiting for message...'
    data, addr = udpSerSock.recvfrom(BUFSIZ)
    udpSerSock.sendto('[%s] %s' % (ctime(), data), addr)
    print '...received from and returned to:', addr

udpSerSock.close()

#!/usr/bin/env python

from socket import *

HOST = 'localhost'
PORT = 21567
BUFSIZ = 1024
ADDR = (HOST, PORT)

udpCliSock = socket(AF_INET, SOCK_DGRAM)

while True:
    data = raw_input('> ')
    if not data:
        break
    udpCliSock.sendto(data, ADDR)
    data, ADDR = udpCliSock.recvfrom(BUFSIZ)
    if not data:
        break
    print data

udpCliSock.close()

一种特殊情况下的优酷广告过滤

Thu, 26 Jul 2012 17:34:14 +0800

最近在看优酷上的电视剧，烦了那个广告，于是想到过滤，一般来说，过滤优酷的广告，网上都有现成的方法，思路是改 hosts 干扰域名解析，这是屏蔽广告的常见思路，而且比较通用，另外，对付优酷的特殊情况，他会使用 flash 判断广告的播放情况，如果没有播放成功，那么会给一个提示，所以需要修改 flash 的一些相关文件，具体的是修改

C:UsersZRJAppDataRoamingMacromediaFlash Player#SharedObjectsBGTENWSR

下面的一个 static.youku.com 文件夹，删掉然后用一个 com 文件占用位置，让其无法写入，于是这个问题也得以解决。

但是我这里的情况稍稍不同，由于处在校内网，所以出入口速度比较慢，刚好知道有一个直连 Internet 的代理服务器，于是对于一些大的流媒体文件，我一般走这个代理去下载，速度还是比较快的，代理的切换方法的话，我是用 chrome 下的 Proxy SwitchySharp，还比较好用，可以自定义规则来控制代理，这个东西一般是用来做本地 goagent 配合着用的，对于被墙的域名走本地搭起的 goagent 代理，对于普通域名走直接连接的通道。

同时他还有一个手动切换的选项，我一般在遇到流媒体的时候会手动切换到那台代理，问题就出在这里：当我们把优酷负责广告的域名通过 hosts 干扰了解析之后，正常情况下是可以奏效的，但是，如果这个时候走代理，那么本地修改的 hosts 就鞭长莫及了。我觉得这是因为他的处理流程导致的：首先当 chrome 有一个 HTTP 请求的时候，被 SwitchySharp 拦截，这个时候，如果 SwitchySharp 在匹配列表中没有发现可以匹配的规则的话，那么就放行，然后 chrome 继续处理请求的流程，这里才发起解析域名的步骤，于是 hosts 才有用，相反，如果 SwitchySharp 能匹配到规则的话，就直接把请求转发到代理去了，于是本地的 hosts 根本就不会被访问，悲剧就这么发生了。

本地 hosts 修改没用的话，那么怎么办呢，沿用这个思路，很快的可以想到，既然如此，我们可以起一个根本不存在的代理，就用本地 127.0.0.1 的地址，来接受那些广告域名，然后配置 SwitchySharp ，让他把所有的广告请求都转到这个不存在的代理，然后正常的视频就从可用的代理那里请求数据。

可惜后面跟着还有问题，问题在于，对于真正的视频文件，优酷并没有使用一个域名来提供内容，而是使用一个类似

http://f.youku.com/player/getFlvPath/sid/134329032214611466521_00/st/flv/fileid/0300020A004FDF4C44FD8A032DBBC7F1B1B40B-AC36-FF2F-113A-10996097DEBD?start=1&K=c1a370c934fd64c2241114b0&hd=0&myp=0&ts=406&ymovie=1

的地址来回应一个 302 到一个 IP 命名的服务器上来提供，估计是 CDN 吧，这样的话，想要使用一个通配的 URL 规则来匹配这些视频就没用了，因为就算那个 f.youku.com 域名下的请求是通过代理得到的，也帮不了什么忙，那只是一个 302 而已，回想之前为什么没有这样的问题呢，因为当时我是把全局的请求都通过可用代理去取内容的，所以根本都不需要关心这些细节。

于是最后就只能配置成：广告域名的请求走一个本地的空白代理，其他的所有请求走可用代理，然后在看完视频后，再把那条规则暂时屏蔽掉。

从评论通知学电子邮件的原理

Sat, 14 Jul 2012 16:48:41 +0800

WordPress 自带了一个当有人评论时发邮件通知管理员的功能，但是没有一个游客的评论被管理回复时通知游客的功能，正好好奇 WordPress 的邮件发送功能是怎么实现的，于是昨天下午就在折腾这个东西，中间由于一些插曲，于是现在才弄好，记录一下。

首先尝试的思路是去找插件，找到一个很久以前的插件，WordPress 官网都已经提示两年没有更新了，估计跟这个版本很难兼容，于是开始自己动手丰衣足食，这样才能学到东西嘛。

找了一下找到这样的一段代码，文章说放到 functions.php 可以生效

/* 邮件通知 by Qiqiboy */
 function comment_mail_notify($comment_id) {
     $comment = get_comment($comment_id);//根据id获取这条评论相关数据
     $content=$comment->comment_content;
     //对评论内容进行匹配
     $match_count=preg_match_all('/<a href="#comment-([0-9]+)?" rel="nofollow">/si',$content,$matchs);
     if($match_count>0){//如果匹配到了
         foreach($matchs[1] as $parent_id){//对每个子匹配都进行邮件发送操作
             SimPaled_send_email($parent_id,$comment);
         }
     }elseif($comment->comment_parent!='0'){//以防万一，有人故意删了@回复，还可以通过查找父级评论id来确定邮件发送对象
         $parent_id=$comment->comment_parent;
         SimPaled_send_email($parent_id,$comment);
     }else return;
 }
 add_action('comment_post', 'comment_mail_notify');
 function SimPaled_send_email($parent_id,$comment){//发送邮件的函数 by Qiqiboy.com
     $admin_email = get_bloginfo ('admin_email');//管理员邮箱
     $parent_comment=get_comment($parent_id);//获取被回复人（或叫父级评论）相关信息
     $author_email=$comment->comment_author_email;//评论人邮箱
     $to = trim($parent_comment->comment_author_email);//被回复人邮箱
     $spam_confirmed = $comment->comment_approved;
     if ($spam_confirmed != 'spam' && $to != $admin_email && $to != $author_email) {
         $wp_email = 'wp@' . preg_replace('#^www.#', '', strtolower($_SERVER['SERVER_NAME'])); // e-mail 發出點, no-reply 可改為可用的 e-mail.
         $subject = '您在 [' . get_option("blogname") . '] 的留言有了回应';
         $message = '<div style="background-color:#eef2fa;border:1px solid #d8e3e8;color:#111;padding:0 15px;-moz-border-radius:5px;-webkit-border-radius:5px;-khtml-border-radius:5px;">
             <p>' . trim(get_comment($parent_id)->comment_author) . ', 您好!</p>
             <p>您曾在《' . get_the_title($comment->comment_post_ID) . '》的留言:<br />'
             . trim(get_comment($parent_id)->comment_content) . '</p>
             <p>' . trim($comment->comment_author) . ' 给你的回复:<br />'
             . trim($comment->comment_content) . '<br /></p>
             <p>您可以点击 <a href="' . htmlspecialchars(get_comment_link($parent_id,array("type" => "all"))) . '">查看回复的完整內容</a></p>
             <p>欢迎再度光临 <a href="' . get_option('home') . '">' . get_option('blogname') . '</a></p>
             <p>(此邮件由系统自动发出, 请勿回复.)</p></div>';
         $from = "From: "" . get_option('blogname') . "" <$wp_email>";
         $headers = "$fromnContent-Type: text/html; charset=" . get_option('blog_charset') . "n";

         // 从全局域获得phpmailer对象
	     global $phpmailer;

		// Make sure the PHPMailer class has been instantiated
		// (copied verbatim from wp-includes/pluggable.php)
		// (Re)create it, if it's gone missing
		if ( !is_object( $phpmailer ) || !is_a( $phpmailer, 'PHPMailer' ) ) {
			require_once ABSPATH . WPINC . '/class-phpmailer.php';
			require_once ABSPATH . WPINC . '/class-smtp.php';
			$phpmailer = new PHPMailer();
		}
		// Set SMTPDebug to level 2
		$phpmailer->SMTPDebug = 2;

		// Start output buffering to grab smtp debugging output
		ob_start();

        $send_result = wp_mail( $to, $subject, $message, $headers );

        ob_get_clean();
         //var_dump($send_result);
         //var_dump($phpmailer);
     }
 }

于是就找到 functions.php 放进去，但是立刻就报错了，重新排查了一下，发现有两个 functions.php 文件，而正确的做法是要放到主题的文件下的 functions.php 文件，于是移过去，再试，还错，于是开始纠结，回去看代码。

这个函数的思路很简单，就是找评论的父节点，然后取得其 email 地址，拼接字符串，调用 wp_mail 函数来发送，最后再使用全局的钩子，把这个函数 hiook 到评论的时候触发。

看来重点在 wp_mail 函数，跟进去看看

function wp_mail( $to, $subject, $message, $headers = '', $attachments = array() ) {
	// Compact the input, apply the filters, and extract them back out
	extract( apply_filters( 'wp_mail', compact( 'to', 'subject', 'message', 'headers', 'attachments' ) ) );

	if ( !is_array($attachments) )
		$attachments = explode( "n", str_replace( "rn", "n", $attachments ) );

	global $phpmailer;

	// (Re)create it, if it's gone missing
	if ( !is_object( $phpmailer ) || !is_a( $phpmailer, 'PHPMailer' ) ) {
		require_once ABSPATH . WPINC . '/class-phpmailer.php';
		require_once ABSPATH . WPINC . '/class-smtp.php';
		$phpmailer = new PHPMailer( true );
	}

	// Headers
	if ( empty( $headers ) ) {
		$headers = array();
	} else {
		if ( !is_array( $headers ) ) {
			// Explode the headers out, so this function can take both
			// string headers and an array of headers.
			$tempheaders = explode( "n", str_replace( "rn", "n", $headers ) );
		} else {
			$tempheaders = $headers;
		}
		$headers = array();
		$cc = array();
		$bcc = array();

		// If it's actually got contents
		if ( !empty( $tempheaders ) ) {
			// Iterate through the raw headers
			foreach ( (array) $tempheaders as $header ) {
				if ( strpos($header, ':') === false ) {
					if ( false !== stripos( $header, 'boundary=' ) ) {
						$parts = preg_split('/boundary=/i', trim( $header ) );
						$boundary = trim( str_replace( array( "'", '"' ), '', $parts[1] ) );
					}
					continue;
				}
				// Explode them out
				list( $name, $content ) = explode( ':', trim( $header ), 2 );

				// Cleanup crew
				$name    = trim( $name    );
				$content = trim( $content );

				switch ( strtolower( $name ) ) {
					// Mainly for legacy -- process a From: header if it's there
					case 'from':
						if ( strpos($content, '<' ) !== false ) {
							// So... making my life hard again?
							$from_name = substr( $content, 0, strpos( $content, '<' ) - 1 );
							$from_name = str_replace( '"', '', $from_name );
							$from_name = trim( $from_name );

							$from_email = substr( $content, strpos( $content, '<' ) + 1 );
							$from_email = str_replace( '>', '', $from_email );
							$from_email = trim( $from_email );
						} else {
							$from_email = trim( $content );
						}
						break;
					case 'content-type':
						if ( strpos( $content, ';' ) !== false ) {
							list( $type, $charset ) = explode( ';', $content );
							$content_type = trim( $type );
							if ( false !== stripos( $charset, 'charset=' ) ) {
								$charset = trim( str_replace( array( 'charset=', '"' ), '', $charset ) );
							} elseif ( false !== stripos( $charset, 'boundary=' ) ) {
								$boundary = trim( str_replace( array( 'BOUNDARY=', 'boundary=', '"' ), '', $charset ) );
								$charset = '';
							}
						} else {
							$content_type = trim( $content );
						}
						break;
					case 'cc':
						$cc = array_merge( (array) $cc, explode( ',', $content ) );
						break;
					case 'bcc':
						$bcc = array_merge( (array) $bcc, explode( ',', $content ) );
						break;
					default:
						// Add it to our grand headers array
						$headers[trim( $name )] = trim( $content );
						break;
				}
			}
		}
	}

	// Empty out the values that may be set
	$phpmailer->ClearAddresses();
	$phpmailer->ClearAllRecipients();
	$phpmailer->ClearAttachments();
	$phpmailer->ClearBCCs();
	$phpmailer->ClearCCs();
	$phpmailer->ClearCustomHeaders();
	$phpmailer->ClearReplyTos();

	// From email and name
	// If we don't have a name from the input headers
	if ( !isset( $from_name ) )
		$from_name = 'WordPress';

	/* If we don't have an email from the input headers default to wordpress@$sitename
	 * Some hosts will block outgoing mail from this address if it doesn't exist but
	 * there's no easy alternative. Defaulting to admin_email might appear to be another
	 * option but some hosts may refuse to relay mail from an unknown domain. See
	 * http://trac.wordpress.org/ticket/5007.
	 */

	if ( !isset( $from_email ) ) {
		// Get the site domain and get rid of www.
		$sitename = strtolower( $_SERVER['SERVER_NAME'] );
		if ( substr( $sitename, 0, 4 ) == 'www.' ) {
			$sitename = substr( $sitename, 4 );
		}

		$from_email = 'wordpress@' . $sitename;
	}

	// Plugin authors can override the potentially troublesome default
	$phpmailer->From     = apply_filters( 'wp_mail_from'     , $from_email );
	$phpmailer->FromName = apply_filters( 'wp_mail_from_name', $from_name  );

	// Set destination addresses
	if ( !is_array( $to ) )
		$to = explode( ',', $to );

	foreach ( (array) $to as $recipient ) {
		try {
			// Break $recipient into name and address parts if in the format "Foo <bar@baz.com>"
			$recipient_name = '';
			if( preg_match( '/(.+)s?<(.+)>/', $recipient, $matches ) ) {
				if ( count( $matches ) == 3 ) {
					$recipient_name = $matches[1];
					$recipient = $matches[2];
				}
			}
			$phpmailer->AddAddress( trim( $recipient ), $recipient_name);
		} catch ( phpmailerException $e ) {
			continue;
		}
	}

	// Set mail's subject and body
	$phpmailer->Subject = $subject;
	$phpmailer->Body    = $message;

	// Add any CC and BCC recipients
	if ( !empty( $cc ) ) {
		foreach ( (array) $cc as $recipient ) {
			try {
				// Break $recipient into name and address parts if in the format "Foo <bar@baz.com>"
				$recipient_name = '';
				if( preg_match( '/(.+)s?<(.+)>/', $recipient, $matches ) ) {
					if ( count( $matches ) == 3 ) {
						$recipient_name = $matches[1];
						$recipient = $matches[2];
					}
				}
				$phpmailer->AddCc( trim($recipient), $recipient_name );
			} catch ( phpmailerException $e ) {
				continue;
			}
		}
	}

	if ( !empty( $bcc ) ) {
		foreach ( (array) $bcc as $recipient) {
			try {
				// Break $recipient into name and address parts if in the format "Foo <bar@baz.com>"
				$recipient_name = '';
				if( preg_match( '/(.+)s?<(.+)>/', $recipient, $matches ) ) {
					if ( count( $matches ) == 3 ) {
						$recipient_name = $matches[1];
						$recipient = $matches[2];
					}
				}
				$phpmailer->AddBcc( trim($recipient), $recipient_name );
			} catch ( phpmailerException $e ) {
				continue;
			}
		}
	}

	// Set to use PHP's mail()
	$phpmailer->IsMail();

	// Set Content-Type and charset
	// If we don't have a content-type from the input headers
	if ( !isset( $content_type ) )
		$content_type = 'text/plain';

	$content_type = apply_filters( 'wp_mail_content_type', $content_type );

	$phpmailer->ContentType = $content_type;

	// Set whether it's plaintext, depending on $content_type
	if ( 'text/html' == $content_type )
		$phpmailer->IsHTML( true );

	// If we don't have a charset from the input headers
	if ( !isset( $charset ) )
		$charset = get_bloginfo( 'charset' );

	// Set the content-type and charset
	$phpmailer->CharSet = apply_filters( 'wp_mail_charset', $charset );

	// Set custom headers
	if ( !empty( $headers ) ) {
		foreach( (array) $headers as $name => $content ) {
			$phpmailer->AddCustomHeader( sprintf( '%1$s: %2$s', $name, $content ) );
		}

		if ( false !== stripos( $content_type, 'multipart' ) && ! empty($boundary) )
			$phpmailer->AddCustomHeader( sprintf( "Content-Type: %s;nt boundary="%s"", $content_type, $boundary ) );
	}

	if ( !empty( $attachments ) ) {
		foreach ( $attachments as $attachment ) {
			try {
				$phpmailer->AddAttachment($attachment);
			} catch ( phpmailerException $e ) {
				continue;
			}
		}
	}

	do_action_ref_array( 'phpmailer_init', array( &$phpmailer ) );

	// Send!
	try {
		$phpmailer->Send();
	} catch ( phpmailerException $e ) {
		return false;
	}

	return true;
}

这个函数做的事情其实比较基本，就是根据 SMTP 协议来准备字符串，处理附件之类的事情。

这里需要温习一下 SMTP 协议，之前安装 WordPress 的时候也说过，想不通邮件的发送机理，于是最近就好好的读了一下 TCP/IP 协议族的东西，现在理论上基本理解了，其实 SMTP 无非就是两个邮箱服务器之间基于 TCP 流上的字符串交互，从 TCP 的层面看，跟 HTTP 没有本质区别，而这些 TCP 也好， UDP 也好的数据包，从 IP 层面也是没有区别的，本质都是一个要送达互联网另一端主机的数据包。

SMTP 协议的交互是类似 FTP ， HTTP 那样的，发起方发送一条开始通信的字符串，接收方开始回应，指示下一步动作，例如要求发送者邮箱，要求回送邮箱，要求正文等等，发送方就依次按照要求，把这些数据一一交付，接收方把这些数据投放到对应的用户的邮箱，然后回应成功或者用户没找到之类的，就这么结束。

根据维基百科，一次典型的交互类似这样

S: 220 www.example.com ESMTP Postfix
C: HELO mydomain.com
S: 250 Hello mydomain.com
C: MAIL FROM: <sender@mydomain.com>
S: 250 Ok
C: RCPT TO: <friend@example.com>
S: 250 Ok
C: DATA
S: 354 End data with <CR><LF>.<CR><LF>
C: Subject: test message
C: From:""< sender@mydomain.com>
C: To:""< friend@example.com>
C:
C: Hello,
C: This is a test.
C: Goodbye.
C: .
S: 250 Ok: queued as 12345
C: quit
S: 221 Bye

可以看到也是类似 HTTP 那样用 200 ， 250 ，之类的数码来表示状态的。

至于附件，这个 HTTP 也比较类似，一开始的时候是没有考虑附件的，后来有这个需求的时候，面临的问题就是怎么兼容，让字符形式可以传递二进制的文件，解决的方案是使用 Base64 编码，顾名思义，Base64 就是一种使用 a-z A-z 0-9 共 26 + 26 + 10 = 62 加上另外两个其他的可打印字符来编码二进制文件的方法，这样的目的就是可以用字符来承载二进制，进行传输，进一步的了解可以看维基百科，http://zh.wikipedia.org/wiki/Base64 ，有意思的是，SMTP 的文本附件混杂也是像 HTTP 那样用 boundary 来分割的。这点在上面的函数中也有体现。

现在再来回头看前面的函数就比较好理解了，无非就是准备一些交流的 SMTP 内容，跟对方服务器交互，最后看到调用了 phpmailer 对象的方法把东西发出去，这个的内容比较长，就不贴出来了，进去是一个封装的很好的发送类，基本上只要关心接口就可以。

另外，由于 SAE 不支持 php 的 mail 函数直接发送内容，于是只能通过第三方的 SMTP 服务，搜了一下发现腾讯有域名邮箱服务，另外还有一个企业邮箱服务，相同点是都可以使用自己的域名，但是据说企业邮箱的功能强大些，于是出于测试目的，先注册企业邮箱，基本的工作原理是我们需要在 DNS 解析服务器那里，把邮件的 MX 类型的 DNS 请求解析到腾讯的邮箱服务器，也就是说，所有发往我们自定义域名的邮件，都会送到腾讯的服务器，于是去改 DNS 记录，改好了，在 SMTP 的插件中填好用户名密码，开始调试。

那么基本理解了 SMTP 的原理之后，要怎么查问题呢，回去看 SAE 上面的 SMTP 插件，里面有一个发送测试邮件的方法，里面会详尽的输出 debug 信息，写法类似这样，

// Set up the mail variables
$to = $_POST['to'];
$subject = 'WP Mail SMTP: ' . __('Test mail to ', 'wp_mail_smtp') . $to;
$message = __('This is a test email generated by the WP Mail SMTP WordPress plugin.', 'wp_mail_smtp');

// Set SMTPDebug to level 2
$phpmailer->SMTPDebug = 2;

// Start output buffering to grab smtp debugging output
ob_start();

// Send the test mail
$result = wp_mail($to,$subject,$message);

// Strip out the language strings which confuse users
//unset($phpmailer->language);
// This property became protected in WP 3.2

// Grab the smtp debugging output
$smtp_debug = ob_get_clean();

// Output the response

于是照猫画虎，搬到发送的函数里面，终于从服务器的回应中看到，回复邮箱出错，看回代码，我们自己的邮箱是 wp 开头的，但是回复的邮箱却是写的 no-reply 开头的，这样就被服务器拒接了，于是把 no-reply 改成自己的邮箱名，就成功的发出了邮件。

昨天下午就到了这个步骤，但是想到注销企业邮箱试试域名邮箱的时候，却发现一直提示说“域名出于24小时保护期内”，不知道为什么要设这个限制，难道是怕 DNS 解析错乱吗，不管怎么用，知道今天才成功的改换到域名邮箱，其实，域名邮箱跟企业邮箱的区别可以这么来说，企业邮箱就是一套完整的系统，甚至包括一些内部的公告，交流等，是实实在在会多一个邮箱可以用，而域名邮箱，本质是只是给自己的 QQ 邮箱增加一个自定义的域名而已，并没有多一个邮箱。

历史评论

把 WordPress 搬出 SAE | ZRJ (2012-12-20 13:02:46):

[…] 最后需要改动的是 /wp-content/themes/twentyeleven/functions.php，加入在管理员在回复访客留言时自动发送邮件通知访客的功能，详情请看很久之前的文章，从评论通知学电子邮件的原理 […]

Tomcat配置多域名虚拟主机

Fri, 13 Jul 2012 23:54:16 +0800

国内的 Java 社区资料确实丰富，可惜大都是抄来抄去，质量也是良莠不齐，比如这个简单的 tomcat 配置多余名虚拟主机问题吧，google 了一下，出来很多文章，前面的点进去看了看，发现解决不了问题，话都没有说清楚，于是关掉，点后面，结果坑爹的发现其实是抄前面的几篇，连错别字都不带改的，于是只能自己狂看好多篇然后挨个试，最后不容易终于试出来了，唉，吐槽完了，转正题。

首先，先明确一些简单的概念，访问一个 web 服务器的根本方法是使用 IP ，但是一般来说，一台主机的计算能力是有余的，于是可以同时运行多个域名下的网站，当然，出于另外的一些目的，我们也需要配置自定义域名，例如，tomcat 默认跑在 8080 ，而且一个网站是在子目录的形式访问的，也就是类似 http://localhost:8080/website/ 这样的 URL ，这样不利于我们在调试阶段使用基于根目录的相对路径形式，所以我的方法是在本地的时候配成 website.local 的形式，等到上线了就是 website.com 的形式，这样所有的相对路径都是可移植的。

主要配置的文件是 tomcat 目录下的 /conf/server.xml 文件，初始的文件结构是类似这样的

<?xml version="1.0" encoding="UTF-8" standalone="no"?><!-- Note:  A "Server" is not itself a "Container", so you may not
     define subcomponents such as "Valves" at this level.
     Documentation at /docs/config/server.html
 --><Server port="8005" shutdown="SHUTDOWN">

  <!--APR library loader. Documentation at /docs/apr.html -->
  <Listener SSLEngine="on" className="org.apache.catalina.core.AprLifecycleListener"/>
  <!--Initialize Jasper prior to webapps are loaded. Documentation at /docs/jasper-howto.html -->
  <Listener className="org.apache.catalina.core.JasperListener"/>
  <!-- JMX Support for the Tomcat server. Documentation at /docs/non-existent.html -->
  <Listener className="org.apache.catalina.mbeans.ServerLifecycleListener"/>
  <Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener"/>

  <!-- Global JNDI resources
       Documentation at /docs/jndi-resources-howto.html
  -->
  <GlobalNamingResources>
    <!-- Editable user database that can also be used by
         UserDatabaseRealm to authenticate users
    -->
    <Resource auth="Container" description="User database that can be updated and saved" factory="org.apache.catalina.users.MemoryUserDatabaseFactory" name="UserDatabase" pathname="conf/tomcat-users.xml" type="org.apache.catalina.UserDatabase"/>
  </GlobalNamingResources>

  <!-- A "Service" is a collection of one or more "Connectors" that share
       a single "Container" Note:  A "Service" is not itself a "Container",
       so you may not define subcomponents such as "Valves" at this level.
       Documentation at /docs/config/service.html
   -->
  <Service name="Catalina">

    <!--The connectors can use a shared executor, you can define one or more named thread pools-->
    <!--
    <Executor name="tomcatThreadPool" namePrefix="catalina-exec-"
        maxThreads="150" minSpareThreads="4"/>
    -->

    <!-- A "Connector" represents an endpoint by which requests are received
         and responses are returned. Documentation at :
         Java HTTP Connector: /docs/config/http.html (blocking & non-blocking)
         Java AJP  Connector: /docs/config/ajp.html
         APR (HTTP/AJP) Connector: /docs/apr.html
         Define a non-SSL HTTP/1.1 Connector on port 8080
    -->
    <Connector connectionTimeout="20000" port="80" protocol="HTTP/1.1" redirectPort="8443"/>
    <!-- A "Connector" using the shared thread pool-->
    <!--
    <Connector executor="tomcatThreadPool"
               port="8080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               redirectPort="8443" />
    -->
    <!-- Define a SSL HTTP/1.1 Connector on port 8443
         This connector uses the JSSE configuration, when using APR, the
         connector should be using the OpenSSL style configuration
         described in the APR documentation -->
    <!--
    <Connector port="8443" protocol="HTTP/1.1" SSLEnabled="true"
               maxThreads="150" scheme="https" secure="true"
               clientAuth="false" sslProtocol="TLS" />
    -->

    <!-- Define an AJP 1.3 Connector on port 8009 -->
    <Connector port="8009" protocol="AJP/1.3" redirectPort="8443"/>

    <!-- An Engine represents the entry point (within Catalina) that processes
         every request.  The Engine implementation for Tomcat stand alone
         analyzes the HTTP headers included with the request, and passes them
         on to the appropriate Host (virtual host).
         Documentation at /docs/config/engine.html -->

    <!-- You should set jvmRoute to support load-balancing via AJP ie :
    <Engine name="Standalone" defaultHost="localhost" jvmRoute="jvm1">
    -->
    <Engine defaultHost="localhost" name="Catalina">

      <!--For clustering, please take a look at documentation at:
          /docs/cluster-howto.html  (simple how to)
          /docs/config/cluster.html (reference documentation) -->
      <!--
      <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"/>
      -->

      <!-- The request dumper valve dumps useful debugging information about
           the request and response data received and sent by Tomcat.
           Documentation at: /docs/config/valve.html -->
      <!--
      <Valve className="org.apache.catalina.valves.RequestDumperValve"/>
      -->

      <!-- This Realm uses the UserDatabase configured in the global JNDI
           resources under the key "UserDatabase".  Any edits
           that are performed against this UserDatabase are immediately
           available for use by the Realm.  -->
      <Realm className="org.apache.catalina.realm.UserDatabaseRealm" resourceName="UserDatabase"/>

      <!-- Define the default virtual host
           Note: XML Schema validation will not work with Xerces 2.2.
       -->
      <Host appBase="webapps" autoDeploy="true" name="localhost" unpackWARs="true" xmlNamespaceAware="false" xmlValidation="false">

        <!-- SingleSignOn valve, share authentication between web applications
             Documentation at: /docs/config/valve.html -->
        <!--
        <Valve className="org.apache.catalina.authenticator.SingleSignOn" />
        -->

        <!-- Access log processes all example.
             Documentation at: /docs/config/valve.html -->
        <!--
        <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
               prefix="localhost_access_log." suffix=".txt" pattern="common" resolveHosts="false"/>
        -->

      </Host>
    </Engine>
  </Service>
</Server>

我们需要做的就是在 Engine 标签下再增加一个 Host 标签，类似这样

<Host name="neddytravel.local" debug="0" appBase="D:DocumentscodeMyEclipse86.metadata.me_tcatwebappsNeddyTravel">
	<Context path="" docBase="." debug="0" />
</Host>

这里的 name 属性就是我们要自定义的域名， appBase 属性是这个网站的文件的真实路径，至于 debug 属性，一般都没看到有解释， tomcat 官网的解释是

The level of debugging detail logged by this Engine to the associatedLogger. Higher numbers generate more detailed output. If not specified, the default debugging detail level is zero (0).

然后就是下面的 Context 标签了，根据 http://www.wangchao.net.cn/bbsdetail_61162.html

path 表示在浏览器里的目录，空表示根目录, docBase 表示在硬盘上所在的目录

当然还要修改端口号为 80 ，修改完成后完整的文件是类似这样的

<?xml version="1.0" encoding="UTF-8" standalone="no"?><!-- Note:  A "Server" is not itself a "Container", so you may not
     define subcomponents such as "Valves" at this level.
     Documentation at /docs/config/server.html
 --><Server port="8005" shutdown="SHUTDOWN">

  <!--APR library loader. Documentation at /docs/apr.html -->
  <Listener SSLEngine="on" className="org.apache.catalina.core.AprLifecycleListener"/>
  <!--Initialize Jasper prior to webapps are loaded. Documentation at /docs/jasper-howto.html -->
  <Listener className="org.apache.catalina.core.JasperListener"/>
  <!-- JMX Support for the Tomcat server. Documentation at /docs/non-existent.html -->
  <Listener className="org.apache.catalina.mbeans.ServerLifecycleListener"/>
  <Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener"/>

  <!-- Global JNDI resources
       Documentation at /docs/jndi-resources-howto.html
  -->
  <GlobalNamingResources>
    <!-- Editable user database that can also be used by
         UserDatabaseRealm to authenticate users
    -->
    <Resource auth="Container" description="User database that can be updated and saved" factory="org.apache.catalina.users.MemoryUserDatabaseFactory" name="UserDatabase" pathname="conf/tomcat-users.xml" type="org.apache.catalina.UserDatabase"/>
  </GlobalNamingResources>

  <!-- A "Service" is a collection of one or more "Connectors" that share
       a single "Container" Note:  A "Service" is not itself a "Container",
       so you may not define subcomponents such as "Valves" at this level.
       Documentation at /docs/config/service.html
   -->
  <Service name="Catalina">

    <!--The connectors can use a shared executor, you can define one or more named thread pools-->
    <!--
    <Executor name="tomcatThreadPool" namePrefix="catalina-exec-"
        maxThreads="150" minSpareThreads="4"/>
    -->


    <!-- A "Connector" represents an endpoint by which requests are received
         and responses are returned. Documentation at :
         Java HTTP Connector: /docs/config/http.html (blocking & non-blocking)
         Java AJP  Connector: /docs/config/ajp.html
         APR (HTTP/AJP) Connector: /docs/apr.html
         Define a non-SSL HTTP/1.1 Connector on port 8080
    -->
    <Connector connectionTimeout="20000" port="80" protocol="HTTP/1.1" redirectPort="8443"/>
    <!-- A "Connector" using the shared thread pool-->
    <!--
    <Connector executor="tomcatThreadPool"
               port="8080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               redirectPort="8443" />
    -->
    <!-- Define a SSL HTTP/1.1 Connector on port 8443
         This connector uses the JSSE configuration, when using APR, the
         connector should be using the OpenSSL style configuration
         described in the APR documentation -->
    <!--
    <Connector port="8443" protocol="HTTP/1.1" SSLEnabled="true"
               maxThreads="150" scheme="https" secure="true"
               clientAuth="false" sslProtocol="TLS" />
    -->

    <!-- Define an AJP 1.3 Connector on port 8009 -->
    <Connector port="8009" protocol="AJP/1.3" redirectPort="8443"/>


    <!-- An Engine represents the entry point (within Catalina) that processes
         every request.  The Engine implementation for Tomcat stand alone
         analyzes the HTTP headers included with the request, and passes them
         on to the appropriate Host (virtual host).
         Documentation at /docs/config/engine.html -->

    <!-- You should set jvmRoute to support load-balancing via AJP ie :
    <Engine name="Standalone" defaultHost="localhost" jvmRoute="jvm1">
    -->
    <Engine defaultHost="localhost" name="Catalina">

      <!--For clustering, please take a look at documentation at:
          /docs/cluster-howto.html  (simple how to)
          /docs/config/cluster.html (reference documentation) -->
      <!--
      <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"/>
      -->

      <!-- The request dumper valve dumps useful debugging information about
           the request and response data received and sent by Tomcat.
           Documentation at: /docs/config/valve.html -->
      <!--
      <Valve className="org.apache.catalina.valves.RequestDumperValve"/>
      -->

      <!-- This Realm uses the UserDatabase configured in the global JNDI
           resources under the key "UserDatabase".  Any edits
           that are performed against this UserDatabase are immediately
           available for use by the Realm.  -->
      <Realm className="org.apache.catalina.realm.UserDatabaseRealm" resourceName="UserDatabase"/>


      <!-- Define the default virtual host
           Note: XML Schema validation will not work with Xerces 2.2.
       -->
      <Host appBase="webapps" autoDeploy="true" name="localhost" unpackWARs="true" xmlNamespaceAware="false" xmlValidation="false">

        <!-- SingleSignOn valve, share authentication between web applications
             Documentation at: /docs/config/valve.html -->
        <!--
        <Valve className="org.apache.catalina.authenticator.SingleSignOn" />
        -->

        <!-- Access log processes all example.
             Documentation at: /docs/config/valve.html -->
        <!--
        <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
               prefix="localhost_access_log." suffix=".txt" pattern="common" resolveHosts="false"/>
        -->

      </Host>

	  <Host name="neddytravel.local" debug="0" appBase="D:DocumentscodeMyEclipse86.metadata.me_tcatwebappsNeddyTravel">
          <Context path="" docBase="." debug="0" />
	  </Host>

    </Engine>
  </Service>
</Server>

配置文件的修改到这里就结束了，下面修改 hosts 文件，至于不理解 hosts 文件的作用的话，请自行搜索，这方面资料很多，也讲的不错

hosts 文件在 C:WindowsSystem32driversetc ，打开，追加一行

127.0.0.1 neddytravel.local

就可以了，然后重启 tomcat ，在浏览器中访问自定义的域名 neddytravel.local 应该就可以看到页面了

进一步理解 DNS

Tue, 01 May 2012 22:54:13 +0800

把空间移到了新浪的 SAE 了，一来是发现海外的空间速度实在慢，在体会过新浪 SAE 的六线接入的一点瞬开的效果之后，直接被秒杀了，另外， DNS 的域名解析也迁移到了 DNSPod ，在 DNSPod 上做了 301 的跳转，于是现在的效果是速度很快，但是 URL 很恶心，不过权衡之后，我还是觉得速度更重要。

说说新浪的 SAE ，SAE 的应用商店里面有专门改写过的 WordPress ，主要是数据库层面改写了，具体的还没有研究，不过这是稍晚的事，我现在的想法是想先读读 WordPress 的源码，昨天安装的时候，发现 WordPress 可以发邮件通知，虽然这不是什么很高深的功能，但是我倒是想到当时我的主机上好像没有主动开启 SMTP 服务啊，那么 PHP 的 mail 函数是怎么发出邮件的呢？于是决定就这个好好研究一下 PHP 的邮件模块。另外一个发现的就是 WordPress 的搜索功能支持空格的，数据库上我当然是没有做全文检索的，那么这个功能又是怎么实现的呢，我目前就只能想到用 SQL 的 like 来拼接，具体的实现还是要读源码啊。

说说 DNSPod ，我们知道，一次 HTTP 请求发起之后，客户机会首先去用各种方法解析输入的域名，这里的各种方法包括本机的 hosts 文件，也包括请求网络的 DNS 服务，而 DNS 在各个层级都有缓存（相应的，缓存当然有过期时间，一般是 600 秒），在原来我的域名是托管在 Godaddy 做 A 解析的，域名的 NameServer 服务器当然是在海外，自然就慢了，后来迁移到新浪的 SAE 后，SAE 提供域名的绑定，但是要求做 CNAME 解析，可惜 Godaddy 是不支持对一级域名做 CNAME 解析的（估计是出于对 MX 解析冲突的考虑），于是没办法，我就把域名转到 DNSPod ，需要注意的是，这个地方的转移，其实只是把域名的解析服务转移出去，但是域名的根本管理权限还是在 Godaddy 的，怎么理解呢，简单的说，就是本来我们的机子向附近的 DNS 服务器请求解析域名，但是附近的 DNS 的并没有缓存我们请求的域名的记录，于是请求被向上转发，这里可能经过一次或者多次转发，然后到了某个层次的 DNS 服务器，一般是根服务器（顺便一提，DNS 的根服务器全球一共 13 台，没有一台在中国，具体情况可以参看维基百科的根域名服务器条目），他会最终处理这个请求，它查询发现我们这个被请求的域名是由类似 ns41.domaincontrol.com 或者 ns42.domaincontrol.com 这样的服务器解析的，但是，此处，根服务器并不会送佛送到西地找 ns41.domaincontrol.com 之类的解析，而是把这个负责域名的服务器地址返回给请求的服务器，于是下级的 DNS 向其中一个要求解析对应的地址，最终得到对应的 IP 地址，再层层下转，当然每层都应该会缓存，最后到了我们的机子，一次 DNS 解析请求就此完成，而在我们把域名转到 DNSPod 解析后，最终的解析请求就是由类似 f1g1ns1.dnspod.net 或者 f1g1ns2.dnspod.net 这样的服务器来解析的，这些服务器都是多线接入网络，自然响应速度比向海外的服务器请求要快了，到此，我们基本就理解了 A 解析了。

但是那个 301 是什么回事呢？其实，一般 301 服务器跳转的实现，都是由 DNS 的 NameServer 服务商自己开一个 HTTP 请求服务器来帮忙实现的，我们可以尝试着发送一个 HTTP 的 HEAD 类请求来验证：

Date: Tue, 01 May 2012 21:54:27 GMT
Server: DNSPod v1.0
Location: http://zrj0.sinaapp.com
Content-Length: 0
Connection: close

可以看到返回的 Server 是写着 DNSPod 的，通过一次 ping 请求我们可以进一步验证这个想法：

C:>ping zrj.me

正在 Ping dnspod-free.urlpod.cn [70.39.107.178] 具有 32 字节的数据:
来自 70.39.107.178 的回复: 字节=32 时间=211ms TTL=47
来自 70.39.107.178 的回复: 字节=32 时间=211ms TTL=47
来自 70.39.107.178 的回复: 字节=32 时间=211ms TTL=47
来自 70.39.107.178 的回复: 字节=32 时间=211ms TTL=47

70.39.107.178 的 Ping 统计信息:
    数据包: 已发送 = 4，已接收 = 4，丢失 = 0 (0% 丢失)，
往返行程的估计时间(以毫秒为单位):
    最短 = 211ms，最长 = 211ms，平均 = 211ms

C:>

可以清楚的看到，响应这个 ICMP 包的，就是 DNSPod 的服务器。至此，我们基本理解了 301 跳转类的域名解析服务了。

其他的比较少用，而且今天刚好发现 DNSPod 上的帮助写的很不错，简单易懂，直接引用过来：

A记录：地址记录，用来指定域名的IPv4地址（如：8.8.8.8），如果需要将域名指向一个IP地址，就需要添加A记录。 CNAME：如果需要将域名指向另一个域名，再由另一个域名提供ip地址，就需要添加CNAME记录。 TXT：在这里可以填写任何东西，长度限制255。绝大多数的TXT记录是用来做SPF记录（反垃圾邮件）。 NS：域名服务器记录，如果需要把子域名交给其他DNS服务商解析，就需要添加NS记录。 AAAA：用来指定主机名（或域名）对应的IPv6地址（例如：ff06:0:0:0:0:0:0:c3）记录。 MX：如果需要设置邮箱，让邮箱能收到邮件，就需要添加MX记录。 URL：从一个地址301重定向到另一个地址的时候，就需要添加URL记录（注：DNSPod目前只支持显性301重定向）。 SRV：记录了哪台计算机提供了哪个服务。格式为：服务的名字、点、协议的类型，例如：_xmpp-server._tcp。

各类型的记录值一般是这样的：

A记录：填写您服务器 IP，如果您不知道，请咨询您的空间商 CNAME记录：填写空间商给您提供的域名，例如：dnspod.cn MX记录：填写您邮件服务器的IP地址或企业邮局给您提供的域名，如果您不知道，请咨询您的邮件服务提供商 TXT记录：一般用于 Google、QQ等企业邮箱的反垃圾邮件设置 URL记录：填写要跳转到的网址，例如：http://www.baidu.com AAAA：不常用。解析到 IPv6 的地址。 NS记录：不常用。系统默认添加的两个NS记录请不要修改。NS向下授权，填写dns域名，例如：f1g1ns1.dnspod.net SRV记录：不常用。格式为：优先级、空格、权重、空格、端口、空格、主机名，记录生成后会自动在域名后面补一个“.”，这是正常现象。例如：5 0 5269 xmpp-server.l.google.com.

========== Update ==========

虽然 DNSPod 的域名解析是多线的，但是可惜的是，据官方说明，他的 301 服务器还是在海外，悲催的是，我转过来第一天就遇到 301 服务器悲剧的情况，于是现在 301 还是在 Godaddy 做。

历史评论

zz (2012-05-02 22:51:50):

请问你这个域名是怎么绑定的？有经过备案么？

ZRJ (2012-05-03 11:53:24):

域名是在 Godaddy 买的，不需要备案。绑定的话，在 Goaddy 的 DNS Manager 面板可以设置把域名指定到哪个 IP ，或者 CNAME 到哪个域名。

从点击到呈现 — 详解一次HTTP请求（2） | ZRJ (2012-10-05 08:28:45):

[…] 如果在本地的 hosts 文件没有能够找到对应的 ip 地址，浏览器会向 dns 域名解析服务器发起域名解析请求，dns 的域名解析是递归的，（还有另外 dns 是迭代的），递归的 dns 首先会查看自己的 dns 缓存，如果缓存能够命中，那么就从缓存中把 ip 地址返回给浏览器，如果找不到对应的域名的 ip 地址，那么就向上转发请求，然后把得到的这个域名对应的 nameserver 的地址取得，再向这个 namserver 去请求域名对应的 ip，最后把这个 ip 地址返回给浏览器，不过怎么说，这个递归查询的过程，对于浏览器来说是透明的，他只要坐等 ip地址送回来就可以了。对于 DNS 的进一步理解，可以看这里，进一步理解 DNS […]

找个地方写学习笔记

Mon, 30 Apr 2012 01:23:52 +0800

最近一段时间感觉自己基础薄弱，沉淀不足浮躁有余，于是心里一直盘算着好好补习欠下的功课。恰逢今晚闲逛一个别人的博客（ glacier 大神的 http://huangxin.me/ ），看到 .me 的域名突然想起自己也有重新写博客（当作学习笔记）的打算，于是顺手搜了搜 ZRJ.me ，发现居然可用，于是脑子一热就买了这个域名，然后就忙着买空间，搭 wordpress 了。

在 wordpress 官网下载安装包的时候速度实在是让人无力吐槽，估量着就这网络环境，等下上传估计也得等到地老天荒，于是干脆在空间上写段 php 下载代码：

<?php
if (isset($_REQUEST["url"])) {
	$url = $_REQUEST["url"];
	$local = fopen("wp.zip", "w");
	$remote = fopen($url, "r");
	$wp = "";
	while ($temp = fread($remote, 4096)) {
		$wp .= $temp;
	}
	fwrite($local, $wp);
	fclose($local);
	fclose($remote);
	echo "<h1>Done</h1>";
}
?>
<form method="post">
	<input type="text" name="url" />
	<input type="submit" />
</form>

在浏览器里面访问，填入 wordpress 的下载 url ，提交到后台让服务器跑去下载，果然瞬间就下完了（记得用完即删，最好顺便把 php 那些危险的函数禁用了），然后就是一路 next 的安装，装好之后，由于是用来写学习笔记的，自然首先考虑代码高亮，简单 google 一下找到 Syntax Highlighter 这个插件，这年代 wordpress 装插件实在方便，后台搜索安装就是几次点击鼠标的事情。

于是新环境就此搭好，以后要努力笔耕不辍。

历史评论

心不再浪迹 (2012-07-15 00:21:16):

杰哥，来看你了~~最近也想找个地写下随感与学习笔记，突然间想起wordpress，但是还无网络建站基础，如此苦逼~如何是好

ZRJ (2012-07-15 12:01:28):

呃，其实 WordPress 不难装的，我第一次装的时候也是网上搜教程出来边看边做的，当时还没有 SAE ，就自己去淘宝买个虚拟主机，然后去 Godaddy 买个域名，把 WordPress 的压缩包上传到虚拟主机解压了，MySQL 里把数据库建好，数据表的话 WordPress 自己会创建，然后把域名的 DNS 指过来虚拟主机的 IP 就可以了，这就是大概的步骤。。教程的话，我看了一下，Google 搜“wordpress 虚拟主机教程”出来的几篇还是不错的，当然这些都是传统的思路，现在有了 SAE ，直接登陆创建一个 SAE 应用，然后在线自动就可以装好 WordPress 了，更简单