《How Tomcat Works》读书笔记(三)--Connector(连接器)

这是《How Tomcat Works》第三四章的读书笔记。主要写了Tomcat4.0默认的连接器(Connector)的处理流程,后面Tomcat的连接器改为Coyote。

概述

架构

  • HttpConector:监听Http请求、维护HttpProcessor对象池、处理Http请求(调用HttpProcessor对象进行处理)
  • HttpProcessor:解析请求(解析连接、解析请求行、解析请求头等等)、调用容器进行处理
  • Request:表示Http请求,实现了org.apache.catalina.Request接口。
  • Response:表示Http响应,实现了org.apache.catalina.Response接口。

其中Request、Response就是代表Http请求、Http响应,实现了Servlet编程中HttpServletRequest、HttpServletResponse接口,这样符合Java EE编程标准。只是Tomcat在这里使用了许多的类、接口等等(如:RequestBase、HttpRequestImpl、RequestFacade),这样做就像Java Util包中哪些类库一样,反正看起来好复杂。

源码

HttpProcessor处理逻辑

只是展示逻辑,省略一些具体的处理方式,省略了try-catch等等。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
/**
* Process an incoming HTTP request on the Socket that has been assigned
* to this Processor. Any exceptions that occur during processing must be
* swallowed and dealt with.
*
* @param socket The socket on which we are connected to the client
*/
private void process(Socket socket) {
boolean ok = true;
boolean finishResponse = true;
SocketInputStream input = null;

// Construct and initialize the objects we will need
// 构造对象,这里省略的许多的try-catch
input = new SocketInputStream(socket.getInputStream(), connector.getBufferSize());
// Http/1.1中新特性:长连接
keepAlive = true;
while (!stopped && ok && keepAlive) {
finishResponse = true;
request.setStream(input);
request.setResponse(response);
output = socket.getOutputStream();
response.setStream(output);
response.setRequest(request);
((HttpServletResponse) response.getResponse()).setHeader("Server", Constants.ServerInfo);

// Parse the incoming request
// 解析请求数据
// 解析连接
parseConnection(socket);
// 解析请求行
parseRequest(input, output);
if (!request.getRequest().getProtocol().startsWith("HTTP/0"))
//解析请求头
parseHeaders(input);
if (http11) {
// Sending a request acknowledge back to the client if
// requested.
ackRequest(output);
// If the protocol is HTTP/1.1, chunking is allowed.
if (connector.isChunkingAllowed())
response.setAllowChunking(true);
}

// Ask our Container to process this request
// 调用Servlet容器处理请求
((HttpServletResponse) response).addDateHeader("Date", System.currentTimeMillis());
connector.getContainer().invoke(request, response);

// Finish up the handling of the request
response.finishResponse();
request.finishRequest();

// Recycling the request and the response objects
// 回收Request、Response对象:把该对象的状态复原使得下一次请求有自己独有的状态
request.recycle();
response.recycle();
}
}

一些解释

有状态的对象线程池

有状态的对象线程池:池中每个对象有一个自己的线程,每个对象有自己的状态(域)。Connector(连接器)中HttpConnector接受请求,调用HttpProcessor对象处理请求。HttpConnector管理一系列的HttpProcessor对象,每个HttpProcessor对象有自己单独的后台线程,这样每次都使用一个线程处理请求。

HttpConnector在最开始就启动池中所有的HttpProcessor对象,然后有请求来时从池中拿一个HttpProcessor对象进行处理。

1
2
3
4
5
6
7
8
9
10
11
12
public void run() {
// Loop until we receive a shutdown command
while (!stopped) {
// Accept the next incoming connection from the server socket
Socket socket = null;
// Hand this socket off to an appropriate processor
// 从池中拿一个HttpProcessor
HttpProcessor processor = createProcessor();
// 调用HttpProcessor进行处理
processor.assign(socket);
}
}

HttpProcessor对象一开始就全部启动,等待HttpConnector分配socket进行处理。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
public void run() {
// Process requests until we receive a shutdown signal
while (!stopped) {
// Wait for the next socket to be assigned
// 在这里HttpProcessor对象会阻塞直到HttpConnector分配一个socket
Socket socket = await();
if (socket == null)
continue;
// Process the request from this socket
// 进行处理
process(socket);
// Finish up this request
// 处理完成,把该HttpProcessor放回HttpProcessor对象池中
connector.recycle(this);
}
}

HttpProcessor对象一开始就全部启动,然后会被阻塞(如上所示)直到HttpConnector分配socket。await()与assign(Socket socket)方法如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
/**
* The socket we are currently processing a request for. This object
* is used for inter-thread communication only.
*/
private Socket socket = null;

/**
* Is there a new socket available?
*/
private boolean available = false;

/**
* Process an incoming TCP/IP connection on the specified socket. Any
* exception that occurs during processing must be logged and swallowed.
* NOTE: This method is called from our Connector's thread. We
* must assign it to our own thread so that multiple simultaneous
* requests can be handled.
*
* @param socket TCP socket to process
*/
synchronized void assign(Socket socket) {
// Wait for the Processor to get the previous Socket
// 如果已经分配,则等待
while (available) {
try {
wait();
} catch (InterruptedException e) {
}
}
// Store the newly available Socket and notify our thread、
// 分配socket
this.socket = socket;
available = true;
notifyAll();
}

/**
* Await a newly assigned Socket from our Connector, or null
* if we are supposed to shut down.
*/
private synchronized Socket await() {
// Wait for the Connector to provide a new Socket
// 如果没有分配,则等待
while (!available) {
try {
wait();
} catch (InterruptedException e) {
}
}
// Notify the Connector that we have received this Socket
// 把分配的socket返回
Socket socket = this.socket;
available = false;
notifyAll();
return socket;
}

最后每一次的Request、Response对象状态是不一样的,所以在处理完完成后,需要把Request、Response对象状态还原。

1
2
3
// Recycling the request and the response objects
request.recycle();
response.recycle();

解析处理

HttpProcessor解析过程是一个耗时的过程,尤其是解析请求参数、Cookies时,所有Tomcat中设计为仅仅我们需要使用Parameter、Cookies时才解析。我们以Parameter为例:

在需要使用Parameter方法时(如:getParameter()方法)时,就验证是否需要解析,如果没有解析就解析,否则直接使用。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
/**
* The parsed parameters for this request. This is populated only if
* parameter information is requested via one of the
* getParameter() family of method calls. The key is the
* parameter name, while the value is a String array of values for this
* parameter.
*
* IMPLEMENTATION NOTE - Once the parameters for a
* particular request are parsed and stored here, they are not modified.
* Therefore, application level access to the parameters need not be
* synchronized.
* 存放Parameter,继承HashMap,与HashMap基本无异
*/
protected ParameterMap parameters = null;

/**
* Have the parameters for this request been parsed yet?
* 是否已经解析
*/
protected boolean parsed = false;

/**
* Parse the parameters of this request, if it has not already occurred.
* If parameters are present in both the query string and the request
* content, they are merged.
*/
protected void parseParameters() {
if (parsed)
return;
ParameterMap results = parameters;
// 具体解析逻辑就不给出来了

// Store the final results
parsed = true;
parameters = results;
}

public String getParameter(String name) {
parseParameters();
String values[] = (String[]) parameters.get(name);
if (values != null)
return (values[0]);
else
return (null);
}

实现

逻辑基本一致,但是自己省略了需要的具体逻辑,如仅仅解析了RequestURL等等。然后许多与Request的有关的类,我仅仅使用了HttpRequest全部代替了。

Container、Connector接口:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
/**
* Created by kanyuxia on 2017/4/27.
* 模拟org.apache.catalina.Connector接口
*/
public interface Connector {
/**
* Return the Container used for processing requests received by this
* Connector.
*/
Container getContainer();

/**
* Set the Container used for processing requests received by this
* Connector.
*
* @param container The new Container to use
*/
void setContainer(Container container);

/**
* Return the scheme that will be assigned to requests received
* through this connector. Default value is "http".
*/
String getScheme();

/**
* Set the scheme that will be assigned to requests received through
* this connector.
*
* @param scheme The new scheme
*/
void setScheme(String scheme);

/**
* Create (or allocate) and return a Request object suitable for
* specifying the contents of a Request to the responsible Container.
*/
HttpServletRequest createRequest();

/**
* Create (or allocate) and return a Response object suitable for
* receiving the contents of a Response from the responsible Container.
*/
HttpServletResponse createResponse();

/**
* Invoke a pre-startup initialization. This is used to allow connectors
* to bind to restricted ports under Unix operating environments.
*/
void initialize();

}
/**
* Created by kanyuxia on 2017/4/27.
* 模拟org.apache.catalina.Container接口
*/
public interface Container {
/**
* Process the specified Request, and generate the corresponding Response,
* according to the design of this particular Container.
*
* @param request Request to be processed
* @param response Response to be produced
*
* @exception IOException if an input/output error occurred while
* processing
* @exception ServletException if a ServletException was thrown
* while processing this request
*/
void invoke(HttpRequest request, HttpResponse response) throws IOException, ServletException;
}

Request、Response相关类:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
/**
* Created by kanyuxia on 2017/4/26.
* Http请求对象
*/
public class HttpRequest implements HttpServletRequest {

// ---------------------------------Http Request Infomations
/**
* 存放Http request headers
*/
private HashMap<String, ArrayList<String>> headers = new HashMap<>();

/**
* 存放Http request cookies
*/
private ArrayList<Cookie> cookies = new ArrayList<>();

/**
* 存放Http请求参数:Query String or Form datas.
* 只有当Servlet取参数时,才解析
*/
private ParameterMap<String, String[]> parameterMap = null;

/**
* Have the parameters for this request been parsed yet?
* 是否解析了Http Parameters
*/
private boolean parameterPared = false;

/**
* The request URI associated with this request.
* 请求URL地址
*/
private String requestURI = null;

/**
* The input stream associated with this Request.
*/
private InputStream input = null;

// ---------------------------------------Servlet some infomations

/**
* The request attributes for this request.
*/
private final HashMap<String, Object> attributes = new HashMap<>();

/**
* The Connector through which this Request was received.
*/
private Connector connector = null;

// --------------------------------------Some methods
/**
* 添加HTTP header
* @param name header name
* @param value header value
*/
public void addHeader(String name, String value) {
name = name.toLowerCase();
ArrayList<String> values = headers.get(name);
if (values != null) {
values.add(value);
}
values = new ArrayList<>();
headers.put(name, values);
}

/**
* 返回门面类
* @return RequestFacade
*/
public HttpServletRequest getRequest() {
return new HttpRequestFacade(this);
}

/**
* 解析HTTP Parameters,如果已经解析则返回。
*/
public void parseParameters() {
if (parameterPared) {
return;
}
parameterMap = new ParameterMap<>();
parameterMap.setLocked(false);

// 解析Query String or Form data

parameterMap.setLocked(true);
}

/**
* Release all object references, and initialize instance variables, in
* preparation for reuse of this object.
* 清空所有数据
*/
public void recycle() {
headers.clear();
cookies.clear();
if (parameterMap != null) {
parameterMap.setLocked(false);
parameterMap.clear();
}
parameterPared = false;
requestURI = null;
input = null;
attributes.clear();
}
}

HttpConnector类:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
/**
* Created by kanyuxia on 2017/4/26.
* Http连接器。
*/
public class HttpConnector implements Runnable,Connector {
/**
* The Container used for processing requests received by this Connector.
* Servlet容器
*/
private Container container = null;

/**
* The current number of processors that have been created.
* 当前已经创建的HttpProcessor
*/
private int curProcessors = 0;

/**
* The minimum number of processors to start at initialization time.
* HttpProcessor对象池最小对象数
*/
private int minProcessors = 5;

/**
* The maximum number of processors allowed, or <0 for unlimited.
* HttpProcessor对象池最大对象数
*/
private int maxProcessors = 20;

/**
* The port number on which we listen for HTTP requests.
* 服务器端口号
*/
private int port = 10086;

/**
* The set of processors that have been created but are not currently
* being used to process a request.
* 存放已经创建但未被使用的Http处理器对象
*/
private final Stack<HttpProcessor> processors = new Stack<>();

/**
* The set of processors that have ever been created.
* 存放已经创建的Http处理器
*/
private Vector<HttpProcessor> created = new Vector<>();

/**
* The request scheme that will be set on all requests received
* through this connector.
* 服务器处理请求模式(协议)
*/
private String scheme = "http";

/**
* The server socket through which we listen for incoming TCP connections.
* 服务器ServerSocket
*/
private ServerSocket serverSocket = null;

/**
* The background thread that listens for incoming TCP/IP connections and
* hands them off to an appropriate processor.
* 启动Http连接器,监听Http请求,把socket分发给HttpProcessor进行处理。
*/
public void run() {
while (true) {
Socket socket = null;
try {
socket = serverSocket.accept();
} catch (IOException e) {
e.printStackTrace();
}
HttpProcessor processor = createProcessor();
if (processor != null) {
processor.assign(socket);
}
}
}

/**
* Create (or allocate) and return an available processor for use in
* processing a specific HTTP request, if possible. If the maximum
* allowed processors have already been created and are in use, return
* <code>null</code> instead.
* 创建HttpProcessor:1. 从栈中拿 2. new一个 3. 返回null
*/
public HttpProcessor createProcessor() {
synchronized (processors) {
if (processors.size() > 0) {
return processors.pop();
}
if (maxProcessors > 0 && curProcessors < maxProcessors) {
return newProcessor();
} else {
if (maxProcessors < 0) {
return newProcessor();
}
return null;
}
}
}

/**
* Initialize this connector (create ServerSocket here!)
* 创建ServerSocket,在原HttpConnector中使用ServerSocketFactory创建,这里就直接创建
*/
public void initialize() {
try {
serverSocket = new ServerSocket(port);
} catch (IOException e) {
e.printStackTrace();
}
}

/**
* Begin processing requests via this Connector.
* 启动Http连接器,并创建HttpProcessor线程对象池
*/
public void start() {
// 启动Http连接器
Thread thread = new Thread(this);
thread.setDaemon(true);
thread.start();
// 创建最小HttpProcessor线程对象池
while (curProcessors < minProcessors) {
if (maxProcessors > 0 && curProcessors >= maxProcessors) {
break;
}
HttpProcessor processor = newProcessor();
recycle(processor);
}
}

/**
* Create and return a new processor suitable for processing HTTP
* requests and returning the corresponding responses.
* 创建HttpProcessor对象,并使用运行它(它运行在一个单独的后台线程中)
*/
private HttpProcessor newProcessor() {
HttpProcessor processor = new HttpProcessor(this, curProcessors++);
created.addElement(processor);
processor.start();
return processor;
}

/**
* Recycle the specified Processor so that it can be used again.
* 回收已经没有使用的HttpProcessor
* @param processor The processor to be recycled
*/
void recycle(HttpProcessor processor) {
processors.push(processor);
}
}

HttpProcessor类

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
/**
* Created by kanyuxia on 2017/4/26.
* Http处理器
*/
public class HttpProcessor implements Runnable {

/**
* The HttpConnector with which this processor is associated.
*/
private HttpConnector connector = null;

/**
* Is there a new socket available?
* 是否有新的socket可用
*/
private boolean available = false;

/**
* The socket we are currently processing a request for. This object
* is used for inter-thread communication only.
*/
private Socket socket = null;

/**
* The identifier of this processor, unique per connector.
*/
private int id = 0;

/**
* The HTTP request object we will pass to our associated container.
*/
private HttpRequest httpRequest = null;

/**
* The HTTP response object we will pass to our associated container.
*/
private HttpResponse httpResponse = null;


/**
* Construct a new HttpProcessor associated with the specified connector.
*
* @param connector HttpConnector that owns this processor
* @param id Identifier of this HttpProcessor (unique per connector)
*/
public HttpProcessor(HttpConnector connector, int id) {
this.connector = connector;
this.id = id;
this.httpRequest = (HttpRequest) connector.createRequest();
this.httpResponse = (HttpResponse) connector.createResponse();
}

/**
* Start the background thread we will use for request processing.
* 启动一个后台线程运行HttpProcessor
*/
public void start() {
Thread thread = new Thread(this);
thread.setDaemon(true);
thread.start();
}

/**
* The background thread that listens for incoming TCP/IP connections and
* hands them off to an appropriate processor.
* 等待HttpConnector分配socket,然后处理该scoket,最后通知HttpConnector回收该HttpProcessor
*/
public void run() {
while (true) {
// Wait for the next socket to be assigned
Socket socket = await();
if (socket == null) {
continue;
}
System.out.println(Thread.currentThread().getName());
// Process the request from this socket
process(socket);

// Finish up this request
connector.recycle(this);
}
}

/**
* Process an incoming HTTP request on the Socket that has been assigned
* to this Processor. Any exceptions that occur during processing must be
* swallowed and dealt with.
* 处理HttpConector分配的socket:1. 解析请求 2. 处理请求(调用容器处理)
* @param socket The socket on which we are connected to the client
*/
private void process(Socket socket) {
InputStream input = null;
OutputStream output = null;
try {
input = socket.getInputStream();
output = socket.getOutputStream();
} catch (IOException e) {
e.printStackTrace();
}
httpRequest.setInput(input);
httpResponse.setOutput(output);
httpResponse.setHttpRequest(httpRequest);
// Parse the incoming request
parseConnection(socket);
parseRequest(input, output);
parseHeader(input, output);

// Ask our Container to process this request
try {
connector.getContainer().invoke(httpRequest, httpResponse);
} catch (IOException e) {
e.printStackTrace();
} catch (ServletException e) {
e.printStackTrace();
}

//
httpRequest.recycle();
httpResponse.recycle();
}

/**
* Parse and record the connection parameters related to this request.
* 解析连接
* @param socket The socket on which we are connected
*/
private void parseConnection(Socket socket) {
}

/**
* Parse the incoming HTTP request and set the corresponding HTTP request
* properties.
* 解析请求行
* @param input The input stream attached to our socket
* @param output The output stream of the socket
*/
private void parseRequest(InputStream input, OutputStream output) {
// 在这里仅仅解析了requestURI
BufferedInputStream inputStream = new BufferedInputStream(input);
StringBuilder stringBuilder = new StringBuilder(1024);
byte[] buffer = new byte[1024];
int b = 0;
try {
b = inputStream.read(buffer);
} catch (IOException e) {
e.printStackTrace();
}
for (int i = 0; i < b; i++) {
stringBuilder.append((char) buffer[i]);
}
int begin = stringBuilder.indexOf(" ") + 1;
int end = stringBuilder.indexOf(" ", begin);
String requestURI = stringBuilder.substring(begin, end);
httpRequest.setRequestURI(requestURI);
}

/**
* Process an incoming TCP/IP connection on the specified socket. Any
* exception that occurs during processing must be logged and swallowed.
* <b>NOTE</b>: This method is called from our Connector's thread. We
* must assign it to our own thread so that multiple simultaneous
* requests can be handled.
* HttpConnector分配一个新的socket
* @param socket TCP socket to process
*/
public void assign(Socket socket) {
synchronized (this) {
// Wait for the Processor to get the previous Socket
while (available) {
try {
wait();
} catch (InterruptedException e) {
e.printStackTrace();
}
}

// Store the newly available Socket and notify our thread
this.socket = socket;
available = true;
notifyAll();
}
}

/**
* Await a newly assigned Socket from our Connector, or <code>null</code>
* if we are supposed to shut down.
* 等待HttpConenctor分配一个新的socket
*/
private Socket await() {
synchronized (this) {
// Wait for the Connector to provide a new Socket
while (!available) {
try {
wait();
} catch (InterruptedException e) {
e.printStackTrace();
}
}

// Notify the Connector that we have received this Socket
available = false;
notifyAll();
return socket;
}
}
}

SimpleContainer类:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
/**
* Created by kanyuxia on 2017/5/3.
*/
public class SimpleContainer implements Container {

@SuppressWarnings("unchecked")
public void invoke(HttpRequest request, HttpResponse response) throws IOException, ServletException {
String servletName = request.getRequestURI().substring(request.getRequestURI().lastIndexOf("/") + 1);
// 创建URLClassLoader
URLClassLoader classLoader = null;
try {
// 创建URL
URL[] urls = new URL[1];
File classPath = new File(HttpServer.SERVLET_ROOT);
String repository = (new URL("file", null, classPath.getCanonicalPath() + File.separator)).toString();
URLStreamHandler streamHandler = null;
urls[0] = new URL(null, repository, streamHandler);
classLoader = new URLClassLoader(urls);
} catch (IOException e) {
System.out.println();
}
Class<Servlet> servletClass = null;
try {
servletClass = (Class<Servlet>) classLoader.loadClass(servletName);
} catch (ClassNotFoundException e) {
e.printStackTrace();
}
Servlet servlet = null;
try {
servlet = servletClass.newInstance();
} catch (InstantiationException e) {
e.printStackTrace();
} catch (IllegalAccessException e) {
e.printStackTrace();
}
servlet.service(request.getRequest(), response.getResponse());
}
}

BootStrap启动类:

1
2
3
4
5
6
7
8
9
10
11
/**
* Created by kanyuxia on 2017/5/3.
*/
public class BootStrap {
public static void main(String[] args) {
HttpConnector httpConnector = new HttpConnector();
httpConnector.setContainer(new SimpleContainer());
httpConnector.initialize();
httpConnector.start();
}
}