本文描述http client使用socks代理過程中需要註意的幾個方面:1,socks5支持用戶密碼授權;2,支持https;3,支持讓代理伺服器解析DNS; 使用代理創建Socket 從原理上來看,不管用什麼http客戶端(httpclient,okhttp),最終都要轉換到 的創建上去,看到代 ...
本文描述http client使用socks代理過程中需要註意的幾個方面:1,socks5支持用戶密碼授權;2,支持https;3,支持讓代理伺服器解析DNS;
使用代理創建Socket
從原理上來看,不管用什麼http客戶端(httpclient,okhttp),最終都要轉換到java.net.Socket
的創建上去,看到代碼:
package java.net;
public Socket(Proxy proxy) {
...
}
這是JDK中對網路請求使用Socks代理的入口方法。(http代理是在http協議層之上的,不在此文討論範圍之內)。
HttpClient要實現socks代理,就需要塞進去一個Proxy對象,也就是定製兩個類:org.apache.http.conn.ssl.SSLConnectionSocketFactory
和org.apache.http.conn.socket.PlainConnectionSocketFactory
,分別對應https和http。
代碼如下:
private class SocksSSLConnectionSocketFactory extends SSLConnectionSocketFactory {
public SocksSSLConnectionSocketFactory(SSLContext sslContext, HostnameVerifier hostnameVerifier) {
super(sslContext, hostnameVerifier);
}
@Override
public Socket createSocket(HttpContext context) throws IOException {
ProxyConfig proxyConfig = (ProxyConfig) context.getAttribute(ProxyConfigKey);
if (proxyConfig != null) {//需要代理
return new Socket(proxyConfig.getProxy());
} else {
return super.createSocket(context);
}
}
@Override
public Socket connectSocket(int connectTimeout, Socket socket, HttpHost host, InetSocketAddress remoteAddress,
InetSocketAddress localAddress, HttpContext context) throws IOException {
ProxyConfig proxyConfig = (ProxyConfig) context.getAttribute(ProxyConfigKey);
if (proxyConfig != null) {//make proxy server to resolve host in http url
remoteAddress = InetSocketAddress
.createUnresolved(host.getHostName(), host.getPort());
}
return super.connectSocket(connectTimeout, socket, host, remoteAddress, localAddress, context);
}
}
和
private class SocksSSLConnectionSocketFactory extends SSLConnectionSocketFactory {
public SocksSSLConnectionSocketFactory(SSLContext sslContext, HostnameVerifier hostnameVerifier) {
super(sslContext, hostnameVerifier);
}
@Override
public Socket createSocket(HttpContext context) throws IOException {
ProxyConfig proxyConfig = (ProxyConfig) context.getAttribute(ProxyConfigKey);
if (proxyConfig != null) {
return new Socket(proxyConfig.getProxy());
} else {
return super.createSocket(context);
}
}
@Override
public Socket connectSocket(int connectTimeout, Socket socket, HttpHost host, InetSocketAddress remoteAddress,
InetSocketAddress localAddress, HttpContext context) throws IOException {
ProxyConfig proxyConfig = (ProxyConfig) context.getAttribute(ProxyConfigKey);
if (proxyConfig != null) {//make proxy server to resolve host in http url
remoteAddress = InetSocketAddress
.createUnresolved(host.getHostName(), host.getPort());
}
return super.connectSocket(connectTimeout, socket, host, remoteAddress, localAddress, context);
}
}
然後在創建httpclient對象時,給HttpClientConnectionManager設置socketFactoryRegistry
Registry<ConnectionSocketFactory> socketFactoryRegistry = RegistryBuilder.<ConnectionSocketFactory>create()
.register(Protocol.HTTP.toString(), new SocksConnectionSocketFactory())
.register(Protocol.HTTPS.toString(), new SocksSSLConnectionSocketFactory(sslContext, NoopHostnameVerifier.INSTANCE))
.build();
PoolingHttpClientConnectionManager connectionManager = new PoolingHttpClientConnectionManager(socketFactoryRegistry);
讓代理伺服器解析功能變數名稱
場景:運行httpClient的進程所在主機可能並不能上公網,大部分時候,也無法進行DNS解析,這時通常會出現功能變數名稱無法解析的IO異常,下麵介紹怎麼避免在客戶端解析功能變數名稱。
上面有一行代碼非常關鍵:
remoteAddress = InetSocketAddress
.createUnresolved(host.getHostName(), host.getPort());
變數host
是你發起http請求的目標主機和埠信息,這裡創建了一個未解析(Unresolved)的SocketAddress,在socks協議握手階段,InetSocketAddress信息會原封不動的發送到代理伺服器,由代理伺服器解析出具體的IP地址。
Socks的協議描述中有個片段:
The SOCKS request is formed as follows:
+----+-----+-------+------+----------+----------+
|VER | CMD | RSV | ATYP | DST.ADDR | DST.PORT |
+----+-----+-------+------+----------+----------+
| 1 | 1 | X'00' | 1 | Variable | 2 |
+----+-----+-------+------+----------+----------+
Where:
o VER protocol version: X'05'
o CMD
o CONNECT X'01'
o BIND X'02'
o UDP ASSOCIATE X'03'
o RSV RESERVED
o ATYP address type of following address
o IP V4 address: X'01'
o DOMAINNAME: X'03'
o IP V6 address: X'04'
代碼按上面方法寫,協議握手發送的是ATYP=X'03'
,即採用功能變數名稱的地址類型。否則,HttpClient會嘗試在客戶端解析,然後發送ATYP=X'01'
進行協商。當然,大多數時候HttpClient在解析功能變數名稱的時候就掛了。
https中需要註意的問題
在使用httpclient訪問https網站的時候,經常會遇到javax.net.ssl包中的異常,例如:
Caused by: javax.net.ssl.SSLException: Received fatal alert: internal_error
at sun.security.ssl.Alerts.getSSLException(Unknown Source) ~[na:1.7.0_80]
at sun.security.ssl.Alerts.getSSLException(Unknown Source) ~[na:1.7.0_80]
一般需要做幾個設置:
創建不校驗證書鏈的SSLContext
SSLContext sslContext = null;
try {
sslContext = new SSLContextBuilder().loadTrustMaterial(null, new TrustStrategy() {
@Override
public boolean isTrusted(X509Certificate[] chain, String authType)
throws CertificateException {
return true;
}
}).build();
} catch (Exception e) {
throw new com.aliyun.oss.ClientException(e.getMessage());
}
...
new SocksSSLConnectionSocketFactory(sslContext, NoopHostnameVerifier.INSTANCE)
創建不校驗功能變數名稱的HostnameVerifier
public class NoopHostnameVerifier implements javax.net.ssl.HostnameVerifier {
public static final NoopHostnameVerifier INSTANCE = new NoopHostnameVerifier();
@Override
public boolean verify(final String s, final SSLSession sslSession) {
return true;
}
}
如何使用用戶密碼授權?
java SDK中給Socks代理授權有點特殊,不是按socket來的,而是在系統層面做的全局配置。比如,可以通過下麵代碼設置一個全局的Authenticator:
Authenticator.setDefault(new MyAuthenticator("userName", "Password"));
...
class MyAuthenticator extends java.net.Authenticator {
private String user ;
private String password ;
public MyAuthenticator(String user, String password) {
this.user = user;
this.password = password;
}
protected PasswordAuthentication getPasswordAuthentication() {
return new PasswordAuthentication(user, password.toCharArray());
}
}
這種方法很簡單,不過有些不方便的地方,如果你的產品中需要連接不同的Proxy伺服器,而他們的用戶名密碼是不一樣的,那麼這個方法就不適用了。
基於ThreadLocal的Authenticator
public class ThreadLocalProxyAuthenticator extends Authenticator{
private ThreadLocal<PasswordAuthentication> credentials = null;
private static class SingletonHolder {
private static final ThreadLocalProxyAuthenticator instance = new ThreadLocalProxyAuthenticator();
}
public static final ThreadLocalProxyAuthenticator getInstance() {
return SingletonHolder.instance;
}
public void setCredentials(String user, String password) {
credentials.set(new PasswordAuthentication(user, password.toCharArray()));
}
public static void clearCredentials() {
ThreadLocalProxyAuthenticator authenticator = ThreadLocalProxyAuthenticator.getInstance();
Authenticator.setDefault(authenticator);
authenticator.credentials.set(null);
}
public PasswordAuthentication getPasswordAuthentication() {
return credentials.get();
}
}
這個類意味著,授權信息只會保存到當前調用者的線程中,其他線程的調用者無法訪問,在創建Socket的線程中設置密鑰和清理密鑰,就可以做到授權按照Socket連接進行隔離。Java TheadLocal相關知識本文不贅述。
按連接隔離的授權
class ProxyHttpClient extends CloseableHttpClient{
private CloseableHttpClient httpClient;
public ProxyHttpClient(CloseableHttpClient httpClient){
this.httpClient=httpClient;
}
protected CloseableHttpResponse doExecute(HttpHost target, HttpRequest request, HttpContext context) throws IOException, ClientProtocolException {
ProxyConfig proxyConfig = //這裡獲取當前連接的代理配置信息
boolean clearCredentials = false;
if (proxyConfig != null) {
if (context == null) {
context = HttpClientContext.create();
}
context.setAttribute(ProxyConfigKey, proxyConfig);
if (proxyConfig.getAuthentication() != null) {
ThreadLocalProxyAuthenticator.setCredentials(proxyConfig.getAuthentication());//設置授權信息
clearCredentials = true;
}
}
try {
return httpClient.execute(target, request, context);
} finally {
if (clearCredentials) {//清理授權信息
ThreadLocalProxyAuthenticator.clearCredentials();
}
}
}
}
另外,線程是可以復用的,因為每次調用完畢後,都清理了授權信息。
這裡有個一POJO類ProxyConfig
,保存的是socks代理的IP埠和用戶密碼信息。
public class ProxyConfig {
private Proxy proxy;
private PasswordAuthentication authentication;
}