net/http: Hijacker for original/redirect IP addresses when using IP_TRANSPARENT or NAT/NAPT

[elico]

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

1.6+1.7

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/eliezer/go-workspace"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build576063994=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"

What did you do?

I use http connection hijack and it locks the connection completly.
https://golang.org/src/net/http/server.go?h=hijacked#L254

What did you expect to see?

An option to hijack the connection so I would be able to extrct from it the local address(for a transparent proxy)
Like I can be done on a regular connection at:
https://github.com/elico/go-linux-tproxy/blob/master/examples/tcpproxy1.go#L167
And I am sure a method that can be a way to do so without any issues.
I was thikning aobut supplying a special listener with a middleware that will pass the local address into the request as an header But would be happy to see one(maybe there is and I don't know about).

What did you see instead?

The only option is to hijack the connection and then manage it.
This is while I don't touch any read or write methods on the connections,

[bradfitz]

Dup of #18686 from my interpretation.

[bradfitz]

@elico, in #18686 you wrote:

#18734 is not a duplicate since it's a special case and socket which this address is always changing du to the diffrent connections that is being intercepted by the proxy.
It's not static but dynamic for a tproxy socker and a server.

If it's not a dup, please clarify what you want.

Maybe https://golang.org/wiki/Questions would be a better forum until the proposal can be refined. Maybe it already exists.

[elico]

@bradfitz I will try to clear up thing.
Maybe I am wrong with my assumption but from the snippets of code I have seen it's not this way.
I am writing a proxy which uses Linux TPROXY socket. It's a transparent proxy and in order to be a full transparent proxy I need to be able to know the original destination IP address of the connection that the software\proxy intercepted.
The localaddress will always be different for such a connection.
To illustrate the idea this line is what I will be using in some oo the cases:
https://github.com/elico/go-linux-tproxy/blob/master/examples/tcpproxy1.go#L83

srvConn, err := tproxy.TcpDial((cliConn.RemoteAddr()).String(), (cliConn.LocalAddr()).String())
when in some cases I will hand over the client connection to the http handler while in other I don't.
When I decided to pass the connection to the http.Serve in someway(leaving this logic aside) I will need to be able to access the LocalAddr() inside the http sessions.
To do that currently the option for me is to Hijack the connection.
But I don't really need to fully hijack but to just look at some property of the net.conn statuc details.
I will try the forum but maybe I missed something and this is why I cannot achive what I need.
Thanks!

[bradfitz]

/cc @mikioh, who knows networking things.

[mikioh]

FWIW, in such case I tend to use a package like github.com/mikioh/tcp.

[elico]

@mikioh but what do you mean by using this package?
Low level TCP and drop all http.* to trash?
(Sorry but I do not understand how you want me to use it in the middle of a http session)

[mikioh]

Simply, 1) make a custom TCP listener type which implements net.Listener interface, 2) make the Accept method of the custom TCP listener type identify the original destination IP address, 3) pass the custom TCP listener to http.Serve function, and let it run with your HTTP serve handler. I don't see any problem to have a map containing remote and original destination IP addresses, and reading it from your HTTP serve handler.

[elico]

@mikioh I am not sure I understood yet.
I want to use the stock http serve handler and feed it with my custom listener.
Now inside a http request handler I have a request and a writter.
Let say I want to pull the local address which is unique to this request and open a custtomized http client with the IP of this specific IP(I will have one per client IP somewhere globally).
The only way I know to access the FD or net.Conn is using the hijack method.
Is there another option to access the net.Conn (with the stock http.Serve) in order to pull the unique local address?( we have the code to do so already... https://github.com/elico/go-linux-tproxy/blob/master/examples/tcpproxy1.go#L167 ) but I do not see(maybe I am missing something) a way to access the net.Conn properties in the middle of the http session without the request being hijacked.
This is why I asked if it would be possilbe to somehow access the net.Conn without fully hijacking it and breaking the http handler option to continue and work with this connection(as it being marked as hijacked).
A n example would be:
#16456

I will try to illustrate the question with code snippet:

package main

import (
	"fmt"
	"net/http"
)

func handler(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintf(w, "Hi there, I love %s!\n", r.URL.Path[1:])
	hj, _ := w.(http.Hijacker)
	conn, _, _ := hj.Hijack()
	fmt.Println(conn.LocalAddr())
	fmt.Fprintf(w, "Hi there, I love hijacker %s! is my original destination address\n", conn.LocalAddr().String())
}

func main() {
	http.HandleFunc("/", handler)
	http.ListenAndServe(":8080", nil)
}

will result in this:

2017/01/26 23:46:08 http: response.Write on hijacked connection

I want to be able to use the stock HTTP Serve and pull the LocalAddr() without breaking the option to handle the connection later inside the handler or leaving open FD.
So I would call it "semi hijack" since it's only for one purpose.
If it's possible to be done I will ask to give an example since I am not familiar with every bit of GoLang,

Thanks,
Eliezer

[mikioh]

I believe this is a duplicate issue of #18686.

@elico,

It looks like what you want is simply "an IP address which is shared between http.Request and net.Conn on a passive-open side established TCP connection ." So you can use the RemoteAddr field of http.Request as a search key for IP addresses stored in somewhere. Using a platform-dependent option like IP_TRANSPARENT doesn't matter. You can write a type that keeps IP addresses for HTTP clients, use it in a custom net.Listener for storing the IP addresses and retrieve an IP address from it by using the RemoteAddr field of http.Request.

If it's possible to be done I will ask to give an example ...

See https://golang.org/wiki/Questions.

[elico]

@mikioh Thanks!!!
This is a great idea.
My first idea was to do something like that but I had hard time to think it over.
Thanks for the time and I will try to publish an example for such a thing.
And if you don't mind, what would be your recommendation to store these pairs of addresses? IE a radix tree maybe? or a simple map?
I will try to ask at the goforum.. and #go-nuts.

[mikioh]

It depends on your circumstances. In general, it could be similar to an algorithm and data structure for IP address prefix lookup. There are tons of tradeoffs, but I guess it would be the most fun part of your software. So enjoy it!

[elico]

@mikioh No the most efficient example:
https://github.com/elico/go-linux-tproxy/blob/master/examples/http-special-handler.go
But indeed should work.