Redevelopment in C

[cheako]

The more I read the source the more I cannot believe this wasn't written in C. Are there any design documents or block diagram lying around that would help in a rewrite?

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[cheako]

Here are some ideas that may make the source more readable.

1. Split off the port 80 listener to it's own source.
2. Perhaps this could all be a lighttpd module, assuming we don't need to accept/fork as we do currently.

[fpietrosanti]

Within Tor's environment the Python Twisted framework is widely used, that's one of the reason Tor2web is based on Twisted Python. Twisted Python additionally is high-performance and self-contained, without requiring any third party daemons, ensuring simplicity.

[fpietrosanti]

If i would re-work Tor2web in C, i would suggest to make a proper modular patch directly to the Tor's core software that's written in C. That way Tor2web would be something that can be enabled/disabled from Tor core code, without installing another application. That would be a huge improvement.

[cheako]

I'm not seeing what twisted has to offer this project, all it's doing is string manipulation. What twisted does not offer is enhancements to the http protocol. I've been looking at this for days and it's not clear to me how to add support for connection upgrade.

Smaller is better. The socks interface tor exposes is sufficient for the task.

[cheako]

I wrote some Design Documentation.

[cheako]

Do we see the need to specify chunked? According to wikipedia anytime using HTTP 1.1 this is an option and it is not necessary to specify. Nore does it seem appropriate to specify it this way.

I'll ask around if anyone knows an offending httpd.

[cheako]

After some testing I've decided that chunked encoding is not important. It may not be supported by 1.0 clients, but we don't need to go out of our way to enable it.

[fpietrosanti]

Consider that there's not a single clear way to reverse proxy many different conditions that are not under the control of the proxy (neither the HTML/CSS/JSS/Headers rewrite), neither the behaviour of the backend HTTP server.

Over 5-6 years we encountered many situations that got fixed iteratively, so please consider and exact 1-to-1 cloning also of the minor rewriting rules.

[cheako]

I've had some success writing a test suite. Currently the tests are passing.

The suite consists of a home-made socks server that prints test results it's collected to a client that connects to the telnet port of host "exit". Other virtual hosts are to be implemented, typically the host names will look like valid tor names.

This should be sufficient to test all the behaviour of both servers and clients, even fictitious cases can be tested.

Where I'll need some help is getting Tor2web started, I've made some example configs and even an ssl certificate.

Here is an example script that starts a daemon and then talks to it, I'll use it as a template for Tor2web.

[evilaliv3]

Thank you for making this @cheako, actually having testing would really help the project.

When you implementation will be ready we could integrate it with Travis and have it all retested on-commit; currently on travis we just verify some code quality metrics and we test the packaging.

Would be nice instead to make the package install and re-test it on commit.

This will require probably applying to Tor2web the same edits we just made to GlobaLeaks in order to instantiate it and control it via txtorcon.

[cheako]

Take a look at this patch. There are a handful of TODO items, once these are done we can begin hacking tx/Dbasic.t and tx/Asocks5.t to actually have a conversation through Tor2web. "git am" can be used to apply this patch to a new branch for the testsuite and I can then pull that branch and continue working.

[cheako]

I'm confused by this error msg:
https://travis-ci.org/cheako/Tor2web-1/builds/265050936#L9414

Unhandled error in Deferred:
Traceback (most recent call last):
Failure: twisted.spread.pb.PBConnectionLost: [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.error.ConnectionDone'>: Connection was closed cleanly.]

strace log doesn't show a connection. It seems like this is in response to an SSL connection, but with no call to accept() it's as though it doesn't like the cert(loads it 20 or 30 times)

[cheako]

I'm camping for the next week, so the pace of development on this will all but stop. This is a good chance to catch up on the changes.

[cheako]

See: https://github.com/cheako/tor2web/wiki/Code-Explination

[cheako]

This is coming along, there is currently more code than tests. I'm really excited about the progress on this.