rsync & SSL 'for real'

[Andreas Kotes]

Dear All,

I've been using rsync with stunnel with success for a while now, but I'm
not necessarily satisfied with it, i.e. having an extra layer externally
and all, no authentication against SSL-subjects ..

I had a look at the patch included with the source, and it doesn't quite
address everything related to SSL, i.e. forcing SSL, verify options etc.
are all missing - besides, it's not included in the main source for some
reason.

I'd be willing to put up the effort to add full (or at least better) SSL
support direclty to rsync if the maintainers would agree with it, and
would seriously consider including it in the main source in the long
run.

My approach would:
- stay backwards-compatible with the existing patch (starttls)
- support SSL-only ports (a port number would have to be allocated)
- support checking the certificate chain (revocation lists would have to
be maintained externally, thou)
- support enforcing the use of SSL (globally as well as for certain
hosts)
- support authentication against x509-subjects instead of usernames
- configuration via existing methods (commandline, config-file)
- support for hardware-aided crypto/randomness via OpenSSL engines
- enough inline documentation to allow for maintenance by the main team

Anything else?

Please let me know wether there would be interest by the community as
well as support by the maintainers by the end of the month.

Thanks,

Andreas

[Wayne Davison]

Post by Andreas Kotes
I'd be willing to put up the effort to add full (or at least better)
SSL support direclty to rsync if the maintainers would agree with it,
and would seriously consider including it in the main source in the
long run.

I'd be interested in getting SSL support into the main rsync codebase
(in such a way that it could be conditionally omitted, if needed). I
haven't had time to really check into it myself (I need to figure out
why the SSL patch didn't work for me, but never had the impetus to delve
into it).

Your list of SSL goals seem quite reasonable to me.

..wayne..

[Lawrence D. Dunn]

Colleagues,
If you do pursue SSL functionality directly in rsync,
please be sure to take a look at Chris Rapier's work
to "fix" standard ssh implementations, at:
http://www.psc.edu/networking/projects/hpn-ssh/

Turns out "-e ssh" using most libraries puts a fixed-window-size
ssh-windowing
behavior on top of TCP - so for large bandwidth*delay product paths,
even if you use large TCP buffers (which Wayne added for such paths),
an "un-fixed" SSL library can clobber overall performance/throughput,
even for a perfectly clean (no errors/loss) path.

Larry
--

Post by Wayne Davison

Post by Andreas Kotes
I'd be willing to put up the effort to add full (or at least better)
SSL support direclty to rsync if the maintainers would agree with it,
and would seriously consider including it in the main source in the
long run.

I'd be interested in getting SSL support into the main rsync codebase
(in such a way that it could be conditionally omitted, if needed). I
haven't had time to really check into it myself (I need to figure out
why the SSL patch didn't work for me, but never had the impetus to delve
into it).
Your list of SSL goals seem quite reasonable to me.
..wayne..
--
https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Carson Gaspar]

Post by Lawrence D. Dunn
Colleagues,
If you do pursue SSL functionality directly in rsync,
please be sure to take a look at Chris Rapier's work
http://www.psc.edu/networking/projects/hpn-ssh/
Turns out "-e ssh" using most libraries puts a fixed-window-size
ssh-windowing
behavior on top of TCP - so for large bandwidth*delay product paths,
even if you use large TCP buffers (which Wayne added for such paths),
an "un-fixed" SSL library can clobber overall performance/throughput,
even for a perfectly clean (no errors/loss) path.

SSL != SSH.

[Aaron W Morris]

Post by Carson Gaspar

Post by Lawrence D. Dunn
Colleagues,
If you do pursue SSL functionality directly in rsync,
please be sure to take a look at Chris Rapier's work
http://www.psc.edu/networking/projects/hpn-ssh/
Turns out "-e ssh" using most libraries puts a fixed-window-size
ssh-windowing
behavior on top of TCP - so for large bandwidth*delay product paths,
even if you use large TCP buffers (which Wayne added for such paths),
an "un-fixed" SSL library can clobber overall performance/throughput,
even for a perfectly clean (no errors/loss) path.

SSL != SSH.

This still applies (depending on the ssl toolkit being used). The
problem referenced here is the TCP window size is hard coded inside
the openssl library. In order to change the window size, one must
patch openssl.

Of course, there is also the question of if openssl is the appropriate
toolkit to use with rsync. I am not sure of the issues with a GPL
binary linking against a BSD library. Perhaps GnuTLS is more
appropriate... (I know... this is probably a whole different can of
worms. :-) ).

[Carson Gaspar]

Post by Aaron W Morris

Post by Carson Gaspar

Post by Lawrence D. Dunn
Colleagues,
If you do pursue SSL functionality directly in rsync,
please be sure to take a look at Chris Rapier's work
http://www.psc.edu/networking/projects/hpn-ssh/
Turns out "-e ssh" using most libraries puts a fixed-window-size
ssh-windowing
behavior on top of TCP - so for large bandwidth*delay product paths,
even if you use large TCP buffers (which Wayne added for such paths),
an "un-fixed" SSL library can clobber overall performance/throughput,
even for a perfectly clean (no errors/loss) path.

SSL != SSH.

This still applies (depending on the ssl toolkit being used). The
problem referenced here is the TCP window size is hard coded inside
the openssl library. In order to change the window size, one must
patch openssl.

TCP window size is not, and can not, be hard coded inside openssl. Do
you know what TCP window size is?

Post by Aaron W Morris
Of course, there is also the question of if openssl is the appropriate
toolkit to use with rsync. I am not sure of the issues with a GPL
binary linking against a BSD library. Perhaps GnuTLS is more
appropriate... (I know... this is probably a whole different can of
worms. :-) ).

There is no license issue. This is just a troll.

[gregorcy]

that was a little harsh

Post by Carson Gaspar

Post by Aaron W Morris

Post by Lawrence D. Dunn

Post by Lawrence D. Dunn
Colleagues,
If you do pursue SSL functionality directly in rsync,
please be sure to take a look at Chris Rapier's work
http://www.psc.edu/networking/projects/hpn-ssh/
Turns out "-e ssh" using most libraries puts a fixed-window-size
ssh-windowing
behavior on top of TCP - so for large bandwidth*delay product paths,
even if you use large TCP buffers (which Wayne added for such

paths),

Post by Lawrence D. Dunn
an "un-fixed" SSL library can clobber overall

performance/throughput,

Post by Lawrence D. Dunn
even for a perfectly clean (no errors/loss) path.

SSL != SSH.

This still applies (depending on the ssl toolkit being used). The
problem referenced here is the TCP window size is hard coded inside
the openssl library. In order to change the window size, one must
patch openssl.

TCP window size is not, and can not, be hard coded inside openssl. Do
you know what TCP window size is?

Post by Aaron W Morris
Of course, there is also the question of if openssl is the appropriate
toolkit to use with rsync. I am not sure of the issues with a GPL
binary linking against a BSD library. Perhaps GnuTLS is more
appropriate... (I know... this is probably a whole different can of
worms. :-) ).

There is no license issue. This is just a troll.

[Aaron W Morris]

Post by gregorcy

Post by Carson Gaspar

Post by Aaron W Morris
This still applies (depending on the ssl toolkit being used). The
problem referenced here is the TCP window size is hard coded inside
the openssl library. In order to change the window size, one must
patch openssl.

TCP window size is not, and can not, be hard coded inside openssl. Do
you know what TCP window size is?

Post by Aaron W Morris
Of course, there is also the question of if openssl is the appropriate
toolkit to use with rsync. I am not sure of the issues with a GPL
binary linking against a BSD library. Perhaps GnuTLS is more
appropriate... (I know... this is probably a whole different can of
worms. :-) ).

There is no license issue. This is just a troll.

that was a little harsh

I'll say. Completely uncalled for and unprofessional. There is no
need for name calling here.

I can admit when I'm wrong. I apologize for any mis-information, I
was just trying to help.

[Andreas Kotes]

Hello,

Post by Carson Gaspar

Post by Aaron W Morris
This still applies (depending on the ssl toolkit being used). The
problem referenced here is the TCP window size is hard coded inside
the openssl library. In order to change the window size, one must
patch openssl.

TCP window size is not, and can not, be hard coded inside openssl. Do
you know what TCP window size is?

well, tuning the TCP parameters can (which is basically what the patch
does), but I think this should be left to the kernel / OS via sysctl and
the like, so I won't touch what's happening there.

Post by Carson Gaspar

Post by Aaron W Morris
Of course, there is also the question of if openssl is the appropriate
toolkit to use with rsync. I am not sure of the issues with a GPL
binary linking against a BSD library. Perhaps GnuTLS is more
appropriate... (I know... this is probably a whole different can of
worms. :-) ).

There is no license issue.

There would be a serious licence issue the other way round, but BSD is a
tad more permissive than the GPL is, so - no problem there BUT: there is
an advertisement clause, so rsync would need to display certain messages
when compiled with OpenSSL.

I'm used to working with the OpenSSL (and ssleay before) API, but I
suppose I'd enjoy learning gnutls as well - albeit it's not as
self-contained as OpenSSL is (hello gnupg!), and the API is quite
different (OpenSSL has grown to it's current state, when gnutls was
started they already knew what's about to be required).

The planned features would have to be revised, as it doesn't support
hardware engines (which the bulk of the users won't use anyway and
seldom can cope compared to raw CPU power in the long run) and could
possibly offer more/other stuff than OpenSSL does.

In the end, both have my confidence (the code is mature and well in
production use, so lots of bugs have already been shaken loose), the
basic featureset is the same - and while I'll look at the OpenSSL-patch,
I'll not necessarily base my work directly on it - not much harm done
changing horses before the real race.

Opinions?

Post by Carson Gaspar
This is just a troll.

just who exactly are you talking about? ;)

Best regards,

Andreas

[Aaron W Morris]

Post by Andreas Kotes

Post by Carson Gaspar
This is just a troll.

just who exactly are you talking about? ;)

Ahem... That would be me. That will be $10 for posting in this list. :-)

[Jamie Lokier]

Post by Andreas Kotes

Post by Carson Gaspar
There is no license issue.

There would be a serious licence issue the other way round, but BSD is a
tad more permissive than the GPL is, so - no problem there BUT: there is
an advertisement clause, so rsync would need to display certain messages
when compiled with OpenSSL.

The modern BSD licenses, which are compatible with the GPL, are not
the same as OpenSSL's license, which is more like the _original_ BSD
license which isn't compatible with the GPL.

Any advertisement clause is widely considered to conflict with the
GPL's "no further restrictions" clause.

This conflict is important enough that Berkeley changed the BSD
license used for BSD itself, and FreeBSD changed their license, and so on.

The OpenSSL FAQ has a section on this question:

* Can I use OpenSSL with GPL software?

On many systems including the major Linux and BSD distributions,
yes (the GPL does not place restrictions on using libraries that
are part of the normal operating system distribution).

On other systems, the situation is less clear. Some GPL software
copyright holders claim that you infringe on their rights if you
use OpenSSL with their software on operating systems that don't
normally include OpenSSL.

If you develop open source software that uses OpenSSL, you may
find it useful to choose an other license than the GPL, or state
explicitly that "This program is released under the GPL with the
additional exemption that compiling, linking, and/or using
OpenSSL is allowed." If you are using GPL software developed by
others, you may want to ask the copyright holder for permission
to use their software with OpenSSL.

The FSF's considered position is that OpenSSL is incompatible with the
GPL. They say:

The license of OpenSSL is a conjunction of two licenses, one of
them being the license of SSLeay. You must follow both. The
combination results in a copyleft free software license that is
incompatible with the GNU GPL. It also has an advertising clause
like the original BSD license and the Apache license.

We recommend using GNUTLS instead of OpenSSL in software you
write. However, there is no reason not to use OpenSSL and
applications that work with OpenSSL.

I think libcurl supports both OpenSSL and GNUTLS for these sorts of
reasons.

-- Jamie

[Lawrence D. Dunn]

Carson,
My mistake- read "SSL", immediately started thinking "ssh",
and issues there. No excuse.
So- my comments aren't applicable to the SSL-for-real discussion -
apologies to the list.

(Aside: the issues with ssh are not about modifying TCP buffers.
They are about a fixed-size ssh-windowing behavior,
which happens "on top of" whatever TCP is allowing.
The end result is similar to having too-small TCP buffers.
That's what Chris' patch addresses - he allows the ssh-windowing
behavior to become dynamic, essentially tracking TCP's window size.
Current Linux releases do a good job of auto-tuning TCP buffers,
without need for manual adjustment. )

Again- sorry for the tangent.

Larry
--

Post by Carson Gaspar

Post by Lawrence D. Dunn
Colleagues,
If you do pursue SSL functionality directly in rsync,
please be sure to take a look at Chris Rapier's work
http://www.psc.edu/networking/projects/hpn-ssh/
Turns out "-e ssh" using most libraries puts a fixed-window-size
ssh-windowing
behavior on top of TCP - so for large bandwidth*delay product paths,
even if you use large TCP buffers (which Wayne added for such paths),
an "un-fixed" SSL library can clobber overall performance/throughput,
even for a perfectly clean (no errors/loss) path.

SSL != SSH.
--
Carson
--
https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html