rsync through multiple ssh hops with password authentication prompt

[Manuel López-Ibáñez]

Hi, I have searched the whole internet and mailing list and I was unable
to find a clear answer to this. The method described in the FAQ [1] are
not useful for me.

Three machines source, middle, target. ssh into middle from source
requires password pass_middle. ssh into target from middle requires
password pass_target. One cannot reach target from source. So I can do:

source $ ssh middle
middle's password: pass_middle
middle $ ssh target
target's password: pass_target
target $

Also I can do it in one step:

source $ ssh -t middle ssh target
middle's password: pass_middle
target's password: pass_target
target $

Now, I want to transfer files from source to target.

source $ rsync -e "ssh -t middle ssh target" :/path/to/remote/file .
Pseudo-terminal will not be allocated because stdin is not a terminal.
middle's password:
Permission denied, please try again.
Permission denied, please try again.
Permission denied (publickey,password,keyboard-interactive).
rsync: connection unexpectedly closed (0 bytes received so far) [receiver]
rsync error: error in rsync protocol data stream (code 12) at io.c(359)

This doesn't work. Neither does:

source $ rsync -e "ssh -t -t middle ssh target" :/path/to/remote/file .
middle's password:
tcgetattr: Invalid argument
protocol version mismatch - is your shell clean?
(see the rsync man page for an explanation)
rsync error: protocol incompatibility (code 2) at compat.c(60)

Finally, the simplest form also won't work.

source $ rsync -e "ssh middle ssh target" :/path/to/remote/file .
middle's password:
Permission denied, please try again.
Permission denied, please try again.
Permission denied (publickey,password,keyboard-interactive).
rsync: connection unexpectedly closed (0 bytes received so far) [receiver]
rsync error: error in rsync protocol data stream (code 12) at io.c(359)

Both ask the first password but they generate an error before asking for
the second one.

I know for sure that if you can ssh into target from middle without a
password (using keys or whatever), then the second rsync command works
perfectly.

I wish to know if there is a way to achieve this, that is, rsync asks
for both passwords and then transfers the files. In case it is not
currently possible, would it ever be possible, that is, should I fill a
bug report or feature request report? Otherwise, it would be interesting
to update the rsync FAQ [1] to state that this is clearly impossible and
one must use a passwordless authentication method in the second hop.

Thanks for your help,

Manuel.

(Please CC me in the reply since I am not subscribed to the list)


[1] http://samba.anu.edu.au/rsync/FAQ.html#6


______________________________________________
Renovamos el Correo Yahoo!
Nuevos servicios, más seguridad
http://correo.yahoo.es

[Matt McCutchen]

Manuel,

The trouble in all three cases is that rsync expects to have the SSH
data stream to "middle" all to itself, but you need to slip a password
into this stream first so that the connection to "target" can be made.

Here's what I think would be easiest. If ssh's input is not a terminal,
but $DISPLAY is set, it tries to use a graphical password-prompting
program specified by $SSH_ASKPASS. Put such a program on "middle" if
you don't already have one, and configure "middle" so that $SSH_ASKPASS
will be appropriately set even though the shell that handles the second
SSH command will not be a login shell (this means ~/.bashrc or
~/.ssh/environment, not ~/.bash_profile). Then, enable X forwarding on
the connection to "middle". The second SSH will pop up a graphical
password prompt, which will rattle through the first SSH to your screen.

This would be the command:
rsync -e "ssh -X middle ssh target" :/path/to/remote/file .
However, I consider the following more elegant because the -e argument
specifies not the remote machine but the way of getting there:
rsync -e "ssh -X middle ssh" target:/path/to/remote/file .

I successfully used this technique to pull a file to my computer from a
computer on my school's network via an intermediary on that network.

[Manuel López-Ibáñez]

Matt,

Thanks very much. However, for several reasons, I cannot apply that
"trick" in my case. Apart from those reasons, there is no X server on
middle or target at all. Moreover, which password-prompting program
should I use? middle is an OpenBSD machine. And what should I put in
$DISPLAY?

Finally, I am not sure if your first paragraph means that this is
impossible and it won't be implemented in rsync.

Thanks anyway for your comments,

Manuel.

Post by Matt McCutchen
Manuel,
The trouble in all three cases is that rsync expects to have the SSH
data stream to "middle" all to itself, but you need to slip a password
into this stream first so that the connection to "target" can be made.
Here's what I think would be easiest. If ssh's input is not a terminal,
but $DISPLAY is set, it tries to use a graphical password-prompting
program specified by $SSH_ASKPASS. Put such a program on "middle" if
you don't already have one, and configure "middle" so that $SSH_ASKPASS
will be appropriately set even though the shell that handles the second
SSH command will not be a login shell (this means ~/.bashrc or
~/.ssh/environment, not ~/.bash_profile). Then, enable X forwarding on
the connection to "middle". The second SSH will pop up a graphical
password prompt, which will rattle through the first SSH to your screen.
rsync -e "ssh -X middle ssh target" :/path/to/remote/file .
However, I consider the following more elegant because the -e argument
rsync -e "ssh -X middle ssh" target:/path/to/remote/file .
I successfully used this technique to pull a file to my computer from a
computer on my school's network via an intermediary on that network.

______________________________________________
Renovamos el Correo Yahoo!
Nuevos servicios, más seguridad
http://correo.yahoo.es

[Matt McCutchen]

Post by Manuel López-Ibáñez
Thanks very much. However, for several reasons, I cannot apply that
"trick" in my case. Apart from those reasons, there is no X server on
middle or target at all. [...] And what should I put in $DISPLAY?

Graphical programs find the appropriate X server on which to show their
windows via the $DISPLAY variable. When you log into X, $DISPLAY is
automatically set to a code for your display, often :0.0, in the
environment of all programs started via X. Conveniently enough, X
programs communicate with X servers through sockets and ports, and SSH
already has code to forward ports. When SSH does "X forwarding", it
sets up a virtual display of sorts on the remote machine and points the
remote $DISPLAY to this display. When you run a remote graphical
program, it reads $DISPLAY and connects to the corresponding port; SSH
is watching this port and redirects the program to the real X server
port on your machine.

In short, you can just tell SSH to do X forwarding and remote graphical
programs will show their windows on "source". You need not set $DISPLAY
yourself, nor do you need an X server on any machine but "source".

Post by Manuel López-Ibáñez
Moreover, which password-prompting program
should I use? middle is an OpenBSD machine.

SSH usually comes with one of these programs, and it is called
"ssh-askpass" or similar. My system has a "gnome-ssh-askpass" and even
sets $SSH_ASKPASS automatically to this program. Failing that, a little
program called Zenity can be scripted to pop up simple dialog boxes; you
might be able to find a BSD version and use "zenity --entry" as your
$SSH_ASKPASS command.

Post by Manuel López-Ibáñez
Finally, I am not sure if your first paragraph means that this is
impossible and it won't be implemented in rsync.

I guess an option /could/ be added to rsync to have it send some data
across the network before it begins its protocol, but rsync's job isn't
to handle SSH authentication; it expects to be provided with a working
transport of some kind so that it can synchronize files. Rsync is not
the only tool that sometimes makes SSH connections without a terminal at
which the user can input the password, which is why there are alternate
techniques like this one, public key authentication, and ssh-agent.

[Manuel López-Ibáñez]

Wow! Dude! This is the most comprehensive, user-friendly, nice,
knowledgeable answer I have ever received. Thanks very much! That
answers everything.

I understand completely that it's not rsync's job to handle ssh password
prompts, however, isn't it the first password prompt handled by rsync?
Why the other password prompts cannot be handled in a similar way? This
is jut curiosity, feel free to NOT answer if you are very busy.

Cheers,

Manuel.

Post by Matt McCutchen

Post by Manuel López-Ibáñez
Thanks very much. However, for several reasons, I cannot apply that
"trick" in my case. Apart from those reasons, there is no X server on
middle or target at all. [...] And what should I put in $DISPLAY?

Graphical programs find the appropriate X server on which to show their
windows via the $DISPLAY variable. When you log into X, $DISPLAY is
automatically set to a code for your display, often :0.0, in the
environment of all programs started via X. Conveniently enough, X
programs communicate with X servers through sockets and ports, and SSH
already has code to forward ports. When SSH does "X forwarding", it
sets up a virtual display of sorts on the remote machine and points the
remote $DISPLAY to this display. When you run a remote graphical
program, it reads $DISPLAY and connects to the corresponding port; SSH
is watching this port and redirects the program to the real X server
port on your machine.
In short, you can just tell SSH to do X forwarding and remote graphical
programs will show their windows on "source". You need not set $DISPLAY
yourself, nor do you need an X server on any machine but "source".

Post by Manuel López-Ibáñez
Moreover, which password-prompting program
should I use? middle is an OpenBSD machine.

SSH usually comes with one of these programs, and it is called
"ssh-askpass" or similar. My system has a "gnome-ssh-askpass" and even
sets $SSH_ASKPASS automatically to this program. Failing that, a little
program called Zenity can be scripted to pop up simple dialog boxes; you
might be able to find a BSD version and use "zenity --entry" as your
$SSH_ASKPASS command.

Post by Manuel López-Ibáñez
Finally, I am not sure if your first paragraph means that this is
impossible and it won't be implemented in rsync.

I guess an option /could/ be added to rsync to have it send some data
across the network before it begins its protocol, but rsync's job isn't
to handle SSH authentication; it expects to be provided with a working
transport of some kind so that it can synchronize files. Rsync is not
the only tool that sometimes makes SSH connections without a terminal at
which the user can input the password, which is why there are alternate
techniques like this one, public key authentication, and ssh-agent.

______________________________________________
Renovamos el Correo Yahoo!
Nuevos servicios, m�s seguridad
http://correo.yahoo.es

[Manuel López-Ibáñez]

Post by Manuel López-Ibáñez
I understand completely that it's not rsync's job to handle ssh password
prompts, however, isn't it the first password prompt handled by rsync?
Why the other password prompts cannot be handled in a similar way? This
is jut curiosity, feel free to NOT answer if you are very busy.

Even though the stdin and stdout of the first SSH are connected to rsync
(so rsync can talk to the rsync on the other end), it can still access
its "controlling terminal" directly (using /dev/tty) to prompt you for
the password. The second SSH, however, does not have a working
controlling terminal, which is why it cannot prompt similarly.

I am not sure if I understood all, why the second ssh does not have a
working controlling terminal? I tried forcing a terminal allocation
using rsync -e "ssh -t -t middle ssh -t -t target" and several
combinations, but none worked.

man ssh(1)
-t Force pseudo-tty allocation. This can be used to execute arbi-
trary screen-based programs on a remote machine, which can be
very useful, e.g., when implementing menu services. Multiple -t
options force tty allocation, even if ssh has no local tty.

Based on the output in your first message, I'm guessing the second SSH
is falling back to using its standard error (which is coming back to
your terminal) and its standard input (which is the rsync data stream)
to prompt for and obtain the password. Perhaps the it is trying to
interpret the beginning of the rsync data stream as a password, which is
why it fails three times in a row; does that "Permission denied" message
mean "wrong password" on BSD?

I think you are right. I have no idea about OpenBSD and, actually,
target is Debian GNU/Linux. I did some experiments. "Permission denied"
messages are produced by middle (OpenBSD) if you provide wrong password.
However, the weird thing is that if you ssh into target (GNU/Linux) from
middle, it asks 6 times for the password, the first three times it just
asks again if the password was wrong, the other three times gives the
"Permission denied, try again" error.

Also, ¿how is the error message reaching me if rsync is not forwarding
the output of the second SSH?


Cheers,

Manuel.


______________________________________________
Renovamos el Correo Yahoo!
Nuevos servicios, m�s seguridad
http://correo.yahoo.es

[Manuel López-Ibáñez]

Martin Schröder's suggestion of ssh-agent might be better in the long
term or if the X business doesn't work, but I got the impression you
didn't want to set up any kind of special SSH authentication.

Yes, that is precisely the point.

There is a FAQ section (which I linked in my first message) explaining
how to do this using keys. That is not the point. For example, isn't it
possible for the root of middle (or some attacker) to get my keys and
use them? That won't be the same using passwords. Yes, you can encrypt
the keys with a passphrase, but then you will end up in the same
situation as with passwords (from the point of view of using rsync),
isn't it?

So the point is "rsync through multiple ssh hops with password
authentication". And now I will add "without X forwarding". Since the
sshd server in middle does not allow X forwarding (and there are not X
libraries or programs there, for example no xauth available).

Perhaps it is just impossible or just too complicated. Those are a valid
answers. :-)

Cheers,
Manuel.


______________________________________________
Renovamos el Correo Yahoo!
Nuevos servicios, m�s seguridad
http://correo.yahoo.es

[Matt McCutchen]

[...] There is a FAQ section (which I linked in my first message) explaining
how to do this using keys. That is not the point. For example, isn't it
possible for the root of middle (or some attacker) to get my keys and
use them?

Yes, root of middle can cause you a lot of trouble. Not only can root
intercept the password going to the second SSH; root can surreptitiously
modify the rsync data going back and forth! Now I get the picture: you
don't trust middle and would want nothing to do with it except that it
is the only way your data can reach target. There's a technique that
can deal with this situation very elegantly: forward target's SSH port
itself to your machine. Here's the general procedure:

Terminal 1:
$ ssh -L 2222:target:22 -N -f ***@middle
Password: middlepass
<ssh just sits there>

Terminal 2:
$ ssh -P 2222 ***@localhost <command, maybe>
Password: targetpass
<interact with target>

Terminal 1:
^C to kill the forwarding ssh

If you use this setup, then middle can do nothing more to you than a
random node on the Internet could. Authentication and data transfer
appear to take place directly between your machine and target; all rsync
and SSH-authentication data is securely encrypted when it passes through
middle. This is really great. The only drawback is that any process on
your machine can piggyback on your forwarding and make its own SSH
connection to target; decide whether you want to worry about this.

Since the second SSH thinks you're connecting to localhost but sees
target's host key, you're going to get dire warnings about host keys.
By editing your SSH configuration file (~/.ssh/config), you can both
solve the host key problem and make the procedure more automatic. Try a
configuration file like this:

Host middle
LocalForward 2222 target:22
User middleuser

Host target
HostName localhost
HostKeyAlias target
Port 2222
User targetuser

Then, say "ssh -N middle" in one terminal, supply the password, and let
ssh run. In another terminal, run rsync without any -e option. It will
call "ssh target", which will know from the configuration file to
actually go to localhost:2222 but to expect target's host key and will
Do The Right Thing.

I tried this technique, using a configuration file like the one above,
and successfully accessed one of my school's machines via forwarded SSH.
Good luck! I'm hoping this will prove to be the solution!

[Manuel López-Ibáñez]

That technique of using a SSH tunnel through middle seems really nice.
Actually it is similar (I think even better because yours doesn't
require a rsync server) to the one described in the rsync FAQ [1] as
method 2.

[1] http://samba.anu.edu.au/rsync/FAQ.html#6

Post by Matt McCutchen

[...] There is a FAQ section (which I linked in my first message) explaining
how to do this using keys. That is not the point. For example, isn't it
possible for the root of middle (or some attacker) to get my keys and
use them?

Yes, root of middle can cause you a lot of trouble. Not only can root
intercept the password going to the second SSH; root can surreptitiously
modify the rsync data going back and forth! Now I get the picture: you
don't trust middle and would want nothing to do with it except that it
is the only way your data can reach target. There's a technique that
can deal with this situation very elegantly: forward target's SSH port
Password: middlepass
<ssh just sits there>
Password: targetpass
<interact with target>
^C to kill the forwarding ssh
If you use this setup, then middle can do nothing more to you than a
random node on the Internet could. Authentication and data transfer
appear to take place directly between your machine and target; all rsync
and SSH-authentication data is securely encrypted when it passes through
middle. This is really great. The only drawback is that any process on
your machine can piggyback on your forwarding and make its own SSH
connection to target; decide whether you want to worry about this.
Since the second SSH thinks you're connecting to localhost but sees
target's host key, you're going to get dire warnings about host keys.
By editing your SSH configuration file (~/.ssh/config), you can both
solve the host key problem and make the procedure more automatic. Try a
Host middle
LocalForward 2222 target:22
User middleuser
Host target
HostName localhost
HostKeyAlias target
Port 2222
User targetuser
Then, say "ssh -N middle" in one terminal, supply the password, and let
ssh run. In another terminal, run rsync without any -e option. It will
call "ssh target", which will know from the configuration file to
actually go to localhost:2222 but to expect target's host key and will
Do The Right Thing.
I tried this technique, using a configuration file like the one above,
and successfully accessed one of my school's machines via forwarded SSH.
Good luck! I'm hoping this will prove to be the solution!

______________________________________________
Renovamos el Correo Yahoo!
Nuevos servicios, m�s seguridad
http://correo.yahoo.es

[Matt McCutchen]

Post by Manuel López-Ibáñez
That technique of using a SSH tunnel through middle seems really nice.
Actually it is similar (I think even better because yours doesn't
require a rsync server) to the one described in the rsync FAQ [1] as
method 2.
[1] http://samba.anu.edu.au/rsync/FAQ.html#6

Here's an interesting variant of the tunneling technique:

~/.ssh/config:
Host target
ProxyCommand ssh middle nc %h %p

This tells SSH that, to connect to an SSHD on target, it is to first SSH
into middle and then connect to target from there using "nc".

You can then say "ssh target" or "rsync myfiles target:/some/path". You
will be prompted for the middle password and then the target password.

Advantages of this approach:
(1) You don't need to remember to set up the port forwarding beforehand.
(2) No one can piggyback on your secure tunnel to target.

Possible disadvantage:
(3) If you want multiple connections to target at the same time, by
default, each will result in a new tunnel through middle. But you can
get SSH to share connections using the ControlMaster option.

Annoyances:
(4) When I try this technique, both password prompts say "Password:", so
when I enter the middle machine's password and get another prompt, it's
hard to tell whether I messed up and need to retype the first password
or the target machine is prompting me for the second password. But the
output in your first message shows that your SSHes produce more
meaningful prompts, so this probably won't be an issue.
(5) In my testing, when I log out, I get a message "Killed by signal 1".
This is some artifact of the logging-out process that doesn't indicate
anything is wrong, but I would like to make it go away.

See if you like this better than the port forwarding version.

[Matt McCutchen]

Post by Matt McCutchen
Host target
ProxyCommand ssh middle nc %h %p

Apparently, ssh uses a SIGHUP to tell the ProxyCommand that it is
finished with the connection. If nc is used as the proxy, it
understands the SIGHUP as a request to exit and all is well. However,
under the arrangement above, the SIGHUP goes to the ssh maintaining the
tunnel, which noisily dies, leaving the nc running on the middle
machine!

I came upon this article on the Internet that recommended the same
technique of using a combination of ssh and nc as a ProxyCommand:
http://www.hackinglinuxexposed.com/articles/20040830.html
The article says to provide the "-w 1" option to nc, which, according to
them, makes it exit a second after the connection is "dead". According
to the man page, however, "-w 1" causes nc to quit if the connection
is /idle/ for one second; furthermore, I don't see how the original ssh
would signal to the ProxyCommand that the connection is "dead" if all it
has is a pair of streams. Nonetheless, passing "-w 1" to nc does the
right thing: nc quits soon after I log out of the original SSH session
but keeps running if I don't type anything for a second. This is very
nice; could someone shed some light on why it works?

Since nc seems to know when to quit, one can just block the SIGHUP
coming from the original ssh to the tunnel ssh. So I recommend this
setup:

~/.ssh/config:
Host target
ProxyCommand nohup ssh middle nc -w 1 %h %p

When one logs out of target, the original ssh sends its SIGHUP (which is
ignored) and quits immediately, returning one to the command prompt.
The tunnel ssh continues running in the background. A few seconds
later, the tunnel nc decides to quit, and the tunnel ssh, seeing that
its command has quit, quits as well.

This setup seems to work well--perhaps it could be added to the rsync
FAQ page as Method 2b. The only annoyance is that one might still get
two indistinguishable "Password:" prompts; could someone tell me how to
configure SSH so the prompt reveals the target ***@host?

[Wayne Davison]

Post by Matt McCutchen
Password: middlepass

Port-forwarding 22 is a great idea as long as ssh is configured not to
complain about the host conflict -- nicely done. I've added this to the
FAQ in the "rsync through a firewall" section.

I also changed the suggested rsync command in the first method to make
the system names more intuitive than the previous method.

Post by Matt McCutchen
<ssh just sits there>

Not when using -f -- that tells ssh to disconnect from the terminal and
run in the background.

..wayne..

[Jay Fenlason]

Post by Wayne Davison

Post by Matt McCutchen
Password: middlepass

Port-forwarding 22 is a great idea as long as ssh is configured not to
complain about the host conflict -- nicely done. I've added this to the
FAQ in the "rsync through a firewall" section.

If you have the keys for both hosts in your ~/.ssh/known_hosts file,
ssh doesn't complain at all, even with host checking enabled.

The fun part is getting both entries into the file in the first place.
I've been known to use methods like "mv ~/.ssh/known_hosts
~/.ssh/known_hosts.tmp ; ssh new-forwarded.host ; cat
~/.ssh/known_hosts.tmp >> ~/.ssh/known_hosts ; rm
~/.ssh/known_hosts.tmp" and simply cutting-n-pasting with a text
editor.

-- JF

[Wayne Davison]

Post by Jay Fenlason
If you have the keys for both hosts in your ~/.ssh/known_hosts file,
ssh doesn't complain at all, even with host checking enabled.

It would complain if the "HostKeyAlias" config item wasn't used (and the
known_hosts file already had a conflicting entry for "localhost") which
was my point -- my statement was (at least intended as) a compliment to
Matt on noticing the proper ssh config items to make the tunnel work and
get the keys associated with the correct hostnames.

Post by Jay Fenlason
The fun part is getting both entries into the file in the first place.

I haven't had a problem with that when using the StrictHostKeyChecking
option to ssh. Setting it to "ask" makes ssh ask me if it should add a
new key, and it then takes care of maintaining the entries in the
known_hosts file. You can either set this option in your ~/.ssh/config
file (and make sure you don't train your fingers to automatically answer
"yes" to an unexpected unknown-hostkey prompt), or you can specify the
option only when needed on the command-line for ssh (use "no" or "ask"):

ssh -o StrictHostKeyChecking=no host

..wayne..

[Carson Gaspar]

--On Thursday, October 20, 2005 12:53 PM -0700 Wayne Davison

Post by Wayne Davison

Post by Matt McCutchen
Password: middlepass

Port-forwarding 22 is a great idea as long as ssh is configured not to
complain about the host conflict -- nicely done. I've added this to the
FAQ in the "rsync through a firewall" section.

Hot keys aren't a problem if you use HostKeyAlias - see ssh_config(4). e.g.:

ssh -L 2022:remote-host.example.com:22 -N -d middle-host.example.com

Host remote-tunnel.example.com
User remuser
HostName 127.0.0.1
Port 2022
HostKeyAlias remote-host.example.com

rsync -aH /foo/ remote-tunnel.example.com:/bar/

[Manuel López-Ibáñez]

Could this also be added to the FAQ ? Please!

Thanks,

Manuel.

Post by Carson Gaspar
--On Thursday, October 20, 2005 12:53 PM -0700 Wayne Davison

Post by Wayne Davison

Post by Matt McCutchen
Password: middlepass

Port-forwarding 22 is a great idea as long as ssh is configured not to
complain about the host conflict -- nicely done. I've added this to the
FAQ in the "rsync through a firewall" section.

Host keys aren't a problem if you use HostKeyAlias - see ssh_config(4).
ssh -L 2022:remote-host.example.com:22 -N -d middle-host.example.com
Host remote-tunnel.example.com
User remuser
HostName 127.0.0.1
Port 2022
HostKeyAlias remote-host.example.com
rsync -aH /foo/ remote-tunnel.example.com:/bar/

______________________________________________
Renovamos el Correo Yahoo!
Nuevos servicios, más seguridad
http://correo.yahoo.es

[Wayne Davison]

Post by Manuel López-Ibáñez
Could this also be added to the FAQ ? Please!

You must not have read the FAQ recently, because this idea has been
there for a week now: after Matt mentioned the HostKeyAlias option back
on the 19th, I replied saying what a nice solution it was, and that I
had added it to the FAQ.

..wayne..

[Matt McCutchen]

Post by Manuel López-Ibáñez
Could this also be added to the FAQ ? Please!

My original explanation included the HostKeyAlias technique, and hence
so does Method 3 of the FAQ. No change needs to be made to the FAQ.

Thanks to whoever it was (Wayne?) for adding the SSH proxy technique.
Now anyone who happens upon the FAQ page should be able to find at least
one way of getting through a firewall that works in their setup. :)

[Manuel López-Ibáñez]

Oh, yes! I am sorry, I should have checked that twice.
Wayne did really a great work on the FAQ.

I would add something instead (of added to) the Method #. For example,

Method 1: no rsync daemon server, passwordless authentication in middle
machine.

Method 2: no rsync daemon server, using SSH proxy

Method 3: no rsync daemon server, using SSH port forwarding

Method 4: no rsync daemon server, using SSH tunnel

Only just to give a quick summary of the differences of the methods.

Cheers,

Manuel.

Post by Matt McCutchen

Post by Manuel López-Ibáñez
Could this also be added to the FAQ ? Please!

My original explanation included the HostKeyAlias technique, and hence
so does Method 3 of the FAQ. No change needs to be made to the FAQ.
Thanks to whoever it was (Wayne?) for adding the SSH proxy technique.
Now anyone who happens upon the FAQ page should be able to find at least
one way of getting through a firewall that works in their setup. :)

______________________________________________
Renovamos el Correo Yahoo!
Nuevos servicios, m�s seguridad
http://correo.yahoo.es

[Matt McCutchen]

Post by Manuel López-Ibáñez
Method 1: no rsync daemon server, passwordless authentication in middle
machine.
Method 2: no rsync daemon server, using SSH proxy
Method 3: no rsync daemon server, using SSH port forwarding
Method 4: no rsync daemon server, using SSH tunnel

The difference between #3 and #4 is not port forwarding vs. tunnel
(we've been using the terms synonymously) but SSH port vs. rsync daemon
port. Here's how I would summarize all the methods:

#1: Rsync runs a chained SSH command as transport; authentication on
middle must be passwordless

#2: Rsync SSH-es to target using a proxy command that first SSH-es to
middle

#3: Forward target's SSH port to a local port; rsync SSH-es to that port

#4: Run rsync daemon on target and forward its port to a local port;
rsync accesses the daemon using that port

I dislike #1 because the middle machine can subvert the connection. I
dislike #3 and #4 because (a) one must remember to set up and take down
the tunnel and (b) others can take advantage of the tunnel. (If, as
many hope, SSH learns to forward filesystem sockets, (b) will go away.)
Except for some technicalities in how the proxy connection closes, #2 is
the ideal technique, and that's what I use to access my school's
firewalled backup machine.

The updated FAQ is very nice, but perhaps the "rsync through a firewall"
section should be factored out into another page because it occupies
more than half of the FAQ page.

Incidentally, I set up SSH on my machine to prefer password
authentication to keyboard-interactive authentication; now the password
prompt shows the target user and host. Thanks, Carson!

[Manuel López-Ibáñez]

Yes, your explanations are better. I will promote #2 to become #1, since
it is more likely the one that will work better in all situations.

Another thing is... if you can use "Method #2 Rsync SSH-es to target
using a proxy command that first SSH-es to middle" also with rsync
daemon servers, then: is there any possible advantage on the other three
methods? If not, can we remove everything but just Method #2 possible
with another example using a rsync daemon server? You know, I remember
some say about keeping things simple "something".

Cheers,
Manuel.

Post by Matt McCutchen

Post by Manuel López-Ibáñez
Method 1: no rsync daemon server, passwordless authentication in middle
machine.
Method 2: no rsync daemon server, using SSH proxy
Method 3: no rsync daemon server, using SSH port forwarding
Method 4: no rsync daemon server, using SSH tunnel

The difference between #3 and #4 is not port forwarding vs. tunnel
(we've been using the terms synonymously) but SSH port vs. rsync daemon
#1: Rsync runs a chained SSH command as transport; authentication on
middle must be passwordless
#2: Rsync SSH-es to target using a proxy command that first SSH-es to
middle
#3: Forward target's SSH port to a local port; rsync SSH-es to that port
#4: Run rsync daemon on target and forward its port to a local port;
rsync accesses the daemon using that port
I dislike #1 because the middle machine can subvert the connection. I
dislike #3 and #4 because (a) one must remember to set up and take down
the tunnel and (b) others can take advantage of the tunnel. (If, as
many hope, SSH learns to forward filesystem sockets, (b) will go away.)
Except for some technicalities in how the proxy connection closes, #2 is
the ideal technique, and that's what I use to access my school's
firewalled backup machine.
The updated FAQ is very nice, but perhaps the "rsync through a firewall"
section should be factored out into another page because it occupies
more than half of the FAQ page.
Incidentally, I set up SSH on my machine to prefer password
authentication to keyboard-interactive authentication; now the password
prompt shows the target user and host. Thanks, Carson!

______________________________________________
Renovamos el Correo Yahoo!
Nuevos servicios, m�s seguridad
http://correo.yahoo.es

[Wayne Davison]

For example, isn't it possible for the root of middle (or some
attacker) to get my keys and use them?

No, that's not how ssh keys work at all. Firstly, you only need to put
the *public key* on the middle host and the destination host, not your
private key (which only needs to be on your local system). Secondly,
you should have encrypted your private key on your own host, so that it
must be decrypted with a pass phrase. This makes everything work
securely. As long as ssh is configured to forward the ssh-agent data,
the remote systems will allow a chain of ssh accesses that originates
from your local system (which will have prompted you for the key's pass
phrase only at the first use of the key). This is a much better way to
configure ssh than to try to do multiple hops using passwords.

..wayne..

[Manuel López-Ibáñez]

Post by Wayne Davison

For example, isn't it possible for the root of middle (or some
attacker) to get my keys and use them?

No, that's not how ssh keys work at all. Firstly, you only need to put
the *public key* on the middle host and the destination host, not your
private key (which only needs to be on your local system). Secondly,
you should have encrypted your private key on your own host, so that it
must be decrypted with a pass phrase. This makes everything work
securely. As long as ssh is configured to forward the ssh-agent data,
the remote systems will allow a chain of ssh accesses that originates
from your local system (which will have prompted you for the key's pass
phrase only at the first use of the key). This is a much better way to
configure ssh than to try to do multiple hops using passwords.

Can rsync work while ssh-agent is forwarding data?

Cheers,

Manuel.


______________________________________________
Renovamos el Correo Yahoo!
Nuevos servicios, más seguridad
http://correo.yahoo.es

[Wayne Davison]

Post by Manuel López-Ibáñez
Can rsync work while ssh-agent is forwarding data?

Of course. All rsync needs is an 8-bit clean pipe to send its data.
What ssh does on its own to authorize connections is of no concern to
rsync.

..wayne..

[Manuel López-Ibáñez]

Post by Wayne Davison

Post by Manuel López-Ibáñez
Can rsync work while ssh-agent is forwarding data?

Of course. All rsync needs is an 8-bit clean pipe to send its data.
What ssh does on its own to authorize connections is of no concern to
rsync.
..wayne..

I think I was not very precise on my question. What I mean is:
Can the passphrase prompt and authorisation for target go through middle
to source without messing up rsync's mind as occurs when using password
authentication?

I am sorry but I am not able to see why this works where password
authentication fails.

(BTW, thanks to all the people who has replied so far, really
informative answers).




______________________________________________
Renovamos el Correo Yahoo!
Nuevos servicios, más seguridad
http://correo.yahoo.es

[Manuel López-Ibáñez]

Post by Wayne Davison

For example, isn't it possible for the root of middle (or some
attacker) to get my keys and use them?

No, that's not how ssh keys work at all. Firstly, you only need to put
the *public key* on the middle host and the destination host, not your
private key (which only needs to be on your local system). Secondly,
you should have encrypted your private key on your own host, so that it
must be decrypted with a pass phrase. This makes everything work
securely. As long as ssh is configured to forward the ssh-agent data,
the remote systems will allow a chain of ssh accesses that originates
from your local system (which will have prompted you for the key's pass
phrase only at the first use of the key). This is a much better way to
configure ssh than to try to do multiple hops using passwords.
..wayne..

OK. Then, should I carry my (encrypted) private key to everywhere? Could
it be possible to leave the private (encrypted) key in middle and still
forward the passphrase? This way I won't need to carry the private key
everywhere, the key in middle would be encrypted and the passphrase
prompt would be forwarded as before without confusing rsync.

I found a nice document [1] about securing rsync connections trough ssh
using keys, however, it doesn't explain anything about ssh-agent
forwarding or passphrase-protected keys.

[1] http://www.jdmz.net/ssh/


______________________________________________
Renovamos el Correo Yahoo!
Nuevos servicios, más seguridad
http://correo.yahoo.es

[Martin Schröder]

Post by Manuel López-Ibáñez
source $ ssh middle
middle's password: pass_middle
middle $ ssh target
target's password: pass_target
target $

Use keys and ssh-agent. :-)

[...]

Post by Manuel López-Ibáñez
Now, I want to transfer files from source to target.

Check out ssh's forwarding options.

Best
Martin

[Manuel López-Ibáñez]

Post by Matt McCutchen
This setup seems to work well--perhaps it could be added to the rsync
FAQ page as Method 2b. The only annoyance is that one might still get
two indistinguishable "Password:" prompts; could someone tell me how to
configure SSH so the prompt reveals the target

Well, actually, I get "Password: " prompts when a ssh into linux
machines and "user (at) hostname's password: " when I log into OpenBSD.
I have not been able to find any option to configure the prompt in man
pages ssh_config and sshd_config, so I would assume that it is an
implementation issue.

The OpenBSD machine has OpenSSH_4.1 and GNU/Linux has OpenSSH_3.9, so
maybe the introduced it between those versions. I really think it is
nice that it tells the hostname so perhaps we can ask the OpenSSH guys.
There is a Usenet newsgroup comp.security.ssh and a mailing list
***@securityfocus.com

If I have time in the following weeks, I will ask them (unless someone
is really curious about this and is willing to ask soon).

Cheers,

Manuel.



______________________________________________
Renovamos el Correo Yahoo!
Nuevos servicios, más seguridad
http://correo.yahoo.es

[Carson Gaspar]

--On Saturday, October 22, 2005 1:56 AM +0100 Manuel López-Ibáñez

Post by Matt McCutchen
This setup seems to work well--perhaps it could be added to the rsync
FAQ page as Method 2b. The only annoyance is that one might still get
two indistinguishable "Password:" prompts; could someone tell me how to
configure SSH so the prompt reveals the target

Well, actually, I get "Password: " prompts when a ssh into linux machines
and "user (at) hostname's password: " when I log into OpenBSD. I have not
been able to find any option to configure the prompt in man pages
ssh_config and sshd_config, so I would assume that it is an
implementation issue.

It's probably a difference between password (shows hostname) and
keyboard-interactive (doesn't unless the remote server sends it).

[Manuel López-Ibáñez]

Post by Carson Gaspar
--On Saturday, October 22, 2005 1:56 AM +0100 Manuel López-Ibáñez

Post by Matt McCutchen
This setup seems to work well--perhaps it could be added to the rsync
FAQ page as Method 2b. The only annoyance is that one might still get
two indistinguishable "Password:" prompts; could someone tell me how to
configure SSH so the prompt reveals the target

Well, actually, I get "Password: " prompts when a ssh into linux machines
and "user (at) hostname's password: " when I log into OpenBSD. I have not
been able to find any option to configure the prompt in man pages
ssh_config and sshd_config, so I would assume that it is an
implementation issue.

It's probably a difference between password (shows hostname) and
keyboard-interactive (doesn't unless the remote server sends it).

Yes, it is. I have checked it with the people of openssh (see below)

Post by Carson Gaspar

Post by Matt McCutchen
As long as the server supports it, the easy way to get it to do what you want is is to tell your client to try "password" authentication first (see PreferredAuthentications in ssh_config(5).

However, apart from using PAM, what is the difference between password and keyboard-interactive authentications?

In OpenSSH 3.9 and up (and 3.6x and below), both use PAM.
The difference is complexity: the "password" authentication allows the client to provide a password (and, optionally, change it) but that's it.
"keyboard-interactive" allows conversations of arbitrary complexity. The classic use for this is a "challenge-response" token: it supplies a challenge which you punch into a little hand-held authenticator then type in what it displays. It could do more than this and more (as can PAM, which is why the two are often used together).

And, what is the difference from the point of view of security? Are both equally secure?

I theory, they're both equally secure.

Post by Matt McCutchen
Maybe there should be an FAQ entry for this.

Yeah, the question would be: "How can I configure the password prompt?", wouldn't?
Unfortunately, I don't know the answer.

Right now, the answers are
a) configure PAM to do it (if possible), and
b) modify the ssh client.

______________________________________________
Renovamos el Correo Yahoo!
Nuevos servicios, m�s seguridad
http://correo.yahoo.es