Build a remote management console using Python and Jupyter Notebooks

Turn Jupyter into a remote administration console.
72 readers like this.
Computer laptop in space

Opensource.com

Secure shell (SSH) is a powerful tool for remote administration, but it lacks some niceties. Writing a full-fledged remote administration console sounds like it would be a lot of work. Surely, someone in the open source community has already written something?

They have, and its name is Jupyter. You might think Jupyter is one of those tools data scientists use to analyze trends in ad clicks over a week or something. This is not wrong—they do, and it is a great tool for that. But that is just scratching its surface.

About SSH port forwarding

Sometimes, there is a server that you can SSH into over port 22. There is no reason to assume you can connect to any other port. Maybe you are SSHing through another "jumpbox" server that has more access or there are host or network firewalls that restrict ports. There are good reasons to restrict IP ranges for access, of course. SSH is a secure protocol for remote management, but allowing anyone to connect to any port is quite unnecessary.

Here is an alternative: Run a simple SSH command with port forwarding to forward a local port to a remote local connection. When you run an SSH port-forwarding command like -L 8111:127.0.0.1:8888, you are telling SSH to forward your local port 8111 to what the remote host thinks 127.0.0.1:8888 is. The remote host thinks 127.0.0.1 is itself.

Just like on Sesame Street, "here" is a subtle word.

The address 127.0.0.1 is how you spell "here" to the network.

Learn by doing

This might sound confusing, but running this is less complicated than explaining it:

$ ssh -L 8111:127.0.0.1:8888 moshez@172.17.0.3
Linux 6ad096502e48 5.4.0-40-generic #44-Ubuntu SMP Tue Jun 23 00:01:04 UTC 2020 x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Wed Aug  5 22:03:25 2020 from 172.17.0.1
$ jupyter/bin/jupyter lab --ip=127.0.0.1
[I 22:04:29.771 LabApp] JupyterLab application directory is /home/moshez/jupyter/share/jupyter/lab
[I 22:04:29.773 LabApp] Serving notebooks from local directory: /home/moshez
[I 22:04:29.773 LabApp] Jupyter Notebook 6.1.1 is running at:
[I 22:04:29.773 LabApp] http://127.0.0.1:8888/?token=df91012a36dd26a10b4724d618b2e78cb99013b36bb6a0d1
<MORE STUFF SNIPPED>

Port-forward 8111 to 127.0.0.1 and start Jupyter on the remote host that's listening on 127.0.0.1:8888.

Now you need to understand that Jupyter is lying. It thinks you need to connect to port 8888, but you forwarded that to port 8111. So, after you copy the URL to your browser, but before clicking Enter, modify the port from 8888 to 8111:

There it is: your remote management console. As you can see, there is a "Terminal" icon at the bottom. Click it to get a terminal:

You can run a command. Creating a file will show it in the file browser on the side. You can click on that file to open it in an editor that is running locally:

You can also download, rename, or delete files:

Clicking on the little Up arrow will let you upload files. Why not upload the screenshot above?

As a nice final tidbit, Jupyter lets you view the remote images directly by double-clicking on them.

Oh, right, and if you want to do systems automation using Python, you can also use Jupyter to open a notebook.

So the next time you need to remotely manage a firewalled environment, why not use Jupyter?

What to read next
Tags
Moshe sitting down, head slightly to the side. His t-shirt has Guardians of the Galaxy silhoutes against a background of sound visualization bars.
Moshe has been involved in the Linux community since 1998, helping in Linux "installation parties". He has been programming Python since 1999, and has contributed to the core Python interpreter. Moshe has been a DevOps/SRE since before those terms existed, caring deeply about software reliability, build reproducibility and other such things.

4 Comments

Great article. This reminds me I need to learn more Python and especially this Jupyter.

Hi there, I did not quite understand how you would access a firewalled remote and start Jupiter there. I assume Jupiter kernel is already running on the remote as a service that can be accessed through port 8888.

My point is that if you already have access to remote using SSH (i.e port 22 on remote is not blocked) then most remote management is already possible using a terminal - so why do we need to access remote terminal using Jupiter running on remote.

Cheers, bou

The Jupyter terminal experience is awful. And if you just need a way to upload files, scp or rsync, or filezilla with sftp for a gui, or even `ssh user@server -XC4` then `nautilus` and congrats you're browsing remote files in a Linux file manager. Drag and drop should work too. Or set up samba and mount your remote as a folder on local. Or, or, or.

Jupyter sucks for anything but notebooks.

As for access behind a firewall, obviously you can't unless you a) expose the ip so it's not behind a firewall, b) get a reverse proxy, or c) reverse tunnel by initiating the ssh connection from the remote to the external local, and port forward the ssh port so you can ssh to localhost on that port.

c) is the only real option as IT won't give a) or b), but c) is definitely against the rules and maybe the law.

In reply to by bou

Thank you for the idea of being able to remotely control network hosts having only a browser installed on you box.
That might be useful for the people who want to use nothing but the browser/Jupyter. Still, they need to run port forwarding ssh command and starting Jupyter service, but after that they are in their comfort zone.
For the rest of us, there are plenty of other options and tools. Between good old terminal with command line interface and plethora of open source GUI tools capable of making SSH connections.

Creative Commons LicenseThis work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License.