Backend Coder Logo

Secure Data Transfer Between Computer and Server

Published: 9th Dec 2016

In this post I want to share what I have learned in regard to secure data transfer between a computer and a web server. This includes remote login, secure website hosting, and the secure transfer of data and files. Disclaimer: I'm no expert on this topic, so do your own further research on more academic sites if you want to gain a deep understanding.

The first thing that we want to do is establish a secure link between us and our destination such as between our computer and the remote server. The best way to do this is with an SSH Tunnel. This is a connection over the Internet where the data is passed in encrypted form that is virtually impossible for anybody along the transmission path to snoop on and make sense of the data that is being passed to and from the end points.

Now a problem arises here. Both parties at the endpoints need a key to unlock the encrypted data and the question arises: "how to share the key"? It cannot be sent over an unsecured connection since it can be intercepted. And it is inconvenient to convey over a telephone call (not secure), or by hand delivery.

But luckily for us, there was a mechanism developed called the Diffie-Hellman (DH) algorithm that is surprisingly simple to implement. It allows two parties to establish a secret key over an insecure transmission path.

The basic formula that is used in DH is n = gkey mod p where p is a prime number and g (generator) is a primitive root modulo p (practically it is usually 2 or 3 in crypto systems I think!). If there are two people Alex and Bob they may agree on p = 31 and g = 3 for example. If we know all of the values to use in the calculation, it is easy to calculate, but if the private values are missing, it is hard (computationally expensive) to get the same result (when we use big numbers).

The following PHP code demonstrates how Alex and Bob may obtain a secret key whilst sharing much of the information in the public domain.


<pre>
<?php
// Alex posts code for a number calculator function to a public forum

function calculate_number($private_key, $generator = 3)
{
	return pow($generator, $private_key) % 31;
}

// Bob copies the code to his computer and makes a private key
$bob_private_key = 5;

// Alex makes a private key also on his home computer
$alex_private_key = 4;

// Alex calculates a number to post to Bob in the public Forum comments
$alex_number = calculate_number($alex_private_key);

// Bob does the same
$bob_number = calculate_number($bob_private_key);

printf("Alex's number: %d Bob's number: %d\r\n", $alex_number, $bob_number);

// Alex calculates the shared secret from Bob's number that he saw on the Forum thread
$alex_secret = calculate_number($alex_private_key, $bob_number);

// Bob calculates the shared secret from Alex's number off the Forum thread
$bob_secret = calculate_number($bob_private_key, $alex_number);

// Both numbers should be the same
printf("Alex's secret: %d Bob's secret: %d", $alex_secret, $bob_secret);

In practice, a very large prime number is used and the calculations use Big Integers (arbitrarily large integers). Since power calculations are used, it is easy to overflow fixed length integers. So with PHP you would need the GMP library or similar as follows:


<pre>
<?php
$a = 5479834; // Just a randomly chosen number for demo purposes
$b = 664896; // Again, a randomly chosen number

$g = 2;

$p = str_replace(" ", "", "0x0" .
    "FFFFFFFF FFFFFFFF C90FDAA2 2168C234 C4C6628B 80DC1CD1" .
    "29024E08 8A67CC74 020BBEA6 3B139B22 514A0879 8E3404DD" .
    "EF9519B3 CD3A431B 302B0A6D F25F1437 4FE1356D 6D51C245" .
    "E485B576 625E7EC6 F44C42E9 A637ED6B 0BFF5CB6 F406B7ED" .
    "EE386BFB 5A899FA5 AE9F2411 7C4B1FE6 49286651 ECE45B3D" .
    "C2007CB8 A163BF05 98DA4836 1C55D39A 69163FA8 FD24CF5F" .
    "83655D23 DCA3AD96 1C62F356 208552BB 9ED52907 7096966D" .
    "670C354E 4ABC9804 F1746C08 CA237327 FFFFFFFF FFFFFFFFF");

$A = gmp_powm($g, $a, $p);
$B = gmp_powm($g, $b, $p);

$s1 = gmp_powm($B, $a, $p);
$s2 = gmp_powm($A, $b, $p);

echo gmp_strval($s1) . "\n";
echo gmp_strval($s2) . "\n";

Having a large prime number makes it impractical to reverse engineer the secret key. Suggested large prime numbers and the generator value may be found here.

C# Diffie Hellman Demo

In C# we can use the System.Numerics library to make use of Big Integers.


using System.Numerics;

namespace DiffieHellman
{
    class Program
    {
        static void Main(string[] args)
        {
            // a, b should be large numbers
            // p should be a large prime number since the mod determines how many possible values there are to guess.
            // Examples are here: https://tools.ietf.org/html/rfc3526
            // g may be a small primitive root modulo p number such as 2, 3, ...
            BigInteger a = 5479834;
            BigInteger b = 664896;

            string hexNumber = ("0" + // Append 0 to avoid the number being made negative
    "FFFFFFFF FFFFFFFF C90FDAA2 2168C234 C4C6628B 80DC1CD1" +
    "29024E08 8A67CC74 020BBEA6 3B139B22 514A0879 8E3404DD" +
    "EF9519B3 CD3A431B 302B0A6D F25F1437 4FE1356D 6D51C245" +
    "E485B576 625E7EC6 F44C42E9 A637ED6B 0BFF5CB6 F406B7ED" +
    "EE386BFB 5A899FA5 AE9F2411 7C4B1FE6 49286651 ECE45B3D" +
    "C2007CB8 A163BF05 98DA4836 1C55D39A 69163FA8 FD24CF5F" +
    "83655D23 DCA3AD96 1C62F356 208552BB 9ED52907 7096966D" +
    "670C354E 4ABC9804 F1746C08 CA18217C 32905E46 2E36CE3B" +
    "E39E772C 180E8603 9B2783A2 EC07A28F B5C55DF0 6F4C52C9" +
    "DE2BCBF6 95581718 3995497C EA956AE5 15D22618 98FA0510" +
    "15728E5A 8AAAC42D AD33170D 04507A33 A85521AB DF1CBA64" +
    "ECFB8504 58DBEF0A 8AEA7157 5D060C7D B3970F85 A6E1E4C7" +
    "ABF5AE8C DB0933D7 1E8C94E0 4A25619D CEE3D226 1AD2EE6B" +
    "F12FFA06 D98A0864 D8760273 3EC86A64 521F2B18 177B200C" +
    "BBE11757 7A615D6C 770988C0 BAD946E2 08E24FA0 74E5AB31" +
    "43DB5BFC E0FD108E 4B82D120 A9210801 1A723C12 A787E6D7" +
    "88719A10 BDBA5B26 99C32718 6AF4E23C 1A946834 B6150BDA" +
    "2583E9CA 2AD44CE8 DBBBC2DB 04DE8EF9 2E8EFC14 1FBECAA6" +
    "287C5947 4E6BC05D 99B2964F A090C3A2 233BA186 515BE7ED" +
    "1F612970 CEE2D7AF B81BDD76 2170481C D0069127 D5B05AA9" +
    "93B4EA98 8D8FDDC1 86FFB7DC 90A6C08F 4DF435C9 34028492" +
    "36C3FAB4 D27C7026 C1D4DCB2 602646DE C9751E76 3DBA37BD" +
    "F8FF9406 AD9E530E E5DB382F 413001AE B06A53ED 9027D831" +
    "179727B0 865A8918 DA3EDBEB CF9B14ED 44CE6CBA CED4BB1B" +
    "DB7F1447 E6CC254B 33205151 2BD7AF42 6FB8F401 378CD2BF" +
    "5983CA01 C64B92EC F032EA15 D1721D03 F482D7CE 6E74FEF6" +
    "D55E702F 46980C82 B5A84031 900B1C9E 59E7C97F BEC7E8F3" +
    "23A97A7E 36CC88BE 0F1D45B7 FF585AC5 4BD407B2 2B4154AA" +
    "CC8F6D7E BF48E1D8 14CC5ED2 0F8037E0 A79715EE F29BE328" +
    "06A1D58B B7C5DA76 F550AA3D 8A1FBFF0 EB19CCB1 A313D55C" +
    "DA56C9EC 2EF29632 387FE8D7 6E3C0468 043E8F66 3F4860EE" +
    "12BF2D5B 0B7474D6 E694F91E 6DBE1159 74A3926F 12FEE5E4" +
    "38777CB6 A932DF8C D8BEC4D0 73B931BA 3BC832B6 8D9DD300" +
    "741FA7BF 8AFC47ED 2576F693 6BA42466 3AAB639C 5AE4F568" +
    "3423B474 2BF1C978 238F16CB E39D652D E3FDB8BE FC848AD9" +
    "22222E04 A4037C07 13EB57A8 1A23F0C7 3473FC64 6CEA306B" +
    "4BCBC886 2F8385DD FA9D4B7F A2C087E8 79683303 ED5BDD3A" +
    "062B3CF5 B3A278A6 6D2A13F8 3F44F82D DF310EE0 74AB6A36" +
    "4597E899 A0255DC1 64F31CC5 0846851D F9AB4819 5DED7EA1" +
    "B1D510BD 7EE74D73 FAF36BC3 1ECFA268 359046F4 EB879F92" +
    "4009438B 481C6CD7 889A002E D5EE382B C9190DA6 FC026E47" +
    "9558E447 5677E9AA 9E3050E2 765694DF C81F56E8 80B96E71" +
    "60C980DD 98EDD3DF FFFFFFFF FFFFFFFF").Replace(" ", "");  // Now that is some big number!

            BigInteger p = BigInteger.Parse(hexNumber, System.Globalization.NumberStyles.AllowHexSpecifier);
            BigInteger g = 2;

            BigInteger A = BigInteger.ModPow(g, a, p);
            BigInteger B = BigInteger.ModPow(g, b, p);

            // s1 and s2 should be the same number which will be used for a shared secret key for Synchronous data transfer.
            BigInteger s1big = BigInteger.ModPow(B, a, p);
            BigInteger s2big = BigInteger.ModPow(A, b, p);

            bool equal = BigInteger.Equals(s1big, s2big);
            string s1 = s1big.ToString();
            string s2 = s2big.ToString();
        }
    }
}

With a shared secret, we may now implement a symmetrical cipher system to create a secure tunnel for our data i.e. SSH. This could use AES or Blowfish for example. Of course you would use an existing code library for this that has been tested in the wild wouldn't you? You would not be tempted to code fall-back code in case a trusted extension was not available would you? :)

Asymmetrical Encryption

Previously we considered Symmetrical Encryption to create a secure tunnel for data. But we still need to authenticate the user that is connecting to the server before giving them access to sensitive data or administrative capabilities. For this, we tend to use Asymmetrical Encryption where a pair of keys, one private and one public is used. These keys are generated with a software tool and the private key is kept on the client computer, and the public key is sent to the server and stored in an authorized keys file for the user.

When a user wants to logon to the server, the server can send a challenge to the client (encrypted with the public key), and this may only be decrypted with the private key to prove authenticity of the user. Then a session may be established between client and server that is trusted.

Key-based authentication is more secure than password authentication because the private key is complex and there is no need to account for user error in entering the details into an online form. Most people tend to use easy-to-guess passwords, and login systems allow multiple guesses from bots so if you can, it's a lot better to use private key authentication when connecting to a remote server for admin purposes.

SSH Connection to a Server

Let's assume that our remote server has SSH enabled on port 22 which is the usual port. So to connect we go to the server's IP address at port 22 and try to establish a secure connection. For a software tool to do this, I use MobaXterm. This has many features such as a file explorer window, command line console, network scanner (to detect ports), and SSH key generator.

The SSH key generator may be used to generate a public and private key pair. I found that the RSA option worked with my Linux server. The public key was copied to the server and the private key saved to my computer. The private key saved as is worked fine with MobaXterm as the client, but with a C# application I needed to convert the private key to Open SSH format using the conversion utility (see below).

SSH Key Generator

Upload a file to an SSH Server with C#

To connect to an SSH server with C# we will need to download and install the SSH.NET Library. This makes it easy to connect with password authentication or key-based authentication. See below for code to upload a file to the server.


using System;
using System.Collections.Generic;
using System.Text;
using Renci.SshNet;

namespace ssh
{
    class Program
    {
        const string HOST = "192.168.1.10";
        const string USER = "joe";
        const string KEYFILE = @"C:\ssh\pk1.ppk";

        static void Main(string[] args)
        {
            var keyFile = new PrivateKeyFile(KEYFILE);
            var keyFiles = new[] { keyFile };

            using (var sftp = new SftpClient( HOST, USER, keyFiles))
            {
                string srcPath = @"C:\files\";
                string destPath = "/var/www/mysite.com/public/";

                string uploadfn = "file.php";

                sftp.Connect();
                uploadFile(sftp, uploadfn, srcPath, destPath);
                sftp.Disconnect();
            }
        }

        static void uploadFile(SftpClient client, string uploadfn, string srcPath, string destPath)
        {
            client.ChangeDirectory(destPath);
            using (var uplfileStream = System.IO.File.OpenRead(srcPath + uploadfn))
            {
                client.UploadFile(uplfileStream, uploadfn, true);
            }
        }
    }
}

SSL Encryption

SSH is great for connecting to a server for admin purposes. But to interact with web pages securely we need SSL (Secure Sockets Layer). Port 80 is the standard port for HTTP traffic which is in the clear, but encrypted traffic is usually on port 443 (HTTPS). No special code is needed to fetch a secure web page. Simple C# code to do this is shown below.


using System;
using System.IO;
using System.Net;

namespace CheckSites
{
    class Program
    {
        const string URL = "https://website.net";



        static void Main()
        {
            Console.Write(getHtml());
        }

        static string getHtml()
        {
            var result = "";

            WebClient myClient = new WebClient();
            Stream response = myClient.OpenRead(URL);
            using (StreamReader reader = new StreamReader(response))
            {
                result = reader.ReadToEnd();
            }
            response.Close();

            return result;
        }
    }
}

How does SSL work?

A container called a Certificate is used to hold a public key, private key, and a subject. The subject refers to the CA (Certificate Authority Organization) or the Website that is secured.

The CA has been audited and added to a list of trusted CA's. This list is stored in web browsers. The validity of a certificate may then be checked by communicating with the server at the CA. The client checks the validity of the certificate that it receives from the server.

Using Asymmetric encryption the Client and Server use their private keys and the public key (sent by the server) to establish a Session Key which is a secret that both of them know. Now this key is used for Symmetrical Encryption (which is faster) for the remainder of the session.

Now, SSL is the old name for the current certificates which are now called TLS.

The main purpose of the certificate system seems to be for the client to be sure that the server is the actual one it claims to be and not a spoofed website, otherwise the same system as used by SSH could be used. But in this case, an extra level of trust is built in.

So on each connection from client to server, there is a fair bit of handshaking going on as follows:

  • between server and client
  • between client and CA
  • between server and CA

So there you go, the basics of SSL.

Finally

Well I hope that you enjoyed reading this long post! If you have any ideas for more posts related to this topic from a backend coder perspective then please mention it in the comments. It's important to use established code libraries to implement secure/encryption tasks so I think that I may do a post about the various code libraries and resources out there. So stay tuned ...