11th January 2020 In Web Development

How to transport and access files securely

The Scenario

The scenario is you have a file situated on another server and you need to give certain users access to that file. This file contains sensitive, personal information which you’d rather not have residing in attachments or languishing in a documents folder on someone’s machine.

These users might be required to approve an expense claim for example. The easiest thing to do would be to embed a link to that file and allow the user access to it. That’s fine provided they don’t need to be on a secure network or VPN to access it. This scenario would require them to access the file from what ever device, regardless if they are on or off the private network.

Another option would be to transfer the file to another server off network and provide a link to that file. That’s ok, but we don’t want to encourage people to download the file. For GDPR compliance you must be in a position to remove a customer’s data on request or otherwise and be assured that there is no way that data can be brought back (Within reason.). Therefore it makes sense to store that data in one place and one place only. In our scenario we will actually remove the data after a certain period because this data does not need to persist beyond being viewed once only by a designated approver.

The Solution

This solution is by no means the best option but does provide a pragmatic option for everyday use.

Step 1

Provide a link within the secure network to the file.

Step 2

Run a scheduled task or cron job to pick up the details and do the following:

  • Create an API in the DMZ of your network
  • Post to the API encoded contents of the file you wish to transfer to a secure location that cannot be accessed via a direct http request
  • At this stage it might be a good idea to include a hashed version of the filename, such as SHA256, MD5 for example.
  • Remove the original file from the secure loction on the network – remember we only want this file stored in one place
  • Send an email and/or SMS to the designated users that there is a transaction that needs to be approved
  • Include in the email the hashed file name with a GET request to your API on the DMZ

Step 3

Here we need to approve the file – this can be done in the following way:

  • Create a hook to listen for an http request to view the file. This is not a direct link to the file, that comes later.
  • The request is received and at this stage you may wish to verify that this user is able to access this request. This is a REST API – so an exchange of keys might be helpful here to ensure that the request is coming from a valid source and the user is a legitimate one.
  • Once you are satisfied that the request is legitimate we can then construct a method to access the file which is outside the public http space. We can then match our file with the hashed filename we prepared in Step 2. If the hashed names match – load the file.

Step 4

This is the clean up phase. In our scenario, to be GDPR compliant, we need to be in a position to remove data securely from ideally as fewer locations as possible, so it cannot be found and viewed easily.

The most pragmatic option in this instance is to remove the files after a time limited period. You could, for example, notify that the designated users only have 24 hours to view the file before it is removed. Then run a scheduled task to remove any files older than 24 hours. By adopting this approach you will always be sure that the files will be removed after a resonable amount of time from your DMZ.

Conclusion

This way of transferring files between servers does offer a convenient way of moving files around securely by for example posting the contents via a REST POST request, thus it cannot be seen in the URL. It also provides a secure way of obfuscating the filename in a link or GET reequest by hashing the name. This prevents an attacker being able to identify the type of file that might be being stored in the DMZ via a man-in-the-middle attack.

The solution also provides a secure option to store files and access them away from the public HTTP space by storing the files outside the web folder. By adding extra security, such as turning off port and directory scanning on the server, you again make it harder for an attacker to gain access to the actual files.

From a GDPR compliance perspective, the solution described goes someway to addressing the need to provide a way of keeping files stored in as fewer locations as possible. This is achieved by removing the file from the secure network and providing a time limited option to view the file on the API before it is removed automatically.

However, this is not a perfect solution because, depending on the browser used there is nothing to stop the user from right clicking, for example, and downloading the file. Though this solution does make it less obvious to the average user that this can be done. In mitigation, this is where user awareness of GDPR and basic information security will need to come in to play.

In essence, this solution offers an attempt to provide a pragmatic solution for most business needs.