Archives: July 27, 2021

V6 UI in development…

Now that all the working parts of Syncplify Server! v6 are in place, and we are very confident in their reliability and fitness for purpose, it’s time to develop all new (and much more user-friendly) UIs for all three of the major components: SuperAdmin UI, Admin UI, and WebClient!.

On top of that, the new initial setup procedure will also be moved to a web UI, for consistency and effectiveness reasons. Here’s a very early sneak peek:

First “seriously serious” stress-test for V6

As the time to release Syncplify Server! V6 approaches rapidly, we now have the pleasure to begin performing real serious stress-tests on our SFTP engine.

The idea is to “throw everything we’ve got at it”, and try our hardest to make it crash… or at least leak some memory, or misbehave in any way. The underlying concept is: if we can find a way to hack/damage/crash our server, someone else may be able to do the same; so we don’t want that.

Well, enough chit-chat, here’s the raw data, followed by a plain English explanation:

    "sessSshSftp":       689841,
    "fileUp":            16552,
    "fileDn":            16433,
    "fileUpFail":        0,
    "fileDnFail":        0,
    "transferUpMb":      "3.156491 TB",
    "transferDnMb":      "3.152610 TB",
    "scriptsRun":        33127,
    "crashOrPanic":      0,
    "successfulAttacks": 0

The test ran for 2 hours, with 56 concurrent attackers designed to introduce the highest possible level of fuzziness in their behavior. This is to ensure that neither the server nor our development team knows ahead of time exactly how the attackers will perform their attacks.

Attacks may include any mix of the following:

  • protocol violations
  • abrupt connection interruptions
  • flooding of concurrent rapid-fire connection/disconnection cycles
  • authentication tampering or bypass
  • permission escalations
  • directory traversals
  • command injections
  • random data packets with wrong size, MAC, or some other wrong syntax/structure/payload

In total, 689,841 client sessions were gracefully handled (~96 sessions per second). The sustained file transfer speed over the course of the 2-hour test was on average 876.26 MB/sec.

Nevertheless, no attack was successful. And, at the end of the test, Syncplify Server! V6’s worker process was still using only ~44.6 MB of RAM.

We feel pretty good about this.

V6 VFSs include: Disk, S3, Azure, Google Cloud, SFTP (all with optional at-rest encryption) Server! versions 1 through 5 have always historically supported only 2 typed of Virtual File Systems (VFS):

  1. Disk (with optional at-rest encryption via DiskAES256)
  2. SFTP (encrypted over the network but not at-rest on the remote site)

When Syncplify Server! v6 is released, though, depending on the edition you use it will include the following Virtual File System (VFS) types:

  1. Disk (local drives and SSDs, and remote NAS, SAN, iSCSI, …)
  2. Amazon S3 (store data in your AWS S3 bucket)
  3. Microsoft Azure Blob Storage (store data in Azure Cloud)
  4. Google Cloud Storage (store data in Google Cloud Platform)
  5. SFTP (store data on another SFTP server, useful for SFTP-proxying)

And all of them will also feature encryption at-rest on whatever local or remote storage they point to.

WebClient! v2: downloading multiple files at once as a “zip” archive

Syncplify Server! v6 will include (depending on the software edition you run) WebClient! version 2. Aside from a brand new set of much more streamlined APIs and a much tighter integration between the two pieces of software, WebClient! v2 will also feature the ability to let users download multiple files and folders in one single “zip” archive, pretty much like some of the cloud storage apps out there do.

For safety and security reasons, though, administrators will be able to set their own limits to that, of course.

Let’s say, for example, that a legitimate yet reckless user requests the download of the zipped contents of a folder containing 100,000 files, each file having an average size of 100 MB. We’re talking about 10TB of data to be compressed. We’re talking about a potential VM-killer.

So, with WebClient! v2, administrators will be able to limit the processing of the zip archive to a predefined maximum number of files and/or a predefined maximum size before compression, whichever comes first.

Here’s a few possible configuration examples:

  • zip downloads may not exceed 1,000 archived files
  • zip downloads may not exceed 200 MB of data (before compression)
  • zip downloads may not exceed 1,000 files or 200 MB of data, whichever limit is hit first

If a limit is hit, the client will still receive a perfectly functional zip archive, but a 206 response code (partial content) instead of the typical 200 response code (OK), to signify that the request was only partially fulfilled, and the limit that was hit will be clearly identified by a set of X- headers in the HTTP response.

aftJS language to be renamed to SyncJS

As you all probably already know, one of the main goals for Syncplify Server! v6 and AFT! v4 is uniformity. And one of the major steps taken to achieve such uniformity is the adoption of the same scripting language in both software products.

Since aftJS (the JavaScript flavor found in AFT!) was way more modern and powerful than any scripting support ever incorporated into Server!, adopting it seemed to be the only sensible choice.

But now we have a problem: the name, aftJS, is so tightly coupled with a specific product (AFT!) that it would spawn confusion if adopted in the context of our flagship product, Syncplify Server!

Renaming the language, therefore, appears to be not only a more uniform choice, but it is also our chance to tie our scripting language to who we are as a company.

So… Syncplify’s JavaScript… SyncJS.

Adding database support to aftJS (in both Server! and AFT!)

In the next version of the aftJS language (that will be included with Server! v6 and AFT! v4) you’ll find, for the first time, client objects to interact with several SQL and NoSQL databases.

At the time this article is being written, this is still work in progress. But for those of you who can’t wait, here’s an actual script that we tested successfully just moments ago:

  var cli = new SqlCli().Driver("sqlite").ConnString(":memory:");
  cli.Exec('CREATE TABLE places (place text NULL,code integer)');
  cli.Exec('INSERT INTO places (place, code) VALUES (?,?)', ['universe', 42]);
  cli.Exec('INSERT INTO places (place, code) VALUES (?,?)', ['milky way', 23]);
  var res = cli.Query('SELECT * FROM places');

And here’s the log line that proves it works:

2021-07-06 13:11:32.290 INF > nodeId="lucid-booth" sender="scripting" 
sessionId="NdhiyPt9NajTewVuegFTi8" protocol="SSH2" event="OnAuthSuccess" 
scriptId="dbtest" scriptName="My DB script" method="Log" 
[{"place":"universe","code":42},{"place":"milky way","code":23}]

Blocklist mem-cache just got even better!

Just a few days ago we shared exciting news about the TTL-aware memory cache we implemented in front of our Blocklist, which was able to reduce by up to 90% the need to query the DB to check whether or not a client IP is already blocklisted.

Well, we worked hard over the weekend, and we tuned the mem-cache algorithm to achieve even better performance. Let the numbers speak for themselves:

  "rejectedConns" : 883,
  "addedToBl" : 4,
  "hitsOnBl" : 857,
  "hitsOnBlCache" : 855

That’s a 99.76% effectiveness rate. On to the next task now…