The team at Autoize was very knowledgeable and professional while helping us deploy an enterprise Nextcloud setup. They were helpful in answering our questions through the process and their turnaround time was quick.
Kent, Health Systems Consultant
The Challenge
A CTO from a healthcare & senior care organization in the Asia-Pacific region was interested in implementing NextCloud to safely store and share digital documents and patient records among their employees.
Expecting a total of 300 users and over 1 TB of stored data once the entire organization is onboarded onto the file sync & share system, our client required a scalable solution to allow the user and storage capacity of NextCloud to grow with their organization’s needs. The solution needed to be reliable, and resilient to the failure of a single server within the cluster.
The majority of the client’s users would interact with NextCloud through the web interface, rather than use the desktop or mobile clients. Therefore, it was critical that the dashboard would support the uploading of large files, and facitiliate the previewing of thumbnails with decent performance. In the future, the client also plans to use NextCloud federation to enable its employees to share data across multiple NextCloud instances using federation addresses.
A web-based office suite integrated with NextCloud for real-time collaborative editing of documents, spreadsheets, and presentation in Microsoft Office formats (.docx, .xlsx, .pptx) was also a requirement.
Like most of the other clients who engage Autoize’s professional services team to implement NextCloud, data protection was the primary consideration which led our healthcare client to go with open-source NextCloud, rather than a commercial cloud storage solution. NextCloud checked all the boxes when it came to privacy, security, and backup options.
Autoize’s Recommendation
The NextCloud cluster we implemented for our client in the Singapore datacenter region, was designed to be horizontally scalable to any number of NextCloud app servers running NGINX and PHP-FPM, in anticipation of a growing number of simultaneous users. The HAProxy (load balancing), MySQL (metadata), and NFS (primary storage) layers were all set up in primary/secondary pairs, facilitating failover and restoration of service within minutes in the event of failure of any single server.
The HAProxy servers leverage Keepalived and the cloud provider API to automatically remap an Elastic/Floating IP to the secondary load balancer, if the HAProxy service on the primary load balancer becomes unreachable at any point. The load balancing layer also handles SSL termination, using the Certbot client to automatically renew the certificate when it’s close to expiry, and share it between the primary and secondary load balancers using an NFS mount. We opted to use HAProxy rather than a managed load balancer from the cloud provider, as it provides full flexibility & control over the HTTP timeouts for uploading large files to NextCloud.
In real-time, over the private network, MySQL replication mirrors any SQL statements executed on the master database server to the slave. There is always a warm replica of the NextCloud database server available, and a failover can be performed by simply stopping replication, promoting the slave to master, and updating DNS.
For primary storage of user data, NextCloud uses an NFSv4 export mounted into the app servers as the NextCloud data directory. The NFS servers have statd and lockd support for file locking, preventing write conflicts between multiple app servers. lsyncd runs as a service on the primary NFS server to facilitate real-time replication of data (over the private network) to the secondary server, which once again serves as a warm replica.
To faciliate no-downtime, consistent off-site backups, a daily backup script is initiated by a backup server located in Amsterdam, an entirely separate datacenter region. First, it connects to the secondary DB server and pauses MySQL replication. Then, it connects to the primary NFS server and pauses lsyncd replication. After, a mysqldump is run on the secondary DB server in single transaction mode, and the SQL export is transferred by rsync to the backup server. The incremental changes on the secondary NFS server since the last backup, are also compressed and transferred by rsync to the backup server. Finally, MySQL and lsyncd replication is resumed, and a summary of the backup job is emailed to the administator.
Collabora Office was selected as the web-based office suite, so the Collabora container was installed on a separate subdomain (and virtual server) running the Docker Engine, with NGINX running as a reverse proxy on the host itself. The Collabora Office app from the NextCloud app store was used to integrate the editing functionality on all the NextCloud app servers.
Additionally, a Redis server was setup to persist PHP sessions across application servers without requiring the use of session stickiness on the load balancer. A separate database on the same Redis server was configured for local and distributed caching to accelerate the performance of the NextCloud web interface.
LUKS encrypted volumes with the dm-crypt kernel driver were implemented on all servers that either stored NextCloud metadata, user data, or both. Operating system level encryption, rather than NextCloud’s application-level encryption at-rest was chosen based on performance considerations. Whenever data travelled outside the private network, it would be securely encrypted in-transit with TLS (NextCloud and Collabora servers) or SSH (backup, database, and NFS servers) protocols.
Result for The Client
Our healthcare client adopted the enterprise NextCloud architecture described above, because it would allow the NextCloud instance to support more simultaneous users in the future by simply cloning the app servers and adding them to the HAProxy backend.
Down the road, the storage can be expanded by simply expanding and resizing the partitions associated with the block storage volumes attached to the NFS and backup servers.
During the pilot stage of the rollout of NextCloud in our client’s organization, the CTO planned to bring 150 users on board storing an aggregate of 500 GB of data, at a cost of $380.00/month ($2.53/user). To scale to 300 users, his department will spin up one to two additional NextCloud app servers ($20/mo per server) and additional storage capacity can be added at a cost of $0.30/GB/mo ($0.10/GB/mo x 3 for NFS primary, NFS secondary, backup volumes).
The employees of the healthcare / senior care organization now enjoy convenience of accessing their data from their NextCloud storage accounts, on any PC or mobile device on-site. The IT department has the peace of mind that patient data is being stored encrypted on servers they control, and that it’s being backed up daily to an off-site location. The maintenance burden of NextCloud is not overbearing, with the creation/revocation of user credentials, installation of additional apps from the NextCloud app store, and minor version updates available through the graphical dashboard. Uptime and availability of the service, which is critical to our client, is enhanced by redundancy throughout the entire infrastructure, allowing no-downtime backups, and addition of new productivity apps.
If you are interested in setting up a similar NextCloud cluster for your organization, please get in touch with an estimated number of users and storage capacity. Our team of NextCloud specialists would be pleased to outline the privacy, security, and cost benefits of migrating to an decentralized cloud storage solution such as NextCloud.
