Autoize streamlines cloud infrastructure for leading real estate search engine, modernizes legacy servers to CentOS 7, and accelerates MySQL performance.
Challenge
Home.ca is one of the leading real estate search engines in Canada, attracting 7 million monthly page views from over 62,000 unique visitors. The company uses Mautic in conjunction with a custom Laravel application that powers the website’s CMS, and a proprietary CRM that facilitates the follow-up of buyer leads from licensed real estate agents via SMS and email.
When Home.ca approached Autoize’s cloud consultants, both the application and database servers were hosted on a cluster of CentOS 6 servers with RackSpace Cloud. The outdated infrastructure had been lifted-and-shifted from their previous web hosting provider, and the database server seemed to be frequently struggling to keep up with the load. More often than not, viewing contacts in Mautic or running a report would time-out, resulting in 503 Service Unavailable errors for Home.ca’s sales & marketing teams – leading to frustration with the IT and degraded productivity.
Furthermore, the organization was concerned with the monthly infrastructure cost on RackSpace Cloud, which had become more burdensome as the database server needed to be scaled up to meet growing demand from the various applications. While Home.ca’s internal teams occasionally reached out to RackSpace Support for help, cutting the cloud bill by moving to a commodity cloud provider – and opting for pay-as-you-go consulting & support services from a provider such as Autoize seemed to make sense.
The client was also looking for a team of knowledgeable cloud consultants to conduct a complete infrastructure review, to identify potential cost savings on block/object storage, load balancing, and bandwidth – in addition to compute savings.
Last but certainly not least, since the Home.ca website has a mechanism for user authentication and stores those logins along with users’ personal information such as names, phone numbers, and email addresses, maintaining the security of the infrastructure during and following the migration was also a key priority.
Technical Overview
As soon as Autoize’s cloud consultants were granted access to Home.ca’s infrastructure, they immediately got to work – evaluating the LAMP services and dependencies which were running on each server in the cluster.
The website and search engine are powered by a Laravel application, as is the CRM which is used to track and assign leads to sales agents. Mautic, which handles all of the lead forms on the Home.ca website, is a Symfony application. Home.ca also uses the Mautic API and webhooks to push captured leads (with points) into their CRM, and pull the contact owner details back into Mautic to send automated follow-up emails.
All of the applications rely on a MySQL database service in the backend to store the MLS listings, sales agent data, and Mautic contacts respectively. The Home.ca website and CRM are also configured to use Redis as an in-memory cache. Also, the CRM depends on the RackSpace Cloud Queue messaging service to process user interactions in the dashboard, such as re-assigning a lead to a different user.
Through scheduled cron jobs, the Home.ca search engine communicates with the TREB API using RETS (Real Estate Transaction Standard) queries multiple times a day to sync the latest listings and images onto the website.
Recommendation & Action Plan
The Autoize team crunched the numbers and presented Home.ca with an analysis that migrating the infrastructure to Linode would result in thousands of dollars in cost savings annually. In fact, it was estimated that over the next 5 years, Home.ca would save US$35,028 at an 8% discount rate, with an internal rate of return of 220% against the migration costs. This was under the assumption that we would maintain the same level of hardware resources as provisioned on RackSpace Cloud.
Over the next 5 years, Home.ca would save US$35,028 at an 8% discount rate, with an internal rate of return of 220% against the migration costs.
Autoize identified that the infrastructure on RackSpace Cloud was unnecessarily using RackSpace load balancers to terminate SSL individually for the publicly accessible web services, as Home.ca was already being served behind the CloudFlare CDN. Under the new infrastructure with Linode, Autoize would eliminate the load balancers and instead, use a CloudFlare origin certificate to terminate SSL directly on the web server in “Full (Strict)” mode, guaranteeing end-to-end encryption over HTTPS.
Also, the database server on RackSpace was not tuned for optimal performance, especially for the InnoDB tables which Mautic uses in its information schema. By tuning the InnoDB cache and page sizes along with other my.cnf variables in accordance with best practices, Autoize was confident it could eliminate the performance issues that Home.ca was persistently experiencing with Mautic. Instead of using only 25% or less of the database server’s physical memory with the default settings, MySQL would store more queries in its cache to take advantage of 65-75% of the available memory – greatly accelerating performance while maintaining stability.
Autoize recommended eliminating the VestaCP control panel which was present on the legacy infrastructure, and upgrading the systems from CentOS 6 to CentOS 7 to ensure long term support through to 2024. On the staging environment, we ran PageSpeed tests which determined that VestaCP’s method of serving static assets using nginx, and PHP pages using Apache and PHP-FPM was actually detrimental to the website performance. The new infrastructure would eliminate the resource overhead and potential security vulnerabilities of the VestaCP control panel.
Instead, the Autoize team tuned Apache for optimal performance based on the anticipated peak traffic load of the Home.ca website, and load tested the configuration using Loader.io. Each PHP module required for the correct functioning of the Laravel applications, and Mautic, was carefully recorded and mapped to the Remi repository for CentOS 7. The migration team hunted down every environment variable in the Laravel and Symfony configuration files to ensure MySQL and Redis were mapped to the new hostnames, and the RackSpace Cloud Queue was now being accessed by the external instead of ServiceNet endpoint.
To ensure the security of the server cluster, Autoize configured a series of internal firewall zones that would allow the servers to communicate with each other over 3306 (MySQL), and 6379 (Redis) but not outside hosts. Furthermore, all of the servers were upgraded from password-based to key-based authentication to deter brute-force attacks.
Even while performing the migration, Autoize remained conscious of the necessity to protect Home.ca’s proprietary code and databases from prying eyes. The replication of the MySQL data from RackSpace Cloud to Linode was strictly done over an SSL connection only with client certificates (the replication user was set to REQUIRE X509). As Redis does not support encrypted connections out-of-the-box, the migration team set up a peer-to-peer VPN between the old and new Redis servers to facilitate secure replication. All other files, such as application code, were transferred using secure utilities such as scp or rsync over SSH.
Result
Home.ca went with Autoize for the cloud migration as we provided an end-to-end service, from planning the move to implementing and testing it. The Autoize team took care of everything from identifying ways the infrastructure could be streamlined, making an action plan to minimize service disruption & downtime, setting up and testing a staging environment on the new provider, then migrating over the live data from the production servers.
To minimize downtime and ensure the new infrastructure did not affect application functionality, Autoize completely cloned Home.ca’s application stack to a series of staging subdomains for the website, CRM, and Mautic, and validated it together with Home.ca’s technical team. After the testing was completed to satisfaction, the maintenance jobs were paused on the production cluster (to ensure data consistency), and replication of the live databases from RackSpace Cloud to Linode was re-initiated. Once the replication slaves (Linode) had caught up with the master (RackSpace), the master was shutdown, slaves promoted to master, and CloudFlare DNS records updated to cutover the production services to the new infrastructure.
Home.ca now enjoys significant, recurring savings on their overall infrastructure cost encompassing compute, network, and storage services, by switching to Linode. Month-to-month, the organization is paying 75% less than it was previously for hosting, freeing up budget to be redeployed towards growth initiatives, such as engineering and marketing. In just under 6 months, Home.ca will recover the entire cost of the migration in infrastructure savings.
Here’s what Edward Cirstea, Home.ca Director, had to say about his experience working with Autoize on this cloud migration initiative:
The team at Autoize went beyond our expectations. They planned meticulously, clearly communicated and executed flawlessly every stage of moving Home.ca's infrastructure.Autoize carefully analyzed our existing bottlenecks, ensuring no detail was overlooked and made suggestions on increasing server capacity and optimization, meanwhile reducing monthly costs. Not an easy feat to accomplish.
I am extremely impressed with the talented team at Autoize and highly recommend you look at their services for managing server infrastructure and open source projects.
