This morning we had another system failure on our loadbalancer server which led to a heliumtracker.io downtime of several hours.
HeliumTracker is already powered by a total of 6 dedicated web servers. To increase redundancy we will add another loadbalancer server today.
While researching the cause, we came across two common triggers:
1: There are some users with several thousands(!!!) of hotspots in their account.
The elaborate calculations in the dashboard are not designed for this scale and thus overload our servers.
For this reason we will have a maximum hotspot limit of 500 per account starting today. If you really need more than 500 devices in one account contact us - we have an enterprise solution in the pipeline.
2nd: The calculations of the city charts is too computationally intensive.
Every morning around 5:00 am UTC our system recalculates the revenue statistics of the cities. This background job now takes several minutes to complete. Together with the above mentioned loads, this job then apparently leads to a crash.
To counter this problem we will stop the calculations of the city charts until further notice.