Estimated read time: 4.5 Minutes
Next generation smart metering devices (SMETS2) will generate billions of data events about customers energy usage and behaviour. , , . Efficiency of big data platforms will be key to retaining existing customers and attracting new ones.
Image source: https://www.smartme.co.uk/technical.html
Signs your teams don’t have it under control
How often do you hear reassuring messages like these from your teams?
“We’ll worry about performance later.”
“Cloud costs are not a priority at the moment, we need to get it built.”
“We have cloud vendor X helping us.”
“It’s in the cloud so we don’t have to worry about performance.”
If you hear any of these, alarm bells should be ringing. It’s not your team’s fault, these myths are perpetuated by over enthusiastic vendors. Cloud provides infinite scale but doesn’t guarantee performance or cost.
Both are critical to delivering value to your customers. Delays in communications will reduce relevancy, increase customer churn and result in losing the market.
To avoid this, your teams need to make the right choices now and avoid the common pitfalls.
Why poor performance will slow you down
Engineers will go for the quickest cloud solution due to time pressure. But what appears to be the quickest option short term is often more time-consuming in the long term.
The easiest solution when you have a performance problem is to buy more expensive hardware. Sounds like a cost problem, right? Not quite. You can hide performance problems with faster cloud technology only for so long. Long term you will find your teams spending more time maintaining performance.
Fixing problems later is more difficult. Original team members would have gone to the next shiny big data project. You will be left to keep the show on the road.
What can you do? Use simple models to validate solutions during design to ensure the appropriate solution is chosen. These need to be supported with small experiments and the right data. They avoid the need for quick fixes and reduce maintenance effort longer term.
Why cloud costs will eat into your margins
The quick solutions and their fixes also cause high cloud costs. You might think that FinOps will address this problem. However, FinOps is reactive  and doesn't address the underlying issues introduced during design that cause high costs. You will consume valuable time and engineering resources trying to keep costs down.
What can you do? Incorporate cost in the simple design models mentioned earlier to prevent this.
The nightmare scenario, poor performance and high costs
In this nightmarish scenario, your engineering teams will spend all their time trying to keep performance and cost under control. This focus on operational cost and performance will sap your engineering teams' morale. Their time is better spent building great products and innovating.
How do you avoid getting into this situation?
Surely you don’t have to think about this now? The roll out already begun, 3M+ users are already connected to next generation smart meters.
CTOs' who have a joined-up approach on performance and cost have a head start.
Organisations using third parties need a framework for independent governance of cloud performance and cost. Do this before signing up contracts. Feel free to use our Capacitas’ NFR template as a starting point to clarify your performance and cost requirements.
Organisations with in-house engineering teams need to build internal capability and methodologies. It takes time to build expertise on how to build cost effective and performant big data systems. You can start by watching the Capacitas’ webinar on how to do advanced optimisations.
 Cloud FinOps. Storment and Fuller.