In one of our previous blogs, we discussed the flexibility and agility of the modern Snowflake data cloud and the importance of observability, automation, and responsible resource management. We addressed the Data Cloud from the perspective of spend and performance control, spend metrics observability via Spendview, and automated recommendations from Snoptimizer by XponentL data. It is very important to set up FinOps capabilities and a recurring practice to take control of your resources and spend.
Today, we want to take a different angle on that problem. In the spirit of today’s cost optimization, we want to look at how you could benefit from other very important capabilities and use them to decrease spending. We will talk about how using one of the best data security and access control (ABAC) platforms—SecuPi—can contribute to optimizing your resource spend through observability and a data access control/security layer!
GDPR, cloud data breaches, and sovereignty laws drastically changed the industry and converting data security from a ‘nice to have’ feature into a mandatory function. Until today, security programs have mostly been perceived as a resource burden.
But what if we told you that you can save money with your data security platform? SecuPi keeps the data of their numerous customers secure and protected by applying data protection policies to customer requests, monitoring, understanding the use of data, and making it secure. What if we could use that access control layer to control costs too?
As SecuPi has already addresses Attribute-Based Access Control (ABAC), can we use “cost” as another attribute to consider when restricting or blocking access? Short answer: yes!
SecuPi and DataOps.live have joined forces to deliver the first Snowflake cost management solution that continuously detects and presents your cost breakdown and takes action by restricting and blocking wild queries from burning your budget. With Spendview and SecuPi transparent gateway, you can quickly get an overview of all accounts and warehouses and recognize improvement opportunities, controlling and constraining your costs while creating the policies to enforce them at-scale.
In this scenario, Spendview acts as a high-level analysis tool to identify the most resource-heavy accounts, improvement opportunities, and challenges. With an understanding of FinOps processes, it helps Privacy Steward create policies to protect costs spiraling out of control. SecuPi acts as a gateway to prevent resource-heavy and anomalous tasks from being performed, alerting the user directly when they try to execute a resource-heavy task!
The DataOps.live + SecuPi ABAC solution gives you superhero powers to analyze why costs are increasing unexpectedly and stop them before they spiral out of control.
For example, with Spendview, you can analyze how your Snowflake accounts are being used and see some common patterns. Perhaps you see your account usage spike during the working week and can set a policy in SecuPi to restrict access to that account on weekdays to prevent situations where analysts might forget and run heavy queries or keep warehouses up in dev environments over the weekend. That can both improve the user experience by preventing warehouse spilling and decrease the cost of running rogue queries.
With Spendview, you can detect a typical usage pattern for a Snowflake analyst and define a policy in SecuPi to stop the analyst request automatically. You can also set it to notify the analyst if it significantly exceeds the previous consumption pattern.
Maybe you observe a high number of queued queries in the warehouse. This may be the result of recurring queries that are executed by impatient users before any of the queries return. In this case, SecuPi will stop the recurring queries while saving on their execution costs.
With control like that, you can let innovation flow in your organization without being concerned that user error will result in significant cost.
Spendview and SecuPi transparent gateway present a Snowflake infrastructure big picture and give control to business leaders within the data team to make data-driven decisions and enforce them immediately about their Snowflake cost and usage patterns. We highly encourage you to reach out to SecuPi or give Spendview FREE a try!
Of course, if you want to do more, the next significant performance and efficiency increase would be to consider building Secure Data Products for Snowflake with SecuPi and DataOps.live. More to come soon; keep an eye on our blog or subscribe here to get notified when new blogs are posted!