Community blog | DataOps.live

Operationalizing Snowflake: Cortex Fine-tuning

Written by Guy Adams - CTO, DataOps.live | Sep 6, 2024 7:59:36 PM

The first in a how-to video series on operationalizing popular and newly released Snowflake features

Each year at Snowflake Summit we’re treated to an assortment of new features. The demos never fail to excite and make clear the advanced functionality they can bring to the enterprise in a simple way. But demos are typically built by one person and they only need to work ‘once’ (maybe more than once, but only in one way).

It’s only when you operationalize the demoed feature that it’ll work 24x7x365 and deliver real business value. 

When you look at the development process behind each, it’s easy to see just how much demos don’t have to worry about.

 

Feature demo

Operationalized

Run by

Vendor (e.g., Snowflake)

The customer

Purpose

Highlight capabilities of the feature itself in specific example

Deliver real business value

Robustness

Works “once”

Works 24x7

Scope

The “happy path”

Considering the edge case

Development

Usually one person, ad-hoc process

Worked on by one or more teams, formal process

CI/CD

Not needed

Critical

Security Model

Not needed

Critical

Scalability

Not needed

Critical

Governance / Auditability

Not needed

Critical

End result

Excitement, possible applications

Business value

 

Probably the biggest difference between the demo and your environment is that no one is making business decisions using a demo. That means the quality — even correctness — of the code isn't actually that important, as long as it works for my demo.

When you operationalize the feature for your environment, it has to run today and be ready to accommodate where your business might be in two years, which could be 10 times the volume. People are going to make real business decisions against it, so governance, change control, and auditability are absolutely critical. 

Below is the first video in our series on Operationalizing Snowflake. In it, I walk you through how to operationalize Cortex fine-tuning. Enjoy! 

PS: If you want to get notified when new videos in this series are released for features like Dynamic Tables, Cortex Search, Serverless Tasks, Streamlit in SPCS, Document AI, and MANY others, sign up here.