As we review data transformation and modernization strategies with our clients, we find many are investigating Snowflake as a data warehouse solution due to its ease of use, speed, and increased flexibility over a traditional data warehouse offering.
Snowflake is typically considered an “easy button” decision for most organizations since there is no hardware or software to install, configure or manage, and ongoing maintenance is handled by Snowflake.
However, potential storm clouds can build on the horizon in two key areas:
- Data migration and ongoing transformation of data.
- Analytics and intelligence for business users and user adoption.
In this post, we focus on data migration and ongoing transformation. Stay tuned for a follow-up post on analytics and intelligence for business users and user adoption.
Whether an organization is building a data warehouse for the first time or migrating off an existing platform, a well-thought-out plan is essential to success. Here are some best practices to get you started in the right direction:
Know your data and consider what you should clean.
It is important to have a clear understanding of what data should be migrated and its format. Snowflake supports data that is structured, delimited (CSV, TSV, etc.) or semi-structured, including JSON, Avro, ORC, Parquet, and XML. Establishing rules and scrubbing data will get you started on the right foot and will save time and money keeping the data clean in the future.
Know how to handle your data.
Ensure that regulatory compliance requirements are understood and followed as part of the data migration strategy and continue to be enforced after the migration. Establish roles and responsibilities for your team members so that someone is responsible for compliance, authorization, security and validation. Snowflake offers security features for network and site access, account and user authentication, object and data security, and security validations.
Know the timeline and be realistic.
The need for setting realistic milestones with minimal disruption to the business cannot be overstated. In almost all migrations, data will need to be deduplicated and cleaned up before it is moved. Include extra time in your timeline for a few unexpected “storm delays” so you have some cushion if complications arise.
Consider tools to help with the data migration and ongoing management.
Keep in mind that tools are there to help automate manual processes to make things easier, but cannot be relied on 100%. Selecting the right tools to meet your migration and ongoing needs can be a big time and cost-saver.
Validate and test through the entire migration.
Don’t wait until the end of the migration; establish and document a testing and validation process throughout the entire process.
Mapping a successful data migration can bring on rough weather. Sirius can help. We offer:
- Full data migration and transformation strategy
- Implementation services
- Data discovery and inventory services
- Data modeling, qualifying and cleansing services
- Agile strategy planning and execution
- Vendor-agnostic recommendations for your specific organizational needs
- Validating and testing data throughout the entire migration process
Snowflake’s cloud-built data warehouse enables the data-driven enterprise with instant elasticity, secure data sharing, and per-second pricing across multiple clouds. Snowflake combines the power of data warehousing, the flexibility of big data platforms, and the elasticity of the cloud. Find out more at snowflake.com.