GUIDE
Snowpark MigrationAccelerator
Move your Spark workloads to Snowpark and take advantage of all that the Data Cloud has to offer.
Code Understanding
The Snowpark Migration Accelerator builds a complete semantic model of your codebase. It does not look for keywords; it builds a functional model.
Step-By-Step Guidance
This tool starts by generating an assessment and when you’re ready, moves you on to conversion – helping you understand what you have and guides you to the next step in the process.
Constantly Evolving
The SMA has been used on millions of lines of code across multiple pipelines and use cases. The tool is constantly updated to improve as each workload is analyzed, increasing the effectiveness of the output.
USING THE SNOWPARK MIGRATION ACCELERATOR
The Snowpark Migration Accelerator has been developed by Snowflake to help you understand the Spark code you have, and get you up and running in Snowpark faster and more efficiently. There are two components to this tool: an assessment and code conversion.
There is an initial assessment based on code files, libraries, and spark references present in any codebase written in Python or Scala. The output allows you to understand what you have and what can be converted to Snowpark. From that assessment, the accelerator can automatically convert some references from the Spark API to the Snowpark API.
HOW DOES IT WORK?
The Snowpark Migration Accelerator is a free tool that you can download and run on a local machine or in a container. This tool takes in code files or notebooks as an input. (It does not connect to any source platform; it only scans files.) The files will be scanned and an assessment report will be created locally that indicates the “readiness” level of your Spark workload for Snowpark.
Once the assessment is created, the Snowpark Migration Accelerator can convert some references from the Spark API to the Snowpark API based on the functional equivalence between the source and the target. What the tool cannot convert will be inventoried and tagged with comments in the output code to indicate the next steps.
For more information on installing and using this accelerator, visit the getting started section of the Snowpark Migration Accelerator documentation.
FAQ
How does it work?
The Snowpark Migration Accelerator creates a model of the source codebase’s functionality by building an abstract syntax tree (AST) and a symbol table. The transformations and information provided are based on this semantic model. It is not a regex find-and-replace tool.
Who can use it?
Anyone! This tool can be downloaded and used by anyone to generate an assessment and convert pieces of the source codebase.
Does this tool connect to my source platform?
No. This is a local application that takes in a directory of files as input. There is no connection to any source platform.
Does this tool connect to my Snowflake account?
No. This is a local application that outputs various reports and code files to a local output directory that you specify in the application.
What file types does the Snowpark Migration Accelerator support?
The accelerator can take any file type as an input, but only code files and notebooks with certain extensions will be analyzed for references to the Spark API. If you choose Python as the source language, files with the following extensions will be processed for references to the Spark API: .py, .python, .ipynb, and/or .dbc. If you choose Scala as the source language, files with the following extensions will be processed for references to the Spark API: .scala, .ipynb, and/or .dbc.
