From the course: Microsoft Azure Data Engineer Associate (DP-203) Cert Prep by Microsoft Press

Unlock this course with a free trial

Join today to access over 24,900 courses taught by industry experts.

Load a DataFrame with sensitive information

Load a DataFrame with sensitive information - Azure Tutorial

From the course: Microsoft Azure Data Engineer Associate (DP-203) Cert Prep by Microsoft Press

Start my 1-month free trial Buy for my team

Load a DataFrame with sensitive information

“

- [Instructor] And the theme here is you're working in a Databricks notebook or an Azure Synapse Spark Pool Jupiter notebook. How do you handle data like personally identifiable info? How do you handle passwords and credentials and pats using encryption and decryption? Well, Databricks has the Fernet encryption library as well as user-defined functions that you can invoke. We don't have to get deeper than that. That's just knowing that it's possible in the Databricks platform, they have libraries and they have built-in functions to help you in terms of obfuscating data, not just credential data but also personally identifiable info, somewhat analogous to dynamic data masking over on the SQL side.

Contents