Databricks row and column level security

    Recently I had a chat with one of client regarding on access control of their reports and dashboards. Interestingly it was found out that client is currently doing this by creating similar reports and granting access to people in different security groups. Obviously this is not the best idea because of redundant reports, the ideal solution is to implement row and column level security on the table so that people in different access groups will have visibility to subsets of the rows in the table or view....

    Created April 30, 2022 · 5 min · Neil Xu

    Azure networking: Hub and spoke topology with terraform

    The hub and spoke topology has been widely adopted for enterprise production deployment. In this lab, let put on our network/infrastructure engineer hat and get our hand dirty on Azure Hub and spoke topology with one of the popular IaC – Terraform. Lets have a look at the high level architecture first. Overall architecture of the lab The essence of the topology is, by the name of it, having all traffic routed to hub before it gets forwarded to spoke....

    Created April 11, 2022 · 15 min · Neil Xu

    Secure Databricks cluster with vNet injection and access resources via Azure private endpoint

    What an interesting topic I had recently regarding on security hardening Databricks using Secure cluster connectivity + vNet injection. This configuration will allow the cluster to access Azure Data Lake Storage (I know right ?! what a popular combination!) and keyvault with private endpoint. In this post, in a lab environment, we will find out how we can put Databricks cluster inside existing Azure virtual network and access private endpoint deployed inside it....

    Created March 28, 2022 · Updated Apr 11, 2022 · 15 min · Neil Xu