Mutable.ai logoAuto Wiki by Mutable.ai
Create your own wiki
AI-generated instantly
Updates automatically
Solo and team plans
Create your own wiki
AI-generated instantly
Updates automatically
Solo and team plans

terraform-provider-databricks

Auto-generated from databricks/terraform-provider-databricks by Mutable.ai Auto WikiRevise

terraform-provider-databricks
GitHub Repository
Developerdatabricks
Written inGo
Stars401
Watchers33
Created02/13/2020
Last updated03/27/2024
LicenseOther
Homepageregistry.terraform.io/providers/databricks/databricks/latest
Repositorydatabricks/terraform-provider-databricks
Auto Wiki
Revision0
Software Version0.0.8Basic
Generated fromCommit e78079
Generated at04/04/2024
• • •
Architecture Diagram for terraform-provider-databricks
Architecture Diagram for terraform-provider-databricks

The Databricks Terraform Provider is a powerful tool that allows engineers to manage Databricks resources, such as clusters, notebooks, jobs, SQL objects, Unity Catalog, and more, using Terraform. This provider simplifies the deployment and management of Databricks infrastructure by enabling infrastructure as code (IaC) workflows.

The most important functionality of this provider is centered around the management of Databricks Catalog, Clusters, Jobs, MLflow, Managed Workspace Service (MWS), Cluster Policies, Repositories, SCIM, Secrets, SQL, Storage, and Workspace. Each of these areas is implemented in a separate directory within the terraform-provider-databricks repository.

The catalog directory provides comprehensive functionality for managing various Databricks Catalog entities, including the ability to create, read, update, and delete catalogs, schemas, tables, views, shares, connections, and permissions. The UnityCatalogPermissionsAPI struct and the corresponding data sources and resources handle the complex task of managing Databricks Catalog permissions.

The clusters directory focuses on the management of Databricks clusters, including creating, resizing, editing, starting, and terminating clusters. It also provides data sources for retrieving information about node types, Spark versions, and available cluster zones, as well as a resource for managing the installation and uninstallation of libraries on clusters.

The jobs directory handles the management of Databricks jobs, including creating, reading, updating, and deleting jobs, as well as managing the lifecycle of job runs. The JobsAPI struct and the JobSettings struct encapsulate the necessary functionality for interacting with the Databricks Jobs API.

The mlflow directory provides resources and data sources for managing MLflow experiments, models, and webhooks within a Databricks workspace. This allows users to integrate their machine learning workflows with the Databricks platform.

The mws directory focuses on the management of the Databricks Managed Workspace Service (MWS), including the creation and management of credentials, storage configurations, networks, permission assignments, private access settings, VPC endpoints, and workspaces. The WorkspacesAPI struct is the main entry point for interacting with the MWS API.

The policies directory handles the management of Databricks cluster policies, including the ability to create, read, update, and delete policies, as well as the differentiation between built-in and custom policy families.

The repos directory provides resources for managing Databricks Repositories, including Git credentials and the creation, reading, updating, and deletion of repositories.

The scim directory implements the management of Databricks SCIM (System for Cross-domain Identity Management) entities, such as users, groups, and service principals, including the assignment of entitlements.

The secrets directory handles the management of Databricks secrets and secret scopes, allowing users to create, read, update, and delete secrets, as well as manage access control lists (ACLs) for secret scopes.

The sql directory focuses on the management of Databricks SQL-related resources, including warehouses, alerts, dashboards, endpoints, global configurations, queries, visualizations, and widgets.

The storage directory provides functionality for managing storage mounts for various cloud storage providers, as well as the ability to create, read, and delete files within the Databricks Distributed File System (DBFS).

Finally, the workspace directory handles the management of Databricks workspace resources, such as directories, notebooks, and global init scripts.

The Databricks Terraform Provider relies on the Databricks SDK for Go to interact with the Databricks REST API, and it utilizes the common.Resource struct and related utility functions to provide a consistent interface for managing the various Databricks resources. The code is well-structured, with clear separation of concerns and comprehensive test suites to ensure the reliability and correctness of the provider's functionality.

Databricks Catalog Management
Revise

References: catalog

The catalog directory contains the implementation of the Databricks Catalog functionality for the Terraform provider. This includes the ability to manage various Databricks Catalog entities, such as catalogs, schemas, tables, views, shares, connections, and permissions.

Read more

Catalog Permissions Management
Revise

• • •
Architecture Diagram for Catalog Permissions Management
Architecture Diagram for Catalog Permissions Management

The …/permissions directory contains the implementation of the permissions management functionality for the Databricks Catalog in the Terraform provider. This includes the ability to retrieve, update, and wait for updates to permissions for various Databricks Catalog entities.

Read more

Catalog Data Sources
Revise

References: catalog

The catalog directory contains the implementation of the data sources for retrieving information about various Databricks Catalog entities, such as catalogs, schemas, tables, views, shares, connections, and storage credentials.

Read more

Catalog Resources
Revise

References: catalog

The catalog directory contains the implementation of the resources for managing various Databricks Catalog entities, such as catalogs, connections, external locations, grants, and Lakehouse Monitors.

Read more

Databricks Cluster Management
Revise

References: clusters

• • •
Architecture Diagram for Databricks Cluster Management
Architecture Diagram for Databricks Cluster Management

The clusters directory contains the implementation of the Databricks cluster management functionality in the Terraform provider. This includes the ability to create, resize, edit, start, terminate, and list Databricks clusters, as well as manage cluster policies and instance pools.

Read more

Cluster Management
Revise

• • •
Architecture Diagram for Cluster Management
Architecture Diagram for Cluster Management

The clusters_api.go file in the clusters directory provides the core functionality for managing Databricks clusters. This includes the ability to create, resize, edit, start, terminate, and list clusters.

Read more

Cluster Data Sources
Revise

The terraform-provider-databricks/clusters package provides several data sources for retrieving information about Databricks clusters, node types, Spark versions, and available cluster zones.

Read more

Library Management
Revise

• • •
Architecture Diagram for Library Management
Architecture Diagram for Library Management

The resource_library.go file in the clusters package defines a Terraform resource for managing Databricks libraries. The main functionality of this file includes:

Read more

Databricks Job Management
Revise

References: jobs

• • •
Architecture Diagram for Databricks Job Management
Architecture Diagram for Databricks Job Management

The jobs directory contains the implementation of the Databricks jobs-related functionality in the Terraform provider. This includes the following key components:

Read more

Job Data Sources
Revise

• • •
Architecture Diagram for Job Data Sources
Architecture Diagram for Job Data Sources

The data_job.go file in the jobs directory defines a Terraform data source for retrieving information about a specific Databricks job. The DataSourceJob() function is responsible for creating this data source.

Read more

Job Resource Management
Revise

The terraform-provider-databricks/jobs/resource_job.go file defines the Terraform resource for managing Databricks jobs. It provides functionality for creating, reading, updating, and deleting jobs, as well as managing the lifecycle of job runs.

Read more

Job Settings and Utility Functions
Revise

• • •
Architecture Diagram for Job Settings and Utility Functions
Architecture Diagram for Job Settings and Utility Functions

The JobSettings struct and the JobsAPI interface provide the underlying functionality for managing Databricks jobs, including settings for tasks, clusters, parameters, triggers, and notifications.

Read more

Databricks MLflow Management
Revise

References: mlflow

• • •
Architecture Diagram for Databricks MLflow Management
Architecture Diagram for Databricks MLflow Management

The mlflow directory contains the implementation of various Terraform resources and data sources related to MLflow in the Databricks platform. The key components in this directory are:

Read more

MLflow Experiments
Revise

The ResourceMlflowExperiment() function in the …/resource_mlflow_experiment.go file defines the Terraform resource for managing MLflow experiments in a Databricks workspace. This resource allows you to create, read, update, and delete MLflow experiments using Terraform.

Read more

MLflow Models
Revise

• • •
Architecture Diagram for MLflow Models
Architecture Diagram for MLflow Models

The resource_mlflow_model.go file in the mlflow package defines a Terraform resource for managing MLflow models in a Databricks workspace. This resource supports the following key operations:

Read more

MLflow Webhooks
Revise

The file …/resource_mlflow_webhook.go defines a Terraform resource for managing MLflow webhooks in a Databricks workspace. This resource allows creating, reading, updating, and deleting webhooks, which are used to trigger actions based on events in the MLflow Model Registry.

Read more

Databricks Managed Workspace Service (MWS) Management
Revise

References: mws

The mws directory contains the implementation of the Databricks Managed Workspace Service (MWS) resources for the Terraform provider. It provides functionality to manage various MWS entities, such as credentials, storage configurations, networks, permission assignments, private access settings, VPC endpoints, and workspaces.

Read more

Credentials Management
Revise

The data_mws_credentials.go file in the mws package defines a Terraform data source for retrieving a list of Databricks Managed Workspace Service (MWS) credentials associated with a specific Databricks account.

Read more

Storage Configuration Management
Revise

The ResourceMwsStorageConfigurations function in the …/resource_mws_storage_configurations.go file is responsible for managing the lifecycle of Databricks Managed Workspace Service (MWS) storage configurations. This includes creating, reading, and deleting storage configurations.

Read more

Network Management
Revise

• • •
Architecture Diagram for Network Management
Architecture Diagram for Network Management

The terraform-provider-databricks/mws/resource_mws_networks.go file defines the functionality for managing Databricks Managed Workspace (MWS) networks. It provides an API for creating, reading, deleting, and listing MWS networks.

Read more

Permission Assignment Management
Revise

• • •
Architecture Diagram for Permission Assignment Management
Architecture Diagram for Permission Assignment Management

The key functionality for managing permission assignments to principals (users or groups) for Databricks workspaces is implemented in the …/resource_mws_permission_assignment.go file.

Read more

Private Access Settings Management
Revise

• • •
Architecture Diagram for Private Access Settings Management
Architecture Diagram for Private Access Settings Management

The ResourceMwsPrivateAccessSettings() function in …/resource_mws_private_access_settings.go defines the Terraform resource for managing the private access settings for a Databricks account.

Read more

VPC Endpoint Management
Revise

The VPCEndpointAPI struct in the …/resource_mws_vpc_endpoint.go file provides the functionality for managing Databricks Managed Workspace Service (MWS) VPC endpoints. This includes creating, reading, deleting, and listing VPC endpoints associated with a Databricks MWS account.

Read more

Workspace Management
Revise

• • •
Architecture Diagram for Workspace Management
Architecture Diagram for Workspace Management

The terraform-provider-databricks/mws/resource_mws_workspaces.go file provides a Terraform resource for managing Databricks Managed Workspace Service (MWS) workspaces. This resource allows users to create, read, update, and delete MWS workspaces, as well as manage workspace-specific Personal Access Tokens (PATs).

Read more

Databricks Cluster Policy Management
Revise

References: policies

• • •
Architecture Diagram for Databricks Cluster Policy Management
Architecture Diagram for Databricks Cluster Policy Management

The policies directory contains the implementation of the Databricks cluster policy management functionality in the Terraform provider for Databricks. This includes the following key components:

Read more

Cluster Policy Management
Revise

The resource_cluster_policy.go file in the policies package provides the implementation for the databricks_cluster_policy Terraform resource, which allows users to manage Databricks cluster policies. The key functionality includes:

Read more

Cluster Policy Data Sources
Revise

• • •
Architecture Diagram for Cluster Policy Data Sources
Architecture Diagram for Cluster Policy Data Sources

The …/data_cluster_policy.go file provides a Terraform data source for retrieving information about Databricks cluster policies. This data source allows users to fetch details of a specific cluster policy, as well as list all available cluster policies.

Read more

Cluster Policy Resource Management
Revise

The resource_cluster_policy.go file in the policies package defines the Terraform resource for managing Databricks cluster policies. This resource allows users to create, read, update, and delete cluster policies in a Databricks workspace, while handling the differences between built-in and custom policy families.

Read more

Databricks Repositories Management
Revise

References: repos

The repos directory contains the implementation of the Databricks Repositories functionality in the Terraform provider for Databricks. This includes the management of Git credentials and repositories within the Databricks workspace.

Read more

Git Credential Management
Revise

• • •
Architecture Diagram for Git Credential Management
Architecture Diagram for Git Credential Management

The main functionality of the file …/resource_git_credential.go is defined in the ResourceGitCredential() function. This function returns a common.Resource object that defines the schema and CRUD operations for a Terraform resource that manages Git credentials in the Databricks workspace.

Read more

Repository Management
Revise

• • •
Architecture Diagram for Repository Management
Architecture Diagram for Repository Management

The terraform-provider-databricks/repos/resource_repo.go file defines the functionality for managing Databricks repositories using Terraform. The main components in this file are:

Read more

Databricks SCIM Management
Revise

References: scim

The scim directory contains the implementation of the SCIM (System for Cross-domain Identity Management) functionality for the Databricks Terraform provider. This includes the management of users, groups, and service principals within a Databricks workspace.

Read more

SCIM User Management
Revise

References: scim/users.go

• • •
Architecture Diagram for SCIM User Management
Architecture Diagram for SCIM User Management

The users.go file in the scim package provides an API for managing users in the Databricks platform. The main functionality includes:

Read more

SCIM Group Management
Revise

References: scim/groups.go

The groups.go file in the scim package provides an API for managing SCIM (System for Cross-domain Identity Management) groups in a Databricks workspace. The GroupsAPI struct encapsulates the functionality for interacting with the Databricks SCIM API to create, read, update, and delete SCIM groups.

Read more

SCIM Service Principal Management
Revise

• • •
Architecture Diagram for SCIM Service Principal Management
Architecture Diagram for SCIM Service Principal Management

The terraform-provider-databricks/scim/resource_service_principal.go file provides the implementation for managing service principals within a Databricks workspace using the Terraform provider.

Read more

SCIM Entitlements Management
Revise

• • •
Architecture Diagram for SCIM Entitlements Management
Architecture Diagram for SCIM Entitlements Management

The scim directory provides the implementation of the SCIM (System for Cross-domain Identity Management) functionality in the Databricks Terraform provider. This includes the management of entitlements for Databricks groups, users, and service principals.

Read more

SCIM Data Sources
Revise

The scim directory contains the implementation of the Databricks SCIM (System for Cross-domain Identity Management) data sources in the Terraform provider. These data sources allow retrieving information about various SCIM entities, such as users and groups.

Read more

Databricks Secrets Management
Revise

References: secrets

• • •
Architecture Diagram for Databricks Secrets Management
Architecture Diagram for Databricks Secrets Management

The secrets directory contains the implementation of Terraform resources for managing secrets and secret scopes in the Databricks platform.

Read more

Secret Management
Revise

• • •
Architecture Diagram for Secret Management
Architecture Diagram for Secret Management

The resource_secret.go file in the secrets directory contains the implementation of the Terraform resource for managing secrets in the Databricks workspace.

Read more

Secret Scope Management
Revise

• • •
Architecture Diagram for Secret Scope Management
Architecture Diagram for Secret Scope Management

The terraform-provider-databricks/secrets/resource_secret_scope.go file defines the functionality for managing secret scopes in the Databricks platform using Terraform. The key components are:

Read more

Secret ACL Management
Revise

• • •
Architecture Diagram for Secret ACL Management
Architecture Diagram for Secret ACL Management

The resource_secret_acl.go file in the secrets package defines a Terraform resource for managing access control lists (ACLs) for Databricks secret scopes. This resource allows users to create, read, and delete ACL entries that grant permissions to principals (users or groups) for specific secret scopes.

Read more

Databricks SQL Management
Revise

References: sql

• • •
Architecture Diagram for Databricks SQL Management
Architecture Diagram for Databricks SQL Management

The sql directory provides a comprehensive set of functionality for managing SQL-related resources in the Databricks platform using Terraform. This includes resources for managing SQL queries, dashboards, visualizations, and widgets.

Read more

SQL Warehouse Management
Revise

The data_sql_warehouse.go and data_sql_warehouses.go files in the sql directory provide data sources for retrieving information about Databricks SQL warehouses.

Read more

SQL Alerts Management
Revise

• • •
Architecture Diagram for SQL Alerts Management
Architecture Diagram for SQL Alerts Management

The resource_sql_alerts.go file in the sql package defines the functionality for managing SQL alerts in the Databricks platform using Terraform. The key components are:

Read more

SQL Dashboard Management
Revise

• • •
Architecture Diagram for SQL Dashboard Management
Architecture Diagram for SQL Dashboard Management

The …/resource_sql_dashboard.go file provides the implementation for managing Databricks SQL dashboards using Terraform resources. The key components are:

Read more

SQL Endpoint Management
Revise

• • •
Architecture Diagram for SQL Endpoint Management
Architecture Diagram for SQL Endpoint Management

The terraform-provider-databricks/sql/resource_sql_endpoint.go file provides the implementation for managing SQL endpoints (warehouses) in the Databricks platform using Terraform. It allows users to create, read, update, and delete SQL endpoints, as well as configure various settings such as cluster size, auto-stop minutes, and custom tags.

Read more

SQL Global Configuration Management
Revise

• • •
Architecture Diagram for SQL Global Configuration Management
Architecture Diagram for SQL Global Configuration Management

The terraform-provider-databricks/sql/resource_sql_global_config.go file defines the functionality for managing the global SQL configuration in the Databricks Terraform provider. The key components are:

Read more

SQL Query Management
Revise

• • •
Architecture Diagram for SQL Query Management
Architecture Diagram for SQL Query Management

The resource_sql_query.go file in the sql package provides the implementation for managing SQL queries in the Databricks platform using Terraform. The key functionality includes:

Read more

SQL Visualization Management
Revise

• • •
Architecture Diagram for SQL Visualization Management
Architecture Diagram for SQL Visualization Management

The resource_sql_visualization.go file in the sql package defines the functionality for managing SQL visualizations in the Databricks platform using Terraform.

Read more

SQL Widget Management
Revise

• • •
Architecture Diagram for SQL Widget Management
Architecture Diagram for SQL Widget Management

The terraform-provider-databricks/sql/resource_sql_widget.go file defines the functionality for managing SQL widgets in the Databricks platform using Terraform. The key components are:

Read more

Databricks Storage Management
Revise

References: storage

• • •
Architecture Diagram for Databricks Storage Management
Architecture Diagram for Databricks Storage Management

The storage directory provides functionality for managing storage-related resources in the Databricks platform using Terraform. The key components are:

Read more

Storage Mounts
Revise

The terraform-provider-databricks/storage package provides functionality for managing storage mounts in Databricks, supporting various cloud storage providers such as Azure Data Lake Storage (ADLS) Gen1 and Gen2, Amazon S3, Google Cloud Storage (GCS), and Azure Blob Storage.

Read more

DBFS File Management
Revise

The terraform-provider-databricks/storage package provides functionality for managing files within the Databricks Distributed File System (DBFS). This includes creating, reading, and deleting DBFS files.

Read more

Utility Functions
Revise

• • •
Architecture Diagram for Utility Functions
Architecture Diagram for Utility Functions

The dbfs.go file in the storage package provides a comprehensive API for interacting with the Databricks File System (DBFS). The main functionality includes:

Read more

Databricks Workspace Management
Revise

References: workspace

• • •
Architecture Diagram for Databricks Workspace Management
Architecture Diagram for Databricks Workspace Management

The workspace directory contains the implementation of the Databricks provider for Terraform, focusing on the management of various workspace entities. The key functionality includes:

Read more

Directory Management
Revise

The terraform-provider-databricks/workspace/resource_directory.go file defines a Terraform resource for managing directories in the Databricks workspace. This resource allows you to create, read, update, and delete directories, with various attributes and options to control the behavior.

Read more

Notebook Management
Revise

The terraform-provider-databricks/workspace/resource_notebook.go file defines the functionality for managing Databricks notebooks using the Terraform provider. The core of this implementation is the NotebooksAPI struct, which provides methods for interacting with the Databricks workspace API to create, read, export, list, and delete notebooks.

Read more

Global Init Script Management
Revise

• • •
Architecture Diagram for Global Init Script Management
Architecture Diagram for Global Init Script Management

The resource_global_init_script.go file in the workspace directory defines a Terraform resource for managing global init scripts in the Databricks workspace. This resource allows users to create, read, update, and delete global init scripts, which are scripts that run during the initialization of Databricks clusters.

Read more

Workspace Configuration Management
Revise

• • •
Architecture Diagram for Workspace Configuration Management
Architecture Diagram for Workspace Configuration Management

The …/resource_workspace_conf.go file provides the implementation for managing the overall configuration of the Databricks workspace, including settings and preferences.

Read more

File Management
Revise

• • •
Architecture Diagram for File Management
Architecture Diagram for File Management

The main functionality for managing files in the Databricks workspace is implemented in the …/resource_workspace_file.go file. The ResourceWorkspaceFile() function defines the Terraform resource for managing files, including the schema and CRUD operations.

Read more

Data Sources
Revise

• • •
Architecture Diagram for Data Sources
Architecture Diagram for Data Sources

The Databricks Terraform provider includes several data sources for retrieving information about various workspace entities, such as directories, notebooks, and notebook paths.

Read more