5X Managed dbt Core dbt™ Environments

Users can build, manage and run dbt projects on the deployment and development environments on 5X Managed dbt Core™ Dbt. The configured environment determines how dbt interacts with data sources to execute the project. Users can test and implement their dbt projects in a Development environment, and run dbt Jobs in production from Deployment environments.

You must configure the environment when setting up a dbt project on 5X Managed dbt Core™ for the selected warehouse connection. This configuration defines the default deployment environment of the project where it will execute jobs.

5X Managed dbt Core™ Dbt executes a project based on its environment configuration. Depending on the environment type it requires the following configurations:

  • The dbt version of the environment to run the project,

  • The warehouse connection information like schema settings or target, and

  • The environment credentials to access the environment.

Multiple users can work on the same dbt project from their 5X Managed dbt Core™ workspace and connect to the environment with the development credentials. Each user needs to enter their credentials to access the development environment of a project on the Console.

Users can create multiple deployment environments to run jobs and execute dbt projects in production. A deployment environment is configured using the deployment credentials using which the dbt project can run dbt Jobs.

5X Managed dbt Core™ environments have the following properties:

Environment propertyDevelopment environmentDeployment environment

Definition

Write and test dbt models in your development affecting production queries made by the end users.

The end user environment has the finalized dbt project data and acts as a source of truth.

Purpose

Write and test code on the Console

Deploy dbt job in production

Number of environments in a project

Single per user

Multiple

Access

Access the development environment using developer credentials

Access from the default deployment environment when dbt project is built

Application

Build, modify and test data models before running deployment jobs.

Create and deploy jobs based on schedule or continuous integration.

Target

Target is defined by the target-path directory from the dbt_project.yml file. The directory stores compiled SQL files.

Target is defined by the schema settings of the vendor connection.

Environments

Navigate to Environments on 5X Managed dbt Core™ to check environments currently on the dbt project. Users can create and view the current environments on the Environments tab, and create and manage environment variables from the Environment Variable tabs.

Select the environment from the overview to check its details. To create an environment, click Create Environment on the right of the tabs to set up the required deployment or development environments. An environment configuration includes general settings to specify the environment details and credentials to run the dbt project.

5X Managed dbt Core™ currently supports:

  • dbt versions: 1.4, 1.5, 1.6 and 1.7

  • Languages: SQL and Python for dbt operations

Note: You can only create new deployment environments if the dbt project has a configured development environment. Users can edit or delete development environment configuration from project settings.

Development environment

The development environment enables data teams to work on building and refining data models and transformation processes. Users can build and test dbt models and data from the Console page and access the Console using the development environment. 5X Managed dbt Core™ supports a single development environment for a dbt project.

When you run 5X Managed dbt Core™, it reads your dbt_project.yml file to find the profile name and then looks for a profile with the same name in your profiles.yml file. This profile contains all the information 5X Managed dbt Core™ needs to connect to your data platform.

The profile setting configures which user profile dbt uses for the project. Since the development environment is configured based on the profile, 5X Managed dbt Core™ can support multiple users to work on a single development environment. The target-path configures the target directory where the complied SQL files are stored from profiles.yml. You can set a target as default to store modified files on the default target dev in a development environment.

To set up a development environment, refer to Set up a development environment.

Update environments

Users can update or delete an existing environment of a dbt project as follows:

  1. To update an existing environment on the dbt project, click on the environment you want to update from the overview.

  1. Click Edit on the top-right to update the general settings, deployment connection and credentials as required. Once done, Save the changes

  1. You can delete an environment by clicking on Delete when editing an environment.

Last updated