Environment Configuration Reference
At the heart of a Toolkit pipeline is the environment configuration. Within a Toolkit pipeline configuration, the environment configuration files are where you define which Toolkit apps are available within different DCCs and customize the settings for each. This document is a complete reference to the structure and function of the environment configuration files. It covers the Toolkit concept of environments for configuring different workflows within a project, configuration structure, file referencing, and ways to discover what customizations are available.
Note : While this document acts as a reference to the environment configuration files, you can see a step-by-step example of editing a configuration setting in the Toolkit Basics Guide on Editing a pipeline configuration.
What is an environment?
The ShotGrid Toolkit platform provides a fully customizable set of integrations for commonly used content creation software with which you can build your studio pipeline. Within a project’s configuration, you can specify which software packages have integrations, which specific Toolkit apps are available in each, and your options for each app—building out artists workflows to suit your studio’s needs.
But often in studio pipelines, it’s common for different types of artists to have different workflows. As a simple example, for artists working on assets, you might want to make texture painting software such as Mari available, whereas for artists working on shots, you might want to make compositing software such as Nuke available.
Beyond just software packages, you might have different settings for the same Toolkit app for different artists. For example, both shot artists and asset artists might use the Workfiles app, but you might want to limit the file navigation to files associated with Shot entities for the former and Asset entities for the latter.
To support these different workflows within a project, Toolkit divides its app and engine configurations across environments. An environment contains the integrations and their settings for a set of software packages, all with a certain context in common.
In the above example, artists working on assets would be working in an asset step environment, whereas artists working on shots would be working in a shot step environment. Each environment is configured independent of any others, allowing you to have distinct workflows within a project.
A note on Toolkit’s Default Configuration
Toolkit gives you a lot of freedom in the way you structure your environment configuration. This document is a reference for all of the options that are available to you, so that you’ll have the necessary knowledge to make choices that best suit the needs of your pipeline.
This document will also occasionally cover some of the specific choices we’ve made in the pipeline configuration provided as a starting point, known as the Default Configuration. When you’re ready to customize your pipeline, the first step is to create an editable pipeline configuration for your project.
While these choices are only conventions and not hardcoded into the Toolkit workflow, it’s helpful to refer to the Default Configuration as an example for learning what features are available once you start customizing your pipeline and best practices for structuring your own configurations. And, since it’s the suggested starting point for new Toolkit users, it’s helpful to know some of its conventions. We will always distinguish between general features of the Toolkit environment configuration and specific choices in the Default Configuration in this document. For specific details on the Default Configuration’s environment structure, see its README file.
Within your Pipeline Configuration, the
config/ directory contains all the files and folders that are meant to be customized. In
config/, you’ll find three sub-directories:
env directory holds the environment configuration files, so this document will refer to the files in
In the Default Configuration, you’ll see the following files in
asset.yml asset_step.yml project.yml sequence.yml shot.yml shot_step.yml
Each of these files corresponds to an environment; having separate files allows each environment to be configured separately.
How Toolkit determines the current environment
Toolkit uses a core hook called pick_environment to determine which environment file to use at a given time based on the current context. The
pick_environment hook’s return values correspond to environment configuration files. For example, if
shot_step, Toolkit will use
config/env/shot_step.yml to configure the Toolkit environment.
The environment configuration files listed above are the ones provided with the Default Configuration. However, some studios may want to employ different or additional environments. For example, a studio might want different configuration settings for every stage in the pipeline—
shot_step_light, and so on. Fortunately, you can fully customize the available environments.
To do so, add the desired environment config files to the
config/env directory. Then, override the
pick_environment core hook, adding to it the logic that defines when to use your new environments.
Toolkit’s configuration files are written in YAML. The general configuration structure for any bundle (an app, engine, or framework) is as follows:
bundle_name: setting1: value setting2: value complex_setting: sub_setting1: value Sub_setting2: value location: type: descriptor_type descriptor_setting1: value descriptor_setting2: value
To demonstrate this structure, here is a very simple example: an environment in which there is a single engine, with a single app defined within it. The following might be the contents of
project.yml in this config:
engines: tk-maya: apps: tk-multi-workfiles2: location: type: app_store name: tk-multi-workfiles2 version: v0.11.8 location: type: app_store name: tk-maya version: v0.9.4
The engines block
Every environment configuration file starts with an
engines block. Nested within it are all of the engines defined for that environment.
In our example, only a single engine is defined,
tk-maya. It has two settings listed,
location is a special setting that every bundle requires. The
apps setting is a list of all the apps defined for the engine, each with its own settings. In this case, only one app is defined for the engine,
The location descriptor
Every Toolkit bundle has a
location setting, which we refer to as the bundle’s descriptor. The descriptor tells Toolkit where to find the given bundle, and depending on its type, whether to access it directly or cache it locally. Some examples of locations that a Toolkit bundle can come from are the ShotGrid App Store, a git repository, a path on disk, or a zip file uploaded to your ShotGrid site. Each of these has a corresponding descriptor type, with settings specific to that type. Here again is the descriptor for the
tk-maya engine from the example above:
location: type: app_store name: tk-maya version: v0.9.4
This is a descriptor of type
app_store, which tells Toolkit to get the given bundle from the ShotGrid App Store. Descriptors of type
app_store have the settings
In contrast, if you are actively developing a custom bundle—say you’re working on writing a Toolkit app for a specific workflow in your studio, you may want to get it directly from a path on disk. In this case you’d use a descriptor of type
dev, which might look like this:
location: type: dev path: /path/to/app
dev descriptor has different settings from the
app_store descriptor. While it can take other settings, it can be set up simply with a
path setting pointing to where the app lives on disk.
See the Descriptor section of the Toolkit Core API docs for details on all available descriptor types and their settings.
The apps block
Apps are Toolkit’s user tools, and each can run independently of any others. You can choose which apps you want to use based on your pipeline needs, and the
apps setting within an engine block is where you define which apps are available in a given engine.
Here again is the
apps setting from our example above:
engines: tk-maya: apps: tk-multi-workfiles2: location: type: app_store name: tk-multi-workfiles2 version: v0.11.8
You can see that we have a single app defined, the
tk-multi-workfiles2 app. It currently only has a single setting defined: its descriptor.
If you wanted to make other apps available in the
tk-maya engine in the
project environment, you’d add them here. Let’s add the Panel,
tk-multi-shotgunpanel, and the About app,
tk-multi-about, to our engine. Our example
project.yml file now looks like this:
engines: tk-maya: apps: tk-multi-about: location: type: app_store name: tk-multi-about version: v0.2.8 tk-multi-shotgunpanel: location: type: app_store name: tk-multi-shotgunpanel version: v1.6.3 tk-multi-workfiles2: location: type: app_store name: tk-multi-workfiles2 version: v0.11.8 location: type: app_store name: tk-maya version: v0.9.4
There are a few important things to note at this time:
- The Default Config lists bundles in alphabetical order, and this example follows that convention.
- The file is beginning to get long, and we haven’t even added any configuration settings yet.
- You might imagine that you’ll be using these same apps in other engines and other environments. For example, you’ll probably have all three of these apps—the Panel, the About app, and the Workfiles app—in different engines (say, Houdini, Nuke, or Photoshop), and in different environments (like
shot_step). Defining common app settings in many places in your config means that when it comes time to make a change, you’ll have to make the modification in many places.
To mitigate the last two issues, Toolkit configurations support includes.
Includes allow you to reference a section of one file in another file in your configuration. Using includes allows you set a configuration setting in one place, but use it in multiple environments.
Includes consist of two parts:
includeslist: a YAML dictionary whose key is
includes, and whose value is a list of all files we want to include from.
- A reference within your configuration settings, prefixed by the
@symbol, and named to point to the name of the section you want to reference from the included file.
To flesh out our above example, you might have a single file where you hold the location descriptors for all of your engines. Let’s put that file in an
includes subfolder, and call it
The contents of
engine_locations.yml would look like this:
engines.tk-maya.location: type: app_store name: tk-maya version: v0.9.4 engines.tk-nuke.location: type: app_store name: tk-nuke version: v0.11.5 ...
This file can act as a single source for all engine locations, and all of your environment configurations can reference it. Using this include file, our example now looks like this:
includes: - includes/engine_locations.yml engines: tk-maya: apps: tk-multi-about: location: type: app_store name: tk-multi-about version: v0.2.8 tk-multi-shotgunpanel: location: type: app_store name: tk-multi-shotgunpanel version: v1.6.3 tk-multi-workfiles2: location: type: app_store name: tk-multi-workfiles2 version: v0.11.8 location: @engines.tk-maya.location
You can see here that the value of the
location setting for the
tk-maya engine is now a reference to a key from the included YAML file.
: Having all engine locations in a
config/env/includes/engine_locations.yml file, as we do in this example, follows the convention of the Default Configuration.
You can add a second include file for app locations, and in fact, the Default Configuration does just that. Let’s expand our example:
apps.tk-multi-about.location: type: app_store name: tk-multi-about version: v0.2.8 apps.tk-multi-shotgunpanel.location: type: app_store name: tk-multi-shotgunpanel version: v1.6.3 apps.tk-multi-workfiles2.location: type: app_store name: tk-multi-workfiles2 version: v0.11.8
includes: - includes/app_locations.yml - includes/engine_locations.yml engines: tk-maya: apps: tk-multi-about: location: @apps.tk-multi-about.location tk-multi-shotgunpanel: location: @apps.tk-multi-about.shotgunpanel.location tk-multi-workfiles2: location: @apps.tk-multi-workfiles2.location location: @engines.tk-maya.location
We’re now getting the
tk-maya engine’s descriptor from the included
engine_locations.yml file, and the descriptor for each app defined for the
tk-maya engine from the included
: The Default Configuration employs a second level of nesting that’s not demonstrated here. Every app or engine that has settings beyond just a descriptor has a settings file in
includes/settings/tk-multi-workfiles2.yml). The engine settings files include app settings from the app settings files, and the environment configuration files include from the engine settings files. For details on the Default Configuration’s structure, see its README file. For a detailed walkthrough of modifying a configuration setting, see the Toolkit Basics Guide on Editing a Configuration Setting.
Every Toolkit bundle has a set of available configuration settings, with a default value for each. Toolkit allows sparse configurations: if a configuration setting is not explicitly specified in the environment config files (and/or the files they include), then the default value from the bundle will be used.
In our example, we haven’t specified any settings for our apps aside from
location. So, in our configuration’s current state, our three apps will use the default values for all of their settings. So, how do we know what configuration settings are available?
Note : While it’s not a requirement that Toolkit configurations be sparse, the Default Configuration is a sparse configuration.
Discovering available configuration settings
With sparse configurations, it’s not immediately evident what configuration settings are available for an app simply by looking at your configuration files. To find out what configuration settings an app has available, you have two choices:
- App documentation: Each of our apps has its own documentation page, and each of these pages has a “Configuration Options” section. This section lists all of the available configuration settings for the app, with description and default values for each. You can see the Workfiles documentation page as an example. The Apps and Engines page lists the documentation pages for all apps and engines.
- The manifest: Every Toolkit bundle includes a file called
info.ymlin its root directory. We refer to this file as the bundle’s manifest, and it defines all of the available configuration settings for the bundle, with a description and default value for each. You can find the manifest in your own cache of the bundle (e.g.,
install/app_store/tk-multi-workfiles2/v0.11.8/info.ymlwithin your pipeline configuration), or in Github (here it is for Workfiles as an example).
Modifying configuration settings
To modify a configuration from the default value, simply add it to the proper block, in the proper environment in your Pipeline Configuration, and set its value.
Going back to our example, let’s say that we want to configure
tk-multi-workfiles2 so that it launches automatically when Maya is launched in the project environment. We can see in the app’s manifest that there is a
launch_at_startup setting that controls whether to launch the Workfiles UI at application startup time, and that its default value is
False. So, we’ll just add the
launch_at_startup option, and set it to
project.yml file now looks like this:
includes: - includes/app_locations.yml - includes/engine_locations.yml engines: tk-maya: apps: tk-multi-about: location: @apps.tk-multi-about.location tk-multi-shotgunpanel: location: @apps.tk-multi-about.shotgunpanel.location tk-multi-workfiles2: launch_at_startup: True location: @apps.tk-multi-workfiles2.location location: @engines.tk-maya.location
Note that if the settings for
tk-multi-workfiles2 were coming from an included file, we’d make this change in that file.