Dagster Authkit: Auth for OSS Deployments

Intro

#

Dagster has become my go-to orchestrator this past year. It has a beautiful UI, it keeps all of my pipeline work version modular and pythonic, and the local dev experience is phenomenal.

The software-defined asset approach also has so many benefits when reasoning about a complex data environment.

Like most of my favorite technologies, Dagster is open-source. In my current role, we are transitioning away from Azure Synapse and using Dagster as our “single pane of glass” for our data orchestration needs. We’re self-hosting the open-source distribution of Dagster on our Kubernetes clusters, and like the rest of our infrastructure, we need to ensure we implement these tools with security best practices.

In this article I’ll describe how we were able to implement our authentication needs with Dagster Authkit and a couple open source contributions I made to solve some issues along the way.

Authentication in Dagster

#

By default, in the open source implementation of Dagster there actually is no auth. This means that anyone who has access to the Dagster URL has full admin access to do what they want with any of the assets/pipelines/resources. Not ideal.

Auth is a first-class feature in Dagster+, their paid hosted option, but for those of us wanting to self-host that leaves us a pretty significant problem to solve.

Dagster Authkit

#

Here is where Dagster Authkit comes to save the day. Their statement in the main README explains it best:

“Inspired by the community’s need for a middle ground between “no auth” and “pay for Dagster+”.”

I couldn’t agree more. Security and proper auth really shouldn’t be a premium feature (see sso.tax and the sso wall of shame for the broader community’s reactions to this idea)

Dagster Authkit solves this need elegantly by wrapping the dagster-webserver command that will intercept the UI’s official GraphQL calls and ensure that the authenticated user has the proper role to do that function.

It uses RBAC (Role Based Access Control) and allows you to define one of four standard roles to perform the proper functions within the Dagster UI:

In our case, we use a simple OAuth2 Proxy so that when users hit our Dagster URL, they have to authenticate with Entra ID. We allow everyone to be a “Viewer”, and have managed app roles for the others and that gives us peace of mind that only specified users have the exact limited permissions we want them to have.

Issues Encountered

#

Dagster Authkit is exactly what our team needed, but when we began exploring how we’d implement it in our environment, we ran into a couple issues.

1. Missing CSP Issue - Newer versions of Dagster have new Content Security Policies (CSPs), which broke a few visuals on the lineage page and produced some console log errors.
2. Multiple UI Injection Bugs - When Dagster Authkit released 0.4.1, there were some changes that caused a couple of UI regressions.

Let’s walk through the solutions for each of these issues!

Solutions

#

Missing CSP Solution

#

This was a pretty straightforward change since the errors that surfaced simply declared what CSPs needed adjusting.

The diff is pretty self explanatory here:

diff --git a/dagster_authkit/auth/security.py b/dagster_authkit/auth/security.py
index 7a65473..d851302 100644
--- a/dagster_authkit/auth/security.py
+++ b/dagster_authkit/auth/security.py

@@ -141,9 +141,10 @@

140 def set_security_headers(response):

141             "Content-Security-Policy": (

142                 "default-src 'self'; "

143                 "script-src 'self' 'unsafe-inline'; "  # Dagster needs inline scripts

-                "style-src 'self' 'unsafe-inline'; "  # Dagster needs inline styles

144+                "worker-src 'self' blob:; " # Dagster graph layout uses blob: workers

145+                "style-src 'self' 'unsafe-inline' https://cdn.jsdelivr.net; "  # Dagster needs inline styles

146                 "img-src 'self' data:; "

-                "font-src 'self' data:; "

147+                "font-src 'self' data: https://cdn.jsdelivr.net; "

148                 "connect-src 'self';"

149             ),

150             # Adds Permissions-Policy (replaces Feature-Policy)

The worker-src 'self' blob:; was missing altogether, and there were updated font and style sources from https://cdn.jsdelivr.net.

Once those changes were in, the console log errors disappeared, and the missing arrows in the lineage graph reappeared.

UI Injection Solutions

#

Dagster Authkit also conveniently adds a small menu in the UI that simply displays the authenticated user and role, also allowing you to manually log out. It’s a very lightweight UI addition, but requires some middleware to work properly.

Problem 1: Unawaited Coroutine

#

Dagster uses newer versions of Starlette, an async web framework and ASGI toolkit. If you’re not familiar with async vs. sync concepts, the main difference is that a synchronous function will immediately do that work and return the results. Alternatively, an asynchronous function will defer that work with a coroutine, which allows you to check in to see whenever that work is done. In an async framework, these coroutines will allow other work to continue that isn’t waiting on some response, and eventually return the answer when it’s able to complete everything.

The original problem looked like this:

if inspect.iscoroutinefunction(original_index_html):
    async def patched_index_html(self, request: Request):
        return await _inject_resilient_ui(self, request)
else:
    def patched_index_html(self, request: Request):
        return _inject_resilient_ui(self, request)

UI injection was trying to handle a conditional situation where if it was async, then await the injection, otherwise return it synchronously. The problem was that the replacement logic it called into (the _inject_resilient_ui function) was always async, even if the original_index_html was not a coroutine function.

That meant that the server would only return the coroutine and not the actual HTTP response it needed, so it threw a 500 error and failed.

The fix was simply to remove that conditional branch altogether and make the function async an await it’s result:

async def patched_index_html(self, request: Request):
    return await _inject_resilient_ui(self, request)

Problem 2: User Menu Silently Missing

#

With the first bug fixed, the UI could now properly render instead of throwing an error, but it was missing the actual injected user menu it was trying to add to begin with. This was a little bit trickier to diagnose because the original code looks like it should work.

Starlette gives every request a request.state object meant to carry extra data (like “who is the logged-in user”) between pieces of middleware. The original code stored that data like this:

scope["state"]["user"] = user

That treats the state object like it’s a plain dictionary where we can add keys and values to the object. In newer versions of Starlette, however, that “state” object is an actual separate class altogether that is accessed with dot notation like this:

request.state.user

Dictionaries don’t have that “.user” property, so it silently returned a None value, which was interpreted as “no user is logged in”, even if a user was indeed logged in. To actually store the user we log in as in Starlette properly, we needed to store that in an actual starlette.datastructures.State object instead of a dictionary.

That middleware change looked like this:

diff --git a/dagster_authkit/core/middleware.py b/dagster_authkit/core/middleware.py
index 6d9afcf..d0fce92 100644
--- a/dagster_authkit/core/middleware.py
+++ b/dagster_authkit/core/middleware.py

@@ -13,6 +13,7 @@

12 import logging

13 from typing import Optional

14 

15+from starlette.datastructures import State

16 from starlette.requests import Request

17 from starlette.responses import RedirectResponse, Response

18 from starlette.types import ASGIApp, Receive, Scope, Send

@@ -251,11 +252,19 @@

251 async def _receive():

252             await response(scope, receive, send)

253             return

254 

-        # Store user in scope for downstream handlers (e.g., UI injection)

255+        # Store user in scope for downstream handlers (e.g., UI injection).

256+        # Use a Starlette State object (not a plain dict) so that attribute

257+        # access via request.state.user works correctly in Starlette 1.3.1+,

258+        # where request.state returns scope["state"] directly.

259         scope = dict(scope)

-        if "state" not in scope:

-            scope["state"] = {}

-        scope["state"]["user"] = user

260+        existing_state = scope.get("state")

261+        if not isinstance(existing_state, State):

262+            new_state = State()

263+            if isinstance(existing_state, dict):

264+                for k, v in existing_state.items():

265+                    setattr(new_state, k, v)

266+            scope["state"] = new_state

267+        scope["state"].user = user

268 

269         await self._passthrough(scope, downstream_receive, send)

270 

Problem 3: Missing Nonce

#

Lastly, newer versions of Dagster has improved Content Security Policy (CSP) handling, which requires a unique “nonce” value for each request to allow inline scripts to execute safely. A “nonce” values stands for a “number used once”, which is usually a random, one-time secret stamped on a request.

The original code wasn’t generating or including this nonce, which caused the CSP to block the injected UI elements. To fix this, since Dagster’s UI was already generating the accepted nonce values on their scripts, we had to simply find that value and include it in the html that we were injecting for the UI.

That particular code looks like this:

_nonce_match = re.search(r'<script[^>]+nonce=["\']([^"\']+)["\']', html)
if _nonce_match:
    injection = injection.replace(
        "<script>",
        f'<script nonce="{_nonce_match.group(1)}">',
        1,
    )

We’re just using regex to find pre-existing nonce values, retrieve those and replace the script opening tag in our injected html.

Altogether, most of these fixes landed on the core/patch.py file here:

diff --git a/dagster_authkit/core/patch.py b/dagster_authkit/core/patch.py
index 22343dc..60f6a33 100644
--- a/dagster_authkit/core/patch.py
+++ b/dagster_authkit/core/patch.py

@@ -12,6 +12,7 @@

11 import inspect

12 import json

13 import logging

14+import re

15 from typing import Optional

16 

17 from starlette.requests import Request

@@ -84,12 +85,12 @@

84 def patched_build_routes(self):

85     try:

86         original_index_html = webserver_module.DagsterWebserver.index_html_endpoint

87 

-        if inspect.iscoroutinefunction(original_index_html):

-            async def patched_index_html(self, request: Request):

-                return await _inject_resilient_ui(self, request)

-        else:

-            def patched_index_html(self, request: Request):

-                return _inject_resilient_ui(self, request)

88+        # Always async: _inject_resilient_ui is async def regardless of whether

89+        # the original index_html_endpoint is sync or async. Calling an async

90+        # function without await returns an unawaited coroutine that Starlette

91+        # cannot invoke as a Response (Bug 1: HTTP 500 on Dagster 1.13.8).

92+        async def patched_index_html(self, request: Request):

93+            return await _inject_resilient_ui(self, request)

94 

95         webserver_module.DagsterWebserver.index_html_endpoint = patched_index_html

96         logger.info("Resilient UI injection patched")

@@ -117,7 +118,15 @@

117 async def _inject_resilient_ui(self, request: Request) -> HTMLResponse:

118         logger.error(f"Failed to get index HTML: {e}", exc_info=True)

119         raise

120 

-    user = getattr(request.state, "user", None)

121+    # Bug 2: The middleware stores the user via dict-key assignment on scope["state"].

122+    # In Starlette 1.3.1+, request.state returns scope["state"] directly, so when

123+    # it is a plain dict, getattr(..., "user", None) finds no attribute and returns

124+    # None — silently skipping injection. Check both access patterns defensively.

125+    _raw_state = request.scope.get("state")

126+    user = (

127+        _raw_state.get("user") if isinstance(_raw_state, dict)

128+        else getattr(request.state, "user", None)

129+    )

130 

131     if not user or not isinstance(user, AuthUser):

132         if user is not None and not isinstance(user, AuthUser):

@@ -168,6 +177,19 @@

176 async def _inject_resilient_ui(self, request: Request) -> HTMLResponse:

177             logger.warning("No </body> tag found in HTML; cannot inject UI")

178             return response

179 

180+        # Bug 3: Dagster emits <script nonce="..."> tags. Per CSP spec, when a nonce

181+        # is present in script-src, 'unsafe-inline' is ignored — only scripts with

182+        # a matching nonce execute. Copy the nonce from the first existing script tag

183+        # so the injected block passes CSP validation. No-ops gracefully when no

184+        # nonce is present (older Dagster or custom deployments).

185+        _nonce_match = re.search(r'<script[^>]+nonce=["\']([^"\']+)["\']', html)

186+        if _nonce_match:

187+            injection = injection.replace(

188+                "<script>",

189+                f'<script nonce="{_nonce_match.group(1)}">',

190+                1,

191+            )

192+

193         html = html.replace("</body>", f"{injection}</body>", 1)

194 

195         headers = dict(response.headers)

Unit Tests

#

Lastly, as part of this change, I included a whole set of unit tests that will catch these same bugs if any future changes re-introduce those by accident.

Conclusion

#

Dagster has been an amazing choice for data orchestration work, and Dagster Authkit fills in the right amount of auth feature gaps without getting in the way. That has enabled hardened security for the Dagster deployments that are self-hosted, and I’m glad I was able to contribute to some fixes as Dagster’s platform continues to evolve!