Talking Drupal #578 - Rosetta

[00:00:00] John: This is Talking Drupal, a weekly chat s- about web design and development from a group of people with one thing in common: we love Drupal. This is episode 578, Rosetta. On today's show, we're talking about better traceability, better debugging, and Rosetta with our guest, Juergen Haas. We'll also cover Maestro as our module of the week.

Welcome to Talking Drupal. Our guest today is Juergen Haas. Juergen started back in 2007 with Drupal 4.6, founded Lakedrops, a Drupal agency in Germany, and is known as a maintainer of a number of Drupal modules, for example, ECA. He's a co-maintainer of the Gen Admin theme that he helps to migrate. He's helping to migrate to Drupal Core.

It works great, by the way. He, his passion for privacy and security got him c- uh, to contribute to Drupal CMS for those topics. Juergen, welcome back to the show, and thanks for joining us.

[00:01:04] Jurgen: Hey, great to be back

[00:01:07] John: Yeah, the, uh, the, uh, Gen Admin theme in, or now the Admin theme co- soon to be, soon to be released is, uh, works great.

Love it. Um, was, was on Gen for a while. We talked about this last week, I think. I was on Gen for a while, then went to, uh, uh, Claro, and now I'm back, back to the Admin theme because I, I wanna, you know, I wanna be closer to core. So I appreciate all your, uh, all your work there.

[00:01:32] Jurgen: Thank you. Noticed that last week, actually.

That was nice to hear. Thank you.

[00:01:37] John: There you go. Uh, I'm John Picozzi, solutions architect at EPAM, and today my co-hosts are Emma Horrell, UX lead for Drupal CMS and UX manager for Drupal Core. Emma, how are you?

[00:01:50] Emma: I'm well, thank you. I'm looking forward to another episode.

[00:01:54] John: Here we go. And, uh, Nic Laflin, founder at nLightened Development.

Nic, what's going on?

[00:01:59] Nic: I'm here. I'm ready. I wanna learn all about Rosetta. I've been hearing a lot about it, but I actually haven't dug in too far, so

[00:02:07] John: I'm excited. I also have this preconceived notion of what it does, and I'm- I'm looking forward to Jurgen correcting all of my misconceptions today. So I, I too am looking forward to it.

But before we start, uh, down that road, let's, uh, talk about our module of the week. Let's turn it over to Martin Anderson-Clutz, a product marketing manager for Drupal at Acquia and a maintainer of a number of Drupal modules. I should just say Drupal things. Drupal things of his own. Martin, what do you have for us this week?

[00:02:35] Martin: Thanks, John. Have you ever wanted to model a multi-step business process on your Drupal site, like a review and approval flow that routes tasks to different people, branches on their decisions, and can even hand steps off to AI? There's a module for that. It's called Maestro, and it was created in June of 2010 by Blaine Lange of Nextide, and is now co-maintained with Randy Kolenko.

It has a 4.6.0 version, which works with Drupal 10.3 and 11, and 3.1.4, which also supports Drupal 9.4 through 11. It is actively maintained. In fact, that 4.6 version came out last week. It has security and test coverage, and for documentation, there's a handbook on drupal.org plus a series of videos including AI and agentic examples.

It has 189 open issues, 55 are which, 55 of which are bugs, but many date back to the Drupal 7 and 8 branches, and a half a dozen issues were closed just last week. It is officially in use by 785 sites according to drupal.org. Now, Maestro's tagline pretty much sums it up: If it can be flowcharted, it can be automated with Maestro.

You build a workflow template in a visual drag-and-drop editor, and 4.6 adds a new React Flow-based editor as an alternative to the older Diagram JS one. Templates are made of tasks, interactive ones assigned to a user or role, plus non-interactive ones like if and or can set a process variable, batch functions, and spawn a subflow.

That gives you parallel approvals, serial steps, and dynamic routing. For example, taking a different path depending on whether someone clicked accept or reject. Each user gets a task console listing what's assigned to them, and tasks can be reassigned or delegated. Under the hood, a background engine Maestro calls the orchestrator and does the work of advancing each process.

You run it via a tokenized URL, cron or Drush Maestro orchestrate, and if it isn't running, nothing moves forward, so that is the first thing to set up. Now, since 4.1, there's an AI task built on the AI module, which can return a yes/no or true/false result if the nef- next if task can root on-- route on or store the response in a process variable.

There's also a tool task for calling tool API tools directly with no AI involved, and AI tools can let an agent list start and check on Maestro workflows. New in 4.6, a trace view and trace archive lets you see a snapshot of the process variables before and after each task, which is great for auditing or debugging.

For editorial and compliance teams, this is a natural fit for multi-stage content review, legal sign-off, or anything where you need a record of who approved what and when. Maestro also has an ECA integration sub-module. A Maestro task can fire an event for ECA to react to, and an ECA action can spawn a new Maestro process.

We covered ECA in episodes, uh, 375 and 402 Both ECA and Maestro lets you visually model automation without code, but the mechanism is different. ECA racks-- reacts to events in the moment. Something happens, conditions are checked, actions run, and it's done. Maestro is stateful. Each process instance lives on for days or weeks waiting for people to complete tasks, with the orchestrator advancing it in the background.

Now, we haven't clevert-- uh, covered Flowdrop yet, but it's the newer option in the space, with an N8n style graph editor where type data flows from node to node, and it leans heavily into AI and data pipelines. It does have human-in-the-loop pauses, but Maestro's model is built around people and their task queues while Flowdrop is built around moving data through a chain of nodes.

Now, the good news is they can work together. Flowdrop works-- or workflows can be published as tool API tools, which Maestro's tool task can call. And of course, the Rosetta Initiative is all about reducing duplication of effort wherever possible and making these solutions work seamlessly together. So let's talk about Maestro.

[00:06:46] Nic: I have to say, I'm very interested in... Sorry. I'm very interested in that tool, the tool application that you said where you can just hand off, um, tasks directly without including AI. Mostly because one of the things I've been finding a lot of my clients are doing is they're building some workflow with AI kind of as a, almost as like a rough prototype or a quick prototype.

And then, um, once they kind of vet the full workflow, they, they pull AI out as much as possible, mo-mostly because of costs. Like, uh, a lot of things, especially now that things are getting automated with AI so much, services are just increasing their costs significantly. And so, it-- this AI is good for that quick prototype, vet the process, and then once it's getting used, you kind of move it off to whatever system makes sense.

And if you have kind of the tool and task API to manage those workflows, and you can swap out those steps, it might be, it might be nice because we're, we're honestly building kind of our own batch harness for, for managing these long-running processes. And if there's an API that we can rely on, that might simplify some of the race conditions that we've, um, had to handle manually.

So I'd, I'd be interested to hear if, if other people have experience with that.

[00:08:16] John: Yeah, I was, I, I was gonna... Go ahead, Martin.

[00:08:19] Martin: Oh, no, I was just gonna say, I mean, I think that, that whole idea of, of, uh, being able to find things as tools and, and is definitely v- really powerful. And also one of the things that, um, you know, Dries was talking about in terms of being able to define tools using metadata, um, even sort of within existing contrib modules to sort of reduce the amount of code re- required to be able to do that.

Um, I think in the context of the, the Dries note where it was mentioned, it was more for saying those can then be exposed via MCP, but, but that would also apply in terms of being able to call them as tools through, through these other systems as well. I also love the example that you gave in terms of, you know, we hear a lot about people sort of, you know, using everything, um, you know, with AI just because it's, it's an easy way to sort of get started and, and sort of start with a problem and then get a solution back.

Uh, but I think a lot of teams are seeing exactly what you're describing in the sense that anywhere where you don't need actual reasoning, being able to solve that same problem with deterministic logic is gonna make it cheaper, faster, more reliable. And so, you know, really being able to sort of, like, look at the overall workflow and figure out which pieces actually require reasoning and only use it selectively there- Yeah

is definitely where things are headed. And I think that, again, is, you know, one of the strengths of Drupal in this space, is that we have these tools that allow you to sort of piece together the right tool for the right part of an individual job.

[00:09:43] Nic: I, I am curious and may- maybe... Sorry, John. Today, today's- It's cool

today's my day, I guess. Um, Jurgen, I, I have a question for you 'cause I think you'll probably know the answer to this with, with these tools. Um, is there a way... 'Cause I know MCP, as Martin said, is one of the primary integration points and plans for this, but I- Uh, for, for most of my clients, they don't want the MCP as- aspect.

They don't want to expose... And I know you can have authentication, that kind of stuff, but they don't wanna expose their internal data or processes. And so, you know, security and privacy are kind of a, a, a crucial aspect. I worry that integrating MCP services with other stuff will lead to kind of like what we've seen with the JSON API, where open by default.

Like, I-- how easy is it to just be like, "Hey, I wanna use this API, but I don't want-- I wanna explicitly disable MCP everywhere so I don't have to worry about accidentally leaving it on, and then all of a sudden agents find all the data, and then it's out in the wild." Is, is it like a submodule or something that you can restrict, or is it baked into the idea?

[00:10:50] Jurgen: That's baked in and comes out of the box. And by the way, we are now already covering 90% of my notes for what I wanted to talk about later on, but that's fine. Um, well, there is, for example, Drush, right? So with either Drush or the new CLI DR, uh, you can call tools directly. You don't have to have MCP. Uh, it can be that deterministic tool.

It can be a batch script that leverages Drush to call it. Uh, and the beauty of that is a tool describes not only what it does, but it also describes what input arguments it requires, what type of data that should be, and what type of data it returns. So it's very easy to, A, find out what tools are available, and B, how to call them, what to give them as arguments, and what to expect as a return.

And you don't need anything extra. It's just that, you know, write it once and everybody who wants to call tools is able to do so.

[00:12:03] John: So, uh, we are in the process of, of scheduling a show with, uh, Mike Lander to talk about tools API in more detail. So, um, we will, we will have that for you most likely in the new year.

So you can, you can look out for that one. I think one thing here, and Martin did a great job of answering this question. I was kind of like, well, Maestro just sounds like ECA. So like, what's the difference? Um, but you, you very clearly stated the, the difference is like, you know, ECA is very action-based. So like I click this button and ECA goes and does something.

Maestro is kind of stateful where it's, it's just kind of listening in the background in some cases and, and moving those things around. Um, so I, I get the, I get the difference, the difference in need there. And I like the fact that they work together. Um, I'm wondering, I don't know. There's so many thoughts right now.

Uh, I will say that we, uh, we had a show on orchestration that was, um, not that long ago. Um- About a year, I would say. About a year? Oh man, time goes by too quickly. Where we kind of talked to in, uh, a little bit more detail about these things. So, um, I think this is all part of, part of that path of, of kind of making, making Drupal a, uh, a more, um Uh, automated, AI-friendly, you know, automatic system.

I'm wondering actually, uh, Martin, if you have any insight or, um, maybe a use case of like would you use Maestro instead of something like Workflows, like content moderation, something like that? Or is it more like business logic like, "Hey, I just created this content. It has to go to legal for review." And, and then when legal says, "Yes, it's okay," then it can go into my publish queue or something like that.

[00:14:00] Martin: So I'm gonna be upfront and confess that I haven't actually used Maestro personally. Um, and so it wouldn't surprise me if actually Juergen is, is probably better positioned to answer this, having been, um, you know, party to a lot more discussions about Maestro. But my understanding would be that Maestro is a better solution than core Workflows where you need things that are sort of less linear.

So core Workflows- Mm ... are great for it's like this, then that, and then the next thing- Yeah ... and it's sort of like a single line. Whereas with Maestro, you can have those sort of like chain off of, well, if, if it gets rejected here, maybe it goes through another process and it can still get published, but more people have to see it.

Yeah. Or different kinds of things that way. So I think it's, it's really for sort of the same idea of it being people at each stage making decisions. Yeah. Um, but in more complex use cases.

[00:14:48] John: Got it. So like less, less linear. Less linear in my head just equated to like, hey, I'm writing a blog post, and like I need legal review or I need, um, you know, uh, before legal can review it, it has to go to, to design for it to look at, you know, for, for some visuals- Mm

'cause I don't have some visuals I need, right? So you can, you know, check a button or, or drop a drop-down, say go to design, and it'll go to the design, design track of that workflow.

[00:15:14] Martin: Or maybe it should go out simultaneously to five different translators for five different languages- Sure ... right? Those kinds of things, too.

[00:15:20] John: Yep.

[00:15:22] Jurgen: I wanted to just- It could also be- Oh, go ahead Sorry, Emma

[00:15:25] Emma: You go

[00:15:26] Jurgen: Uh, it could also be that decision means that the flow needs to go back to a previous state. Like, if somebody applies for something and it's rejected, uh, with a reason, it goes back, and then the application gets- Yeah ... uh, you know, modified.

And then it's reviewed again, and then the decision either approves it or sends it back yet again.

[00:15:50] John: Emma, I'm gonna, I'm gonna let you chime in here with your thing, and then I'm gonna, I'm gonna, um, hit another fast ball at, uh, Martin and see, see what he does.

[00:15:58] Emma: That's fine. I'm glad I'm not Martin right now, then.

So, um, my thought is looking at these two, Maestro and ECA versus Flowdrop, and how Flowdrop is talking about data flow. Mm-hmm. I think that's interesting because I think whether it's prompted by AI or whether it's always been a thing, but I think these are two different ways that users bring to this kind of problem.

Am I doing one task followed by this task, and that kind of dependency conditions idea, um, versus how is my data moving through a system? I think those are both valid ways that people approach this. Mm-hmm. So it's interesting that those tools take those different approaches. Was all I was gonna chuck in on the UX side of things there.

[00:16:41] John: Yeah. That's a, that's a great point. And I think in our, if I'm remembering correctly, or maybe I saw a webinar where, where Jurgen went over this, but I feel like in our, in our orchestration episode, we might have talked about kind of the different, different use cases of, of when you might use something like Flowdrop or Maestro or ECA or, or, you know, all of the above.

Um, so that's definitely, definitely a good, a good call-out. Um, so Martin, uh, looking and thinking about events and the fact that you are, um, you know, the master of the events platform, um, I'm wondering if, like, Maestro is something that you might consider for, like, flows of, um, you know, uh, talk acceptance and, and rejection and that sort of thing?

[00:17:28] Martin: Yeah, it's an interesting question. I mean, I know there have been a couple of little gremlins. Like, I, I saw, I think it was even for an EdCamp this past week where one of the, uh, speakers went in to, you know, change something on this- Every year I get

[00:17:40] John: at least three of those And- Somebody's

[00:17:42] Martin: like- Right. And- "

[00:17:42] John: Ah, I edited one word and now, uh, it's unpublished"

[00:17:46] Martin: Yeah.

So it's definitely possible that Mais- Using something a little more robust like Maestro would allow us to, to better model what actually have become fairly complicated workflows in ways- Yeah ... that, that don't have some of those, you know, unintended surprises when, when things get changed, so.

[00:18:04] John: But-

[00:18:04] Martin: Yeah, definitely possible

[00:18:05] John: a last time

[00:18:07] Nic: The, the, I mean, and, and truthfully, and I understand why you didn't do this, Martin. I'm not saying that you should have because-

[00:18:14] John: You should take this as a slight. God ...

[00:18:16] Nic: no, no, no, it's way, it's way too much work, right? He's already put in hundreds of hours into maintaining it. But, but really, the, the right, the right solution for that kind of thing is to have actual content moderation, right?

Once it's been published, you want, you want it to stay up. If somebody makes an, uh, an edit, you want that to go in front of a different, you know, somebody who maintains the camp to look at it and be like, "Yes, that's appropriate. No, it's not," and publish it, rather than unpublishing the whole thing. But content moderation is extremely complex.

It's very specific to the implementation. And so, like I said, it's n- it's not that you should've gone that way, but for, for, like, a real full solution, that's what you do. I think the workaround of like, "Hey, they edited it. Let's unpublish it," that's probably... that- that's fine 'cause like you said, John, you get three, two to three emails a year.

That's really- Hey,

[00:19:02] John: my time's valuable, Nic.

[00:19:04] Nic: Everybody's time is valuable. Uh, I think

[00:19:07] John: that- Accepting people that have already been accepted is tiring.

[00:19:09] Nic: Yes, I, I understand. But also the, from, from the other side, you know, Martin, Martin is supporting not just NetCamp. He's supporting, at this point, how many? A dozen, two dozen camps.

So-

[00:19:22] Martin: Yeah, it, it's an interesting question. I'm not 100% sure, particularly because I know, like, every year there's a new, um, you know, Drupal Dev Days site. And I think at this point, most of them seem to use the, um, well, hopefully now Summit site template moving forward. Yeah. But not all of them do, and so it's sort of hard to, to keep track, to be honest, in terms of, you know, who, how many sites are actually using it and are still sort of, like, uh, live in the sense of have ongoing events and so on.

[00:19:49] Nic: Yeah. Yeah, exactly.

[00:19:51] John: All right. Well, uh, Martin, thank you for a, uh, riveting, um, module of the week. And if folks wanted to suggest a module of the week, how best could they do that?

[00:20:02] Martin: We're always happen, happy to have lively discussions about candidates for module of the week in the Talking Drupal channel of Drupal Slack, or folks can find me directly as mandclu on a variety of Drupal and social platforms.

And of course, I'll be at NetCamp this coming weekend, and at Evolve Digital, uh, New York City, the week after. So by all means, if you're in the area, uh, stop by and say hello.

[00:20:26] John: Fabulous. Look forward to seeing you in person.

[00:20:30] Martin: I'll see you then. Thanks,

[00:20:31] Angie: Martin.

[00:20:31] John: Have a good one.

[00:20:32] Martin: See you.

[00:20:36] John: All right, before we jump into our primary topic, here's some cool cats that want, uh, your help making DrupalCon great

[00:21:20] Angie: It's all the rage I can't stop dancing when I'm in here 'Cause DrupalCon's talent show is crystal clear So while you sit back and wonder why We got these performers reaching the sky

[00:21:26] Mike: Oh my God, let's go, let's go I'm telling y'all it's a- It's a

[00:21:30] Angie: talent

[00:21:31] Mike: show

[00:21:33] Angie: All right, so please, please submit the form so that we do not have to do this, 'cause you have way better talent than us.

So please, please, we're begging you, submit the form to talk about your talent

[00:21:49] John: Thanks Mike and Angie. That, we'll put that, uh, link in the show notes for anybody out there that has a talent and wants to share with, uh, with all your Drupal friends. Um, you know, that, that would be great. I highly recommend it. Looks like it's gonna be a, it's gonna be a good time. Um, all right, before we jump into our primary topic, I do wanna remind anybody that might be listening live, you can drop a, uh, comment into the, uh, YouTube chat or the LinkedIn chat and we'll, we'll see it here, and we'll be able to, uh, to ask that question live.

So feel free to, feel free to drop that in there if you, uh, if you have questions. All right, Juergen, uh, here comes, here comes the big one. Um, what is Rosetta and what problem is it trying to solve for Drupal?

[00:22:35] Jurgen: Okay. So Rosella is Dries' initiative that he introduced in his Driesnote in Rotterdam. And the idea in a single sentence is de- describe it once, and everything else reuses it.

And that can apply to several things. We have discussed about tools before, and tools is really very well underway, and it's great to hear that you are planning a session with Michael because he has done a great deal in hammering out that API and building something which is, well, citing Dries again, the most important and interesting module underway right now in the Drupal world, right?

But describing it once and everything else reuses it can also apply to things like events. We do have events in Drupal, most of which come from Symfony, and Drupal brings, uh, hooks as well that can be compared to events but are not the same thing. But they are things that can trigger other, or that allows people to subscribe to something that has happened to then do something else.

And events and hooks are great, but they are-- they have not a specific API, so they are not in a common way describing what their input is that they require or expect, and they are not telling anything about what they are doing and what they are giving back once the event might be completed. So having something that allows people to easily specify what an event is, when it triggers, what it needs or what it provides as data, and what it expects to, uh, return back, that would be something that could make things much, much easier.

I come back to that in a minute, but there is two more, one of which is think about forms, config, input and output, and schemata, and type data fits into that category as well. All of those today are defined in many different places. When it comes to forms, as an example, you have a module, uh, that comes with a config schema.

You define all the variables there with a label, with a description, maybe with a data type, and also with some validation and constraints If you build a configuration form for the same thing, you do the same thing yet again. It's almost exactly the same description. You define a form with the fields, with the descriptions, with the validation, and so on.

Uh, if you define a tool, you have to define input variables, and they look very similar yet again. Like, you need to define data type, you need to define label, you need to define a description so that the caller really knows what all those variables are there for. So it's very easy. A- as a maintainer, you have to define the same thing over and over again.

Uh, so that is a candidate to come up with an idea to only describe it once, and then use it in many different places. And candidate number four in Rosetta is tracing. Like, if you have all those engines, modules, external tools, all the, uh, participants in a workflow, each of which, uh, keeping track of what they are doing, what the context was, uh, that's a lot, but everybody does it in their own way today, which is a nightmare.

And if you later want to review or replay a workflow, it's very hard to do that. So Rosetta wants for each of those to make it possible to declare it once, and the name Rosetta actually tells it. It comes from Rosetta Stone, where you have one text and many readers. And yeah, go ahead.

[00:27:14] John: I have so many questions.

So I think, I mean, I think you've, you've hit on, um, a lot, uh, as far as what, what Rosetta is, and I think that it makes, it makes sense. I'm wondering, is it, is it just internal to Drupal, or is this something that's gonna be used externally as well? And I'm assuming it is gonna be used externally because as we were talking about with the module of the week, you know, there are MCP servers and AI trying to figure out how to interact, right?

Um, and then if that is the case, is there some sort of, like, standard schema that we're gonna start using or is it something that we're actually developing and communicating out to services that wanna interact?

[00:28:03] Jurgen: There is not one single answer to that question. That is, uh, different for all the different components that are currently part of Rosetta, like the four building blocks that we know today.

Nobody's saying that's all of Rosetta. While we are working on those, I'm pretty sure we're gonna identify many more. So today, if it comes to tools, events, config, and tracing, each of those are living in different, uh, context, right? Talking about tracing, there are well-established standards today, so we don't wanna reinvent the wheel- Mm

and build something new. Uh, we're gonna look out what others are doing and adapt to those. When it comes to tools, um, I would say what Tool API is already doing is, uh, a nice declaration that everybody can read.

[00:29:04] Angie: Mm.

[00:29:04] Jurgen: But there is no such thing outside of Drupal. Uh, each workflow engine does their own thing, and we haven't found, actually, in the orchestration working group, we haven't found, and we tried really hard, a common, uh, taxonomy that even uses, uh, bespoke terms for the building blocks that make a workflow.

Mm. Everybody is calling that differently So- Got it ... for tools and forms, I would say there is no common standard. Tracing, yes, and events is probably pretty close.

[00:29:44] Nic: Mm. So I- I'm, I'm curious, are these building blocks defined somewhere? Is there, like, a project or a documentation page or...?

[00:29:55] Jurgen: Rosetta is currently a Slack channel with a lot of people in there.

So that channel is open, it's just called Rosetta, and Dries has written a blog post recently about it, uh, I think we have the link in the show notes as well, where he describes the scope as he sees it at the beginning of the initiative, uh, of what Rosetta is. But Rosetta as a working group is open to everybody.

Everybody's invited to join, become part of that Slack channel where we currently do our work, and part of the work is to determine what is the best way to formalize that process, right? So we haven't even defined that one yet. We just know that we wanna address those four topics because those four have been submitted or proposed by individuals in that channel.

But it could well happen that while we talk, there is somebody else in that channel to propose an additional building block. So nothing's written in stone. Uh, it's open to everybody.

[00:31:08] Nic: Awesome. And we, we have a-- So we, we, I think we kinda highlighted this a little bit at the beginning, but Jürgen, we have a question from Carlos, uh, on LinkedIn.

He wants to know what the goal of Rosetta is. Like, what's, what's the end state that we're trying to achieve?

[00:31:25] Jurgen: Um, well, that's a bit like, um If we can engineer out redundancy in Drupal, that would help a lot, right? We talked about it in the module of the week, Maestro as well. There are so many different things that do the same thing.

Like there is Maestro, there is ECA, there is Flowdrop. Everybody is doing it their own way. If we can bring that down to single APIs with no redundancies, with solid interfaces, um, for all the workflow components, and then being adopted by the majority of modules, both internally and externally, that would then help fast adoption, adoption where, um, we are prepared for everything after agents.

Like, you know, today a lot of people are focusing on, how can I make my functionality available for agents, either internal or external? But there is probably a time after them where we realize that agents are probably not the thing. There is something else, and Drupal wants to be ready for that time.

[00:32:48] John: I mean, I, I think you did a great job, uh, a- at least for me, you, you really explained it.

Like, when you look at, like, a module's config versus, like, its config form and the fact that you kind of have to define the, the, the, the expectations for the module in two kinda different places. Like, streamlining that to defining it once and then being able to use it in both places or have both places interact with it, like, really makes, really makes a lot of sense from a, from a module standpoint.

And then, you know, I also understand from a larger systemic standpoint of like, "Hey, let's integrate with an MCP server," or, "Let's integrate with another system." Like, it's a lot easier to just say like, "Here you go. Here, here are all my, here are all my values, my inputs, my outputs, my expectations. Like, here you go.

You can interact with me in these ways," right?

[00:33:42] Mike: Mm-hmm.

[00:33:42] Nic: I, I do have one really big request, Juergen, that please keep this in mind, 'cause there, there, there is also the, um, we, we wanna make sure that these things also have some, uh, some form of failing gracefully. Mm-hmm. Uh, and, and my example, the example that I'm gonna go to right now is SDCs, you know, single directory components.

I mean, single directory components are great. I, I think one of, one of their big benefits is they just reduce the size of your library's file. Like, you, you can kinda just really compact how much JavaScript and CSS you have, and, and you don't have to worry about that. But, uh, in one of the more recent versions of the Drupal, I think 11.4, um, they turned on some validation for the schema For SDCs and that cau- and that causes, um, in a lot of common configurations, a...

You just get a white screen. If you, if you, if you define a particular prop as a string and you send no, for example, or sen- and it's not... Even if it's not required, your screen is just gonna... You're just gonna get a white screen instead of just, like, that prop being ignored or trying to be typed or like...

Mm-hmm. So i- if we're working on an initiative to kind of programmatically define how to interact with things, we do wanna have kind of a... We wanna have some thinking around how to make sure that it, if it's wrong- Mm-hmm ... it doesn't just completely blow up. Because that, that, you know, the, the idea behind making everything validatable and making everything defined is that it makes it easier to interact with because you know it's expected, right?

That, that's the whole purpose of it. I'm not... That's a great goal, but we also wanna make sure that if something... I think that's something that Canvas has been dealing with, too, right? When a component version changes, um, they had to do a lot of work to make sure that the older versions of the components on pages that already exist don't blow up, and then they still sometimes do, I think.

[00:35:42] Jurgen: Yeah.

[00:35:42] Nic: But we wanna make sure we're keeping in mind that use case as well, so, so that you can, you can version it essentially.

[00:35:50] Jurgen: 100%. Yeah, absolutely. Uh, so error handling is a big topic. Uh, validating inputs and outputs is equally important, but invalid input arguments, as an example-

[00:36:07] Nic: Yeah ...

[00:36:07] Jurgen: should not mean that it fails.

It should just mean that it doesn't execute and tell the caller why, and ideally how to correct it.

[00:36:16] Nic: Yeah. Yeah, I, I think that's ideal. Okay, we are 36 minutes into the show, and we're ready for question number two.

[00:36:23] Jurgen: Yay.

[00:36:25] Nic: Sign, this is the sign of a good show, though. Uh, so w- uh, let's say, you know, this is a new initiative.

Let's say we're a year down the road. Uh, uh, assuming that, you know, s- the, the four pillars that you talked about kind of we find some structure, we get that in place, modules started adopting it, how do you think it's gonna change how modules are kinda built and interact with each other?

[00:36:49] Jurgen: Well, today, if two modules wanna work together, let's take the example ECA and Maestro.

What we usually do as maintainers is we talk to each other, and we build integrations ideally in both directions. That's feasible, that's glue code, and that's happening all the time. If then a third module comes along and wants to work with those two as well, like for example, Flowdrop, we talk about three integrations that are required.

Four modules require six, and five modules require 10. Now, that grows very quickly and is just not maintainable Having something like tools means ECA and Maestro don't have to integrate into each other anymore, because ECA can call a tool, and that tool could well be a Maestro workflow. ECA doesn't need to know that.

ECA just calls the tool, and the tool executes and returns according to the contract what was agreed in that API. And that scales up really quickly, because with tools as an examples, we have, uh, readers that are ECA, Maestro, Flowdrop, AI, internal and external. We have Drush, we have MCP, we have external systems, and everything that goes through orchestration.

Um, but what about all those modules that already exist and have some reusable functionality? How can we open the door for all the existing modules and their maintainers to actually leverage that as well and become part of the party? Like, if there is a tool like Smart Date and they have some date validation rules in them, wouldn't it be nice if any of those participants could just call a tool in Smart Date and find out if the date is valid or not?

Now, talking about developer experience, asking every single maintainer like Martin for Smart Date to build and re-architect his module to provide a tool plugin is too much to ask. That would not happen. But what could well happen if Martin has a service in his module that does the validation, if it would be enough to just add some attributes to the existing methods in that service, and the framework then exposes that method in the service as a tool, it will become very, very easy for every maintainer to expose existing functionality to the new framework.

And if lots of maintainers get excited about that idea, we get a quickly growing environment where every module can call tools from all the other modules without knowing anything about implementation detail.

[00:40:16] Nic: The loppy goal

[00:40:18] Emma: Can I ask off the back of that, it feels to me that AI agents are gonna be a piece of this. So do we know yet, or maybe this is still to be worked out, but what role, how will Rosetta support AI agents interacting, you know, from the automation perspective? Are there, like, defined outputs that we're looking for for agents to be able to use as part of this work?

Juergen, can you speak to that a little bit?

[00:40:48] Jurgen: Yeah. I would say an AI agent is just another UI to Drupal functionality. And let's think about an external agent. Uh, for them to be able to leverage all the tool functionality, the only thing that's required is a recipe that installs MCP in Drupal, the OAuth module for authentication, and the tool API.

And that's all that's required, and then every outside agent will be able to find out what tools are available on a specific Drupal site, how to call them, what they are ... First of all, what they are doing, what the purpose is, how to call them, and what data and structure to expect back as a result from calling it.

So the tools bring functionality. Drupal brings the access control, and the user chooses their UI. It could be interactive to do something. It could be an internal workflow or AI, or it could be an external scripted, uh, tool, like Drush or Becht Script or whatever, or it could be an agent. So the agent is in no way different from all the other tools.

I really see it as a, an additional UI or an optional UI where somebody can choose which way they wanna use to interact with their Drupal site.

[00:42:24] John: So I'm still working through this in my, like, in my brain. Uh, I was gonna, I was gonna be self-deprecating, but, um, anyway, um- Huh. So, all right. So let me, let me take this down to, like, you know, explain it to me like I'm a fifth grader.

So in, like, today's world, right, you build a module, and if you want somebody to interact with your module, right, they kinda have to build a se- a sub-module of, in their module to interact with your module's pieces, make your module a requirement, right? In a future state with Tool API and Rosetta and this whole initiative, right, once we get down the path of that, it'll be able to basically scan the module.

If the module's enabled based on the services that are enabled, it'll be able to say, "Hey, we have all of these services that you can use," and then a, another module might not need a sub-module. They would just be able to use that Tools API as kind of a standard communication layer to say, like, "Okay, go look at the Tools API and go see if there's a..."

Um, I'll use your date, your date, uh, example, right? "Go see if there's a, a smart date service that I can use to, to, you know, I don't know, format my date correctly," right? Mm-hmm. And it'll say, "Yep, there is. Here, here's the inputs, outputs, all the things it can do. Here's how you can, you can use it," right? So, uh, and the reason I'm, I'm thinking this way is 'cause, as, you know, Carlos was talking about, um, kind of like making it better for non-Drupal folks, but also making it good for Drupal to Drupal, and I think, I, I think this kind of like- Does that, right?

Because it's, it's making interaction between modules within Drupal a lot easier, but then by doing that, you're also making interactions with the outside world and Drupal easier. Mm-hmm. Right? Did, did, did I ... Any of that make sense? Am I even on the right train tracks here?

[00:44:30] Jurgen: That's absolutely perfect, and I would just add one thing to this, that you don't even have to think about or find out which module is providing which functionality.

You know? There is one pool of tools in your Drupal site, and there is a discovery mechanism that lets you find out what's available.

[00:44:56] John: Oh, so it's completely flat. Like, you could have, like, 100 modules, and it's just like, "Here's the menu. You- Yeah ... order off of this menu however you need to to do whatever the thing is you need to do."

[00:45:06] Jurgen: Yeah. There- And to you as a developer, it shouldn't matter whether the date validation comes from Drupal Core or the Smart Date module. If it's present, it's not important to you, right? S- You just want to use it.

[00:45:21] John: So how does that change, like, m- module dependencies, right? Like, so, I don't know. We'll go back to

Well, we'll j- Martin's not even here. We're just gonna keep using events and dates as, as, uh, as- Mm ... as examples here, right? So, like, you have something like, you know, event platform that requires Smart Date, right? So in this, in this, you know, new world, the Rosetta, Rosetta improved world, like, if the event platform is thinking, "Hey, I know I need Smart Date, and I need that to be available," like, does it still need to define that dependency, or can it just say, like, "Ah- Yeah

we're using Tool API, so whatever date thing we have available is what we're gonna use"?

[00:46:06] Jurgen: Mm-hmm. I would put it this, this way. If you build a deterministic tool like Martin builds the event platform, that's entirely deterministic, and h- the dependencies have to be determined as well and declared, right? Got it.

That tool only works if it precisely describes what are the prerequisites to be able to function.

[00:46:34] John: Got it.

[00:46:36] Jurgen: On the other hand, if you use AI- It's totally non-deterministic, and the AI is built to determine itself what's available and how can I use it.

[00:46:51] John: Got it. Got it. Okay. That makes, that makes sense. So like, so,

[00:46:54] Nic: so the tool API is providing a language that says like, uh, "I'm a date field, I accept these things in this situation," or the smart da- "I'm a smart date field, I accept recurring rules."

But th- there's still a couple of ... I think there's an architectural thing I'm sure you guys have talked about, but I'm, I'm curious about, which is a lot of the glue code is about, um, interception points, which as you said are, have typically been hooks or events. I don't see how ... The, so the tool API can describe those intersection points, but those, the tool API can't interact with those, right?

Because to interact with a hook, you need to implement the hook, or to interact with event subscriber, you have to create one. So you're still gonna have code that is specific to particular modules if you're trying to modify the behavior.

[00:47:51] Emma: Mm-hmm.

[00:47:52] Nic: Um, so it's, but it, but it, from what I'm hearing, the tool API is more about what are the current capabilities, maybe these are integration points, but it's not sidestepping that glue code all that mu- or, or am I misunderstanding something?

[00:48:07] Jurgen: No, you're absolutely right. Uh, the glue code is d- is, uh, driven by the specification of each tool. Like, a tool is telling you, it's like a method, uh, in a class that tells you what the input arguments are and what the return values are, and what the purpose of the method is, but you don't look into the code itself when you are developing something yourself that leverages such a tool.

You're not looking into the code itself line by line to find out what it does and how it does it. You rely on the description on it solely. And if you, let's say, use ECA, for example, to build, uh, a workflow to modify Drupal's behavior, you can call tools only by their description knowing that this is what they do, and you don't have to develop it yet again yourself

You're muted.

[00:49:15] Emma: Oh, you're muted, John

[00:49:16] John: Sorry, I was using the mute button so you guys didn't have to listen to me kinda like sick breathing. But, um, so yeah. Okay. So like the, it's, it's kind of like a little bit of a, um Like, it, it's like just a little bit of a, a, a blur layer on the underlying services.

Because ultimately, like, if I'm using, like, Easy Email or Mail System or whatever, like, the requesting service doesn't really care. They're like, "I just need to send an email. Like, c- can you do that?" "Yes, I can. Here's how." "Okay, great. That's the thing I need to use." Yeah. Send that email, right?

[00:49:59] Nic: Um- Yeah.

[00:50:00] John: I,

[00:50:00] Nic: I think that's the piece that, that I think is, is really powerful.

[00:50:04] John: Yeah. No, that's super, that's super, uh, super powerful. And I, and I think it, it adds that kind of abstraction layer, right? That says like, "Hey, listen, I don't really care what the underlying tool or module or thing is as long as it can do this task," right? Can it post something to Google Docs? Can it post something to the internet?

Can it send an email? Can it, you know, fill out a web form? Can it place an order? Whatever. Whatever the, the, the thing is, right? It just kind of gives that, that ability to, um, provide that interaction layer. Uh, you know, I think the, uh, obviously the, the, the moniker Rosetta makes a lot of sense there because you are definitely providing a, a common language to systems that might not necessarily understand like, "Oh, you're using Easy Email or Drupal Commerce under the hood.

And, like, how do I interact with that?"

[00:50:58] Nic: So if, if we shift gears for a second, h-how... So you, you mentioned the four, four pillars. One of them was kinda debugging or tracing, right? How are you thinking about co- creating kind of a single trace across the multiple systems and, and how will that make debugging issues easier?

[00:51:19] Jurgen: Well, don't we all know that reviewing logs is painful? Like, you know, if something misbehaves on your Drupal site and you have to look through the logs, it's just not telling what you are looking for in a meaningful way. Now, if we talk about Rosetta and multiple engines working together, that would mean reviewing five logs from five different engines, and manually linking together and finding out which one called which other one, and in which order the log should be interpreted.

In ECA, we have built something like that earlier this year, and we have learned a lot of lessons, because a log trace also has to come with state information. Like in ECA terms, there are tokens available at every single step, and those tokens are very important if you wanna debug a misbehaving ECA model.

Now, storing all that data together with the trace, we have seen, uh, situations where users reported 700-plus megabyte traces for a single page request. And we have learned a lot from that experience, and we went through several iterations, and each of which brought us 90% or more savings, both in processing and in storage.

Um, and that turned it into a real production-ready solution. Now, we have to talk about storage here as well, like storing in a SQL database, like what we do by default in Drupal is, in most cases, not really performant, so we need to think about external storage solutions, and there are great candidates available.

And before you ask the question, no, we are not inventing a new standard here. There is a W3C, uh, recommendation for all that, which is, there is open telemetry, which even has great support inside Drupal already as a Drupal module at this stage. Um, and that assumes s- external storage. Well, that brings certain difficulties to us because we don't just wanna store the traces and then go through them externally.

In ECA, for an example, we have built a replay functionality, which is really, uh, well, well-received by all the users. They really love it because it's very nice in the Drupal UI To go through, see what happened, and find out what needs to be fixed to behave correctly. That's only possible if we take the trace and visualize it inside Drupal.

Whether that means we have to read back the externally stored data into Drupal, or if we have to find a way to mirror the data for a short period of time to make it available for that interactive, um, scenario for, for the user while they are debugging stuff, that's not yet determined. And that's part of, uh, the existing standard that's not finally defined anywhere.

So we wanna contribute to that standard to solve Drupal use cases. And that would be one of the other success criteria for Rosella if we are actually achieving that, that the existing standard, uh, um, takes on our requirements as well and finds a solution for them. Uh, other things, like Randy has a great example.

He always talks about, like, if you have a Maestro workflow, and the end user at 3:00 AM in the morning, uh, the next task is to upload an Excel spreadsheet so that the workflow can continue. And by accident, they just don't pick the spreadsheet, but the picture of their baby, the whole workflow after that is failing.

Now, what we wanna do then is use the trace, A, to find out what, what went wrong, and ideally restart from that point with the same context and continue playing the workflow. So that's another benefit, hopefully, that tracing and debugging will allow us to do so that we exactly know what was the state at a certain point in the workflow, correct the mistake that maybe the user made, but it could have been another error as well, and continue processing from that point on.

[00:56:31] Emma: I guess a point related to that is how How we can handle, and I think this is, this is something that applies to whatever you're doing with workflows, how do you balance the transparency with exposing the data while, like, there'll be points where there's sensitive data. So how... I guess, like, the point of Rosetta is opening things out, publish once and then it's available.

But within that there will, I don't want to use the word guardrails, but there will be need, the need for sort of sensitive data to be handled. And maybe this is something that comes beyond, you know, making it work, but has, has that been considered as yet?

[00:57:17] Jurgen: It has. And I would even say it should be, uh, considered and solved right from the beginning.

Yeah. From the get-go. As an afterthought will most likely fail. Uh, the good thing is that the OpenTelemetry standard has conventions, uh, for that, in particular in the context of generative AI, where OpenTelemetry is used as well. It looks like all the AI companies have consolidated behind that standard.

They are using that all themselves. Uh, and one of the conventions is, for example, especially because we store data externally, most likely, uh, that large or sensitive data is stored separately, if at all. It could be filtered out upfront, but most likely it's stored somewhere separately where only limited access is granted to specific users, and trace itself only stores a reference to the sensitive data.

So there's the- That most players are doing it today. Whether that covers everything that we need is unknown at this point, but I think it's a good starting point.

[00:58:34] Emma: So it, I guess it's about preparing for Rosetta. You know, there'll be a piece to- Yeah ... to do that to, which is something Drupal handles very well, right?

But it's, it's again, making sure your setup is there to handle the telemetry.

[00:58:51] John: So-

[00:58:51] Nic: Well, and... Sorry, sorry, John.

[00:58:53] John: Go ahead.

[00:58:55] Nic: I, I just wanna add on to what Emma just said. I, I think it's not just about the telemetry though, right? It's also about the actual access to tools and things, right? So you need a way to be...

'Cause, um, to Emma's point, like, the date, the date may not, generically isn't sensitive, right? But maybe the date of publication of something is sensitive, or maybe the date, uh, some other... So, like, you need a way to be able to say, "Hey, this is sensitive. Even though this is an open API defined by this module, it's not on this site.

Do not ever reply to any requests about that." Um, is that, are, are you thinking about both sides of it, or are you, are you just talking about the traceability side?

[00:59:38] Jurgen: Uh, no, no. Uh, everything, uh-

[00:59:41] Nic: Okay ...

[00:59:41] Jurgen: in, in

[00:59:42] Nic: Rosetta- I, I assumed so, but I wanted to ask- Yeah ... be,

[00:59:45] Jurgen: be explicit.

[00:59:48] John: So clearly, I mean, this, this all sounds pretty straightforward.

I'm sure it's, I'm sure it's super easy, right? And you guys have already gotten it all figured out, and it'll be, you know, it'll, we'll roll it right into Drupal 12, and it'll be, it'll be good to go. Um, obviously I'm kidding. Uh, this is super complicated. I'm wondering if you can highlight what the biggest challenge you see ahead for this is, and, and then just talk a little bit about how the community can, can help to contribute to kind of push this, push this forward or, or over the finish line.

[01:00:26] Jurgen: Yeah. Well, in, in my view, tool API is the most important part to get started, because everything else just surrounds it afterwards. But I would say that this part is really well underway. There is Michael Lando, there is, uh, Matt Clement, who is working on that now as well. Uh, and both of them are doing an amazing job.

Um, where they do need help in other areas, or we do need help, is with specifying, you know, regarding events, settings, and tracing. Uh, specifying and then developing, and not to forget the user experience. And in this case, maybe also developer experience, which is not necessarily the end user, but developers are playing, are, are really a target audience for that kind of development as well.

Um, and who knows which other building blocks we're gonna identify along the way. Um, where people can come in and are more than welcome today and tomorrow is, for example, an issue that Dries opened over the weekend, or was it last week? Can't remember. Time flies. Uh, I've put the link in the show notes as well, which is about settings, and he really asks for feedback on that.

How can we move forward in getting rid of all the redundancies when it comes to settings, config, forms, schema, validation, and so on and so forth. That's a big issue. Don't be afraid if you go into it. Uh, just the, uh, intro is really, really long, but there is a lot of meat in there. And other than that, every helping hand and/or brain is very welcome.

The best place to get involved is the Rosetta channel in Drupal Slack. But also if you are wanna work on your own and if you are a module maintainer, as I said before, there is now a way that you could expose existing functionality in your modules to make them tool API compatible. So I would like to call out to everybody who is interested, just give that a try and see how it works, and tell others about it so that others can actually give it a try and call it from their modules and see how things gonna interact with each other.

[01:03:02] John: Are there any, uh, uh, are there any modules that are already tool API enabled? Like, are, are the A- is the AI team kind of building that functionality into the AI modules? Or, um, is there a list of, like, tool enabled AI modules that we can, we can see?

[01:03:20] Jurgen: Um, other than those that we already mentioned, like ECA, Maestro, and Flowdrop, I actually only know about a custom module that lives on Dries' website-

[01:03:33] Nic: Mm

[01:03:33] Jurgen: where he provided search functionality for his pictures on the website.

[01:03:38] Nic: Yep.

[01:03:39] Jurgen: Um, honestly, I don't know if there is- Okay ... anything else yet that is... Well, we also have to remin- remind ourselves the tool API is currently in beta 12, if I'm not wrong. But it could well- Yeah ... be that the first release candidate is coming out maybe this week or next, and I think this is when things consolidate and where the API is then stable, and that's the point where people can really jump on it and, and use it.

[01:04:13] John: Makes sense.

[01:04:15] Nic: Hm. Interesting. So we, we kind of asked this a little bit earlier, but let's, let's be a little bit, uh, I think about one aspect of this, but let's be a little bit broader. What does success look like for Rosetta? 'Cause I, I, I don't think it's like many initiatives where if it takes a few years to get there, it's fine.

I, I, this feels like something that needs to be... I- if it's going to be helpful for the community long term, it needs to either be accepted or rejected rel- relatively quickly, right? And find some other solution for these types of problems. W- what does successful look like, and how quickly do you think it needs to be adopted?

[01:04:59] Jurgen: Adaption speed, in my view, is the number one success criteria actually, because, you know, we find ourselves in a world where the surroundings are changing so quickly. We wake up every morning in a different context, it sometimes feels like, that to stay relevant with Drupal means we have to put ourselves, i.e.

Drupal, in a position where it can be used by that fast-moving, uh, world of tools and processes and so on and so forth, ideally by Orlando, I mean DrupalCon, right? It will not be finished by then, that's for sure. But we wanna provide something that makes difference as quickly as possible, because this is what today's world is asking for.

[01:06:00] Nic: Hmm. That's quick.

[01:06:04] Jurgen: Yeah.

[01:06:07] John: Well, Jurgen, as always, thanks for joining us and, uh, educating us about, uh, Rosetta. I feel like, um, y- we definitely checked the box. I now know exactly what it's, uh, supposed to do, and, um, I look forward to seeing the progress that, uh, the Drupal community can make

Don't forget, you can reach out to, to us on Drupal Slack in the Talking Drupal channel or with, uh, email via [email protected]. To promote your Drupal event, sponsor an episode, or become a guest, visit talkingdrupal.com and click the buttons in the sidebar. And thank you patrons for supporting Talking Drupal.

Your support has kept us talking for now 578 episodes. Juergen, if folks wanted to get ahold of you, uh, talk about Rosetta, talk about ECA, talk about any, any of the great stuff that you're doing, how could they best do that?

[01:07:01] Jurgen: Uh, best place is Drupal Slack or a few social networks. My handle is the same everywhere, which is, uh, Juergen Haas, J-U-R-G-E-N-H-A-A-S.

[01:07:13] John: Perfect. Ama, what about you?

[01:07:16] Emma: Uh, you can get me on Slack. I'm just Ama Horel. That's

[01:07:19] John: the best place. There you go. And Nic?

[01:07:22] Nic: You can find me pretty much everywhere at nicxvan, N-I-C-X-V-A-N.

[01:07:26] John: And I'm John Picozzi. You can find me personally at picozzi.com or on social media and drupal.org @johnpicozzi. And if you're interested about EPAM, you can find out more about, at, about, of, but yeah, there you go, about them at epam.com.

[01:07:43] Nic: And if you've enjoyed listening, we've enjoyed talking. See you guys next week.

[01:07:46] John: Yeah, have a good one everyone.