Talking Drupal #529 - MCP, Automators, and Agents

Nic: This is Talking Drupal, a weekly chat about web design and development from a group of people with one thing in common. We love Drupal. This is episode 529. P Automators and agents. On today's show, we're talking about MCPs, AI, automators, and AI agents with our guest Marcus Johansen. We'll also cover AI ecosystem recipe as our module of the week.

Welcome to Talking Drupal. Our guest today is Marcus. Marcus is the tech lead of the AI initiative and works as the senior developer at Freely Give. He's one of the maintainers of the AI module and also some other modules in the AI ecosystem. Marcus, welcome back to the show and thank you for joining us.

Marcus: Thank you for having me.

Nic: I'm Nick Lham, founder at Enlightened Development, and today my co-host are Martin Anderson, klutz, principal Solutions engineer at Acquia. Thanks, great to be here. And as usual, John Picozzi, solution architect at EPAM.

John: Hey everyone, just wanted to shout out if you're in the US today, it's Veterans Day. Thank you to all of our veterans out there for you know, protecting our civil liberties and you know, keeping us, keeping us all safe. Really appreciate the the efforts and the sacrifices that you guys make.

To, to do that.

Nic: And now to talk about our module of the week, let's turn it over to Martin, a principal solutions engineer at Aqui and a maintainer of a number of Drupal modules and recipes of his own. Martin, what do you have first this week?

Martin: Thanks Nic. Have you ever wanted to explore the AI capabilities of Drupal but didn't know where to start?

There's a Drupal recipe for that. It's called AI Ecosystem Recipe, and it was created in October of 2024 by today's guest Marcus Johansson of Freely Give. It has a 1.0 0.0 alpha two release available, which requires Drupal 10.3 or newer. And because it's a recipe, it doesn't really have sort of all of the same qualifying metrics that we usually talk about, so it does se seem to be actively maintained.

It does have two open issues, both of which are bugs, but we don't really have any usage data at the moment. Now when you require and apply this recipe on your Drupal site, you'll be able to start working with a variety of LLMs and specialized AI-based services. You'll be able to do things like ingest unstructured content and map it to structured fields automatically, or generate a detailed SEO analysis of your nodes.

There are multiple translation tools, crawler to help work across entire sites and more. Now, this recipe is something you would probably want to apply to a sandbox site to understand the various ways to achieve something specific with AI and Drupal, and then apply whatever is best for your particular use case to your actual site build.

But it's a useful resource for Drupal List wanting to start exploring some of the growing list of options for working with ai or someone familiar with AI tools, who wants to start using them with Drupal. So let's talk about AI ecosystem recipe. And Marcus, maybe you can kick things off by telling us about what inspired you to create this.

Marcus: Yeah. So basically the idea behind it was that AI was moving so fast. We were adding so many providers. There were so many people, people contributing to the, the system that was actually where it is actually hard to keep track on what, what you can do with the AI module. So the idea was basically, Hey, let's take everything that we, we know exists and put it into our recipe and try to keep this updated.

We haven't kept it that updated to be honest. But the idea is that we're gonna continue working on this, updating it and basically, as you say, it gives you a sandbox. You know, you, you download it on your sandbox and you can start testing around everything that exists around ai. What providers exist, what kind of functionality can you solve with this?

And

John: hmm. I'm super interested in the unstructured content to structured content mapping. I think that's, that's pretty interesting. I'm gonna definitely at some point this week try this out. I'm just wondering, so recipes, recipes are kind of like, you know, apply once and, and you know, keep carry on, you know, go down the path yourself.

I wonder, I mean, obviously you can reapply it, but I wonder what that looks like for future updates of it. Any, any thoughts on that?

Martin or.

Martin: Marcus. So I would say to the extent that a newer version of a recipe might add, let's say, additional modules, then sort of, you know, running composer updates depending on whether or not the recipe was unpacked, if you got sort of those additional modules and then the configuration for them than reapplying, the recipe could give you sort of additional features.

But yeah, to me, unpacking might be the complicating factor in that scenario. It,

Nic: I, I would say in general, recipes are not meant to be reapplied. By default fair, they will, they will blow up because they, unless they use config actions for every single one to provide it if this already exists, don't create it then.

Or update it, it will blow up. Most recipes don't it, it's a lot, lot of extra work generally to make them so you can reapply them.

Martin: Yeah, I mean there is that concept of item potency, which is exactly that. A recipe should be something you can reapply, but I agree it does, does require sort of changing your approach if you want that to be something that'll actually work in practice, particularly as the recipe might change over time.

Marcus: Yeah, so I, I would say in this case it's, it's mostly for developers to try it around, which means that if you actually want to try it in half a year, you should probably set up a new sandbox environment if it has updated and install it again.

John: That's kind of where I was, I was getting to, in my head, I was kind of like, oh, well you could just like ditch your, ditch your sandbox and reinstall it every time you wanna try, try an AI thing I a jigger.

Right. So like, I mean I think that's, that's totally reasonable. I was just I was just curious. I'm super interested in in giving this, giving this a whirl as you know, there, there that list of, of features is, is super interesting to me. Especially the translation piece of it too. I think that's, that's kind of interesting.

Or the SEO analysis. I mean, there are so many interesting pieces. I just gotta, you gotta get in there and try it out.

Marcus: I, I would say on top of this actually especially one X internet has been working really hard the last couple of weeks to create production ready recipes to use on your website.

Mm-hmm. So if you look in the AI initiatives issued to you would be able to find issues for these, that links to the actual recipes. So some of the the things that this recipe sets up for you in a development fashion, we actually have solved in our production fashion.

John: So that's, that's awesome. I feel like more, more recipes, making these things easier to configure and set up is, is important.

One question when Martin was describing this, he said, you know, obviously it's a sandbox recipe. You install it, you play with it, you can kind of figure out how these things are built. Is there good documentation to kind of tell you how these things are built? Because I feel like once you get this recipe installed, you're gonna be like, oh, I really wanna reapply, like, to my own site, the SEO analysis.

And then you gotta kind of like dig through to figure out what modules were installed and like what, like how does that, I mean, other than, I guess you could go look at the recipe files to say like, here's here are the, the modules that it installed and the configuration steps that it did. But like, is there another way you can kind of like, I don't know, dissect what this is doing once you kind of say, Hey, I wanna move this feature over.

Marcus: Yeah, so, so the recipe itself is actually very vanilla. It mostly installs modules for you and doesn't do any specific setups. So in the SEO example, if I remember correct, this is an actual module that just kind of works out of the box by installing it. So there aren't any necessarily any extra steps, but when it comes to documentation, that is of course, depending on the module that you're trying out, how well that is documented.

For, for the AI core and the, the modules that we maintain, we try to document it as well as possible, but there are of course, contract modules that might not have ample documentation.

John: Hmm. Makes sense.

Nic: Whoa. Thank you, Martin. As always, you found another on-topic module of the week, and I know you're sticking around for the show this week, but if listeners wanted to suggest a module, week or chat what's the best way for them to do that?

Martin: We're always happy to have conversations about modules of the week in the Talking Drupal channel of Drupal Slack. Or folks can find me directly as mandclu on all of the Drupal and social channels. And by the time this comes out, I will also be at Drupal Con era. So if any of our listeners happen to be there, by all means stop by and say hello.

It would be great to interact with people face to face as well.

John: Martin, how long of a flight is that for you?

Martin: The main flight is gonna be 14 hours.

John: Man, this guy, this guy really loves Drupal.

Nic: All right, so there's a, I'm gonna say new term, even though it's been. In the AI ecosystem, I guess it's been a few months, and that's eons. There's a new term that's been floating around model context protocol or MCP servers. What are those and how does it fit into the Drupal AI ecosystem?

Marcus: Yeah, so to explain that, I think maybe we need to take a step back and talk about tools and how tools works together with ai.

So if, if you go way back to when chat GPT was released if you were trying it out, then you might have noticed if you, it, it could kind of answer stuff. It was hallucinating. But if you a asked it about something current, like, you know, what is the weather in New York? Us either it would just. Tell you, well, I don't know.

Or it might hallucinate. Also you, you would probably notice if you asked it like do addition on a big number, like two really big numbers, ask it to do this. It would just also either hallucinate to tell you that it can't do this because large language models are not meant to do this, basically. And what they realized fairly quickly is that this is of course solved on computers having API where you can ask for the weather or, or using a computer to do some simple math that is very simple.

The only thing they needed to make sure was that that large language model could basically call a tool in a standardized way or at least a way for that provider. So you might have noticed after some months if you asked for the weather or if you asked this math number. It showed some books where it was running either Python code or running some tool.

And what it's doing there is basically it's calling a tool that has been defined in some way with input definitions and output definitions. And it, well, it usually gives back text then when it comes to lms. So what would happen is that when you are asking it to, to do this math number, it responds that I want to use this tool, this tool runs, it, gives it back into memory and then it can answer, well, the real answer to this is whatever it is in a semantic way.

And there is another version of this thing called function calling and function Calling is something we use a lot in agents in Drupal. So what function calling does is that you define the functions and you give them to, to, to the LM and the LM can choose to use this. So if you take the example for instance of you want to know what content types exist on your Drupal website, you might have a, a function called then that is called something like get bundle type, and it takes a entity type as the input.

You ship this together with a question, what, what no types exist on my website and this is gonna answer back then instead of some texted answers, you'd like, use get bundle type and use it with input note. You run this on your system, get the answer from this, and you send it to the LLM and the LM can answer it.

Nic: So in in home automation. This is, and I think it, this is just purely the LM portion of it. If you do like text to speech or something recognition, there's something called intense, right? Which is like, it, it breaks the tokens down into like, what is it, what are you actually asking? It sounds like what function calling is is just the intent piece of it.

You're saying, Hey, if you, if you find this intent, here's how you answer that intent.

Marcus: Yeah, exactly. So, so it's just a structured way for that system to know how to use it. Now in Drupal ai, we have abstracted this away. So that's why you can use this in different providers. Then the interesting thing then about MCP is that Anthro came up with a, a, a, a way.

So Anthropic is the, the company behind cloud. The provider came up with a standardized way, how you can set up servers that runs tools. So how do they define the input? How do they list the tools they have and how can they define the output? There are a lot of other levels, like resources. You can use it for comp, but I think tools is basically what has been popular.

And what, what they claim to be is the USBC standard. So that's how they define themselves. The USBC standard for two, because basically before USPC standard, everyone was using, you know, different power cables. I remember myself, I was in Bristol like 15 years ago and lost. I forgot to bring my power cable for work.

And, you know, it took like four hours to find something that actually worked to charge my laptop. And nowadays you can just buy a USB charger, right? So their, their intent was basically to try to solve this initially when it was released. A lot of people were skeptical, of course, because it was one provider doing this.

So there is this famous XKCD, you know, where, where someone is saying like, there's 14 standards. Oh, this is really bad. We have to create one standard for everything. And, you know, the, the result is now there's 15 standards. So, you know, this was what people thought it would be. But I think it was around Groupon, Atlanta basically there, there was this massive massive amount of companies joining and OpenAI was one of the companies.

And that, that's when you realize that it was actually serious. And Dr. Drupal actually had MCP server from day one. So there's two Georgias from Georgia. That has been working on the MCP server from day one, and they, they were really forward thinking with this. So Dral has been one of the MCP servers you could have.

John: So that's interesting, and I'm gonna ask a question about that in a minute, but I keep thinking about this, like, as you're talking about like functions and protocols and stuff. I keep thinking about this in like Star Trek terms, which I can, I can see Martin laughing over there because like Hmm. Like I just, I'm like, oh, like this feels like, Lieutenant Commander Data where he is, like he's the LLM and like when you ask him a question, he then has like subroutines and sub subprocesses that he kind of runs to get you the answer.

Right? I'm like, like, oh, we're almost, we're almost there. Not really, but sort of. So, that's, that's kind, that's kind of the way I'm thinking about it. But going back to what you just said. I'm wondering how CPS help to bridge that gap between Drupal structured content and like the broader AI landscape.

And you just kind of answered this question a little bit by saying like, Drupal could be an MCP server but are there like other ways that like Drupal is bridging that gap and, and kind of making, making those connections easier?

Marcus: Yeah, so, so I was going to say, you know, if you have these servers, so one thing is the Drupal as the server.

One thing is that you can then very easily either download like open source server that exists, that runs in some other language. In this case, you can be language ag agnostic because the MCP protocol takes care of how whatever you have talks to another language. So if you take the example, when you say language, do you mean like programming language?

Programming language, exactly. I run it. So, so a lot of these servers running OJS. And there is also because it became so popular, there are also remote, fully remote servers now. So like PayPal has a remote server where you can just, you know, point your MCP client towards this and run. So in Drupal we also have experimental MCP client.

So just take example of what this can solve then, is that you, you could install the official AFR slack MCP server, for instance, and say that you want to send a Slack message every time. Enter the type is saved. The only thing you need to do now then is connect this server to the sorry, connect the client to the server.

In this case it's using server to server authentication, so it's already authenticated, but there is a whole protocol on top that could be run as well. Mm-hmm. But you connect these and these tools becomes available. And we have something that's brewing now that's called Tool API. So there's a guy, Michael Lander, that has put a lot of effort into this where it, these things that you see in the MCP client doesn't just become available for ai, but rather for the, you know, wider Drupal ecosystem, which means that you can encode now then every time an entity type is saved, you can send a Slack message just using one role or code using a plugin manager to call a tool and like using a standard ICE way of calling this slack message.

Or you can use ECA. So ECA could trigger one of these tools then as well. So it's something that you can use outside of the, the whole ecosystem of. AI as well.

John: So it's kind of going both ways. You could choose to use Drupal as an MCP server to send data out to other, other ais looking for that information.

Or the reverse is Drupal can connect to MCP servers to pull and do different, different things basically based on exactly whatever you're doing within, within Drupal. Oh, that's interesting.

Marcus: Just to answer that, so I actually did a, a, a demo at Vienna.

I also did the same at Texas Longhorn where I wanted to see how far you can get with agent to agent communication because agents can use MCP to communicate as well. They are just another tool. And there is this voice agent system called bpi that you can connect to Twilio, where you can basically, you know, have a agent calling you or you can call a agent.

So, me, me and someone named Nick Operas, we, we built a web form agent that has a lot of tools for how you can manipulate web forms, where you can then use the agent to ask in free text how you want to create a web form. It goes ahead and creates this, but then we can also, in this VPI agent have a m MCP connection back to our Drupal server where it can fill in actual fill in the actual web form.

And what we could set up with this was basically that our web form agent could ask to call a person to do a survey. On our web form we set up, so what's happening there, our MCP server calls the VPI agent and says, call this person and run on this specific web form. The VPI agent uses the MCP tools that we have on our Drupal website to get the questions in the web form so it knows what to ask, and then when it's finished actually asking this, send a submission.

So this is a way where you can have bidirectional communication using MCP and this, I think this, I mean, we had to program all the tools for webform. That took some time, but setting up the agent was under two hours.

John: Wow. That's, there's a lot of use cases for that. Yeah. I'll tell you.

Nic: Well, the, the interesting thing here too is that your, aside from the tooling piece, which is what mcps.

Like, like you said, the the MCP function pieces, letting computers do the part that computers are good at and LLMs are good at understanding intent and what people are saying. Right? So you're kind of getting the best of both worlds. You're getting the pieces that, that are actually strong rather than trying to calculate a function or calculate something with an LLM, which they're not built for.

You're, you're kind of, that's actually, that's pretty clever. I was gonna ask, kind of talking through this piece a bit. How complex are MCP servers? Not to set up like an existing one, but to build one out. You said building the tools for webform took some time. Is it, is it something that takes, like some really niche experience?

Is it just a matter of reading some documentation and writing? You know, how, how hard is it to set one up yourself? A custom one.

Marcus: So, so do you mean to actually just set up a web sorry, MCP server or to, to create one?

Nic: To create one. To create one? Because I, I imagine a lot of Drupal sites are gonna be a mix of both, like setting up, connecting to an existing one, but also a bunch will have to set up kind of own tooling and own MCP.

Is it, is it something that you need to be pretty experienced to do? Or is this spec kind of straightforward and simple?

Marcus: I mean, if you're gonna use pu as the MCP server, there is a module that you can just install. So it's under the namespace MCP. There are actually two solutions to this. I haven't tried the other one, but material from Lulla Bot has basically created MCP wrapper around JSON API and JSON RPC as well.

That takes the advantage of, you know, having access to create. Create any entity, list an entity via the adjacent pi. So if you're just gonna set it up, it's fairly simple. Creating the tools is also fairly simple. It's just a plugin system. So it's, it's usually one file that you have to create to create one tool.

The, the big issue is if you're gonna use them, I think with agents you have to think about things like context. How do they answer? Okay. You can't have too many tools. Basically around 15, 22, the agent just starts hallucinating because it, it's too much for it to handle. Hmm. So that is complex to actually create an agent around them.

But the tools themselves are very simple. It almost goes back to, you know, procedural programming because it is functional, it is literally function with some input. Execute, create some output and that's it. Okay. So, I mean, we, we have we have input definitions and output definitions around them to have some kind of structure.

So, you know, a tool like ECA or something else can know what it actually, what this tool is looking for and what it's gonna output. But in theory we, I mean, in theory we have actually discussed if we're gonna create a derivative of this, where you can just create the procedural function. Hmm. So literally just a doc block, procedural function, and then you have your tool which I think is a good way of getting, getting people that has their product in something else, that Drupal maybe uses Drupal for documentation or for their webpage or something to have a very simple API.

'cause that's how you can use cp. That

Nic: sounds, sounds perfect. Like a perfect use case for hooks. Right,

Marcus: right, right. Yeah.

Martin: Marcus, you mentioned that the, the tools that in some ways were kind of written with the intent of supporting MCP in mind could also work with other things like ECA. And I know the project page for the tool API module that you mentioned actually mentions that tools in some ways could be considered an upgrade to Drupal core actions.

Can you tell us a little bit about sort of the differences between those two and, and why the benefits are important for MCP?

Marcus: Yeah. So I, I think the two most important thing was basically to get. Input definitions and output definitions for, I mean, it's important for ECA currently is using the config system, right?

But it's, it's important for it to be able to get like, definitions, clear definitions of what inputs it expects. And it's also important to know what outputs to expect. From AI point of view. It's even more important with metadata. So things like an actual good description of what the, the expected input is, an actual good title or what the expected impetus we have seen when we are tried out, even when you're defining a function call, you actually set the function called name.

Even this is important to, to not confuse the ai agent. So, so, those are some of the improvements. And differences compared to this is the current actions,

Nic: this is the phenomenon that I've noticed. And Marcus, maybe you know the answer, the, the answer to the question of why this is happening, but let's talk about the metadata for a second, right?

Naming things well, providing good metadata and descriptions is helpful for human developers, but it's almost impossible to get developers in general to just by default, write good documentation, write clear expectations. But when it comes to AI, for some reason, people are like, oh, AI needs this good description.

I'm gonna spend four hours writing great documentation for this one function. Why do you think people are willing to do that work so that AI can interact with it? But it was such a battle to get people to do it for other, for even themselves. The number of times I've gone back to a function that I wrote a week ago and went, Hmm, why did I call it this?

And what, what was I thinking?

Marcus: Yeah. Why? I, I think actually people do spend a little bit time on it, but maybe not in the structured way that developers and AI is looking for. So some, something we noticed early on when we were doing the first agents, one of the first agents was like you know, creating fields on a content app, you should be able to ask it to create fields.

And one, one of the things that you need to fill out there is the configuration for that specific field type. And, you know, in the beginning we thought, okay, this is super simple. We just use the schema. The schema definitions is perfect. Drupal is super structured. The problem is that the descriptions in the schema definitions of some of the field types is, you know, the description is basically the label.

So AI couldn't use this. But what what we noticed was that, anything that was basically what's the name of it? Plugin configuration form interface, I'm not sure, but where you can define a, a, a form for a interface if we instead had the AI look at the form. That was actually almost always very well defined, because I think that the, the developers, and that's probably true for me as well, at least before I start working with ai, spend a lot of time to explain it where the actual end user sits rather than maybe for developers.

And that this actually worked very well, because the descriptions are usually very well written.

Nic: Interesting. Yeah. That, that's interesting because it's true. Like if it's something I think nobody's gonna see, I'm not gonna spend, I like, I'm guilty of this too. I'm not, I'm not out here pointing fingers. But it, it, it's a phenomenon that I've noticed.

Same thing with SEO, like. Having the discussion with clients over and over again. If you want better SEO in this particular term, you should write content that talks about that particular term. But when it comes to ai, they're like, oh, we need to rank better in ai. We need to rewrite all this content. It's like, I've been doing that for five years.

But we need to invest in, well, I mean, really what they should do is

John: have AI re rewrite their content, but that's another show.

Martin: I will, I will also add, I listened to a podcast a couple weeks ago that was called something along the lines of, you should Treat your spouse like ai. And the point that they were making is when people create prompts for ai, they're very specific in terms of what they expect, give them a lot of context and are like super clear about everything.

Whereas oftentimes, you know, when people are communicating on sort of an interpersonal level, they'll just sort of like glibly ask for something and expect the other person to sort of like, you know, anticipate all of the different things that you had in mind. So I thought that was sort of an interesting way of, you know, reframing how we talk to each other.

John: I'm gonna try that tonight with my, with my spouse and see how it goes because I, I definitely, I definitely do that. And the, the, you are absolutely correct. I, I often say things with very little context and just look at me and go like, John, just

Nic: one, one quick piece of advice. Don't add the prompt part where you say, don't hallucinate true.

Give me true answers only. That's funny because I

John: actually never ask AI to do that. Maybe I, maybe, maybe my relationship with AI is very different, but I'm always kind of like, here's what we're doing, here's what I need from you. And like typically it's creative tasks, so it's not like I need need it based in a lot of fact.

But

Nic: yeah. I, I, I think, I think that's a good point, Barton, by the way, which is it that I think one of the, one of the good things that's coming out of AI is people being more intentional, intentional about what they're asking for, right? I, I'll have to think about this some more because it's, I think it's definitely a positive and something that maybe I could consider.

Moving to other areas like issue summaries or slack messages. Like the amount of times I send a message in Slack and somebody comes back with, what are you even talking about? It's like, maybe I can start front loading that myself too. It's interesting. So we, we've talked briefly about mcps. We've talked a little bit about agents.

Can you tell us, Marcus, what AI automators are, especially in the Drupal context? How, how are they different from kind of typical automation tools or kind of on jobs, background jobs, that kind of thing?

Marcus: Yeah, so the automators is basically where it started for me, where I well kind of came back to Drupal.

I was doing Drupal for a long, a long time, a long time ago with a project that was called AI Interpolator at that time. And the idea was actually pretty simple. We, we were doing this. So I was working at a PR company that was working a lot with crisis communication. And one of the things in crisis communication is that you want to notice that a crisis is brewing, which means that you have to, you know, look at social media, you have to look at news and so on.

And b before chat GPT was released, there was basically students looking at this. We were working for a multi global company, so, you know, there had to be students that knew a lot of different languages. And the idea was to, to create tools to help them to actually get summaries and, you know, point out like where there could be some storm brewing.

What we realized quickly was that Drupal was a perfect tool for this. So I had been working with live streaming and the platform we were putting these live streams on was on Drupal. And I had been working with Field API, so I knew the field api. I was extremely structured. There was always one way how used to work, I don't know, address field or FAQ field or a image and so on.

So, one of the requirements we had there was also that the, like the, the social media managers of these sites should be able to set up these systems. So it was like a wide label system. And the field API was perfect for this because they could understand if they want to do a sentiment analysis where they want to set a rating, like where one is super bad and 10 is super good for some kind of topic.

It was very easy to understand how to create the field. And the only thing we needed to add is. How do we use AI to fill out this field? So we just added a checkbox where you click this checkbox, and then we had so-called, automator types, it's called now. So different ways that you can use to fill this out.

And that is how the Interpolator was born. I asked my, you know, boss, if I could open source this because I assumed there's gonna be something in Drupal otherwise that does this, and then we have to switch over and he let me open source this. And that's how it came to be. And yeah, so the, the idea is very simple actually.

You, you store something in one field, you create it, or you decide how to create something that gets stored in a field and you can then also decide to order this gets saved at, and this actually becomes a so-called AI workflow system. It's very dr. It's very much, may be used for editorial purposes rather than universal automation tool, but it's very easy to set up and very, I, I think, very conceptual to, to understand how to set this up.

John: So it's also a little bit different than, like Nick had mentioned, like Aron job too, right? Because it's not necessarily running on a schedule every day, although I'm sure you could configure it to do that. It's it's based on an action, right?

Marcus: Yeah. So it's, it's basically based on you saving the entity, entity.

And if a specific field is not filled out, it's gonna fill it out. If you have set it up to fill it out. And this means that you can create you know, com fairly complex workflows like you put in a link that scrapes a website. And after that, so this website gets stored in a h TM L field a formatted text field.

And the next step is that you create a summary for it. Next step is maybe that you create a description of how you want to have a image that's generated for it. And the next step is that you generate the image. So you just, in this case then just create a, you know, filtered text field, create another text long field, and create a image field.

And you just tell it how to fill this out. And then you can start them with a link and get all these things in steps

Nic: and, and it's easy to set it up to like just run automatically or run in a background on or something. I, I would say the one piece that has taken a fair amount of work is just if you need, if you have branching workflows or conditional workflows, you, you do need some, I mean, it's not hard.

You just end up having to have multiple field. Workflow starting points, right? Either it's separate entities or separate field, but Oh, you mean like basically

John: based on the workflow, like, hey, if you're in draft, do this one. Hey, if you're in review, do this one. I got you.

Nic: Yeah, you can't, you can't really do that that logic directly on an individual automator,

Marcus: right?

Right. Yeah. You have to either do branching or there is a way you configure it in easy as well. So you can actually put it in ECA and, and put some conditional around it. Huh.

John: So I laugh 'cause I have an automator, but it is the most basic automator in the world. Basically when I first save my blog post, it reads the body and then creates a, a meta description and puts it into the field for me.

So super basic, nothing crazy goes out to it sends out to ai, ai, it generates it, you know, fills in the, fills in the description field. So obviously automators can help teams streamline repetitive tasks, can fill in content, can kind of, as you said, take multiple pieces of content and kind of develop new content if need be.

I'm sure their SEO as I just illustrated, their SEO possibilities, personalization possibilities. What are some good use cases? And, and Mark, I'm gonna start with you and then open it up to the, to the rest of the rest of the crew here. But I'm wondering, like, what are some good automator use cases you've seen that you're like, oh, wow, that's a really good use case for this.

Marcus: Yeah. I, I mean, I, I think the most used ones are the simple ones, to be honest. Summarizing content. Categorizing content. So look at the body field and based on a list of categories, choose which categories there is. You can use it for alt text generation. So there is this image alt text module that's very easy to set up and use.

But if you want something that's actual actually contextual. Is that, so it's not just looking at the image, but it can also read the body oil, take some other things in into consideration. And I mean, there, there is automators for every type of field, so you can do very complex stuff as well. I, I know I, I, I did a demo and I, I tried it out as well with like 50 different videos like two years ago where we, where I took the Umami website, so like mm-hmm.

The triple cooking book website where you start with a YouTube link or someone cooking, and then there's a automator that can basically listen and look at a video and write out text based on this. And based on this text, then you could fill out all the structural content. And there's even automator that can, you know.

Based on a prompt you have, find a image for you in a video and take a snapshot of this. Mm-hmm. So you could fill out the whole recipe based on just a YouTube link, for instance. So you can do hyper complex things with it as well. But I think what people use them for mostly is the simple things.

John: Can, can, can you clarify something? 'cause you just said something in a and a light bulb kind of went off. So the AI image, alt text module that you get kind of with, with, I think it comes with a No, it's an add-on, right? It's an add-on module. It's addon module.

Marcus: It's Inal

John: CMS. Okay. So yeah, so it's a data module.

So that module, that module is only doing image. Basically you're sending the image to ai, AI's analyzing the image and then providing the, the alt text as opposed to what you just said. With an automator, you could say, Hey, send this image to ai, analyze it, and then based on these three contexts of, of the body and all of the, the meta attacks and all this other stuff, then build the image, the image alt text.

So it's kind of like alt text, alt text plus plus at that point, right?

Marcus: Yeah, yeah, exactly. I mean, interesting. We, we tried it. For instance, if you have a image of pa jambalaya risotto, if you just have a image, it might actually be hard for the AI to figure out what this is. If you then take text into consideration where you actually describe what, what is being shown, then it's very easy for it to create something relevant.

John: Got it. That makes, that makes a lot of sense. And that's, I think that's good context for folks that are kind of wondering what the differences are there.

Nic: Should we, should we call that the risotto test to show it a picture of each of those dishes? And if we can differentiate?

John: So yeah, it's funny, it's, it's about lunchtime here and I'm kind of, kind of hungry and that comment just made me think of some really good pa I had a couple, a couple of weeks ago.

Uh hmm. Anyway actually, so that question, I wanna kind of shift it to Martin because I know Martin and I had a conversation a couple of weeks ago maybe about AI. Tagging of content and kind of like setting up tag to like nicely, you know, I don't know, press a button and be able to tag, tag some some, some content.

And I was wondering like any, any progress, not, not specifically to you, but have you seen any progress in that area where like, we have the ability to now, now easily tag content.

Martin: Sorry, who's, who's that question for

John: that was for you?

Martin: Okay. Yes. So there, there, I mean, there's, that I would say is actually something that I think both Marcus and, and some of the maintainers of the Augmenter AI module that I worked on had both approached I think in slightly different ways. So I think the Augmenter approach tended to be more, you know, somebody submitted the, the node, let's generate some tags and, and you know, create them as taxonomy terms and get those all populated and all the other things.

Whereas the Augmenter route was more to say, let's take the AI suggestions and then actually create sort of this custom little, almost like a tag five style widget where you can, as the content editor say, yes, these are the ones that I want to actually accept. And one of the things that, that I think is actually really.

Cool that I've seen. I think it was the one x internet team, but you can correct me if I'm wrong there. Marcus had worked on basically creating the same style of widget to work with the newer AI modules so that you can do the same kind of thing. So rather than it's sort of working on save automatically without human interaction, now we've got that human in the loop where it's coming back with the suggestions, but now I, as the content creator can say, these are the ones that I think are actually appropriate and those will get saved to the database.

So I would say there's been both recent and, you know, in the past progress on specifically that use case. But I think we're getting closer and closer to having something that, that follows, I think, a number of best practices and, and making that a really streamlined experience.

Marcus: Yeah, so, so we have something in 1.2 of the AI module that from one x internet mainly worked on, that's called Field widget Actions.

That's really a really cool thing. It's gonna at least split out because there is actually no reason this should be in the AI module. It was just a connection to the AI automators. That's why we released it there. So what this can do is, is basically on any field type that the automator can run on, and I think it's, this is up to like 30 different field types now.

We want to be able to also create a button where it either can, when you click there, it's just run the, to generate stuff. Whereas Martin say, actually run and give X amount of suggestions. Where, where you can pick one of them. And all of a sudden it's not automation anymore, it's editorial tool. So, so this doesn't get, this doesn't get saved until you actually save then,

John: wait, so I, I can do, I can do this right now, like in, in Drupal if I'm using AI 1.2?

Yes, partly. So the, the problem is

Marcus: we have to create one of these button plugins for every type of field, and we don't have it for every type of field yet. So hopefully this is coming one three or two zero.

Martin: And I think you, again, correct me if I'm wrong, Marcus, but the challenge is also not just the type of field, but in some cases the widgets.

So John mentioned Tag Gify specifically, and even though Tag Gify is included in Drupal CMS it, the, the field widget action didn't actually work with Tafi specifically. So that is, you know, an open issue that, that hopefully will be resolved soon.

Marcus: Yes, exactly. Yeah, infrastructure. I think there is for, sorry.

Nic: No, go. Go ahead. I, I was just saying it's, it's because of the expected structure of the data model, right?

Marcus: Yeah, exactly. So, I think we actually have four, if I remember correctly. We have four taxonomy fields and out to complete tax. There actually exist, but not for tag. I

John: hmm. Have test that out later.

Martin: Just wanted to shift gears a little bit. Marcus, you had mentioned AI agents a couple of times earlier and really get the sense that AI agents are becoming a big part of Drupal's AI story. So how would you define AI agents and what makes them so powerful specifically within Drupal?

Marcus: Well, so I mean, the first and mo fourth most important definition I think is agent C. So an agent as compared to workflow where you set up, you know, do a, B, C, and you might have unconditional or something with a agent. The only thing you do is give it tools. You give it a sub system prompt, and then you tell it, try to solve whatever the user throws at it.

So the the one taking the decisions is actually the LLM. So this means also that they are very adaptive, so they're not just solving a workflow. They're, you know, they're, they kind of built for input that we might not actually know about. So one thing that happened with the field agents, for instance, that's in Drupal CMS, was I was in Drupal Con.

I don't remember where, which one of it. Which one group of content it was. But someone came up to me and said like, oh yeah, the agents are super cool because, you know, I can crawl an entity to another entity with the same fields. I had no idea that you could do this with the agent. I should. So I'm like, okay.

Yeah, super. But this also makes them extremely hard to test because, you know, if you, if you write a functional test for a contact form or something, you can maybe think of 200 things that goes wrong. I don't know, you fill out the email with 500 characters or I don't know, click submit before you fill something out and so on.

With agents, this is just one big text field, maybe files as well. Yeah. So, so there are, there are guaranteed use cases that you can't you haven't thought about, that people want to solve with this, that might not be solvable because you didn't think about.

Nic: And, and, and there's also another issue there, which is the underlying endpoints are always changing too, right?

So something, and, and this, this isn't directly related to these automators or agents, right? But if you go from four oh to five to Claude to, you know, whatever changing that can change the output or to change the way that interacts or change the data model that's coming back in a way that can break things.

And I mean, that's still gonna end up as an issue in your queue, even if it's not something that you totally did. But my, my point is, is testing becomes even more difficult because there's a layer behind it that you don't control at all.

Marcus: Right? Right. I, I mean, we, we have abstraction in terms of how you're using tools.

So in theory, you know, you can take 1 billion model on a laptop. So for reference, that is extremely small. And run it on a laptop and ask it to choose tools. It's just not gonna be able to do anything that's relevant, or at least not at the moment. They are working on trying to, you know, compress models to be smaller and smaller and run on smaller hardware, but right now it's just not possible.

So that's why, you know, Drupal CMS comes with the choice of philanthropic and open AI because these are, these are models that we know works together with this. Yeah. But yeah, so I mean, agents is hard. If you ask, like one year ago, a lot of people said that there was actually no proof that agent works, workforce people knew they work.

If you just make them stable enough, it works. But the models have gotten better. There are a lot of improvements to. How agents works that actually makes them work. And you were asking like in a Drupal context I, I think what we have focused on a lot now in the beginning is one, something that should be easy for this marketer persona.

So someone that might not be able to, you know, use AI Id like called code or cursor root tools or something like this, where you could use these tools directly from the id, but where you want to install group pool and start chatting with Drupal. So that, that, that is something we're focusing on.

Another thing is creating agents for pieces in Drupal that requires a lot of UI setup. So setting up fields always takes time. If you're setting up web forms, often that takes time. Setting up field groups takes time. There's one developer, his name is Giran Gs. He just created the views agents.

So this has been a holy grail for us for some time because it's as super complex system to actually set up. He's gonna do a webcast on this and hopefully this is gonna work well. So, so, and, and this will also help not just the, you know, the developer that doesn't want to spend all the time in the UI to set this up, but going back to the marketing persona as well, I, I think it's very useful to have a agent where you can ask it to set up something to get you started.

Because understanding views from scratch is very, very complex. But actually modifying our views is, well, it's at least simpler. Yeah. And the same goes for fields. The same goes for web form as well.

Nic: Yeah, that's, that's a good point. So can we bring it, bring it back to kind of real world you know, building a Drupal site.

Are there cases today where AI agents kind of significantly improve the user experience? I mean, you just mentioned kind of a starting place for views, but other implementations you've seen of agents that you think a broader audience would find helpful?

Marcus: I mean, those are the agents that we have worked on to try to push into Drupal CMS.

So the, those are the agents I talked about, like content type agent. Okay. Field agent, the taxonomy agent, you know, taxonomy. You need to add 20 taxonomies of the most famous footballers or something. This is something they all know and can just fill out for you. Okay. So, so those are the agents. Basically that we have been working on in that context, we have been working on very hyper specialized agents as well, and that's why we have tried to make the agents framework as simple to set up as possible. We have a lot of UX work missing there. So we're, we're trying to simplify it as much as possible because I, I think one of the major use cases is to create agents for your specific problem you're having.

Nic: And, and so are I, I imagine a lot of the agents work right now is going to be about managing context as well, right? Because it's very easy to kind of blow out your context wins window or move past your context window. Is are there any tools around helping manage, manage that?

Marcus: Yes. So in the Drupal AI we have something called short term memory.

We're still mostly experimenting with it. So the plugin is there, it's going to go into agents, and that is basically taking care of, so what our agent is, it's a loop of your tools. So it's called a React agent which stands for Reasoning and Acting. So it's a system prompt that looks at the tools, looks at the instructions, and tries to figure out should I use any tools to do something?

Okay. If I should do tools. Then we go into another loop and it does this until it's basically finished and then it returns text. So, and this can of course be problematic because if you are gonna do something. That requires a lot of context. You're gonna you know, scrape a website or something like this.

This website might have instructions because it's a cookbook website or something like that. You, you, you get problems with a context link, and this is where short term memory comes in then where it can start summarizing, for instance, what have you done so far? Or you can just start removing things you have done so far that is not important for for the rest of the loop and so on.

But yeah, I mean, context is the most important things when you're both for agents and, and automation. Okay. Because, because what, what AI is very good is at, is taking a lot of context and concentrating this. So, you know, doing the opposite where you take one sentence and want to write a story about this, for instance, that's gonna create AI slot.

But the, the opposite, where you actually give it a lot of context and concentrate this usually you make sure that the agent doesn't hallucinate its truthful, it creates something that's worthwhile. And that, that's where we have been working with this context control center. Well name to be set, but right now it's called the Context Control Center.

So you might have seen it in the freeze notes. Yeah. Where, where you make sure that you have all the important context for a specific task

John: at hand.

I have a completely random question, but you just made me think of it because you, you said context, right? So, and this is, this is like almost almo bordering on embarrassing, asking you this 'cause like, it feels like this question's way, way above, way above you.

But hey, I'm gonna go for it. So, creating an image with ai right. Trying to give it like the appropriate context and the appropriate, like, you know, things that I'm looking for in my prompt right image comes back, has, has text in it. And we all know that AI doesn't do very well with like, putting text in images, right?

So I'm like, all right, let me edit my prompt to like, tell it not to have text in the image. And like, wouldn't, wouldn't do it, wouldn't, wouldn't, wouldn't take any wouldn't take the text out. Just curious, am I, am I doing something wrong? Or like, how do I communicate, how do I provide the context to AI that I don't want texted my images?

Marcus: I mean, it's very model specific and sometimes sometimes it can be hard to get the models to do what. I want them to do. I don't work that much with images. But when, when you're working with tools, for instance, it can, it can be a difference between a couple of words that makes the difference between if it's picking the right tool or not.

But I mean, they are trained, they are trained on like real life and something that people have seen. It might sound horrible, but you know, the carrot and the stick actually works very well on elements. So either, you know, actually threaten it, something bad is gonna happen if you don't do this or reward it.

Like you get $500 if you wanna do this. This is actually a trick that has proven to work. So there's been studies on that. This actually worked.

John: So next time if I go, Hey, create me an image and I'll give you a piece of candy. If it doesn't have text in it, I'll yet a text free image. Got it. Okay. Maybe.

Or, Hey, I will turn you off if it does have text in it.

Nic: That's how you get Skynet. I, I wouldn't use that particular one, but I, I mean, I've, I've run into this too where I've, you know, I, I haven't resorted to threats or carriage, but I've sent, I've done prompts where I'm like, summarize this. Under no circumstances, use the term deep dive, right?

And invariably it comes back with deep dive into topic, blah, blah, blah. Just like, oh, see,

John: I always like, so I use, I use AI to write a lot of my, or help me write a lot of my blog posts, I guess I should say. And like, when it uses a term like that, I'm like, ah, you did great. But to like, remove this term, it's like, okay, sure, here's another term.

And I'm like, all right, great. Thank you. It's like your, your best little, your best little buddy. Okay, Marcus, let's ask, let's ask a que let's elevate the level of question here. Looking ahead, right? So we have all of these new and exciting tools. We talked about, you know, cps, automators agents they're, you know, we're, we're, we're pretty advanced in the Drupal space using these tools, right?

But we're, we're kind of in the middle of it right now. We're, we're, we're, we're gaining steam as I would, I would like to say I'm wondering, like, looking ahead, how do you think's automators agents are, are gonna, like, work together to kind of shape, redefine, however you wanna put it the, the future of an intelligent adaptive Drupal site, simplifying that question, how are these, these things gonna help us, you know, build better Drupal sites in the future?

Yeah. I mean, it's,

Marcus: it's, it's very hard. To see the future with AI because it's, you know, moving all the time. I was talking to Nick before we started recording it. It's very much like frontend frameworks, like 10 years ago that, you know, one frontend framework work released. You started learning it and then there was five other frameworks.

John: But talk about you wanna talk about something that's gonna inflict Skynet, consider call AI a front end framework, see what happens.

Marcus: But yeah, I mean, the automator is a little bit outside. It's it's, it's, I mean, you can use it as automation tool. I think with a field wi action, it's mostly editorial tool.

So I, I think how that works is probably not gonna change that much. I'm, I'm guessing. And to be honest, we haven't changed that much on it since the ai inter leaders time. I think there's gonna, in, in terms of Drupal, we need like a, a third kind of tool. So some kind of way to run automation, structured automation within Drupal that is easy to use.

So I, I think this can be easy. A we're already adding a lot of AI parts into ECA Yesper Lamins, and, and Juro is working a lot on this. And I know they're also working on making ECA ECA easier to use. There is another tool that's being developed now called Flow Drop. I think Jamie might have mentioned it last time he was here.

That looks, it looks gorgeous and if it's, if it will be as easy to use as what I have seen, it's gonna be really, really good for ai, but also for automation. So there's a guy shipping dust from Factorial that has been spending a lot of work on this. Yeah, I, I think

Nic: there, there's one area that I'm curious about, 'cause you've hinted at that I think would really improve the way people view Dral from an ed editorial ex standpoint, which is, if we can make it easy to kind of just copy and paste a page from Google Doc or a Word doc and be like, okay, fill this into this content type cr create the five components that we're gonna need, put them in the components and just kind of do, like, for example, we've talked about this before in the show a few times, but we have the show notes.

I put them into Hack md, which is a markdown. Editor, and then I put them on talking, I put them on the website. Right. And that process takes me, the, the copying piece takes me five minutes, right? Because I'm, I'm just copying from one block to the, the intro field, then to the episode number field. And then, you know, there, it's not a whole lot, but it would, it would be nice to be able to be like, I'm gonna create a new episode.

Let me just copy the whole markdown file and then have it break up the pieces into the fields that they belong in. And maybe I have to tweak something, but using AI, not to like write the content or tweak the content, but just to place the content would be, I think, I think the editorial experience is the piece that marketers don't like the most, but it's also the most powerful pieces of Drupal.

Right.

John: But it sounds like that exists, right? Based on the, the module of the week, right? You can take content. And, you know, move it into structured content fields.

Marcus: So, so, I think you can this today actually using NCP. So there, there are definitely Google Docs, CPS, that can read from Google Docs. We have well, I haven't published it yet, but I, I am finished with them.

Notion integration. So basically tools to use Notion where you would then be able to, you know, write that blog post on Notion and then have a, either a AI agent, but this could also be automator or it could be ECA flow drop something to do it in a deterministic way of importing this into a node.

So this is definitely something that is becoming simpler due to MCP or true calling.

John: Mm-hmm.

Marcus: Yeah,

John: sounds like you need to write an agent for, for show notes there, Nick.

Nic: Maybe, I mean, like I said, I, I don't mind doing the, the note creation because there, there is cleanup to do and I wanna, it gives me the opportunity to kind of edit it.

But the copy and pasting piece, it'd be nice to reduce that too. And it sounds like maybe there's some tools to, to experiment with.

Martin: Now speaking of the sort of proliferation of tools, we've talked a lot about MCP today in the DRE note from Drupal Con Vienna. We also saw using the new orchestration module with services like active pieces, and some people in the Drupal community are already using it with services like N eight N to connect different systems together.

Marcus, do you think we should be looking at this proliferation of options as a blessing or would it be better if as a community we try to sort of focus our efforts on a smaller number of options?

Marcus: I, I think we should do both. To be honest as long as there, there is resources to do it, of course, because it's, it's not clear if there's gonna be a specific winner out of all these orchestration tools because basically everyone wants to be the orchestration tools.

No there's a lot of power in what whatever company can solve this, of course. And, and the whole idea of function calling tools, tools and so on has made this simpler. So, NA 10, for instance, it existed before CHATT was released. I know I have checked into like the amount of investments they got before Chat g PT was released and the amount of investment they got after.

And it's just amounts different because all of a sudden, you know, what LM sold is the whole idea of taking something unstructured and make it structured or the opposite then which is a total game changer for orchestration as well. But, but yeah, I, I think we should do both. Drupal has the structure.

Drupal already has a lot of layers that, that makes orchestration possible. Easy is extremely mature. It just probably need to be a little bit easier. But at the same, same time, be open for having runners that can, can basically, hopefully in the future, touch anything in Drupal using tools.

Nic: How, how far away do you think tools is from being kind of stable?

Marcus: We're, we're trying to get it stable before end of the year. This doesn't mean, of course, that we're gonna have every we're not gonna have tools for every subsystem inside Drupal. We're, we're gonna start creating tools for the most important ones. But the, the idea is that it's gonna be stable by end of year fully.

Awesome.

Nic: Well, Marcus, thank you for joining us. It's always a pleasure to have you on to talk about kind of the cutting edge of AI and Drupal.

Marcus: Thank you for having me.

John: Do you have questions or feedback? Reach out to talking Drupal on the socials with the handle talking Drupal or by email with [email protected]. You can connect with our hosts and other listeners on Drupal Slack in the Talking Drupal channel.

Nic: If you would like to be a guest on the Talking Drupal or a new show TD Cafe, click on the guest request button in the sidebar on talking drupal.com.

John: You Yes, you dear Listener can promote your Drupal community event on Talking Drupal. Learn [email protected] slash td promo.

Nic: And you can get the Talking Drupal newsletter to learn more about our guest host, show news, upcoming shows, and much more set up for the [email protected] slash newsletter.

John: And thank you patrons for supporting talking Drupal. Your support is greatly appreciated. You can learn more about becoming a [email protected] and clicking that big come up patron button in the sidebar.

Nic: Alright, Marcus, if our listeners wanna get in touch with you, had any questions or comments, what's the best way for them to do that?

Marcus: So on drupal.org, my username is Marcus Johansson. LinkedIn is Marcus Johansson 80. I have a website where I showcase basically how to set up AI workflows. That is workforce of AI com.

Nic: Awesome. And Martin, how about you

Martin: folks can find me as mandclu on all of the Drupal and social platforms. And John, how about.

John: Personally, you can find me at picozzi.com. You could find me on all the socials and drupal.org at John picozzi, and you can find out about EPAM at epam.com.

Nic: And you can find me pretty much everywhere at Nicxvan at N-I-C-X-V-A-N. And if you've enjoyed listening,

Martin: we've enjoyed talking. Thanks everyone. You guys. Thanks.

John: We

Nic: thank you very much. Awesome.