Full transcript of “Promptmetheus – Prompt Engineering Demo”. Click a timestamp to open the video at that moment. Need another video? Use the YouTube transcript generator.
0:00 okay let's look at the real world example of how to use Prometheus to build a chat GPT integration for an app I have this micro Journey app and I want to build a really cool integration for it called Freud which will read the entry and extract sentiment mood and emotions from it and present to use as a small summary and also plot it over time in order to do that let's open the forge playground and create a new product let's call it Freud sentiment or sorry Freud emotions
0:47 you can choose the platform you want to use in the future we'll have more options for now it's only open AI the mode and the model let's go with the most powerful one and find unit you can set the temperature to zero to get um objective outputs you don't want to have any creativity in this task can also hover over the different settings to see what they do can at a stop sequence if you want and give it a system message
1:39 and then we construct a prompt you can minimize this with blocks and first of all we need to tell you what's going on following is an entry from micro Journal and we have this concept of fragments so for each block we can have different options that are called fragments um I go into detail in a bit but for now let's call this
2:25 first one simple and then we need an example um an entry we want to experiment with and play around we could actually add a new block to hard code it but even better we can use data sets and data blocks so let's create a new data set it's called journal entries and that holds some examples create some here we can just add a few options so we can see how the thing performs with different entries
3:14 and then we go back to the prompt and we can use this data set as part of our prompt data block and selected and here you can see now that we have the different options that we can swap back and forth and then some instructions two motions are present in the entry you can see here like each of the blocks um will get automatically done the token count
3:59 and total one and then see how much it will cost when we execute this front with the given model to get a better result you might also wanna add some output instructions like print only uh emotions as um comma rated list nothing else
4:44 it's gonna be called Simple I guess or maybe let's do unsupervised I will do a supervised version in a bit and that's it it's um the prompt you can preview it here how it looks in full you can also told token count and then we can just execute it and see what it comes up with okay so we get an output which doesn't look even bad and here on top we see how the prompt was constructed which blocks and which fragments we used for this specific output also which model parameters how much input and output tokens we had and how much interacting
5:30 costs and now one more interesting part is we can rate this you can say for instance this is really good one um I like it and then all the fragments that were used in this prompt the generators output you can see this color coding that will represent the quality of it let's run it again another output let's see for instance this one was bad you read it and you can see this will build up over time like visual statistics so that you can immediately see how each block and each fragment performed let's do um a target one here
6:18 and copy it gives some more instructions say bullet Journal Focus stick writing execute again and then if you rate this one you can see we get like um statistics for this one as well once we have a few options and play around for a while we can see which one is green which one is red and which one performs better in the bottom you can also see how much
7:04 money you spend or in total and requests how many we sent and um what is the total token count all right let's do a supervised version so instead of letting it come up with its own emotions I give it a set that it should choose from and we could hard code them but again we can use a data set let's do a new one
7:49 and I have already a set of emotions here this is like set 40. if I want to play around the different options I can say I just use half of them and set because or that's the five no it's available here as a data set can select it drag it into place and here you can see I can now play with
8:36 the different options to see how the output changes and let's go with the full set for now and execute it and get any output we can see um set 40 was used just like before you can write it again you get to see you can see the name the ratings one interesting thing might be also if for instance I don't want to use a block I can just remove it and it will be excluded from down from the printer
9:23 okay so one interesting thing also is we can see this versioning here once fragment was executed and I change it afterwards which one do we change let's just remove something here you can see him the version changes and if I execute it again we see that the version is updated so we always have full traceability of what
10:09 happened which prompt which block and which fragment produce which output and how it performed we have some options too change the display if this is too cramped for you yeah that's about it for now um I hope you like it I'm working on cool new things like embeddings where you can just drag and drop a PDF and use the contents as a context for the prompt and aipis which will allow you to to take the prompt you design and publish it to an API endpoint that you can use directly in your app
10:55 where you send for instance the the entry is a request body which will be inserted instead of the data set version and you get the output back so this way you build your application you only drop down API endpoint and from that point on never have to touch it again you just go to Prometheus update and play around and your app will just use the change version without republishing it to the Play store or something like that I guess this will be very powerful once it's ready all right that's it for now I'm really excited to see what you guys are
11:41 building with Prometheus and yeah have a good one
