Prompt Interpolation

Original wiki text from AUTOMATIC webui wiki

Prompt interpolation makes it possible to interpolate between the embeddings of multiple prompts over time. The syntax is a list of prompts followed by a list of step numbers. Each number indicates the specific step at which the corresponding prompt should be used:

[prompt1:prompt2:prompt3:...:when1,when2,when3,...]

prompt1, prompt2, prompt3 and so on are separate prompts, and when1, when2, when3 and so on are numbers that indicate at which steps which prompt should be used. If the sampling step is in-between when1 and when2, for example, then an embedding that is a weighted average of both when1 and when2 is used instead.

A smaller number for whenN has a bigger impact on the structure and placement of all objects in the final image, because it will be sampled earlier. A greater number will have a more fine-grained impact on the whole picture, affecting mainly the specific details and features of the image.

Note also that the further away the interpolated embeddings are from each other, the more sampling steps is required to get good results.

In case there are at least 3 prompts used in the interpolation, one of the following interpolation curves can be used to interpolate between the embeddings of the prompts:

• linear : interpolate linearly between the embeddings corresponding to each prompt
• catmull : an artificial embeddings is created at each step that follows a catmull-rom curve that passes through all prompts' embeddings
• bezier : an artificial embeddings is created at each step that follows a bezier curve with every prompt embedding used as a control point

By default, the interpolation curve is linear. If you want to use another function, write its name as an additional argument to the interpolation expression.
Here is an example with prompts p1, p2, p3 at steps 0, 1, 9 using catmull as the curve type:

[p1:p2:p3:-1,0,8:catmull]

Please note that step numbers are off by one n-1 to stay coherent with the existing prompt features for AUTOMATIC webui.

It is also possible to use step values between 0 and 1 to indicate a ratio of the total number of samples. For example, if the total number of steps is 20, [a:b: .25, .75] will start interpolating at step 5 (inclusive) and stop at step 15 (exclusive).

Corner cases:

• [to:when] - adds to to the prompt after a fixed number of steps (when)
• [from::when] - removes from from the prompt after a fixed number of steps (when)
• [from:to:when] - starts with from and switches to to after a fixed number of steps (when)
• [a:b:from,to] - interpolate from a to b starting at step from and stopping at step to
• [a:b:,when] - same as [a:b:-1,when] ; interpolate from a to b from the beginning of the sampling process to when
• [a:b:when,] - same as [a:b:when,total] where total is the total number of steps of the sampling process
• [a:b:,] - same as [a:b:-1,total] ; interpolate during the whole sampling process from a to b

Example: a [fantasy:cyberpunk:3,7] landscape

• At start, the model will be drawing a fantasy landscape.
• After step 3 (so at step 4) still only includes fantasy.
• Between step 4 (inclusive) and step 8 (exclusive), the embeddings corresponding to fantasy and cyberpunk are interpolated linearly. For example:
  • At step 6, the embeddings corresponding to the interpolation expression will be .5*embeddings('a fantasy landscape') + .5*embeddings('a cyberpunk landscape').
  • At step 7, the embeddings corresponding to the interpolation expression will be .25*embeddings('a fantasy landscape') + .75*embeddings('a cyberpunk landscape').
• After step 7 (so at step 8), it will switch to drawing a cyberpunk landscape, continuing from where it stopped at step 7.

Here's a more complex example with multiple texts:

masterpiece, best quality,
[exploding huge bright fire ball in the night sky
:a magical landscape filled with fairies and dragons
:a mysterious and menacing medieval landscape
:,0,18
:catmull]

(sampler has 20 steps)

• at start (after step -1 (since it is omitted), so step 0):
  • masterpiece, best quality, exploding huge bright fire ball in the night sky
• after step 0 (so at step 1):
  • masterpiece, best quality, a magical landscape filled with fairies and dragons
• between step 1 and step 19 (both excluded), sample the catmull curve passing by all prompt embeddings from t=1/19 to t=19/19
• after step 18 (so at step 19):
  • masterpiece, best quality, a mysterious and menacing medieval landscape