Letta-powered AlgoRave

For the last few months I have been thinking about building this (epic) “DJ” with Strudel powered by Letta, where live stream viewers can chat and the agent tunes the shared audio stream collectively. I finally sat down to build a prototype.

Strudel is a music live-coding environment for the browser. It ports the TidalCycles pattern language to JavaScript, so a performance is just a buffer of code that you edit while it plays. The idea is to point a stateful agent at that buffer and let the room steer it.

See the demo: Letta-powered AlgoRave

Building the agent

I started in the Letta ADE, and a few ideas showed up immediately.

One I think would be particularly cool: let users specify custom sound packs. In Strudel you can already load custom samples from GitHub, like this. If chat can name a pack, the agent should be able to pull it in instead of being stuck with the default kit.

The agent also needs some kind of seed. Its SOP is always to mutate rather than start from scratch. I skipped that for the prototype, since I’m all-in on stateful agents and the memory can carry the pattern forward, but a seed would be nice for future use-cases. A fresh room, or a fresh agent, needs somewhere to begin.

The seed

I started from this “cover?” of John Coltrane’s Giant Steps, from the Strudel examples. Open it in the REPL.

// John Coltrane - Giant Steps

let melody = seq(
  "[F#5 D5] [B4 G4] Bb4 [B4 A4]",
  "[D5 Bb4] [G4 Eb4] F#4 [G4 F4]",
  "Bb4 [B4 A4] D5 [D#5 C#5]",
  "F#5 [G5 F5] Bb5 [F#5 F#5]",
).note()

stack(
  // melody
  melody.color('#F8E71C'),
  // chords
  seq(
    "[B^7 D7] [G^7 Bb7] Eb^7 [Am7 D7]",
    "[G^7 Bb7] [Eb^7 F#7] B^7 [Fm7 Bb7]",
    "Eb^7 [Am7 D7] G^7 [C#m7 F#7]",
    "B^7 [Fm7 Bb7] Eb^7 [C#m7 F#7]"
  ).chord().dict('lefthand')
  .anchor(melody).mode('duck')
  .voicing().color('#7ED321'),
  // bass
  seq(
    "[B2 D2] [G2 Bb2] [Eb2 Bb3] [A2 D2]",
    "[G2 Bb2] [Eb2 F#2] [B2 F#2] [F2 Bb2]",
    "[Eb2 Bb2] [A2 D2] [G2 D2] [C#2 F#2]",
    "[B2 F#2] [F2 Bb2] [Eb2 Bb3] [C#2 F#2]"
  ).note().color('#00B8D4')
).slow(20)
.pianoroll({fold:1})

Melody, lefthand voicings, and bass, stacked and slowed way down. That’s the pattern the agent is supposed to mutate, not replace.

Where I left it

I got so distracted playing with the proof-of-concept agent that I didn’t finish the rest. The missing piece is a way for the agent to edit the text in the Strudel REPL. Chat can already steer the idea. Custom samples and a seed can wait. Until the agent can write into that buffer, it isn’t actually DJing.