ericguizzo / reckon

Generate sound dreams

  • Public
  • 259 runs
  • GitHub
  • Paper
Run with an API
Playground API Examples README Versions

😵 Uh oh! This model can't be run on Replicate because it was built with a version of Cog or Python that is no longer supported. Consider opening an issue on the model's GitHub repository to see if it can be updated to use a recent version of Cog. If you need any help, please hop into our Discord channel or Contact us about it.

Run time and cost

This model runs on CPU hardware. We don't yet have enough runs of this model to provide performance information.

Readme

Reckon

This repository supports Eric Culm’s Reckon, released under Krisis Publishing. In this Replicate demo you can use the Reckon engine to generate sound dreams, exploiting the same pretrained models that I created to compose the X, Y and Z axes of the album release.

Feel free to use this code as a creative tool. You can think of it as a neural synthesizer that generates sound dreams of an AI. You are allowed and encouraged to download the sounds generated by Reckon and use them in your own music. Please, cite me if you do so.

Description

Reckon is an algorithmic system aimed at making tangible, in acoustic terms, the dream of an Artificial Intelligence . The audio information is entirely generated by a digital dreaming apparatus. Inside the latter, the psychic operations of sensorial memories re-elaboration, proper of human dreaming, are substituted with purely mathematical operations between digitalized sensorial stimuli. Reckon exploits the cooperation between algorithms: a system able to stock and analyze acoustic stimuli to generate acoustic memories and a parallel mechanism that re-elaborates and organizes these memories, building oneiric textures. On the one hand, a neural network model called SampleRNN analyzes audio data collections to learn how to reconstruct the essence of the experienced stimuli (for example piano or guitar). The mathematical procedure at the basis of this model, as well as the resulting sound, has much in common with the process through which we are able to mentally recall a perceptual experience, evoking its salient characteristics but omitting its meticulous details. In fact, the sound elements present in Reckon resemble blurred memories, remotely evoking the main characteristics of musical instruments and natural soundscapes, but discarding their most accurate elements that would make them real. On the other hand a semi-entropic apparatus controls SampleRNN and selects, re-elaborates, concatenate and overlaps sound memories to compose digital audio dreams. To perform this operation, the system auto-defines a hierarchy of complex semi-random rules through which create a semantic evolution of the sound material in a similar way to how happens in human biologic dreams.

Usage

With the UI parameters you have broad control on many macro details of the generated dreams. Although a big part of the process is aleatory, so the output will be always unpredictable.

User parameters

  • memories: list of sound memories that a dream or episode can contain. This gives you control over the timbre types that will be included in the sound dream.
  • dream_length: length of the generated sound dream in minutes. The actual duration may be shorter than this setting due to post-processing. Up do one-hour dreams can be generated!
  • density: this parameter is connected with maximum number of simultaneous sounds that can occur [range 0-1]. Larger values produce fuller mixes.
  • diversity: this parameter sets maximum duration of memories in a dream [range 0-1]. With higher values the sound dreams will contain more diverse segments.
  • output_type: File format of the (downloadable) generated sound dream. Can be wav or mp3.