[REQ] Delossifier function

[MarcoRavich]

Hi there,
after the recent cool super resolution feature add, it would be great to have a tool able to "delossify" - as far as possible - compressed audio.

@kroll-software has recently updated their Audio Delossifier and a brunch of pre-trained models (for mp3 delossification) are provided too.

Last but not least, I've just created a simple script to inference on Colab: kroll-software/AudioDelossifier#5 (comment)

Hope that inspires !

[RyanMetcalfeInt8]

Thanks for the idea. I also see some MP3 restoration models listed here: https://github.com/ZFTurbo/Music-Source-Separation-Training/blob/main/docs/pretrained_models.md#single-stem-models

As always, it would be very helpful to get some help from the community to evaluate & validate quality of the projects & available pre-trained models.

[MarcoRavich]

Thanks for the hint and happy new year, @RyanMetcalfeInt8 !

Just added Apollo by @JusperLee under AUDIO \ AI-based \ Restorers:

Apollo consistently outperforms existing SR-GAN models across various bit rates and music genres, particularly excelling in complex scenarios involving mixtures of multiple instruments and vocals. Apollo significantly improves music restoration quality while maintaining computational efficiency.

Anyway I honestly didn't understand exactly how it can work since it relies on music separation nnet, even if:

During data preprocessing, we drew inspiration from music separation techniques and implemented the following steps:

Source Activity Detection (SAD):
We used a Source Activity Detector (SAD) to remove silent regions from the audio tracks, retaining only the significant portions for training.

Data Augmentation:
We performed real-time data augmentation by mixing tracks from different songs. For each mix, we randomly selected between 1 and 8 stems from the 11 available tracks, extracting 3-second clips from each selected stem. These clips were scaled in energy by a random factor within the range of [-10, 10] dB relative to their original levels. The selected clips were then summed together to create simulated mixed music.

Simulating Dynamic Bitrate Compression:
We simulated various bitrate scenarios by applying MP3 codecs with bitrates of [24000, 32000, 48000, 64000, 96000, 128000].

Rescaling:
To ensure consistency across all samples, we rescaled both the target and the encoded audio based on their maximum absolute values.

Saving as HDF5:
After preprocessing, all data (including the source stems, mixed tracks, and compressed audio) was saved in HDF5 format, making it easy to load for training and evaluation purposes.

Hope that active nnet-based "audio restoration" projects' devs (such as @asjad895, @AakashRevankar, @matthewmcq, @shaws34, @bkraad47 and @kroll-software of course) will join this discussion too.

If can help I'll try to engage Hydrogenaudio's experts again for qualitative evaluation of restorers' outputs.

[JusperLee]

Thank you for your attention, our publicly available pre-trained model for this method was trained using only open source data. If you guys train with more data, you are able to get better generalization performance.

[Brodvd]

If want to try fast: https://huggingface.co/spaces/patriotyk/Apollo
(there's also the checkpoint "apollo-universal" is more or less Audio SR but not the same)

[MarcoRavich]

@yoongi43 - the Exploiting Time-Frequency Conformers for Music Audio Enhancement author - just noticed about this discussion.

[MarcoRavich]

Just discovered this interesting Audio Super Resolution implementation by @jakeoneijk:
FlashSR: One-step Versatile Audio Super-resolution via Diffusion Distillation

Abstract from the relative paper:

Versatile audio super-resolution (SR) is the challenging task of restoring high-frequency components from low-resolution audio with sampling rates between 4kHz and 32kHz in various domains such as music, speech, and sound effects. Previous diffusion-based SR methods suffer from slow inference due to the need for a large number of sampling steps. In this paper, we introduce FlashSR, a single-step diffusion model for versatile audio super-resolution aimed at producing 48kHz audio. FlashSR achieves fast inference by utilizing diffusion distillation with three objectives: distillation loss, adversarial loss, and distribution-matching distillation loss. We further enhance performance by proposing the SR Vocoder, which is specifically designed for SR models operating on mel-spectrograms. FlashSR demonstrates competitive performance with the current state-of-the-art model in both objective and subjective evaluations while being approximately 22 times faster.

Hope that inspires !

[Brodvd]

Well, listening the examples FlashSR seems better of Audio SR about quality.

[RyanMetcalfeInt8]

Very interesting!

[kroll-software]

I think, you all doing a great job pushing this idea further.

[Neovolter]

Have someone objectively compared quality of these models (AudioSR, Apollo, FlashSR and maybe something else)? I like both Apollo and FlashSR in terms of quality)

[MarcoRavich]

hydrogenaudio's listening test is needed for, IMHO.

[MarcoRavich]

Bump.

Seems that @JusperLee's Apollo is gaining results (and needs optimization on Intel-*PUs)...

[matthewmcq]

Hi all!

I apologize for not contributing much to the thread until now, but I just wrapped up my thesis which I feel has some fairly profound implications for audio restoration. Unfortunately, I did not get the change to directly integrate any of these algorithms into a DL pipeline, but I feel y'all are the type of people who would know what to do with it better than me. TLDR is I developed an optimized algorithm to convert from a DFT spectrum into the set of constituent "true" continuous components with more or less arbitrarily good accuracy. In my thesis I used it to "clean" an STFT and for a pretty novel resampling approach.

Code is available here. If you're at all curious to read the actual thesis to get a better idea of the theoretical foundations or to see the results visualized, the pdf is in the folder titled "thesis."

Have a great weekend!

[MarcoRavich]

Bump.

Just discovered @rajasekarnp1's / @31jay's Neural Audio Upscaler project: models aren't present (but the Project Summary claims "Voice, Music, Ambient and General") and I don't exactly understand inference platform they relies on, but the approach seems interesting.

[MarcoRavich]

Ok, with the help of GH-Copilot I've finally "ported" the latest @kroll-software's AudioDelossifier (and, of course, converted provided models) to run - locally - exploiting OpenVINO.

https://github.com/MarcoRavich/OV-AudioDelossifier#readme

It's still pretty raw of course, but it seems to infere without errors.

I'm waiting for your feedbacks and suggestions to improve it further.

[Brodvd]

Strange, I downloaded your fork and installed the requirements.txt, now with your latest commit I get this:

Available devices for inference:
1: CPU
2: GPU
3: AUTO (Automatic Device Selection)
Choose the device to run inference on (number):
1
Enter the path to the models directory (default: ./models/OpenVINO):
models/OpenVINO
Enter the path to the audio file you want to infer on:
audio/64k.mp3
Detected lossy input bitrate: None kbps
Is this correct? Enter different kbps value or press Enter to confirm:
64
Using model: models/OpenVINO\WaveletModel-64k-2048.weights.xml
Using device: CPU
Processing audio/64k.mp3 ...
Traceback (most recent call last):
  File "C:\Users\User\Downloads\OV-AudioDelossifier-master\OV-AudioDelossifier-master\Deloss.py", line 189, in <module>
    main()
  File "C:\Users\User\Downloads\OV-AudioDelossifier-master\OV-AudioDelossifier-master\Deloss.py", line 170, in main
    inputs = preprocess_audio(audio_file)
  File "C:\Users\User\Downloads\OV-AudioDelossifier-master\OV-AudioDelossifier-master\Deloss.py", line 119, in preprocess_audio
    data, _ = af.read_audiofile(file)
ValueError: too many values to unpack (expected 2)

Can you check that the one you use locally is the same as the fork?

[MarcoRavich]

Remember that \ matters...

[Brodvd]

Ok but it's the same thing, I corrected the error like this:

def read_audiofile(filename, target_sr=44100, channels=2):
    # ...
    data, samplerate = sf.read(filename, dtype='float32', always_2d=True)
    # ...
    return data, samplerate

and deleted dtype=np.float32

af.write_audiofile_wav(output_audio, out_fname) #dtype=np.float32

so:

Available devices for inference:
1: CPU
2: GPU
3: AUTO (Automatic Device Selection)
Choose the device to run inference on (number):
1
Enter the path to the models directory (default: ./models/OpenVINO):
models\OpenVINO
Enter the path to the audio file you want to infer on:
audio\128k.mp3
Detected lossy input bitrate: None kbps
Is this correct? Enter different kbps value or press Enter to confirm:
128
Using model: models\OpenVINO\WaveletModel-128k-2048.weights.xml
Using device: CPU
Processing audio\128k.mp3 ...
Inference complete. Output saved as audio\128k_deloss.wav (32bit float PCM WAV)
Inference time: 92.09 seconds

but:

[MarcoRavich]

32bit fp audio output - using dtype=np.float32 - works for me:

Anyway, as claimed above, you have to generate a null mix (in Audacity select one track and perform Effect -> Special -> Invert, then select both and perform Tracks -> Mix -> Mix and Render to New Track) to be able to get the differences between lossy source and delossifyed result:

[Brodvd]

Okok, sorry for my misunderstanding :)

[kroll-software]

I really appreciate all your efforts and progress.

Some ideas for improvements:

1. Better training data could lead to better results
2. A 'sliding window' during inference could improve the algorithm.
   see audio_predict.py, from line 36, # process data in chunks

Detlef

[MarcoRavich]

Bump.

Thanks to @jarredou's Apollo-Colab-Inference repo I've discovered the Universal model for any lossy files by Lew bringed by @deton24.

I've failed to convert it to ONNX format, anyway I'm trying to inference with it as is.

Any suggestion @RyanMetcalfeInt8 ?

[MarcoRavich]

@RyanMetcalfeInt8: I wonder if @matthewmcq's models (aka upscalemp3) will be included too in the next version...

...anyway, with this huge prolification of supported models, something like ChaiNNer's approach will be needed at some point IMHO:

[RyanMetcalfeInt8]

Hi @MarcoRavich -- I think the set of new models / features for next release is pretty much locked in at this point, as I'd like to get it out as soon as possible and is already taking me much longer than I wanted.

Regarding this kind of DAG UI -- I guess we can address it when the time comes. If it ever happens, it would need to be with AU4.

[MarcoRavich]

I think the set of new models / features for next release is pretty much locked in at this point, as I'd like to get it out as soon as possible and is already taking me much longer than I wanted.

I still can't wait for !

[MarcoRavich]

Waiting (impatiently) for the next release, thanks to @olilarkin I've discovered this cool Node-based GUI for CDP by @j-p-higgins that, as cited ChaiNNer, could inspire:

(just a note)

[RyanMetcalfeInt8]

These 2 denoising models made by @aufr33 could also be great addition to Audacity: https://huggingface.co/jarredou/aufr33_MelBand_Denoise (it's the same models than in https://github.com/ZFTurbo/Music-Source-Separation-Training/blob/main/docs/pretrained_models.md#single-stem-models)

Hey @jarredou -- Sorry, I think I missed this one, but yes absolutely -- this one is being targeted for the next release.

And maybe just a quick update / outlook on next release -- It's coming along, apologies it's taking me much longer than I anticipated. Hoping to get it out in a month or so.

Note that, all development is happening 'in the open' though, on this branch: https://github.com/intel/openvino-plugins-ai-audacity/tree/next

Nothing really stopping you from building it / trying it out yourself:
Linux: https://github.com/intel/openvino-plugins-ai-audacity/tree/next/doc/build_doc/linux
Windows: https://github.com/intel/openvino-plugins-ai-audacity/tree/next/doc/build_doc/windows

I finally was able to post HuggingFace spaces for most of the new models (e.g. https://huggingface.co/Intel/drumsep_mdx23c_jarredou_openvino) , and so they are installable via the new model manager UI:

(I still need to add residual output for that 6th stem, hence the screenshot.. but I'll do that next couple of days)

The model card info is generated from a set of .md's here: https://github.com/intel/openvino-plugins-ai-audacity/tree/next/model_cards

Very happy to accept any tweaks / info that you think is missing from those.

[Neovolter]

Sorry for maybe stupid questions but is it possible to use this on a Nvidia gpu (on a cpu it's soo long) and is there any better delossifier in terms of quality ?

[RyanMetcalfeInt8]

You're asking about our AI plugins in general? If so, then unfortunately not, OpenVINO doesn't have a 'backed' that supports NVidia (CUDA).

[MarcoRavich]

Sorry for maybe stupid questions but is it possible to use this on a Nvidia gpu (on a cpu it's soo long)

Ehm, did you notice we're into @intel repositories ?
Check source repos instead...

and is there any better delossifier in terms of quality ?

Well Apollo maybe, but not for OpenVINO at the moment.

Anyway, if you own an NVIDIA GPU, why don't you train models yourself (and share please) ?!?

[deton24]

I'm sorry, but no. I'll write back if they come up. pon., 7 lip 2025, 17:21 użytkownik Ryan Metcalfe ***@***.***> napisał:

…

Hi @deton24 <https://github.com/deton24> -- Any chance you heard back from Lew? I've been occasionally checking your GDoc (which is amazing btw), but didn't see any info there. I guess chances are, you didn't hear anything -- but I figured I'd check :) I just really want to release this model with our plugins, as it's pretty amazing.. — Reply to this email directly, view it on GitHub <#356 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AIJ3EHGBPMKOMM5I23NNJXT3HKGBDAVCNFSM6AAAAABUPP3MJCVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTGNRYGU2TGOI> . You are receiving this because you were mentioned.Message ID: <intel/openvino-plugins-ai-audacity/repo-discussions/356/comments/13685539 @github.com>

[MarcoRavich]

[BREAKING] Shut up, the enemy is listening! 😄

A2SB: Audio-to-Audio Schrodinger Bridges (by Nvidia) implementation released:

A2SB is an audio restoration model tailored for high-res music at 44.1kHz. It is capable of both bandwidth extension (predicting high-frequency components) and inpainting (re-generating missing segments). Critically, A2SB is end-to-end without need of a vocoder to predict waveform outputs, and able to restore hour-long audio inputs. A2SB is capable of achieving state-of-the-art bandwidth extension and inpainting quality on several out-of-distribution music test sets.
We propose A2SB, a state-of-the-art, end-to-end, vocoder-free, and multi-task diffusion Schrodinger Bridge model for 44.1kHz high-res music restoration, using an effective factorized audio representation.
A2SB is the first long audio restoration model that could restore hour-long audio without boundary artifacts.

Git: https://github.com/NVIDIA/diffusion-audio-restoration#readme
Demo: https://research.nvidia.com/labs/adlr/A2SB/
HF checkpoints: https://huggingface.co/nvidia/audio_to_audio_schrodinger_bridge/tree/main/ckpt

[MarcoRavich]

Sorry, I forgot to mention the other contenders cited in the A2SB's test (which are interesting too):

• CQTDiff: Solving audio inverse problems with a diffusion model by @eloimoliner;
• Diffusion-Based Denoising of Historical Recordings by @bvm810;
• BABE-2: A Diffusion-Based Generative Equalizer for Music Restoration by @eloimoliner.

Hope that inspires !

[RyanMetcalfeInt8]

Thank you @MarcoRavich! I'm always a little delayed to respond these days but I do love all of this stuff that you share. I am inspired -- Keep it coming!

[MarcoRavich]

End of Aug25 bump

Here's Audio Restoration Studio by @duchannes19, an interesting Gradio-GUI for comparing restoration models:

[MarcoRavich]

It's maybe barely OT, but I've just updated the HyMPS project \ AUDIO \ AI-based \ Enhancers page resources collection with direct links to available pretrained models (in the "Models" column of course): enjoy and let me know how to improve it.

[deton24]

Incredibly impressive work with HyMPS. Still, it would be nice to have the best stuff sorted somehow by currently the best solutions, or with more info on what to actually use, then just have a massive list with no clue where to start. Look around here. There's some stuff lucking on your list from what I've checked a month or so ago: https://docs.google.com/document/d/1GLWvwNG5Ity2OpTe_HARHQxgwYuoosseYcxpzVjL_wY/edit?tab=t.0#heading=h.i7mm2bj53u07 śr., 10 wrz 2025 o 12:29 Marco Ravich ***@***.***> napisał(a):

…

It's maybe barely OT, but I've just updated the HyMPS project <https://forart.it/HyMPS> \ AUDIO <https://github.com/FORARTfe/HyMPS#-> \ AI-based <https://github.com/FORARTfe/HyMPS/blob/main/Audio/AI-based.md#--> \ Enhancers page <https://github.com/FORARTfe/HyMPS/blob/main/Audio/AI-Enhancing.md#---> resources collection with direct links to available pretrained models (in the "Models" column of course): enjoy and let me know how to improve it. — Reply to this email directly, view it on GitHub <#356 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AIJ3EHHQ7YGXUWSXGXGQK633R74R3AVCNFSM6AAAAABUPP3MJCVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTIMZWGEZTSMY> . You are receiving this because you were mentioned.Message ID: <intel/openvino-plugins-ai-audacity/repo-discussions/356/comments/14361393 @github.com>