Skip to content

Add AudioEffector #3163

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

Add AudioEffector #3163

wants to merge 1 commit into from

Conversation

mthrok
Copy link
Collaborator

@mthrok mthrok commented Mar 10, 2023

This commit adds a new feature AudioEffector, which can be used to
apply various effects and codecs to waveforms in Tensor.

Under the hood it uses StreamWriter and StreamReader to apply
filters and encode/decode.

This is going to replace the deprecated apply_codec and
apply_sox_effect_tensor functions.

It can also perform online, chunk-by-chunk filtering.

Tutorial to follow.

closes #3161

@mthrok mthrok mentioned this pull request Mar 10, 2023
facebook-github-bot pushed a commit that referenced this pull request Mar 23, 2023
#3192)

Summary:
OPUS encoder and VORBIS encoders require "strict=experimental" flags. This commit enables it automatically.

The rational behind of it is typically we care if we can encode these formats at all and not how they are encoded. (This might be concern when these encoder becomes more mature on FFmpeg side and providing flags would result in weird behavior)

Also when writing high-level functions that uses StreamWriter, if we do not set these flags, then these high-level functions have to add new options that should be passed down to StreamWriter, which turned out to be very painful in #3163

Pull Request resolved: #3192

Reviewed By: nateanl

Differential Revision: D44275089

Pulled By: mthrok

fbshipit-source-id: 74a757b4b7fc8467c8c88ffcb54fbaf89d6e4384
@mthrok mthrok force-pushed the effector branch 3 times, most recently from bd0d52f to 1f8dbd1 Compare March 24, 2023 17:14
@sorgfresser
Copy link

You're using process_all_packets from time to time. While I get that collecting all packets is necessary for some of the filters, it poses a threat since one can not tell how large the underlying file will be. Do you plan on adding an option to limit the amount of packets processed at once - even though the filters would obviously become worse? I would be very glad.

@mthrok
Copy link
Collaborator Author

mthrok commented Mar 27, 2023

You're using process_all_packets from time to time. While I get that collecting all packets is necessary for some of the filters, it poses a threat since one can not tell how large the underlying file will be. Do you plan on adding an option to limit the amount of packets processed at once - even though the filters would obviously become worse? I would be very glad.

@sorgfresser

My implementations are aware of the potentially long audios, and they don't force the one-go operation. In this particular feature, I am providing two options. one-go or chunk-by-chunk process. It's up to users to pick which fashion they want to process their data.

@mthrok mthrok force-pushed the effector branch 11 times, most recently from 6a43ad2 to 6e99aac Compare March 31, 2023 13:34
@mthrok mthrok changed the title wip: add effector Add AudioEffector Mar 31, 2023
@mthrok mthrok marked this pull request as ready for review March 31, 2023 14:02
@mthrok mthrok requested a review from a team March 31, 2023 14:03
@facebook-github-bot
Copy link
Contributor

@mthrok has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

(applied,) = reader.pop_chunks()
return Tensor(applied)

def stream(self, waveform: Tensor, sample_rate: int, frames_per_chunk: int) -> Iterator[Tensor]:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does every effect require the specification of sample rate?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, without a user provided sample rate, we need to assume some default value, which is not universally applicable.

@facebook-github-bot
Copy link
Contributor

@mthrok has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D44576660

mthrok added a commit to mthrok/audio that referenced this pull request Mar 31, 2023
Summary:
This commit adds a new feature AudioEffector, which can be used to
apply various effects and codecs to waveforms in Tensor.

Under the hood it uses StreamWriter and StreamReader to apply
filters and encode/decode.

This is going to replace the deprecated `apply_codec` and
`apply_sox_effect_tensor` functions.

It can also perform online, chunk-by-chunk filtering.

Tutorial to follow.

closes pytorch#3161

Pull Request resolved: pytorch#3163

Differential Revision: D44576660

Pulled By: mthrok

fbshipit-source-id: 27e2a2af626188934a25e66d33c693ddf5bc580e
mthrok added a commit to mthrok/audio that referenced this pull request Mar 31, 2023
Summary:
This commit adds a new feature AudioEffector, which can be used to
apply various effects and codecs to waveforms in Tensor.

Under the hood it uses StreamWriter and StreamReader to apply
filters and encode/decode.

This is going to replace the deprecated `apply_codec` and
`apply_sox_effect_tensor` functions.

It can also perform online, chunk-by-chunk filtering.

Tutorial to follow.

closes pytorch#3161

Pull Request resolved: pytorch#3163

Differential Revision: D44576660

Pulled By: mthrok

fbshipit-source-id: 42097e758598c098313ff5a6b9563183604d6842
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D44576660

mthrok added a commit to mthrok/audio that referenced this pull request Mar 31, 2023
Summary:
This commit adds a new feature AudioEffector, which can be used to
apply various effects and codecs to waveforms in Tensor.

Under the hood it uses StreamWriter and StreamReader to apply
filters and encode/decode.

This is going to replace the deprecated `apply_codec` and
`apply_sox_effect_tensor` functions.

It can also perform online, chunk-by-chunk filtering.

Tutorial to follow.

closes pytorch#3161

Pull Request resolved: pytorch#3163

Differential Revision: D44576660

Pulled By: mthrok

fbshipit-source-id: 1ac9613b3e5e5fa51dcc19e54978f23d82f5fa96
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D44576660

Summary:
This commit adds a new feature AudioEffector, which can be used to
apply various effects and codecs to waveforms in Tensor.

Under the hood it uses StreamWriter and StreamReader to apply
filters and encode/decode.

This is going to replace the deprecated `apply_codec` and
`apply_sox_effect_tensor` functions.

It can also perform online, chunk-by-chunk filtering.

Tutorial to follow.

closes pytorch#3161

Pull Request resolved: pytorch#3163

Differential Revision: D44576660

Pulled By: mthrok

fbshipit-source-id: e6794d1d434c95db5cd24b3bd11f5e5e2a9671da
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D44576660

@facebook-github-bot
Copy link
Contributor

@mthrok merged this pull request in a403624.

@github-actions
Copy link

github-actions bot commented Apr 1, 2023

Hey @mthrok.
You merged this PR, but labels were not properly added. Please add a primary and secondary label (See https://github.com/pytorch/audio/blob/main/.github/process_commit.py)

@mthrok mthrok deleted the effector branch April 1, 2023 01:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Apply filter function
4 participants