Update README.md
Browse files
README.md
CHANGED
|
@@ -19,10 +19,12 @@ This work was granted access to the HPC resources of IDRIS under the allocations
|
|
| 19 |
|
| 20 |
Models currently available are:
|
| 21 |
|
| 22 |
-
- Wav2vec2 base model () pretrained (no fine-tuning) on Librispeech (English speech), FMA (music), subset of audioset, or all of them together. It also includes a model pretrained on VoxPopuli french dataset.
|
| 23 |
- Wav2vec2 tiny model, where we used only 3 transformer layers. Models' performances are surprisingly high.
|
| 24 |
|
| 25 |
Scientific papers using the models provided in this repository:
|
| 26 |
Orhan, P., Boubenec, Y., & King, J.-R. (2024). Algebraic structures emerge from the self-supervised learning of natural sounds. https://doi.org/10.1101/2024.03.13.584776
|
| 27 |
|
| 28 |
-
Models are pretrained using HuggingFace's trainer.
|
|
|
|
|
|
|
|
|
| 19 |
|
| 20 |
Models currently available are:
|
| 21 |
|
| 22 |
+
- Wav2vec2 base model (https://huggingface.co/facebook/wav2vec2-base), but pretrained (no fine-tuning) on Librispeech (English speech), FMA (music), subset of audioset, or all of them together. It also includes a model pretrained on VoxPopuli french dataset.
|
| 23 |
- Wav2vec2 tiny model, where we used only 3 transformer layers. Models' performances are surprisingly high.
|
| 24 |
|
| 25 |
Scientific papers using the models provided in this repository:
|
| 26 |
Orhan, P., Boubenec, Y., & King, J.-R. (2024). Algebraic structures emerge from the self-supervised learning of natural sounds. https://doi.org/10.1101/2024.03.13.584776
|
| 27 |
|
| 28 |
+
Models are pretrained using HuggingFace's trainer.
|
| 29 |
+
These models pretraining are often shorter (100,000 steps compared to 400 000) than original pretraining because of resource scarcity.
|
| 30 |
+
In my experience, most emergences I studied had happened before 100 000 steps.
|