xvasynth

Technical (?) post #3 - xVA modding support for other games

xVASynth is built to be very easily extendable to other games. It is mainly just a framework to serve voice models for games, so nothing in the code is actually specific to any specific game. I have already done everything necessary to include support for the games I have listed on the description page, for which I train voice models. However, if people wish to provide support for other games, I fully support, and actually encourage this (I would prefer to take care of the existing xVA b...

Technical post - explainer on the models used

I thought I'd quickly write up a semi-technical post, to explain the models used, in more detail. This should be useful to understand the current plans, as well as future plans for how voices are released.

There are four models being used in the xVA process (for now). The models are Tacotron2, FastPitch, HiFi-GAN, and WaveGlow, and they can be categorised as follows:

• Pre-processing: Tacotron2
• Sentence generation: FastPitch
• Audio generation (vocoder): WaveGl...
  2021-02-20 15:40:00 +0000 UTC View Post

Batch #6 of 8 early access models

Group #6 is ready, following the 6th poll. The options were split for the 7th voice (between FemaleGhoul and Glory), so they are both ready. The full voice list for this batch is as follows:

- Fallout 4: MaleOld
- Fallout 4: Shaun
- Fallout 4: X6-88
- Fallout 4: Gage
- Fallout 4: FemaleGhoul
- Fallout 4: Glory
- Fallout 4: MaleBoston
- Fallout 4: Maccready

All the voices turned out quite well! I'm actually really surprised at how good the female ghoul came out...

Batch #5 of 8 (6+2 bonus) early access models

Round 5 of early access models is here, following poll #5. I've taken the feedback from Discord to break apart the voices done by Stephen Russel into separate models, fine-tuned on their individual game voices, as they were different enough to require it. I thought I'd also take this occasion to finish off the other voices done by this actor, namely DiMA and MrHandy (both Fallout 4, and also 3 for MrHandy). The final voice list in this batch is as follows:

- Fallout 4: NickValentine
...

Batch #4 of 5 (4+1 bonus) early access models

Round 4 of early access models is here, following poll #4. The models are:

- Fallout 4: FemaleRaider01
- Fallout 4: DoctorAmari
- Fallout 4: KasumiNakano
- Fallout 4: Sierra

I'm also throwing in a finished Skyrim: Maven, a model which I've been using to play with the development of the FP16 model saving (as you may have tested in the Discord server). All 5 models are using this FP16 mode, cutting file size by half.

I ran Tacotron2 pre-training for all of these (...

v1.2 beta - the Efficiency/Speed/Quality update

Following feedback from v1.1 where I changed the vocoder model, this update focuses on the quality of the output. The UI has been changed to accommodate for multiple vocoder models, thus allowing people to pick between which WaveGlow model they prefer to use for synthesis (the old one, pre v1.1, and the newer bigger one). 

Additionally, I've added support for bespoke Vocoders on a per-voice basis. You may have seen the two side-by-side audio samples on the Discord server for Nate's...

v1.1 beta - Big update + GPU news

This post is two-fold, first a pre-release of the biggest update to xVASynth yet, incorporating many feature requests, fixing numerous bugs, and making several large improvements to existing components/features. The second is a big announcement on the GPU situation, and a new rough plan for the future of development of new voice models.

Let's start with the v1.1 pre-release changelog:

* Ability to edit previously generated samples
* Numerical inputs for sliders
* Upgraded Wa...

v1.0.9 pre-release: Audio post-processing

Following the last updates post, I will release this new version on here first, for people to play around with. This update contains a few tweaks/fixes, as well as a big(ish) new feature, audio post-processing. Do note, you need to install ffmpeg version 4.3 or later to toggle this on for use. The download link is at the bottom of this post, do let me know if it works alright! I will post it to the nexus once I've played with it some more, or someone reports that it works ok for them!

C...

v1.0.8 - A week's worth of updates, and updates going forward

So just over a week has passed since the initial v1.0.0 release, and I wanted to do a quick recap of the main additions to the app, as well as some ideas for updates going forward.

So far:

- An updates menu to notify people of new updates, when they are released

- Re-organized the UI following some feedback, to keep letter-specific settings on the top bar, sequence-wide settings on the bottom bar

- Added a "Reset Letter" button, and added red colouring to the letter ...

Batch #2 of 5 early access models

The second round of early access models is here, following poll #2. The models are:

- Skyrim: FemaleShrill

- Skyrim: FemaleDarkElf

- Fallout 4: ElderMaxson

- Skyrim: Vex / Fallout 4: Proctor Ingram (different models/training runs)

Unfortunately I had some issues with convergence on the robot version of Curie, so I may need to leave that for when I can do T2 training.

Again, I have attached the models here first, and will post them on the Nexus ~2 weeks from...

Technical post - new data cleaning/filtering

The quantity of data used during training is very important for converging a model well. But more important still is the actual quality of the data used. As such, there are a number of data cleaning/filtering scripts that I use, to make sure the quality of the data is as high as possible. There are scripts for adjusting the audio, and scripts for filtering out lines.

Having given it some more thought, I came up with an additional filtering step which I hope will help clean out the data ...

First batch of early access models [3]

Poll #1 overwhelmingly chose these two voices:

- Skyrim: FemaleCommoner / Fallout 4: Alana Secord / Allie Filmore

- Skyrim: Delphine

I have attached the models here first, and will post them on the Nexus ~2 weeks from now. 

Quality is quite good for these two models I think! Check the discord channel for a sample of the two.

v1.0.1

Quick update as I'm preparing for release: Added an updates button/menu to notify you (via a yellow-coloured arrow) when there are available updates, and a change-log in a menu.

I've just launched this patreon page so it may be a bit early for it, but I'll be posting the first poll shortly for voices to go in the next update.

Hello world

Here goes for the first post.

If you are new to this page and you haven't yet seen the main showcase video, I'd recommend watching this video to get a quick overview of what I am working on:

https://www.youtube.com/watch?v=xqJ-ujWvu7s

In short, xVASynth is a freely available app making use of AI to generate realistic speech for voice acting, using existing voices from video games (...