xvasynth

Batch #21 - 6 new voices; 5 new games!

The results are in for poll #21. The winning voices are:

- Bioshock: Elizabeth
- GTA 4: Niko
- GTA 5: Franklin
- GTA 5: Trevor
- Red Dead Redemption 2: Arthur
- Resident Evil: Lady Dimitrescu

This does mean that there are now an additional five new games supported by the app. I've already sorted out the theme asset files for these in v1.4.1 which I've uploaded a small patch for, on the Nexus sites.

I've already compiled the datasets for the other voices, so do...

Introducing: xVAScribe

xVAScribe is the first companion app for xVASynth. This small tool is for use in creating/improving speech datasets, for use during model training. 

There are two primary ways you can use the app:

1) For adjusting existing datasets, by changing the text transcript associated with audio data

2) For creating a brand new dataset of your own voice, via the microphone&nbs...

Batch #20 - 6 new voices

Closing off poll #20, the winning voices are as follows:

- Skyrim: MaleBandit
- Skyrim: MaleEvenTonedAccented
- Skyrim: MaleDrunk
- Cyberpunk: Takemura
- Fallout 3: President Eden
- Oblivion / Morrowind: Male Nords

These came out well! 

Keep your eyes out for the next poll, because I think it will be a really fun one!

AI source separation - live action and echo-y voices now possible

If you recall the first release of the C&C: Kane voice, I mentioned plans to research some additional data pre-processing techniques, to help improve the training data for voice models. This was motivated by unclean audio such as the live action cutscenes from which I pulled data for Kane. The background humming and computer beeping in the training data made it into the trained data, meaning the quality wasn't amazing.

So this is where AI source separation comes in - an AI model whi...

Interim update - 9 voices; +1 game

A somewhat larger interim update, this includes some requested voices:

- Dragon Age: Cailan
- Dragon Age: Duncan
- Dragon Age: Morrigan
- Fallout 3: Three Dog
- Fallout 3: Three Dog (radio voice)
- Mass Effect: Liara
- Mass Effect: Tali
- Skyrim: Riekling
- Grand Theft Auto - San Andreas: K-DST host

I've also included an update to Kane's voice from Command and Conquer (now much much better, and without any background noise, such as cheesy computer beeping -...

Batch #19 of 8 (6+2 bonus) early access models

The next round of voices is here! 6 from the poll:

- Skyrim: Neloth
- Skyrim: Delvin
- Cyberpunk: Alt
- Cyberpunk: Delamain
- Fallout New Vegas: Arcade
- Fallout New Vegas: FemaleAdult03

as well as two public requests:

- Final Fantasy: Sazh
- Fallout New Vegas: MaleAdult04

They all came out fairly well. Though Sazh needs a little more work, and may get an update later on.

5 requested models (+1 game); v1.4.0 now out on Nexus

Getting back to the voice training, there was also a spike in requests. Most were private, but there are 5 here which were not:

- Fallout New Vegas: MaleAdult03
- Mass Effect: Miranda
- Dragon Age: Zevran
- Final Fantasy: Hope
- Final Fantasy: Lightning

There is a new game (series) supported: Final Fantasy

To use models for this game, grab the newest version of the app (v1.4.0) from the Nexus. I made some changes to how asset files are used to allow secondary the...

v1.4.0 pre-release: Speech-to-Speech; Plugins; Much more...

A long time coming, the next version of the app is nearing release. This is by far the largest single update to the app (except for the v0.1 -> v1.0, of course). There's a couple of big new features, but let's get the changelog out of the way.

Changelog:

- Added a speech-to-speech mode
- Added a third party Plugin system. Developer reference: https://github.com/DanRuta/xVA-Synth/wiki/Plug...

Batch #18 of 6 early access models

A fresh new batch of voices is here! 6 voices this time:

- Skyrim: Harkon
- Skyrim: Ancano
- Skyrim: Gelebor
- The Witcher: Keira Metz
- Cyberpunk: Claire
- Skyrim: Garan

Keep your eyes peeled over the next day or two for a big pre-release for the next version of the app. 

Interim update - 3 requested voices

Quick update with three requested voices:

- Fallout 4: Dara Hubbell
- Skyrim: MaleElfHaughty
- Fallout New Vegas: MaleAdult02

Batch #17 of 7 (6+1 bonus) early access models

The next round of voices is here:

- Skyrim: MaleCoward
- Fallout 3: Amata
- Cyberpunk: Panam
- Witcher: Zoltan Chivay
- Witcher: Cerys
- Fallout New Vegas: Raul
- Oblivion/Morrowind: Male Redguards

There are a few requests on the way also. This one had a bigger focus on some of the other games, to give them some attention as well, we'll do some more Skyrim ones in the next poll again.

Interim update - 4 requested voices + 1 game

Quick update while I finish the poll voices, with some voices which were requested. The voices are as follow:

- Fallout 4: Gen1Synth03
- Fallout New Vegas: MaleAdult01DefaultB
- Dragon Age: Leliana
- Command and Conquer: Kane

They all came out well, apart from Kane, which came out with some beeping in the background. I ended up using data from all the games, and cutscenes for this voice. Being mostly (amazingly cheesy) live action, there was lots of background audio (a l...

Batch #16 of 9 (5+4 bonus) early access models; +1 game!

The next batch is here, including the top 5 voices from poll #16, and 4 requested voices. The poll voices:

- Skyrim: Arngeir
- Skyrim: Mirabelle
- Skyrim: Esbern
- Skyrim: Isran
- Skyrim: Kodlak Whitemane

The requested voices:

- Mass Effect: Ashley
- Mass Effect: EDI
- Fallout New Vegas: MaleAdult01Default
- Dragon Age: Alistair

There is now also support for the Dragon Age game series, following the request. You need to add the attached image into ...

Batch #15 of 8 (6+2 bonus) early access models; 200+ voices!

The next round of voices is here! This one has 6 from the poll, as well as two extra that were requested:

- Fallout 3: FemaleAdult04
- Fallout New Vegas: Lanius (requested)
- Skyrim: Alduin
- Skyrim: Dragon
- Skyrim: Paarthunax
- Skyrim: FemaleArgonian
- Skyrim: MaleArgonian
- League Of Legends: Swain (requested - "other" category)

This batch now puts us at over 200 voices trained for xVASynth!

The dragon voices actually turned out not TOO bad. Usable I ...

Third party plugins pre-release

I've just finished putting together the initial implementation of support for third party plugins in xVASynth. Plugins are supported for either/both Python/JavaScript scripts, for either/both back-end/front-end, respectively. There is also support for .css files, to change the app visuals. You can think of them as chrome extensions, but for xVASynth.

I've written up an exhaustive, but hopefully...

Batch #14 of 6 early access models

The next batch is here! This time these are the poll winners:

- Skyrim: Tulius
- Skyrim: Ulfric
- Skyrim: MaleDarkElfCynical
- Skyrim: FemaleCoward
- Skyrim: MaleCondescending
- Cyberpunk: Judy

I've also still been re-training older voices. Alongside these, the public versions of the following voices have been updated:

- Fallout 4: MaleGhoul
- Skyrim: Serana
- Fallout 4: Nate

I found out why Nate/Nora had inexplicably lower quality than other voice...

Hardware upgrade

I wanted to make a quick announcement, about an update on the hardware side of things. Thanks to everyone's incredible support, I have now completed the next bit of hardware upgrades, composed of a new motherboard, and a new CPU.

This is the third upgrade since the start, the first being the new GPU+PSU, and second (unannounced) being a system RAM upgrade.

I managed to somehow find a 5950x in stock, which is insane - the upgrade over my old CPU will be night and day. I've implemen...

The Witcher nexus page now live; +2 requested voice models

The Nexus page for the Witcher models is now live, at: https://www.nexusmods.com/witcher3/mods/5676

---

In other news, a further two requested models are now finished:

Mass Effect: Kaidan
The Witcher: Triss

The Triss voice came out good. The Kaidan voice seems to have a fair bit of variatio...

Batch #13 of 6 early access models

New voices! From the poll, these are:

Oblivion: Martin
Fallout 3: FemaleAdult02
New Vegas: Cass
Skyrim: MaleDarkElfCommoner
Skyrim: MaleOldKindly
Skyrim: MaleOldGrumpy

These all came out well, and each include a HiFi vocoder. Expect a large wave of re-trained updates to come to the older voices as well, in the following days. 

Interim update - 3 requested voices

Another quick mid-batch update with a few more of the requested voices. The non-private ones:

Fallout 4: Kellog
Mass Effect: Thane
Mass Effect: Shepard (Male)

The poll voices will be out shortly. I've also re-trained and released the following two voices (requests):

Skyrim: Frea
Skyrim: Karliah

The others are coming along also. I'm also part-way through re-training the Nate/Nora voices. I realized there was a mistake in the transcripts (namely in the DLC lines...

Batch #12 of 7 (6+1 bonus) early access models

The next batch of voices is ready:

- Skyrim: MaleCommonerAccented
- Skyrim: MaleCommoner
- Skyrim: FemaleKhajiit
- Skyrim: MaleKhajiit
- Skyrim: FemaleOrc
- New Vegas: Benny
- The Witcher: Vesemir

All these voices came out well, and they have HiFi vocoders (apart from Vesemir, which will have it soon). I've also finished the Cyberpunk: Johnny HiFi vocoder, which I've attached.

I have also re-trained and updated the following models with Tacotron 2 and HiFi ...

Todd Howard! - And other voices.

It was inevitable, and here it is: Todd Howard's voice! It came out far better than what I had hoped (comes with HiFi, of course).

About 6 different AI models went into making this, all doing different things. 

Pretty please don't share this one around early if people ask for it - I will release it to the Nexus after the 2 weeks early access period.

--

Also in this post are some other voices, mostly from requests (the non-private ones):

- The Witcher: Yennef...

Transcript no longer required

As you may have seen from previous posts, and messages on Discord, I've been experimenting in the background with ways to get around unavailable transcripts for voices. This was spurred on by the Cyberpunk requests, for which the transcript is not (yet) available.

Given just the audio files, I've been working on automating the transcription process. At the core of the method is another AI model doing speech recognition. I simply run the audio lines through such a model and get an initia...

Second half of Batch #11 - 7 models (4+3 bonus) early access models (+1 game)

The next wave of voices is here, including a few voices from requests. The voices are as follows:

Fallout 4: RobotCompanionFemaleProcessed
Fallout 4: RobotCompanionMaleDefault
Fallout 4: RobotCompanionMaleProcessed
Fallout 4: FemaleOverseerDLC6
The Witcher: Cirilla (requested)
Fallout 76: MODUS (requested)
Civilization IV: Narrator (requested)

All the voices came out good, with MODUS being maybe just ok - the voice is a little strange, and sometimes it adds short...

Batch #11 of 8 (7+1 bonus) early access models

The first round of voices from the last Fallout 4 poll is ready. I've also been spending some time re-doing some of the earlier voices. I released an update to RobotAssaultron (now with T2 and HiFi) to the nexus, along with an update to Curie (also T2 and HiFi).

As for new voices, they are as follows:

Fallout 4: MrGutsy (with HiFi)
Fallout 4: SuperMutant (with HiFi)
Fallout 4: SuperMutant02 (with HiFi)
Fallout 4: SuperMutant03 (with HiFi)
Fallout 4: RobotCompanionFemal...

v.1.3 beta - the Batch synthesis/partial regen update

The next version of xVASynth is ready for early testing! The main new features are the Batch synthesis mode for synthesizing many lines in one go, and changes to the normal synthesis process to allow for partially generating the audio (to retain editor changes when editing the text). The changelog is as follows:

- Added batch synthesis mode! (From a .csv or .txt file)
- Added partial re-generation! For changing only parts of the text without losing editor progress
- Fixed silenc...

Batch #10 of 10 (9+1 bonus) early access models

A new batch of voices! This time:

- Fallout 4: GuardMaleDiamondCity01
- Fallout 4: GuardMaleDiamondCity02
- Fallout 4: Strong
- Fallout 4: GangOperatorMale01
- Fallout 4: GangOperatorMale02
- Fallout 4: Gen1Synth01
- Fallout 4: Gen1Synth02
- Fallout 4: MagsBlack
- Fallout 4: Hancock
- Cyberpunk 2077: V (female) - requested

I actually trained Hancock a while ago as a personal request, but I didn't deem the quality good enough at the time for publishing. W...

Batch #9 of 10 (9+1 bonus) early access models

A new batch is done. The voices this time are:

Fallout 4: Preston
Fallout 4: BobbiNoNose
Fallout 4: Clem
Fallout 4: GangDiscipleFemale01
Fallout 4: GangDiscipleFemale02
Fallout 4: GangDiscipleMale01
Fallout 4: Mason
Fallout 4: MaleBoS01
Fallout 4: MaleBoS02
Cyberpunk 2077: V (Male) 

All voices are good quality apart from GangDiscipleFemale01 which is meh. I've also started group training the HiFi model on the male voices, so I'm hoping quite soon th...

v1.2.3 and Batch #8 of 14 (11+3 bonus) early access models

A big one this time, following the options from the poll, as well as the second half of the  surge in personal requests (of which 2 additional voices not in this list were private) from earlier.

The new voices are:

- Fallout 4: MaleGhoul
- Fallout 4: Longfellow
- Fallout 4: MaleMaineAccent
- Fallout 4: LancerCaptainKells
- Fallout 4: FemaleBoS01
- Fallout 4: FemaleBoS02
- Fallout 4: MaleGhoulCombatant01
- Fallout 4: JackCabot
- Fallout 4: GangOperatorF...

Batch #7 of 10 (7+3 bonus) early access models

I had a surge in personal requests from Producers/Investors this week, three of which made it into this batch. Following these and the options from the poll, the new voices are:

- Skyrim: MaleOrc
- Skyrim: MaleEvenToned
- New Vegas: MrNewVegas (radio voice)
- Fallout 4: Sturges
- Fallout 4: Shank
- Fallout 4: MaleRaider01
- Fallout 4: MaleRaider02
- Fallout 4: ChildrenOfAtomMale01
- Fallout 4: ChildrenOfAtomMale02
- Fallout 4: MaleTriggerman

Most of the...