Beacon of the Walkyries: Artificial Metal Lyrics Generator

Posted on 28 February 2020 in machine-learning

After having analyzed a bunch of power metal songs to see if it was really about dragons, I wondered for a moment what could be the next step. Actually there was two next steps. The first one was to augment my power metal dataset from 58 bands to 506 bands. The second one was to use this dataset to generate power metal lyrics.

Let's have a look at the following lyrics.

Beacon Of The Valkyries

The night is dying
The sun is rising
The clouds are closing in
The battle is on
The countdown is done
The twilight is near
The battle is on

The blood on the altar is now
The blood on the altar is now
The sun is rising
The sun is rising

We're calling for the warriors
We're calling for the whispers
We're calling for the warriors
We're calling for the gods

Beacon of the valkyries
Beacon of the valkyries
Beacon of the valkyries
Beacon of the valkyries

What do you think? Pretty metal huh.

It was one of the first lyrics produced by the model I fine-tuned on my power metal dataset.

The dataset

The new dataset I prepared contains 10 times more bands than the first one for a grand total of 508 bands. With that amount of data, I hope to capture the spirit of what makes a good power metal song.

The model

At first, I had the idea of creating the following lyrics generators and compare them:

HMM,
RNN,
transformer.

Since less is more and I wanted to do something with what is considered the state of the art, I decided to go with a transformer model, and more specifically the GPT-2, a model created by Open AI.

GPT-2

GPT-2 is a transformer model. This type of model is usually made of stacks of decoder and encoder blocks, piling up like smart pancakes. In the case of GPT-2, we use only decoder blocks. Inside these blocks, we find something called self-attention. I could try to explain all of this but other people did it better, so if you want to know more please go to the References section.

Fine-tuning

My understanding of what is happening here is that it is more about continuing the training of the model than a classic last layer freeze and re-train that last layer type of fine-tuning.

To fine-tune GPT-2, I had to format the lyrics to confirm to what the model is expecting. Other than that, I ran as is the code for training GPT-2.

The results

The Seeds

The model can take as an input a few words. I've decided to seed it with the titles of one of my favorite music albums of all time: Serpent's Egg by Dead Can Dance. Dead Can Dance is not a power metal band but a dark wave / world / gothic / something type of band.

The Host of Seraphim
Ring of Fire
Severance
The Writing on My Father’s Hand (part I / part II)
In the Kingdom of the Blind
The One-Eyed Are Kings
Chant of the Paladin
Song of Sophia (part I/part II)
Echolalia
Mother Of A Thousand Vows
Ullyses

Bonus Tracks

Angel Of The Morning
Poem of the Priest
The Flame And The Water
Giants Of The Light

I generated for each title 3 outputs and cherry-picked myself the most coherent or relevant one. A few times, when more than one set of lyrics generated were great I saved them. These lyrics are in the bonus tracks.

The album

The Host of Seraphim

Come with me and you will see the sunrise
Bring the Angel to my eyes

Fight the night and survive the day
Hear the call of the Seraphim

I was named by the fortress of the underworld
The Archangel of the Light
Shining in the light of the night
And I am the Angel of the morning

All the angels will be here
Sons of angels
All the children of angels
All the gods will be here

Come with me and you will see the sunrise
Bring the Angel to my eyes

Fight the night and survive the day
Hear the call of the Seraphim

Ring Of Fire

The pit is closed for me
The pit is filled with rage
The fire is burning deep inside
This is the pit of fire
The bell is ringing
This is the price for my soul for eternity

No more words for peace and love
My soul is burning with fear
I'll face the world inside me

The pit is closed for me
The pit is filled with rage
The fire is burning deep inside
This is the pit of fire
The bell is ringing
This is the price for my soul for eternity

Severance

You're standing up
Looking for a sign
Give it one second
Tell me what's wrong
Let's just call it insanity

Guess the time has come
Take your leave
From the heathens

Seconds are testing your faith
You're standing up
Looking for a sign
Keep it up
Tell me what is wrong
Let's just call it insanity

You're standing up
Looking for a sign
Give it one second
Tell me what's the wrong
Let's just call it insanity

The Writing on My Father’s Hand (part one)

I live in a world of spinning hearts
Where the living creatures were all in their minds

So long ago my father had told me
All this time and all the times I still remember
I can't help but to wonder
Should I cry for a reason
When I was young and weak

I saw the writing on my father's hand
And I now can't help but to wonder
Should I cry for a reason
When I was young and weak

Why can't I hear the voices
Of the wise men and the wise men
Tell me why don't you feel the pain

I live in a world of spinning hearts
Where the living creatures were all in their minds

So long ago my father had told me
All this time and all the times I still remember
I can't help but to wonder
Should I cry for a reason
When I was young and weak

I see the writing on my father's hand
And I now can't help but to wonder
Should I cry for a reason
When I was young and weak

The Writing on My Father’s Hand (part two)

The stories they tell are as untrue as they are untrue
And what is the truth, stay away from me

What will you do when you see the writing on your fathers hand?
What will you do when you see the writing on your fathers hand?

A man of innocence, who doesn't understand
On from the day his father succumbed to his disease
A thorn and a thorn and a thorn, have murdered the story of his life
His brothers and ones whom he lost but who survived
He feared the end of his life
His soul would not be washed away

In hot and cold he could not sit down
He felt his lust was strong
He felt his blood was cold
He wished his life to be free
His blood was cold

Forever and ever he would wait
For the writing on his fathers hand
The writing on his fathers hand
And the stories that he wrote are as untrue as they are untrue
"And what is the truth, stay away from me"

I will cleanse the world of sin in all its filth
I will be your guardian, and feed upon it like a lion
I will destroy this virgin

In the Kingdom of the Blind

All the rest of the world is creating a scheme
To destroy the white sky

Stalking the dark land we'll find another way
This time it is by law
To attack the task
Of the ruler
And make him a stand
And then the world is plunged into darkness

Abandon all the right
To divide the World
Around us

So the ruler controls the world
And
He's the one who can't see
The future will end
Now we're ruined

Stalking the dark land
We'll find another way
The world is plunged into darkness

The One-Eyed Are Kings

Amore we'll make them die

Abandoned in the wild
Facing the dark
A militant priest
Reinvented from the grave

I've taken the path of the righteous
I've not only learned the truth but made all my friends fall

The One-Eyed Are Kings
Aeron, the one-eyed are kings
I'm the one-eyed are kings

Abandoned in the wild
Facing the dark
A militant priest
Reinvented from the grave

The One-Eyed Are Kings
Aeron, the one-eyed are kings
I'm the one-eyed are kings

Abandoned in the wild
Facing the dark
Holding the gate

The One-Eyed Are Kings
Aeron, the one-eyed are kings

Chant of the Paladin

Wield on yourself
The mighty warrior is glorifying the slain

The black sun has begun to set
The prince of the glory
Cries out for the might of the earth

The sun will light the way
Walls will fall down
Pushed by Lightning
The great hunter's one and only goal

The black sun has begun to set
The prince of the glory
Cries out for the might of the earth

The sun will light the way
Walls will fall down
Pushed by Lightning
The great hunter's one and only goal

Song of Sophia I

Where do you come from
Where are you going to stay
And where can you be? I don't know, I can't explain
I'm not the person you are
I am not the people you can't save

I can't say, there's no way I could make it through
I don't know, I'm not the person you are
I am not the people you can save

Don't you ever look at me
Don't you ever look at me
I'm not the person you are
I am not the people you can save

I am the man you can't save
I am the people You can save

Song of Sophia II

Into the forest
Inside the burning flames

Beware of this witch
She will make you fain alive
She is the witch of the night

She waits for you
While you watch her pucker

I am the witch of the night

I watch you through the flames
I am the witch of the night

I'm the witch of the night

I am the witch of the night

Echolalia

Bloody madness
I hit the ground
The air was full of fire
It took my breath away
The rain and the snow
I felt the burning of my soul
My soul would fade away

The chasm has found its way
To the place where I can feel
A break in time and time again
I am the winner of the trial

Hundred unknown gates
Are here everywhere
I've approached them all
And I may be in the time
When you'll see their evil
The time is only right
And I wonder how they've come
To make this journey
So far away

The northern land
Is full of fear
As I fall
I can't find a way back
To the river of life
I'm falling so
Too far

The winter was a long time ago
Now it's fading
And you won't see my eyes
In the coldness of the air

The chasm has found its way
To the place where I can feel
A break in time and time again
I am the winner of the trial
Silence speaks my name
The time is only right
And I wonder how they've come

Mother Of A Thousand Vows

Aproach to our mother
She's a liar, the tyrant, the poisoned name
I am the one who controls the world

Hell has no balance, no defence
You're the one who's fighting in the fight

I'm a leader, I'm the one who knows what's wrong in Hell
I'm the one who's calling the people

Hell has no balance, no defence
You're the one who's fighting in the fight

Your god is evil and you are the god
I'm your God, and you are my Son

Hell has no balance, no defence
You're the one who's fighting in the fight

You're the one who's fighting in the fight

Ulysses The Serpent

As I walk through
The portals of dark and light, Look at the glow of the moonshine
I've seen in time what the blackest of black can be

Cursed I felt the calling
I am the Serpent of the Ethereal
Cursed I felt the storm

Open the doors to your realm
Came the promise of a new life
The lure of your hand
Has bled the ground
Killing warlocks and captives

Cursed I felt the calling
I am the Serpent of the Ethereal
Cursed I felt the storm

Open the doors to your realm
Came the promise of a new life
The lure of your hand
Has bled the ground
Killing warlocks and captives

Cursed I felt the calling
I am the Serpent of the Ethereal
Cursed I felt the storm

Open the doors to your realm
Came the promise of a new life
The lure of your hand
Has bled the ground
Killing warlocks and captives

Bonus Tracks

Angel Of The Morning

The battle cry of angels
The sound of the wind
In color the colors of our light

Rid of the beast, robed and armored
To the realm of the dark
Some say it's so real

We will fight, we will win
We will create the future
Until the end of time

The battle cry of angels
The sound of the wind
In color the colors of our light

Rid of the beast, robed and armored
To the realm of the dark

Poem of the Priest

He's the one who is the one
Who is the one who knows
We may die in vain

We will chant with our eyes
When time is near
And darkness comes to the end
We'll dance in the river

We will pray for our lives
But not for our souls
We are to be free

We will chant at the flames
But hear them sing
We will attack from the top

The sacred Priest
He's the one who is the one
Who is the one who knows
We may die in vain

We will chant for our lives
But not for our souls
We are to be free

The Flame And The Water

Before the dark we conquered
And then we felt the fire that we
Killed
And the sea of sand
We could feel the pain

Walk of glory
The only way is to fight
Now the Gods
They will we fight
To bring you
With their will

Speak the language of the gods
Together with the earth
And the dragon will fly
To bring you the fire and the cold
So it's easy and so easy

Wings of hope
We can feel your mind
Now the Gods
Will we fight
To bring you fire and the cold
So it's easy and so easy

Wings of hope
We can feel your mind
Now the Gods
Will we fight
To bring you fire and the cold
So it's easy and so easy

Giants Of The Light

My eyes are filled with confusion
My mind is trying to shatter
The one who's been lost inside

Like the dragon in the midnight sky
I searched the sky but I cannot find
The god of the dark

My eyes are filled with confusion
My mind is trying to shatter
The one who's been lost inside

Like the dragon in the midnight sky
I searched the sky but I cannot find
The god of the dark

Like the dragon in the midnight sky
I searched the sky but I cannot find
The god of the dark

And I can not find my way back home

I search for my way back home

Before I can say a word
I fall into a deep sleep

Like the dragon in the midnight sky
I searched the sky but I cannot find
The god of the dark

Like the dragon in the midnight sky

Conclusion

I was surprised that the finetuning of the metal lyrics worked so well. I shared a few lyrics with my head-banging friends and their reaction was unanimous: they loved what was generated.

This is opening so many possibilities to artists and creative people. It is possible to become one-person band (no pun intended). Music generation is still young and singing voices never produced anything good but who knows? Maybe in a couple of years we could hear something close enough to the reality. If so, I will start my band 🤘🤘🤘

References