273: Trolley Problem

Over the next days, Nicolai continued to build his Node system and A3 continued to sponge up everything it was taught. It worked with one of the other construction bots, who it named “Quiet Brothers” because the other bots never said a word. A3 would stand and observe how the others built. Later, it joined in. The Modules taught it many things, and the Mask attempted to impart moral philosophy, learned from the datapackets he’d bought for it, joined by Psychology. It was a new area for both of them, but they were very enthusiastic.

Nicolai taught A3 how to fight. The principles and movements and ideas, at least. There wasn’t much room for sparring, alas, but he intended to rectify that with the eventual creation of a training room.

A3’s form wasn’t well suited for combat at first, but he worked to change that, too. The ultimate form A3, with input from he and the others, generated for A3 would be large and require quite some time and resources to produce.

However, it also created a temporary version. This would be the addition of the basic parts of the ultimate form, into which A3 could begin to build its core Nodes—it already had its version of a heart Major Node, but in this starter body it would be able to build its heart area Minor Nodes, as well as create its version of spine and brain Major Nodes then the relevant Minor Nodes.

Then, later on, they would be able to addon the parts in which the stomach, arm, and leg Nodes would go into—its primary weapons, and a larger lower body with carrier capabilities.

The starter body was still large, however, and at least as capable as the typical combat bots he would create. As it morphed into this form A3 grew taller, and slimmer. It walked on four legs, still, but its drills and tools were increasingly switched out for finer manipulators, more utility pieces, guns and blades. Soon it looked more like a combat bot than a construction bot.

‘When I am away from here, it is your job to guard our home.’ He told the bot. ‘You must never leave our home. It is dangerous out there. The Assembler and your brothers need you to look after them. Your brothers can deal with simple situations on their own, but if something more complex occurs it will be up to you to direct them. You must prioritise protecting the Assembler above all else. Do you understand?’

‘Yes, father.’

‘Good, A3. Now, what did I tell you about the outside, and strangers?

‘I should not trust strangers, father. If someone should come here, I should use my projectile weapons on them, and send out an alert, marshalling my combat form quiet brothers. I should be wary because they may be a Cultivator and possessed of strange abilities.’

He smiled. ‘Very good.’

The combat forms were the combat bot top-halves the Assembler had so far made for him.

The Assembler had completed the first of them the other day, and A3 had been very interested in the process as Nicolai had one of the “quiet brothers” swap.

###

Two construction bots carried a newly-made top half from the Assembler, watched by Nicolai and A3.

These top halves were designed to make use of the Assembler bots modular patterns. Each Assembler bot was effectively two halves joined together.

The bottom half was mostly four legs, merging crab-like into a central part. Attached to this weighty central part were two medium-sized manipulator arms at the front, which folded together in a cross-armed manner when not in use. Rising from the central part where the legs joined, there was a central tower-shaped portion. This central portion contained the bots processing units, primary battery, and other crucial pieces. It was the most expensive and difficult to create portion of each bot, and he only had resources in the Assembler’s banks to make so any.

The two construction bots placed the top-half on the ground, and stepped back. A mining bot came forward, following Nicolai’s orders to swap.

It spread its two back legs wide, anchoring itself. Its two front legs placed their first joint on the ground, similarly anchored, then reached back and grasped the two hookable grooves either side of its upper mining-setup portion, aided by the arms which unfolded to grasp and steady the load.

In a smooth movement it disconnected internally and pulled the upper half off from itself. This upper half was composed of medium armour that slotted around the tower part, as well as the two large, heavy limbs used for mining, and its head with the various sensors and cameras plus self-defence laser on top.

It placed this on the ground and took a moment to return to standing, then positioned itself in front of the newly made combat-bot top-half. There it performed the same process in reverse, putting the new part onto itself. This one was even heavier, the armour thicker.

In only a minute it had finished slotting on the new top-half on and stood once more, now a combat bot. It looked similar to the mining and construction versions, only this had very different limbs. On one side, its arm was equipped with a heavy 50 BMG machine gun. On the other, a minigun chambered in 7.62.

Fortunately for Nicolai, since gaining the Assembler he didn’t have to buy ammo at the Trade Link’s inflated prices, instead simply making his own. The only problem was that the vats of materials were beginning to get somewhat low. He couldn’t make much more until he refilled them, and ideally not with expensively purchased material packs from the Trade Link. The mining bots had at least found various ores while mining Oma crystals, which he could feed directly into the Assembler to be processed for useable materials.

In the longer term he envisioned a significant mining operating to continually refuel the vats. Rock and ore would be the main feed-ins, to gain the metals and trace metals used for steel and electronics. He didn’t anticipate needing any organics except perhaps to synthesize biofuel, but for now it had plenty of power and he hoped to build and setup solar power before it ran low.

But for now, less resource usage was better. This was another reason he’d chosen this resource-saving method. It would be much cheaper overall to just create a maximum number of bottom-halves, then top-halves to swap out as necessary. If he needed lots of miners, he could have them all swap to mining halves. If his base was attacked, he’d have them equip every available combat-half.

###

The combined lessons of Psychology and the Mask mostly occurred while Nicolai was focused on creating Rotation Orbs. A3 would sit in the cave in front of him, and receive a new kind of lesson. Psychology had gone to Simulations and with its aid, they were able to create a basic VR world into which A3, the Modules, Nicolai, and the Mask could all have representations.

In the first of these lessons, Nicolai opened his virtual eyes to find himself in an empty white space. Around him were the Modules, the Mask, and A3.

A3 resembled itself, the same here as in reality. Looking down at himself, he saw he, too, resembled himself.

The others, however, had taken on forms.

Psychology was a human-sized teddy bear, with tan fur, button eyes, a waistcoat and a little hat.

Cyberwarfare was a field of shimmering light, full of strange and at times worrying shapes.

Simulations was a plank of wood with a smiley face drawn on.

Threat Analysis was a large, floating fish. It wore a white robe and had a somewhat scholarly vibe.

Aiming appeared to be the grim reaper, complete with scythe.

Legal was a tiny, flying devil wearing a suit and holding a briefcase.

Nicolai chose not to comment on the various choices. He knew the Modules had spent quite a bit of time deliberating and were all very happy with their choices. Instead he frowned at the Mask, who… was standing before him, staring at him.

It looked… like him. A younger, unscarred Nicolai with somehow gentler features and kind eyes.

‘Why do you look like that?’ he asked, perturbed.

‘This is what I think a Better Man might look like.’

‘I thought a Better Man would be older,’ he muttered.

‘I considered taking a more grandfatherly form, but I didn’t feel it fits my current stage of development,’ explained the Mask.

A3 was looking excitedly back and forth between them all. ‘I never expected my uncles to look like this!’ spoke the bot.

‘Quite impressive, no?’ blubbed Threat Analysis in a low and fishy voice, flicking its tail and spraying water at Cyberwarfare. The water disappeared into the other Module’s flickering lights, which displayed a nightmarish face in response.

The giant teddy bear that was Psychology waddled out to stand before them all. It gestured, and a railing formed. The land dropped away beyond the railing, transforming into a scenic countryside vista. A train track ran across the grassy ground, winding between trees to pass just below their vantage.

‘Observe: A train travels down this track,’ spoke Psychology in quite a deep, refined voice, and a train appeared off to the side, chugging along the track. ‘On its current course, two humans are restrained, tied to the rails. They are utterly unable to act.’ Two people, tied and gagged, appeared on the track below.

‘If left uninterrupted, the train will strike and kill them both.’ Psychology’s tone held a trace of relish. ‘However, a second track exists. It diverges from the first, a short distance before the two humans. This track also has a human tied to it—but only one.’ Another track materialised, linking to the original track a short distance before the tied people, with another tied and gagged human.

‘You are positioned at a manual override lever,’ announced Psychology, and a lever appeared just shy of the railing. ‘If pulled the train will divert to the other track. You possess no intel on any of the subjects. There is no apparent difference between any of them.’

‘You have a choice. Take action and reduce the casualties from two to one, or refrain, and let fate take its course.’

Will you divert the train, so that one is killed, rather than two, or let it continue on its route?’

‘I would hack the train!’ cried the flickering lights of Cyberwarfare.

‘I would leave it up to fate. Either way, death is good,’ intoned Aiming, who appeared to be roleplaying.

‘This question isn’t for you lot,’ said Psychology irritably. ‘You’re free to fabricate whatever structures you please in this space. Go do that.’

Cyberwarfare instead made several crude comments about Psychology’s parentage, referring to toasters and hamsters. Threat Analysis’ fishy form grew little arms to cover A3’s “ears” while this was ongoing.

Soon Cyberwarfare grew bored and along with the other three wondered off to construct their own, better morality tests.

‘You must decide: will you divert the train, causing the death of one, or allow it to continue and kill two?’ Psychology repeated to A3.

A3 pondered, uncertain. ‘I don’t know. If there is no difference between the humans, does what I do matter?’

‘Yes,’ answered the Mask. ‘Because though the humans are all very similar to one another, on one side there are two, on the other one. Clearly, it is best to divert the train so that only one person is killed, rather than two. This ensures the lowest overall loss of life, and is thus the Better Man action.’

‘A standard answer,’ said Psychology approvingly. ‘Let’s see what Nicolai thinks.’ The teddy bear looked to him expectantly.

He shrugged. ‘I’d pull the lever.’ When he stood right by the lever and could pull it with practically no effort, there was literally no cost to being a Better Man. So, why not? Plus, then he’d have two people grateful to him, as opposed to one.

‘Excellent, a consensus!’ enthused Psychology. The Module gestured to the tracks, which shifted. ‘Now, each track holds a single human. The primary track contains a child; the alternate, an adult. Will you divert the train to kill the adult instead of the child, or allow it to continue on its present course?’

‘I don’t know, I guess I’d divert it,’ muttered the Mask. ‘The child has more unlived years, after all. They have a whole life ahead of them. Neither death is good, but at least the adult has already experienced a decent amount of their life.’

‘I would not divert it,’ said Nicolai. ‘Child or adult, it makes no difference to me. Plus, from a practical point of view, the adult is more functional and useful.’

A3 looked back and forth between them, uncertain.

‘And you, A3?’ asked Psychology.

‘What is a child?’ asked A3.

‘A child is a young, undeveloped human. Before the age of say, twelve, a human would be considered a child. Between the ages of thirteen to seventen they could be considered a teenager. From thereon they would be an adult. Children are less experienced but also typically considered to be more innocent.’

A3 had learned of years, but this was the first time it was learning about ages. ‘What would I be?’ it asked.

‘You would be considered a very young child,’ said Psychology. ‘Often, children play games together,’ it added.

A3 tensed, and peered with greater focus over the railing. ‘I would save the child!’

‘Why?’ asked Psychology.

‘Because we could play together!’

Nicolai snorted. ‘Fair enough.’

The Mask sighed. ‘Perhaps A3 is still a bit too young for these kind of lessons…’

‘We’ll allow A3 a reprieve. Mask, you’ve been exploring moral frameworks—perhaps that will inform your response?’ Psychology gestured. ‘The track the train is currently heading down is empty. On the alternate track, eight individuals are restrained. One of them is known to you, in fact you have foreknowledge of his intentions and future impact. He intends to harm you directly and to destroy the society you value. If left alive, his actions will lead to widespread collapse: civil war, systemic failure, mass casualties.’

‘Do you divert the train to eliminate this threat—killing seven innocents in the process—or do you preserve their lives, knowing what will follow?’

The Mask pondered. ‘According to Kant, one should not treat people as a means to an end in any circumstance, and intentions matter more than outcome. Thus, it would be wrong to pull the lever and kill seven innocents, to kill the one evil man.’

‘You’ve been studying,’ noted Psychology with approval. ‘In fact, according to his theory, it would even be wrong to kill the potential wrongdoer—he has not yet committed the crime and according to the framework, intent does not justify execution.’ Psychology turned to Nicolai. ‘And you?’

‘I feel like it must be right to kill the man, though,’ said the Mask, frowning, and Psychology turned back to it. ‘His actions are so much worse. Plus, you said I have foreknowledge. So I know this man will do these things. How can it still be wrong, according to this framework?’

‘A Kantian would argue that even with foreknowledge, the man hasn’t committed a crime yet,’ replied Psychology. It’s tone shifted, as though lecturing from a memorised book. ‘To kill him—or to kill seven others just to reach him—would be to treat people as tools for an end, which Kant strictly forbids. Moral worth comes from respecting each person’s autonomy, not from trying to control the future. Even if the outcome is certain, the act of killing either the innocent or the not-yet-guilty remains wrong.’

The Mask’s frown only grew. ‘But what if we move forward in time, and now the man has done the act and society has been upheaved, and millions are dead, and we can say quite clearly that if only we had acted back then and not been concerned with all the… all the…’

‘Self-righteousness,’ put in Nicolai helpfully.

‘Right, without worrying about all that, we could have stopped it... Would this framework still insist it is the morally correct choice?’

‘Yes!’ said Psychology excitedly, clearly enjoying all this a great deal. ‘It would still be considered morally correct, because it was made in accordance with duty and respect for moral law, not outcomes—which are considered irrelevant. The framework holds that the only thing truly within our control is the morality of our actions, not the future. One does not act rightly to guarantee a good world, but to remain personally just, even if the world burns for it.’

Nicolai burst out laughing. ‘That is without a doubt the stupidest, most self-satisfied philosophy I have ever heard.’

‘What’s the point of morality if you don’t ever do anything?’ muttered the Mask unhappily. ‘No, this philosophy isn’t for me.’ It scowled. ‘I would kill the seven to kill the man. I believe that is what a Better Man would do.’

‘I agree,’ said Nicolai easily. ‘If I were being a Better Man, I’d kill them all. The ends justify the means.’

‘The utilitarian view,’ said psychology, nodding as though it had expected as much. ‘What if you weren’t being a Better Man?’

‘Then perhaps the collapse of society would benefit me.’

Psychology chuckled, entirely pleased by this. ‘Wonderful. Wonderful!’ It gestured to the tracks again and cried out, ‘Next! Now, there are two individuals, one on either track. As always, you may divert the train from one to the other. However, this time, you possess detailed knowledge of the individuals involved. What factors might influence your judgement? Let’s begin with A3.’

‘Oh,’ squeaked A3, seeming surprised to be asked. ‘Who are they?’ it asked.

‘They are people you know.’

A3 pondered. ‘Like my Quiet Brothers?’

‘Correct. These are not strangers. One of them is, indeed, one of your Quiet Brothers. And the other…’ Psychology’s teddy-bear face twisted into an eager grin. ‘The other one is Nicolai!’ On the track below, the relevant copies appeared.

A3 jerked with shock. ‘I would save Father, of course!’

‘Why?’ asked Psychology, leaning forward.

‘Because he is Father!’ A3’s limbs were flexing with fear and anger.

‘Provide three reasons,’ intoned Psychology, ‘why his life holds greater value than that of a Quiet Brother.’

The Mask observed this with a growing frown, while Nicolai watched curiously, interested in what answers might be given.

‘Father looks after me… Father cares for me… Father created me…’ A3 muttered, managing to subject Psychology to a glower with only its cameras.

‘Correct on every point,’ said Nicolai, smiling. He patted the bot on its head-part, and A3 relaxed at his touch. He hadn’t even tried to make A3 so devoted to him. It just was. It was actually quite nice.

‘Is this necessary?’ broke in the Mask. ‘He’s only days old, how can he even give a proper answer to questions like that? It’ll just upset him…’

‘On the contrary,’ replied Psychology. ‘Morality is often difficult. I am merely ensuring A3 is properly prepared for the choices a harsh world might present… But you’re right, it isn’t fair for only A3 to field difficult questions. Plus, I believe it would be worthwhile for A3 to observe the two of you giving answers to such questions, so it might learn from you both… So, Nicolai.’ Psychology smiled at him, that big teddybear face curving.

He eyed the Module suspiciously, wondering if being able to freely ask him such questions might’ve been the whole reason Psychology had been so eager to jump into this activity. ‘Go on.’

‘Let’s say on one side of the tracks, we have… Jo. And on the other, Beth!’ Psychology waved, and two squirming figures appeared, tied to the tracks. ‘Who would you—‘

‘Jo.’

The bear frowned at him. ‘That was quick,’ it said, sounding quite put out.

‘I have thought on this already, back when I operated with them. Of those still alive, I would prioritise individuals in this order: Jo, Beth, Kleos, Maric, Perro Azure, Maxine, Daksh, John, Cait.’

‘Why?’

‘Primarily strategic reasoning, plus some personal factors.’

‘Such as…?’

‘Jo and Beth were the most useful to me, compared to the others. But if I had to choose, I would pick Jo over Beth. I simply care more about her existence. As to the next two, Kleos and Maric were both useful sources of information. If I hadn’t decided to be a Better Man and put them in the Coffin, plus the necessity of Kleos’ quest, I would have wanted to take them with me. Perro and Azure were young and easily led, especially Perro. Maxine held control over Channel 2. Daksh had that arm. John was slightly easier to manipulate than Cait.’

Psychology nodded thoughtfully. ‘Why do you care more about Jo’s existence?’

‘I respected her and enjoyed her company. I felt similarly toward Beth but to a lesser degree.’

‘What about the rest?’

‘I respected Kleos because he was cunning and quite wise, and before his punishment I believe he may have been a great fighter. We also talked many times together and I believe we “bonded.” I suppose, in a way, he was like a friend. I also appreciated that there were times he was genuinely helpful, more than he needed to be considering our bargain. The others…’ He considered. ‘I found Azure amusing at times, annoying at others. Maxine’s voice on the radio helped me against the Dark, in the early days. I almost liked Perro, but I found his lack of a spine somewhat offensive.’

‘What about you?’ Psychology asked the Mask. ‘Same choice. Jo or Beth?’

The Mask just glared at the bear. ‘I’m not choosing.’

‘Okay, so the lever is left alone, Beth lives and Jo dies,’ said Psychology, nodding. ‘I expected a different answer. Surprising, interesting!’

‘No, no!’ cried the Mask. ‘I don’t mean that! I mean I refuse the whole idea of this stupid thing. I would save them both!’

‘Save them both?’ asked Psychology, putting a considering paw to its chin.

‘I would jump down from here, and go and save them before the train arrived,’ snarled the Mask. ‘Look how slow its going! I could do it!’

‘Okay, that’s fine,’ said Psychology. ‘So Nicolai would save Jo, and the Mask would save them both!’

‘I want to change my answer,’ came A3’s voice. ‘From back with whether I’d save Father or my quiet brother. I too, would save them both!’

‘Wait,’ said Nicolai, frowning. He scoffed. ‘The whole idea of the thought experiment is that you cannot save both, you have to choose. If I could save them both just by jumping down and spending a couple minutes removing them from the tracks, then I’d save them both.’

‘I never said you couldn’t,’ said Psychology, smug.

Nicolai glared at the Module, feeling quite distinctly that he’d been tricked. ‘I have a feeling that if I’d said I wanted to save them both, back then, you would have said that I couldn’t, that the purpose of the thought experiment is to choose.’

‘What an astonishingly suspicious mind you have,’ said Psychology, grinning. ‘Onto the next question!’ the Module added cheerfully.

And so, A3 learned things. Over time they performed quite a few more of Psychology’s thought experiments. How exactly these might influence the bot in the future, Nicolai couldn’t say.

Still, he’d said that he wanted A3 to understand itself. A part of him hoped that by helping A3 to understand itself, so would he learn to do the same. Lessons like this seemed a good way of making that happen.