What If Software Had a Soul?

We inspect people for who they are and software for what it does. Yet software also has biases, values and a recognizable personality, and those characteristics affect the outcomes it produces. A product can satisfy every requirement in a tender and still be wrong for the future we are trying to create.

I have sold learning software for twelve years. During this time, I have been asked where the data is stored, whether the product supports single sign-on, how it integrates with existing systems and whether administrators can export the reports. I have been asked about accessibility, implementation timelines, branding, security and, eventually, whether it can cost less.

I cannot remember anyone asking what the software believes. Not whether it produces biased answers or has been trained on the wrong data. I mean something much more fundamental: What does it believe learning is? Does knowledge come from an expert and move into the learner? Is it produced between people? Is the purpose of learning to remember, comply, become employable, change your behavior or develop the ability to think for yourself?

None of those questions normally appear in an RFP. That is strange because software is rarely the thing anybody actually wants. Nobody wants a learning management system (LMS). They want people to learn. Nobody wants recruitment software. They want to find the right employees. Nobody wants hospital administration software. They want patients to receive the right care. The software sits between the person and the desired outcome. It is a proxy.

We understand this perfectly well in other parts of our lives. A person using Bumble does not actually want Bumble. They may want a relationship, a marriage, sex, companionship for the evening or simply confirmation that somebody attractive is still prepared to swipe right. The app is only the intermediary.

A dating profile then becomes a crude proxy for compatibility. Does this person smoke? Do they want children? Are they religious? What kind of relationship are they looking for? None of those answers can tell us whether we will love somebody. They can tell us whether we are trying to build remotely compatible lives.

An RFP is, in some ways, a dating profile for software. It contains a long list of characteristics intended to establish compatibility. The difference is that the software profile is almost entirely functional. It tells us what the product can do, but very little about what the product thinks we are trying to become.

Two learning platforms can support the same integrations, contain the same administrative features and satisfy every requirement in the tender while embodying entirely different ideas about learning. One may treat learning as the transfer of correct information from an authority to an individual. Another may treat it as something people construct through participation, disagreement and reflection. Both may include courses, assessments, dashboards and discussion boards. Both may work exactly as promised.

But they are not producing the same thing. Software is not a neutral route toward an outcome. It is a theory of the outcome, translated into code.

We have learned to call some of this bias. That is useful, but bias is not a big enough word. Bias sounds like a defect: distorted data, an unfair recommendation or a system that treats one group differently from another. It suggests that if we could identify and remove the distortion, we would eventually arrive at neutral software. I am not convinced neutral software exists.

When I moved to the United States at sixteen, I encountered a multiple-choice test for the first time. My immediate reaction was: Why would they provide you with the answer? To me, a test required you to know the answer and produce it yourself. The American test asked you to recognize it among several alternatives. That felt completely insane to me and entirely normal to everybody around me.

Neither format is merely a container for assessing knowledge. Each contains an idea about what knowing means, how it should be demonstrated and which mistakes matter. One rewards the ability to produce an answer. The other rewards the ability to recognize it. One tolerates greater variation in how knowledge is expressed. The other makes answers easier to standardize, compare, and score at scale. The multiple-choice test did not need to announce a philosophy. Its philosophy was built into the format.

Software works in the same way. Somebody decides what appears on the first screen, which actions require permission, whose contributions become visible, what the dashboard measures and what the system calls success. These choices may be deliberate expressions of a philosophy. They may also be inherited from the founders, the market, the business model or simply the first customer who paid enough to influence the roadmap.

The buyer normally sees the result as a collection of features, but two products can contain the same features and still feel as though they come from different worlds. We already describe software as rigid, intuitive, controlling, collaborative, demanding or forgiving. These are not descriptions of what the product does. They describe how the product behaves toward the people using it.

Features are the anatomy of a product. Its personality appears in how those features behave together: what they make easy, what they make difficult and how they respond when people do something unexpected. Its soul, if software has one, may be found in the future it quietly treats as desirable.

This is not unique to learning. Hiring software also contains an idea of what a promising employee looks like. A conventional career path, recognizable qualification and familiar employer may be easier for a system to identify than an unusual sequence of experiences. Nobody has to explicitly decide that the unconventional candidate is worse. The system simply makes one kind of person easier to see.

That matters because software does not remain politely inside the procurement document. It enters the organization. People change their workflows to accommodate it. Managers begin relying on its dashboards. What it measures becomes easier to discuss, while what it cannot see becomes harder to defend. Eventually, the product may not simply reflect the organization. The organization may begin reflecting the product.

AI makes this more urgent, but it did not create the problem. Software has always structured our choices. AI increasingly recommends, prioritizes and sometimes makes them. The personality and values buyers ignored when selecting the system are gaining more influence over the outcome they hoped to achieve.

Dating profiles cannot tell us whether a relationship will work. They are useful because they force a more basic question into the open: What are you looking for? The answer changes the meaning of everything else.

The same should be true when we buy software. What are we trying to make happen? What assumptions does the product make about the people involved? Which behaviors does it encourage? Whose judgment does it privilege? What does it recognize as progress? What kind of organization will become easier to operate once the system is inside it?

These questions do not replace questions about security, accessibility, reliability or price. They are what make the technical answers meaningful. A product can be secure, scalable, compliant and perfect on paper. It can possess every feature we requested and perform exactly as promised.

We keep buying software as though it has no character of its own. Then we act surprised when its character begins shaping the outcome.