Last Friday I travelled (all the way!) to the Microsoft Reactor community
hub in Shoreditch to attend the first ever Coed:Ethics conference, the
first event of its kind dedicated to exploring the ethical issues that face
software developers and other technologists. The conference consisted of eight
talks, followed by a panel and open discussion.
Eight half-hour talks made up the bulk of the day, from a range of speakers
across numerous topics around ethics in technology. The videos should be
available in the near future, but I've written my summary of each talk below.
When Data Kills
The first ever Coed:Ethics talk was given by Cori Crider, a human rights
lawyer. Her talk focused on tech companies' involvement with the military,
covering topics like machine learning for drone strike targeting. She also
talked about the efforts to draw up principles governing the development and
use of AI, for example prohibiting its usage in weapons systems.
One event she talked about was Google's involvement in Project Maven, a
research program aimed at improving object recognition in miltary drones.
Following an open letter signed by thousands of employees and at least a
dozen resignations, Google announced that it would not be renewing the contract.
This illustrates the power that employees in technology have to affect the
decisions their companies are making, especially if they are willing to stand
up in public for what they believe in. The people who are building these
technologies need to have a say in how it is being used.
For me, a key question from her talk was: "what is the line between lethal
and non-lethal help?" My employer recently worked with the US Air Force
on a project to improve the efficiency of scheduling aircraft refuelling
flights. Is that far enough removed for comfort?
What is a Data Citizen?
Next up was Caitlin McDonald, talking about what data citizenship looks
like by drawing parallels between our civic lives in relation to law
and the government and our relationship with data science.
She cited "Weapons of Math Destruction" by Cathy O’Neil, which makes
argues that there are particular kinds of application that cause problems;
those that lack feedback cycles for improvement, for example. Caitlin
suggested that we as "data citizens" deserve to know what the rules we are
being evaluated by are and have a structure to challenge them (just as we
can read the law and appeal a legal decision in civic society).
She introduced several existing tools to support ethical decision making,
including the Open Data Institute's Data Ethics Canvas, but made the
point that you also need a culture of ethics in a company. It's not enough
to apply the toolkit, you also need people willing to listen to the results
and act on them.
Data Science in Action
The third talk was by Emma Prest and Clare Kitching from DataKind, a
group that "[brings] together top data scientists with leading social change
organizations to collaborate on cutting-edge analytics and advanced algorithms
to maximize social impact". In particular, they talked about the need to
embed ethics throughout the data science process, illustrating their points
with an example of working with a food bank to identify individuals in need
of further counselling.
They suggested building ethical concerns into the project right from the
scoping and kick-off stages, beginning by considering what the high-level
risks around worse-case outcomes and data could be. Then, as the project
continues, building bias assessments into the data and algorithm checking.
They highlighted the importance of involving a diverse range of people in the
process; not just the clients and the data scientists, but also the
developers, the users and other stakeholders.
One particularly interesting point they made was the need to consider whether
the model being developed is appropriate for the "data maturity" of the
organisation in question. In case study there was a board member with
experience in maths and data science, but what if the model continues to be
applied after they have left? How do you ensure ongoing governance of the
model and embed understandability and transparency into the process, so that
it doesn't end up getting misused or misapplied in the future?
Another was the impact of open sourcing tools and models; their open question
was whether it is better to make these things open source so that the broader
community can gain maximum benefit, or whether it's actually preferable
overall to keep a higher level of control and governance by keeping them
proprietary? Thinking about the context and the value is important here, are
we making things better enough?
Psychology of Ethics 101
The final talk of the morning was by Andrea Dobson, a psychologist
interested in what makes good people make bad decisions. She talked about
Milgram's work on obedience, highlighting the result that around 75% of people
will conform to something that they know is wrong, and that conformance is
actually more likely in more ambiguous circumstances. Given that we are
generally taught to defer to figures of authority, how can individual
contributors effectively raise the concerns to management?
This applies to technology, too; Andrea related the results of the Stack
Overflow survey, where nearly 60% of respondents said that upper
management was ultimately most responsible for code that accomplishes something
She advised listening to your emotions: do you feel guilty about what you're
doing? Why are you doing it? What matters to you personally?
Companies become ethical one person and one decision at a time.
One thing that can help this is providing an environment of psychological
safety, where individuals feel comfortable speaking up and voicing their
concerns. If you don't feel you have that, she suggested speaking to other
people (HR, regulatory bodies, friends, family, ...) who can help you figure
Thinking Ethically at Scale
After lunch Yanqing Cheng talked about how and why we might care about
ethics, and how we can manage to both make the world a better place and
feel like we're doing so.
She highlighted the issue of human intuition and its failure to scale - for
example, people would donate roughly $80 dollars to help birds in oil slicks
whether they were told it affected 2,000, 20,000 or 200,000 birds. Part of the
problem is simply being bad at imagining large numbers; when she reframed
worldwide road traffic deaths as being more than every British Airways flight
from London to New York for a year crashing into the sea, that really brought
home the magnitude in a way the number alone didn't. The key lesson in that the
most effective actions are not always the most intuitive.
She talked about the Effective Altruism movement, trying to measure the
impact of different charities so that individuals can maximise the impact of
their personal donations. The Who (can have an impact on your goal),
How (can they help or obstruct), What (can I do to affect their
behaviour) model, a useful brainstorming technique, can be used to try to
identify the actions that give the biggest payoff for the largest number of
people and focus on what you can achieve as an individual.
(Bonus points for being the only conference talk I've ever seen that included
lessons from Harry Potter fan fiction.)
Harry Trimble was next, talking about the power that designers hold in
a world where software is everywhere, and their responsibility to give power
to those without it. He commented that ethical decisions are unavoidable,
even if the only option you may have is refusing to do the work or quitting
entirely. In particular, he talked about the issues around authentication and
consent in data management.
Two topics stood out for me:
Informed, shared and delegated consent, allowing groups and communities to
participate effectively in decision making or involve other parties that
might have more appropriate information; and
Accountability and transparency in automated decision making, for example
providing a "snapshot" that identifies the version of the system and its
state so that you can review and appeal the decision.
One interesting tool he talked about was the Data Permissions Catalogue,
a collection of design patterns for sharing data. The idea of tools like this
is to bring a shared language to ethical issues, making them easier to
describe and discuss. This reminded me of one of the core ideas of
domain-driven design (DDD), the importance of having a shared, consistent
vocabulary (the "ubiquitous language") within teams and projects.
A Responsible Dev Process?
This was a two-part talk, starting with Adam Sándor talking about how
the idea of breaking down silos within companies and giving responsibility
and autonomy to teams can help people move from the idea that they're just a
small cog in a large machine to actually having ownership of what they are
building. These multi-disciplinary teams are more inclusive and having the
responsibility means you're more likely to make the right ethical decisions.
Next Sam Brown from Doteveryone talked about their work in Responsible
Technology, defined as technologies that:
Do not knowingly create or deepen existing inequalities;
Recognise and respect everyone’s rights and dignity; and
Give people confidence and trust in their use.
She talked about the power that technologists have as people with the skills
to turn ideas into reality and introduced the "3C model", framing ethical
conversations around Context, Consequence and Contribution. Doteveryone are
working on assessments and tools to support the discussion, providing a model
for responsible practice and making responsibility the new normal, a key
business driver for growth and innovation rather than a side concern.
One question from the audience was whether adding more regulation and
associated cost to conform with security and other standards would entrench the
positions of larger companies and discourage smaller companies and startups.
One suggestion in response was being open about products' status, giving users
information about what has been done and what is planned. But as a company or
group and as individuals it's important to know your limits; if you're a
bridge-building startup, your bridges still have to work.
How to Build a Good Product
The final talk of the conference was from Steve Worswick, creator of the
Mitsuku chatbot (and possibly the only man to have both three Loebner
prizes and two tracks on Scottish Clubland 3). He talked about some of the
ethical decisions he made when developing the bot, which is a general-purpose
chatbot rather than a virtual assistant like Siri or Alexa.
One specific decision was to train Mitsuku using supervised learning, rather
than allowing unsupervised learning like the infamous Tay chatbot. This
is extremely time-consuming, especially when trying to keep up-to-date with
current affairs so Mitsuku can respond appropriately, but allows him to keep
the bot family-friendly (despite the revenue temptation from more "romantic"
users). It can learn from a specific user, but only for that user, then
Steve chooses which rules to add to the general set.
Another set of interesting decisions was in dealing with the ~30% of users
who are abusive in some way (50% are "normal" users, the other 20% skeptics
and academics). Tame responses only seemed to encourage bullies, so the bot
currently uses a "5 strikes and you're out" system, combined with a flag on
the user that allows the bot to respond more strongly to abusive users
(although it never swears back at them).
The day finished with a panel and open discussion. A lot of different points
were talked about, I've summarised a few below:
Education vs. regulation - where do we start? We need to educate
politicians in the issues before we can talk sensibly about regulation. But
laws are often behind the times, and in a fast-moving industry we have a
responsibility to do something.
Ethics as a product - can companies use ethics as a competitive
advantage? Are there any downsides to doing so? People want to engage
with ethical companies, but misuse can lead to cynicism if those
principles aren't seen to be upheld.
Not Invented Here - what prior art is there, is this really something
we need to figure out outselves? Psychologists and doctors, for example,
already have mandatory ethics training and advice panels. We don't
currently ask about ethics in interviews, so even where CS students have
ethics modules they aren't seeing the benefit when it comes to entering
Is Apple ethical - should I get a different phone? The most ethical
smartphone available is keeping the one you already have. Issues exist
around the supply chain, e.g. conflict minerals, but Apple are doing good
work around privacy and federated machine learning. There is a bottom
line below which we won't accept a company, but it's hard to have a
binary yes or no for most, especially when their products give a lot of
A few key thoughts:
Understanding power: it's important for tech workers of all kinds to
understand the power they hold. Software is, as we're often reminded,
"eating the world", and as the industry grows skilled people will continue
to be in high demand and have power as a result.
Collective action: it would be easy to think that, because the
working conditions and benefits in tech companies are already generally
good, we have less need of unions and professional bodies, but it was clear
from the discussion that there is a role for these groups in driving issues
like ethics across the community. For example, the Tech Workers
Coalition is working toward an inclusive and equitable tech industry,
covering all kinds of workers involved in the industry. In a timely
fashion, as I write this, the ACM is about to publish an updated
ethical standard for the computing profession.
Diverse groups: a few of the talks touched on the idea of involving
more people, and more diverse groups, in discussions and decision making.
This helps make sure we aren't missing simple problems that are just not
obvious from our own point of view. Especially when building consumer
products it's important to build groups that reflect broader populations
and can bring more ideas and ways of solving problems to the table.
Finally, I'd like to thank the organisers for their work; the day went very
smoothly and it was great to see such a diverse group of speakers and
attendees talking about this important issue. Hopefully this is just the
start of something much bigger!
Disclosure: Pivotal paid for my attendance of this conference as part of
my professional development budget, and were a Gold level sponsor of the
conference. However, I have written this article as an individual; it
reflects my position and opinions, not necessarily those of my employer.