One “natural” good intention that can end up undermining data privacy is enabling customer support. For example, say a website was built so that its website owners could not access passwords even if they wanted to (no “master users” or “superusers” of the database.) This would seem to be a great design decision from the perspective of security and privacy. But if a customer forgot their password and wanted to get back into their account, without such “superuser” access this could technically be impossible (depending on the technical architecture of the website.) So companies do end up introducing such “database superusers” since users end up making mistakes in this vein or wanting to undo their actions. To aid customer support personnel further, they often introduce “God Mode” tools that allow them to perform any actions on behalf of any user. But the problem is that such tools then get abused by employees, far beyond the purpose they were created for, for example to stalk users. This is further exacerbated by smaller companies de-prioritizing proper controls on such tools because of pressure to deliver quickly, and even for companies that do have such controls, I believe those may not be enough; for example, some companies do auditing and after-the-fact disciplining, but I would argue that by that time the damage is already done. Scandals connected to this have in fact happened in major companies like Uber.
Also, from a development perspective, developers would like to test and analyze their apps in environments that are as close to the “real world” as possible (how users would actually use them.) It is well-known that even with developers’ best efforts to guess all the scenarios of possible usage patterns, they can fall short of the breadth of what happens in reality when the app is released. This means that developers would like test data to be as close to real user data as possible, even possibly a small randomized sample of actual user data. Also, when troubleshooting real customer issues, the more information developers have to work with, the more effective they can be in helping out. This leads to incentivizing the hoarding of user data, with the reasoning that “it could be used later to make the user experience better and help users more.” But such troves of data that are amassed are then attractive targets for hackers and bad actors, and in companies that store even more data, the potential for subsequent damage from any hack is far greater. And as a company operates for longer and longer, the probability that they could slip up gets higher, especially with the necessity of continual maintenance of their systems.
Companies usually address this with internal data privilege enforcement systems, where only certain employees are specially authorized to deal with user data, and only the data that they specifically need for a legitimate business purpose. Use of “God Mode” tools is tracked, producing detailed audit trails, and the price of any misuse is severe disciplinary action, including termination. However, while this is well-intentioned as a setup, it still doesn’t feel adequate to me: after-the-fact disciplinary action is not enough when the damage is already done, and there does not usually seem to be much external transparency into who internally is granted access to user data internally for what purpose — this is all owned within the company, and for the purpose of a smoother customer experience it is supposed to be invisible to the customer (as if it is something they shouldn’t need to worry about.) A more robust system towards the requirement of providing the transparency that customers deserve would be one that externally exposes and audits these internal data privilege grants better.
I also have a few general points on communication and discussion of this issue. First, I think too much of the discussion is symptoms-focused rather than root cause-focused. It’s easy to criticize a certain company for having data privacy violations, but the more interesting and important question to me is what incentives led to those violations happening, especially if such incentives could apply to anyone and redirect them, even those who started with good intentions. This especially applies to companies that were once hailed as privacy supporters, and then in sudden heel turns (often derived from one or a few news articles) condemned as privacy saboteurs. This point can also be generalized beyond data privacy to many other issues people often criticize the tech industry on today.
That said, I think there is too much pressure rampant throughout the industry to deliver quickly, and while some of that pressure is a good thing, it cannot be to the point that it crosses certain fundamental boundaries, especially for example those concerning security and privacy. I do believe these boundaries can be crossed too often sometimes these days, and while this doesn’t necessarily happen with malicious intentions (“We’ll fix it later”), regardless it is not acceptable, and we should be much stricter in upholding those boundaries, no matter what.
In companies that do have proper controls (what I described with an internal grant system and a thoroughly audited God Mode tool), I wonder if a “God Mode” tool could be built that has customer consent technically baked in. At first glance, this seems to not make much sense — say for example that such a tool was built, where a customer support agent technically could not access it without a customer’s consent. How would the customer provide that consent, and how would their provision of it be technically verified? After all, if the customer is the one who forgot the very thing that allows them to access their account (e.g. their password), then from the perspective of the website, how can it know that they are who they claim they are, and therefore that the consent was valid? And if they are able to prove that, then they could just access the website directly using that proof without any customer support in the loop. In fact, this would essentially just be the same as logging into the app directly, possibly with a “forgot password” flow that still requires some other authentication. With a robust enough “forgot password” flow (which must be secure in requiring enough alternative proof but also covering enough cases of what users could forget — this seems actually to be a tricky problem that we’re glossing over here), this could presumably work for many cases, but for those customers where that flow doesn’t work, they would never be able to get back into their account again, no matter how much a human tried to help. Maybe some partial technical verification of consent could be possible: for example, the system could use voice parsing via AI to check whether the customer agent is speaking to the intended user, and if so then grant consent; but with AI today this could also potentially be faked, and this would also require storing user voice data in order to perform the comparison. In general, any alternative authentication mechanism would require storing user data along the lines of that authentication method in order to perform such a comparison.
I’m not sure what a better solution here could be. But external auditing of internal data privilege grants would be a start, and in general more external transparency around how these systems are built and why the tradeoffs exist would be important to provide customers with the fundamental knowledge that they should have when navigating technology.
