Liveblog by Alexis Hope, Sam Klein, Willow Brugh and myself
Karrie has been a pioneering researcher on how technology shapes our lives. She is also an expert on algorithmic auditing, looking into ways that these technologies are shaping our social lives.
As we think about the work Karrie has been doing to address the legal barriers to producing research — and the legal barriers to consuming research — we will also talk about how we think about our roles and responsibilites adjusting the systems.Three weeks ago, Karrie teamed with other researchers on a lawsuit to challenge the barriers to doing algorithmic auditing because the data is tied up by Terms of Service.
This work started for Karrie around 2012, when she helped start the Center for People and Infrastructures. We looked at cases like public transit, where machine learning might be trained on a camera operator that always stop the film when a person is black.
We worried for instance that people being black would be constantly observed. In reality, we found that black people weren't being observed at all. The camera tracked one researcher perfectly; it did a perfect job. But a black researcher wasn't tracked at all; leading to the quip that HP's cameras were racist.
Karrie describes some similar issues:
The StreetBump project for tracking potholes by leveraging crowdsourced data. In 2014 it was revealed that some poor parts of Boston weren't surfaced by this, and only worked in wealthy suburbs. The problem was later reported to be solved.
Amazon Prime: Roxbury is the only central boston neighborhood that wasn't getting same-day delivery. When this was revealed (publicly), it was fixed the same day. But the information has to get out for this to happen.
Predictive Policing: we need to know which regions are being observed and tracked. Is it only poor areas? Is it leading to bias?
Pokemon Go: Why are there fewer stops in poor neighborhoods? There are many reasons for this... crowdsourcing plays a role.
Self-driving cars: What decision does a car make when the owner dies, if the car has to choose what object or person to hit while braking?
Fair housing: are minorities less likely to find housing via algorithmic matching systems?
AirBnB: A black person renting on AirBnB was found to get less for their unit than someone who was white (via the automated algorithm).
We've addressed some of these things with the Civil Rights Act (1964), Fair Housing Act (1968), Housing and Community Development Act (1987). It took 20 years to add disability status.
At that time, HUD further supported special projects, including prototypes to respod to new or sophisticated forms of discrimination.
Auditing:
Traditional audit - match pairs on family and economic features, varying race; successively visit realtors. They found that minorities (blacks, hispanics, asians) were told about fewer units than whites.
Digital audits -
We wanted to explore what happens with the new proliferation of algorithmic realtors — she and collaborators wrote a paper auditing algorithms. It's hard to get this data and code from companies. Although, researchers were willing to give their code. Ex: finding naked people (original code from a 1996 research paper). There are explicit lines added to find black people (more specifically, special-casing different ranges of skin hue)
Ex: algorithmic awareness. It took 18 months to get a probability sample, which was too long. Collecting data manually is also not ideal. Some people won't provide it, and it's not reliable when provided. Latanya Sweeney searched for names online; for white people, you got automated ads saying "NAME found!". For black people, you got automated ads saying "NAME arrested?" linking to similar white-pages or online search tools.
You can also scrape everything. This is a violation of the CFAA but many rsearchers are doing this all the time. You also see this at hackathons and published in major conferences around data mining like KDD, WWW, etc.
CFAA and Terms of Service are a problem. We had the FeedVis project, which was successful, that we could have continued by scraping FB, but we shut it down intentionally because of their Terms of Service.
Sockpuppets was another project in which you could create bot accounts with different demographic qualities and see how they were treated by different algorithms online like real estate or price discrimination.Wilson et al - price discrimination - buying things in mobile phone costs more than on desktop, the more times you go to a site the more expensive it is - used 300 real people and a lot of additional sock puppets to get enough data.
Collective audit - Finally, you could do a collective audit where lots of people provide common information that offers the data to do the kind of auditing we are describing.
We were excited when our work was recently cited by a White House report on Big Data and Algorithms. The report considered: transparent algorithms, spot testing, the right to appeal if the algorithm is not fair.
Block: CFAA, the Computer fraud and abuse act of 1986.
1986 was a different environment then than today. A lot of people didnt have broadband or access to email.
With the ACLU and colleagues, we have sued the US Government, a few weeks ago. We are waiting to get a response from the governemtn to see what happense. The CFAA prohibits unauthorized access to proected computers - it also includes any website accessible on the internet. The thng that is most confusing is that you "exceed uathroized access" - the specific point targetted in the lawsuit.
Violating this: 1 1 year ax prison sent and fine. 2nd volation is prison sentence of up to ten years and a fine - but your intentions dont play a role. Even if you dont intend to do harm.
Many people are violating this every day without even realizing it.
In some cases, companies ban manual collection of data. One might interpret this as sitting by a computer with a pen and paper and writing things down - this might be a violation. Other things you can't do include reverse engineering. It is frustrating because terms of service are not static. Just by going to the website you agree to the terms of service. Just by going to pokemon go site you are waiving your right to participate in class action suit.
Why is this so important? Researchers, journalists, everybody is affected by this. As a professor, Karrie cares about protecting her students. it can even lead to inability to publish work! IRB may not approve of work, finding employment (ie. at facebook), might not be possible because of a study that you did.
Reputation of students & faculty are at stake.
Norms are crucial here because lots of computers ceintists scrape and use bots and do so for social good. last fall we had event here called Freedom to Innovate. And I was frustrated by the ACM around this issue. But now thanks to Amy Bruckman and others there is now an ACM SIGCHI Ethics Committee that will be looking into this and other issues. Recently, on Amy's blog she talked about how documenting your intent might actually be bad for your research until this problem of breaking ToS and CFAA is resolved.
What the community does matters. What the community cares about matters. Hopefully we can reach some kind of policy change together that will benefit not just researcher but everyone.
Nathan: I'm fascinated by the different approaches you have taken. You both face environments where researchers are constantly scraping site without following their terms. Karrie, you're taking part in these legal cases to change the rules; Alexandra, you are trying to build on the breadth of other efforts to share research, and build a system that is effective and widespread. How do you see the established institutions and your relationship to them?
Karrie: One issue for researchers: should Institutional Review Boards consider legal as well as ethical issues? I ended up on an IRB board because this topic came up. It was nice to see topics get approved within a day, rather than 6 months or more (due to legal uncertainty) as it was before.
Moving to dealing with bigger corps, I've had great interactions with FB's data scientists. But their plan to share data didn't come to pass. That involved the organization as a whole, and their lawyers, not just their data science, which I don't really understand.
Q: Alexandra, when people talk about scihub they often want to rehash Open Access debates and what their instutitons should do for sharing openly. But your project started for outsiders, outside those instuitutions. Do you see your work helping change the publishing industry, or just sharing your [system/resource]?
A: I think that Sci Hub if it continues to exist then the publishing industry will have to adjust because they will not be able to reap the big profits they currently enjoy from subscriptions.
If you ask my personal opinion about open access, I am for it; trying to promote it.
Q: I'm curious about the dynamic in both these situations, where we have lots of people already doing the thing (disobeying) and Alexandra and Karrie have become more visible because they are trying to address the problem in a large scale way?
How has this been - Alexandra, to start out - did you expect this attention, has it changed how you work?
A: Well, I have to say from the beginning of sci-hub's existene there was a lot of attention paid, but it was largely covered by local (Russian) media more than international media. [so: not much!]
Q: And you have also faced legal actions from US publishers. How does that affect you and Sci-Hub?
A: For obvious reasons, there are many challenges: domain registration (and updating), one domain was closed, and legal actions taken. For other resources it's not even necessary to take a legal action. I'm not going to the united states or europe, trying to be cautious.
Q: Part of your strategy, Karrie, is to reach out to other researchers in this space and ask them to come forward? Can you talk about what that means for them?
A: It's similar to Amy Bruckman's blog post. When I talk to the media, they ask me to find more people. There are many (not my job to out them!) doing similar work. Many have gotten Cease & Desist papers and stopped doing the work, but they don't want that to be public, for fear there might be something negative for their work that results.
6 out of my 8 phd students are not US citizens. Many of my colleagues are international as well.
For any of you whose visa gets revoked (or know people from your country whose visas are revoked) for unexplained reasons, it's very hard to admit to something like this.
Q: I'm curious what changes you think can be made to the commercial interest of providers, so they aren't so strict about enforcing draconian Terms of Service [which most set up presumably to protect their business, not to prevent spot checking or researching algorithms for bias]. Can we add incentives for companies to have research-friendly ToS?
A: Many have a cut-and-paste ToS, it doesn't seem very custom.
I have not asked a company to change their ToS. I have talked to lawyers who have said that they can't be touched; they have to be there. My talks with data scientists end up talking to lawyers and they end there. [so maybe I should talk to lawyers next]
Q: Thanks Alexandra for the great work you've done for the community. Do you get any support from academia for your website, particularly from top institutions like MIT who do have their own access? do you feel any are supporting your actions; anticipating the answer is no, do you think there's anything that they should do? that applies to other panelists as well. Can universities leverage their power to help in these cases and promote auses we care about?
A: I'll try to take pauses while answering :) I think that MIT has poor access to publications at least that was a frequent user complaint. Second, as far as support is concerned, there is support from users who offer donations. But there is no official support because of obvious legal reasons.
Forbidden