I’m going to try to offer an accessible explanation of the topic modeling I’ve been doing over the last couple of weeks. First, a massive shoutout to Joanne for scraping the dataset for us, to Martin and Kevin for their help with the cleaning/sorting/tagging, and to Micki for her suggestion to topic model just the syllabi (the “syllabus” is basically an abstract for each case) instead of the full text of each court decision. The data gathering/production has been an awesome group effort, and the topic model just feels like one piece where I’m steering the ship.

This will be Part 1, covering what LDA topic modeling is, and my first attempt at doing it on this dataset. I’ll link to the R code once we finalize which parts of our project are public and private — for now that’s only in our private data repository but I’m happy to share the code with anyone privately if that’s of interest. Part 2 will cover how I tweaked the model to improve the results.

So, what’s a topic model? The one I’m using is Matt Jockers’ LDA model, which he explains with a lunch buffet metaphor in this blog post. I’ll be honest… it’s not my favorite, but worth a read for a first taste. I’ll see if I can do any better here, though, by walking through my experience.

**Pre-firstly, I do some basic text cleaning. I take out all the punctuation, the numbers, and the whitespace, and make all letters lowercase (otherwise “taco” and “taco,” and “Taco” and “TACO!” are all different words). I also remove common words, based on a pre-determined/widely recognized list of stop words. For this project, if I leave in words like “and”, “because”, and “he”, it will be hard to get clarity about words that are more interesting to me, like “communism” or “obscenity.”

Firstly, I set some parameters for the model. (After that I basically just run a couple lines of code and then wait a few minutes for the computer to execute a series of computations, so “understanding topic modeling” is largely about knowing what the parameters mean.) One parameter I set is how many topics I want the model to return. This step is, as far as I can tell, a bit soft — i.e. I have a feeling that 20 is a more appropriate number of topics than 5 or 100, because Joanne and I brainstormed the topics we saw in the cases and came up with about 12-15. I want to give the model enough breathing room that it comes up with different, distinct topics (for example, to separate “school” and “flag” as two topics) but not so much that we lose the thread of a common idea (I don’t want “school” and “teacher” in two different topics). I started with a number that felt right, 20, knowing that I could adjust after looking at the topics I got.

Another parameter I set is how many times I want the model to sample over the topics. (This is the Gibbs Sampling part of Jockers’ blog post, FYI.) The model starts by taking all the non-stop words that occur at least 5 times in all the corpus, and dumping that list into 20 bins in 20 different random orders. This is the first iteration of our 20 topics: 20 groups of the same words, but randomly ordered as if the words were all equally unrelated. The topics will eventually be ranked from top to bottom, so words at the top are more “important” and words at the bottom are less important in that topic. All words in the whole corpus are present in each topic, they are just ordered by the statistical likelihood that they occur together. In the end, I’ll end up taking the top 10 words from each topic, but I could also examine the top 50 or top 200 if that would be useful to me.

Back to our 20 random buckets: because I set this parameter to 5,000, the model basically “re-creates” the topics 5,000 times, decreasing the amount of randomness with each iteration. Randomness is decreased by examining one word at a time, in the context of its original document (“which other words are present in that syllabus?”) and its current topic (“which other words are important in this topic?”).

So… the model eventually ends up computing that, for instance, the word “communist” co-occurs disproportionately often with the words “party”, “foreign”, “control”, and “soviet,” and those all rise to the top of one of the topics. I can export a spreadsheet that has 20 columns, each with the top 10 words in the topic. Here’s a selection of the topics my first model returned:

A human could read a case syllabus and tell you that that particular freedom of speech case is about communism and foreign policy; the computer can compare its words to all the other words it could have had, but didn’t, and tell you that it has a relatively high proportion of Topic 11.

This brings us to the next step. At this point, we have 20 topics — we also want to know how those topics are distributed over the 573 case syllabi. Not every case will fit perfectly into one of those 20 topics, of course, and some may be a mix of multiple topics. That’s totally fine, and I can make a second spreadsheet to explore that. This one is much bigger: each column still represents one topic, so I have one “obscenity” column, one “communism” column, up to my 20 columns. Now, though, each row is for one of the 573 case syllabi, and each cell in the row is the percentage of the case taken up by the corresponding topic. Each row adds up to 1 (i.e. 100%), giving us the distribution for each case across 20 topics.

Here, we can see a few cases that might potentially be labeled as “obscenity” or “communism” cases, because they have relatively large amounts of those topics. It’s possible to use this data to figure out if two topics consistently occur together, or if a topic ebbs or flows in importance. (For example, in the screenshot above, all of those cases register a 0 for the “university” topic — that is an important speech topic, but it’s not “popular” until later cases.)

These topics and topic distribution are a promising start, but Topics 14 and 15 in particular tell me that I can make the model better. That will be part 2!

For me, it all feels a little messy and hard to wrangle in right now – which it has all along, but I think I expected that to change a bit as time went on. I have really struggled with getting this post up this week – I have been stuck by getting stuck. What is fair to ask of students at this time? How far to push colleagues? How far to push colleagues to push others? Who can help us gauge this? Our deadlines for the semester feel aggressive in the face of the emotional labor we are asking of our students. How could I have not realized that for the sake of the students and also the sake of our group? Our team has been wonderful – but I am struggling not to see some degree of my own failure here. On the other hand, our intention is something we trust and the students who have understood it have expressed a sense of empowerment at the idea. Is it right to ask/push (what is the word) for work from the students I know best? What would be wrong about it? And if there is something wrong, is that absolved by driving outreach to those I don’t know personally? There is something larger to be asked here. Before this project goes wherever it goes after this semester, I am going to have a long, hard look at how we ask what we do of students and how we engage them after they have submitted work. I am also struggling with why, in the face of all of these questions, I am still having a hard time scaling back the vision of how far to reach this semester, which has just a few short weeks remaining. I wish we had 4 more weeks…at least. This is perhaps also complicated by good news, which is that we may have confirmation from a school in Peru, our first global connect. And as another possible outreach angle, I have anonymous work (word clouds and digital images) to share from a teacher – having asked her students the questions from our student submission form. Another possible angle to bring to our group.

At a more process-oriented level, we have been sorting through work flow and diagraming our internal processes for the next round of submissions and/or if we scale to one more level/prepare to scale after the term is over. Our students currently live in NYC, upstate NY, Vermont, Texas, and Virginia – a really nice start of geographic distribution. Their work has made me pause, think, and step back. Some of it is hard to let fully sink in. All are moving in different ways.

This week’s tasks were all about organization. Each of us have been gathering so many links and images to rebus sources that our Google folders were becoming unruly. As a result, we were at a standstill as to how to go about curating the rebuses to add to the website. We settled on a method and have recently finalized the metadata thanks to Ostap. So, now the not-so-fun part of creating docs, copying, pasting, putting into folders, etc. And then we’ll finally be able to see everything we’ve collected and narrow down a final corpus. I’m a bit behind on doing this due to a busy work week but hope to catch up this weekend.

Bianca’s discovery of “rebus” in the crossword puzzle last week had me thinking about rebuses which use type only, different from the pictorial rebuses that I have been focusing on. I found an article on the English literal rebus and will try to type some clever examples. It’s surprisingly difficult to type some of these given the limitations of computer word processing (typewriter, where are you?) and I’m not even sure how they will turn out in WordPress… Answers at the bottom of the post.

B
—–
TIME

DOSE DOSE

B
E
A
T

THEHANGRE

Answers: be on time, double dose, down beat, hang in there!

This week our team focused on incentives and rewards for our contributors. This conversation reminded me of my Open Education and Open Educational Resources studies in my Digital Pedagogy course. There, I learned about co-creation practice, when the educator/instruct and the learners work together to construct a piece of material that others can use. During this practice, educators have to make sure the learners are still learning while contributing to the collaborative work. They also have to make sure to scale their work and that each learner has the necessary skill and equipment to contribute to the creation. With care and inclusivity, the instruct must note the amount of time, effort, and dedication the class takes to assemble the product that may be used for years to come. With this comes the issue with incentives.

In this vein, our team made a point to discuss the incentives and rewards not only for our awesome young ladies that contributed, took the time to meet with us, create a sample project, and assisted in some of the decision making for our site, but also the students who took the time to create wonderful works of arts for our archive. We are still deciding how best to recognize their achievements and make them feel part of the archival process, making truly a by the youth for the youth project.

This week, I’m focusing on producing what will eventually amount to appraisal of my collection of heraldry that I’ve amassed. The first step was going through my entire collection and selecting the most appropriate and viable images. This was somewhat time-consuming, but relatively easy. Generally, I selected the images that were larger, more vibrant, and more detailed. For instance:

I narrowed the collection down to roughly 33 images from the 400 or so I have collected. By appending metadata to an image, one can give it empirical context. In other words, in the case of our project, a user that comes to the site and sees a rebus will have less trouble interpreting or analyzing it if something of a “given” is established by way of metadata.

As Amanda points out, our recent outreach efforts have surfaced new learnings related to the well being and concerns of our contributors. The process of soliciting contributions has raised questions of reciprocity and the importance of understanding not just the general contexts but more significantly the immediate perceptions of our participants. Similarly, shifts from initial commitments to reticence to contribute highlight the risks of sharing personal experiences with the public at large. These learnings once again foreground the importance of engaging with a praxis of ethical care.

As one approach to an ethic of care, Rita Manning describes an ethic of care as “a way of understanding one’s moral role, of looking at moral issues and coming to an accommodation in moral situations”. She further defines care as involving “a basic human capacity to recognize and respond to the needs of others and to moderate our behavior in light of the good or harm it might cause to others.” Manning highlights four aspects of care: (1) moral attention, (2) sympathetic understanding, (3) relationship awareness, and (4) accommodation and harmony  (Manning 2009, 105-107).

In constructing a website and archive dedicated to the amplification of under-represented voices, it is easy to overlook the time and effort taken by contributors to create contributions. This raises the question of volunteerism within an environment of pressures and stresses, resulting in large part from the consequences of regimes of neoliberal austerity. It could be argued that our time under these regimes of exploitation and oppression becomes the ultimate site of extraction. As perhaps the most existentially finite resource without which life is impossible, time for living and being could arguably be counted as one of the most precious characteristics of life within the biosphere. Given the extent of already extracted time, setting aside additional time often involves difficult evaluations. Moral incentives and volunteerism need to be understood in the context of these and other pressures and stresses.

In terms of vulnerabilities resulting from publicly sharing personal experiences, the environment of public spaces under current adversarial and competitive conditions is fraught with potential abuses that often result in negative consequences. As digital media increasingly reduces the possibilities of anonymity, sharing personal experiences leads to justifiable ambivalence. This ambivalence prompts an imaginary of a world of public spaces designed to protect the privacy of individuals and their public personas. Until these protections are in place, it would seem to be a practical matter to carve out smaller protectable spaces within the world wide (and wild) web. As content creators consider these issues, the dilemma remains between the potential benefit through public visibility and the costs of such visibility.

Reference

Manning, Rita. 2009. “A Care Approach” in A Companion to Bioethics, 2nd Ed. edited by Helga Kuhse and Peter Singer 105-116. Chichester, UK: John Wiley & Sons.