Long in the making, I just put out a blog post on downloading social media data from platforms like X, Facebook, YouTube, and TikTok, at scale! Check it out here: https://lnkd.in/gnhsKDAy The context: social media companies don't make it easy to study them. Even the research APIs they flaunt often have major limitations, which obstructs desperately-needed research & journalism. So, I wrote up my notes on how I personally did social media data collection over the course of the last 2 years, gathering 100s of millions of tweets, posts, and videos. The approach I describe is super generalizable. I hope it's useful to students and other independent researchers embarking on this work themselves. A few points I cover: ➡ Breaking down these big data projects into small pieces can be trickier than you think! When you're dealing with finicky APIs and weird scraping tools, this can take a *lot* of trial-and-error. ➡ Anticipating, identifying, and responding appropriately to errors is the crux of how you design everything else. The name of the game is designing subtasks that can succeed or fail quickly and clearly! ➡ Some cloud compute tools can make this work easier, but aren't necessary. I talk through situations in which I've used tools with all the bells and whistles, and situations where I've done stuff pretty much by hand. There's tons more, with code snippets and fun little doodles 😁 Shoutout to the Coalition for Independent Tech Research for sustaining incredible community and support for people doing this kind of work!
Dr. Milagros Miceli’s Post
More Relevant Posts
-
We had an amazing launch yesterday. Thanks to everyone who participated! 📽 If you missed the launch, you can watch the recording here: https://lnkd.in/db3YCcQe 💫 And don't miss the next talks! You can register for July 22 here: https://lnkd.in/d29xHs4c
We're superexcited 🤩 about finally launching the Data Workers' Inquiry. 👉 Join in now: https://lnkd.in/drK_kQGA Thank you for your important work! 💪 Dr. Milagros Miceli, Camilla Salim Wagner, Adio-Adet Dinika, Krystal Kauffman #AI #dataworkers #DWI
-
-
Check out this work at data-workers.org and join us on Monday for the launch event! tinyurl.com/data-workers
🔉🔉🔉 Last week, I highlighted Fasica’s report on the toll of content moderators experiencing #TigrayGenocide, and Ranta’s zine highlighting the harrowing experiences of the African women in content moderation. Find my post at https://lnkd.in/eTj_keHf. ➡ Today, I want to highlight two other researchers, this time from Latin America. Once again, you can find all of these data workers telling their stories through documentaries, comic books, zines, reports, animated shorts, podcasts and more at https://data-workers.org. ➡ We’re introducing the project on a virtual launch event on July 8 at 8am PDT/5pm CET where I’ll also moderate a panel. Register at https://lnkd.in/dixxWRQr. ➡ Alex Chávez, a data worker from Venezuela, wrote his report on how Amazon Mechanical Turk pays data workers with gift cards instead of money. Based on seven interviews with affected workers, Alex’s report underlines how this payment method fosters workers‘ dependency. Read more at https://lnkd.in/eH_S6EwA ➡ From Colombia, Oskarina Fuentes joins the Data Workers’ Inquiry to ensure that data workers like her are recognized not as mere tools but as human beings who make significant contributions to the advancement of tech. Oskarina’s animated video highlights structural issues exacerbated by economic and political crises in Latin America, including how platform workers face irregular hours, uncertainty, low wages, and unpaid time. Watch the film on https://lnkd.in/eqDf_WrK. Again see you on July 8 (register at https://lnkd.in/dixxWRQr).
