Time: 12:00 - 1:00 PM ET | 9:00 - 10:00 AM PT | 6:00 - 7:00 PM CET

Recent advancements in natural language processing have led to the proliferation of large language models (LLMs). These models have been shown to yield good performance, using in-context learning, even on tasks and languages they are not trained on. However, their performance on African languages is largely understudied relative to high-resource languages. We present an analysis of four popular large language models(mT0, Aya, LLaMa 2, and GPT-4) on six tasks (topic classification, sentiment classification, machine translation, summarization, question answering, and named entity recognition) across 60 African languages, spanning different language families and geographical regions. Our results suggest that all LLMs produce lower performance for African languages, and there is a large gap in performance compared to high-resource languages (such as English) for most tasks. We find that GPT-4 has an average to good performance on classification tasks, yet its performance on generative tasks such as machine translation and summarization is significantly lacking. Surprisingly, we find that mT0 had the best overall performance for cross-lingual QA, better than the state-of-the-art supervised model (i.e. fine-tuned mT5) and GPT-4 on African languages.  Similarly, we find the recent Aya model to have comparable result to mT0 in almost all tasks except for topic classification where it outperform mT0. Overall, LLaMa 2 showed the worst performance, which we believe is due to its English and code-centric (around 98%) pre-training corpus. Our findings confirms that performance on African languages continues to remain a hurdle for the current LLMs, underscoring the need for additional efforts to close this gap.

Time: 12:00 - 1:00 PM ET | 9:00 - 10:00 AM PT | 6:00 - 7:00 PM CET.

Thousands of users consult digital archives daily, but the information they can access is unrepresentative of the diversity of documentary history. The sequence-to-sequence architecture typically used for optical character recognition (OCR) – which jointly learns a vision and language model - is poorly extensible to low-resource document collections, as learning a language-vision model requires extensive labeled sequences and compute. This study models OCR as a character level image retrieval problem, using a contrastively trained vision encoder. Because the model only learns characters’ visual features, it is more sample efficient and extensible than existing architectures, enabling accurate OCR in settings where existing solutions fail. Crucially, the model opens new avenues for community engagement in making digital history more representative of documentary history.

Time: 12:00 - 1:00 PM ET | 9:00 - 10:00 AM PT | 6:00 - 7:00 PM CET

The Asylum Seeker Advocacy Project (ASAP) is the largest membership organization of asylum seekers in history. Through ASAP’s participatory model, 600,000+ members have set collective priorities, pursued lawsuits, and led advocacy campaigns. And they have won—securing work permits for hundreds of thousands of immigrants! ASAP believes solutions will often have the highest impact when they are designed by the highly impacted. In this talk, Swapna Reddy, co-founder and Co-Executive Director of ASAP, will discuss opportunities, challenges, and lessons learned from engaging with a large member base to achieve meaningful change. Swapna will share concrete examples of how ASAP has engaged in participatory decision-making and collective action at scale.