Published in arXiv, 2024
Developed in partnership with the University of Pretoria's DSFSI, this paper discusses the development of the Bilingual Open Tswana Suite of Langauge Models (BOTS-LM), a suite of LLMs trained for Setswana and English.
Recommended citation: Brown, Nathan and Marivate, Vukosi (2024). "BOTS-LM: Training Large Language Models for Setswana" arXiv https://arxiv.org/abs/2408.02239
Published in Empirical Methods in Natural Language Processing (EMNLP), 2023
This paper discusses the distillation of long-context, efficient attention BERT-based models to yield models that are smaller, faster, and cheaper to deploy.
Recommended citation: Brown, Nathan and Williamson, Ashton and Anderson, Tahj and Lawrence, Logan. (2023). "Efficient Transformer Knowledge Distillation: A Performance Review" Empirical Methods in Natural Language Processing. https://arxiv.org/pdf/2311.13657.pdf
Published in N/A, 2023
Trained a multi-head attention vision architecture to improve performance on the classification of FAST ultrasound exams given limited training data.
Published in N/A, 2022
Fine-tuned embeddings and BERT-style models for tasks such as text clustering, sentiment analysis, and named entity recognition on hospital reports.
Published in N/A, 2019
Developed a novel method to estimate the circumference of an ellipse and π by exactly calculating the perimeter of stretched regular polygons.