alon-albalak.github.io Open in urlscan Pro
2606:50c0:8000::153  Public Scan

Form analysis
0 forms found in the DOM

Text Content

 * About Me
 * Publications
 * Experience
 * Photography
 * CV


ALON ALBALAK

Ph.D. candidate, NLP, University of California, Santa Barbara

Follow
 * Santa Barbara, CA
 * Email
 * Twitter
 * LinkedIn
 * Github
 * Google Scholar
 * Semantic Scholar


ABOUT ME - ALON ALBALAK


** I AM CURRENTLY ON THE INDUSTRY JOB MARKET FOR RESEARCH SCIENTIST POSITIONS **

Please reach out by email if my skills match up with your team’s research goals!



I am a fifth year Ph.D. candidate in the NLP Group at the University of
California, Santa Barbara, advised by professors William Yang Wang and Xifeng
Yan. During the first year of my Ph.D. I was gratefully supported by an NSF
IGERT Fellowship. While pursuing my Ph.D. I took a year off from research to
work at a financial technology startup, Theta Lake. Prior to my Ph.D. I received
my B.S. in mathematics at Wayne State University, with research advised by Gang
George Yin.

My primary research focus is on applying ML methods to NLP to improve data
efficiency and model performance. In my research I have explored the use of
methods including multi-armed bandits, data selection, multitask learning,
transfer learning, reinforcement learning, and neuro-symbolic methods.
Additionally, I have a wide array of interests in other topics including model
efficiency, logic and reasoning, conversational AI, and multilingual models.





** NEWS **


[10/2023] OUR NEW WORK ON “EFFICIENT ONLINE DATA MIXING FOR LANGUAGE MODEL
PRE-TRAINING” IS ON PRE-RELEASE

This work presents an extremely efficient online data mixing algorithm that
reaches the same model perplexity of the next best method (DoReMi) with 19%
fewer iterations, and improves downstream performance by 1.9% while adding a
miniscule 0.000007% overhead.
Check out the pre-print


[10/2023] ACCEPTED TO EMNLP 2023 - “RWKV: REINVENTING RNNS FOR THE TRANSFORMER
ERA”

RWKV is a new model architecture that combines the efficient parallelizable
training of Transformers with the efficient inference of RNNs.
Check out the paper and code for more information.


[10/2023] ACCEPTED AT EMNLP 2023 - “LOGIC-LM: EMPOWERING LARGE LANGUAGE MODELS
WITH SYMBOLIC SOLVERS FOR FAITHFUL LOGICAL REASONING”

This work demonstrates that combining Large Language Models (LLMs) with symbolic
solvers makes for a strong method for solving logical problems.
Check out the paper and the code.


[09/2023] ACCEPTED TO NEURIPS 2023 - “IMPROVING FEW-SHOT GENERALIZATION BY
EXPLORING AND EXPLOITING AUXILIARY DATA”

This work presents 2 methods of few-shot learning with auxiliary data, inspired
by multi-armed bandits. These methods show significant improvement over
multi-tasking followed by fine tuning (9% improvement).
Check out the paper and the code for more information.


[05/2023] ACCEPTED TO ACL 2023 - “MODELING UTTERANCE-LEVEL CAUSALITY IN
CONVERSATIONS”

Check out the paper for more details.


[04/2023] ACCEPTED AT IJCAI 2023 - “NEUPSL: NEURAL PROBABILISTIC SOFT LOGIC”

NeuPSL is a neuro-symbolic framework that unites the powerful symbolic reasoning
of PSL with the representation learning of deep neural networks.
Check out the paper for more.


[02/2023] THE FETA BENCHMARK ON TASK TRANSFER WILL BE A SHARED TASK AT THE NLP
FOR CONVAI WORKSHOP AT ACL ‘23!

*Awards* The FETA benchmark will have prizes for top scorers and most innovative
approaches!

*Purpose* The FETA benchmark shared task aims to bring together researchers from
a variety of backgrounds and compare their best ideas for task transfer. The
benchmark allows for comparing many different methods including:
instruction/prompt fine-tuning, source-task selection, multitask learning,
continued pre-training, meta-learning, and many more!

See detailed rules, starter code, and submission instructions on the website.


[02/2023] “ADDRESSING ISSUES OF CROSS-LINGUALITY IN OPEN-RETRIEVAL QUESTION
ANSWERING SYSTEMS FOR EMERGENT DOMAINS” WAS ACCEPTED AT EACL 2023!

This work addresses the low-resource question-answering setting where supporting
documents may not be in the same language as the query, cross-lingual
Open-retrieval QA. In particular, this is an important problem in emergent
domains, where the majority of supporting documents are more likely to be in a
limited number of languages. Check out the paper and code for more information.


[01/2023] THE TRANSFER LEARNING FOR NLP WORKSHOP (TL4NLP) WORKSHOP IS AVAILABLE
TO WATCH!

TL4NLP explored insights and advances on transfer learning, including insightful
talks from our guest speakers and hot takes from our debaters.
TL4NLP features talks from Mike Lewis, Percy Liang/Ananya Kumar, Graham Neubig,
David Adelani, and Jonas Pfeiffer
as well as a debate between Sara Hooker and Kyunghyun Cho.
Check out the talks, topics, and more at tl4nlp.githb.io.
Find recorded talks here.


[10/2022] MY PAPER ON BENCHMARKING TASK TRANSFER WILL BE AT EMNLP ‘22: FETA

FETA is the largest NLP benchmark for intra-dataset task transfer, where task
transfer is isolated from domain shift.
Check out the paper, and our github repo for more.


Sitemap
 * Follow:
 * GitHub
 * Feed

© 2023 Alon Albalak. Powered by Jekyll & AcademicPages, a fork of Minimal
Mistakes.