Matthew Lease

Orcid: 0000-0002-0056-2834

According to our database1, Matthew Lease authored at least 150 papers between 1999 and 2024.

Collaborative distances:
  • Dijkstra number2 of four.
  • Erdős number3 of four.

Timeline

Legend:

Book 
In proceedings 
Article 
PhD thesis 
Dataset
Other 

Links

On csauthors.net:

Bibliography

2024
Human-centered NLP Fact-checking: Co-Designing with Fact-checkers using Matchmaking for AI.
Proc. ACM Hum. Comput. Interact., 2024

Fairly Accurate: Optimizing Accuracy Parity in Fair Target-Group Detection.
CoRR, 2024

Diverse, but Divisive: LLMs Can Exaggerate Gender Differences in Opinion Related to Harms of Misinformation.
CoRR, 2024

Annorama: Enabling Immersive At-Desk Annotation Experiences in Virtual Reality with 3D Point Cloud Dioramas.
Proceedings of the 2024 ACM Symposium on Spatial User Interaction, 2024

Benchmark Transparency: Measuring the Impact of Data on Evaluation.
Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), 2024

Snapper: Accelerating Bounding Box Annotation in Object Detection Tasks with Find-and-Snap Tooling.
Proceedings of the 29th International Conference on Intelligent User Interfaces, 2024

Promoting Constructive Deliberation: Reframing for Receptiveness.
Proceedings of the Findings of the Association for Computational Linguistics: EMNLP 2024, 2024

A General Model for Aggregating Annotations AcrossSimple, Complex, and Multi-object Annotation Tasks (Abstract Reprint).
Proceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence, 2024

2023
Editorial: Human-centered AI: Crowd computing.
Frontiers Artif. Intell., February, 2023

A General Model for Aggregating Annotations Across Simple, Complex, and Multi-Object Annotation Tasks.
J. Artif. Intell. Res., 2023

The state of human-centered NLP technology for fact-checking.
Inf. Process. Manag., 2023

DMLR: Data-centric Machine Learning Research - Past, Present and Future.
CoRR, 2023

Interpretable by Design: Wrapper Boxes Combine Neural Performance with Faithful Explanations.
CoRR, 2023

How Crowd Worker Factors Influence Subjective Annotations: A Study of Tagging Misogynistic Hate Speech in Tweets.
CoRR, 2023

Learning Complementary Policies for Human-AI Teams.
CoRR, 2023

Same Same, But Different: Conditional Multi-Task Learning for Demographic-Specific Toxicity Detection.
Proceedings of the ACM Web Conference 2023, 2023

Designing Closed-Loop Models for Task Allocation.
Proceedings of the HHAI 2023: Augmenting Human Intellect, 2023

New Metrics to Encourage Innovation and Diversity in Information Retrieval Approaches.
Proceedings of the Advances in Information Retrieval, 2023

2022
Data excellence for AI: why should you care?
Interactions, 2022

In Search of Ambiguity: A Three-Stage Workflow Design to Clarify Annotation Guidelines for Crowd Workers.
Frontiers Artif. Intell., 2022

Voices of Workers: Why a Worker-Centered Approach to Crowd Work Is Challenging.
CoRR, 2022

longhorns at DADC 2022: How many linguists does it take to fool a Question Answering model? A systematic approach to adversarial attacks.
CoRR, 2022

Fairly Accurate: Learning Optimal Accuracy vs. Fairness Tradeoffs for Hate Speech Detection.
CoRR, 2022

Designing Closed Human-in-the-loop Deferral Pipelines.
CoRR, 2022

Measuring Annotator Agreement Generally across Complex Structured, Multi-object, and Free-text Annotation Tasks.
Proceedings of the WWW '22: The ACM Web Conference 2022, Virtual Event, Lyon, France, April 25, 2022

Learning a neural Pareto manifold extractor with constraints.
Proceedings of the Uncertainty in Artificial Intelligence, 2022

Understanding and Predicting Characteristics of Test Collections in Information Retrieval.
Proceedings of the Information for a Better World: Shaping the Global Future - 17th International Conference, iConference 2022, Virtual Event, February 28, 2022

You Are What You Tweet: Profiling Users by Past Tweets to Improve Hate Speech Detection.
Proceedings of the Information for a Better World: Shaping the Global Future - 17th International Conference, iConference 2022, Virtual Event, February 28, 2022

The Effects of Interactive AI Design on User Behavior: An Eye-tracking Study of Fact-checking COVID-19 Claims.
Proceedings of the CHIIR '22: ACM SIGIR Conference on Human Information Interaction and Retrieval, Regensburg, Germany, March 14, 2022

ProtoTEx: Explaining Model Decisions with Prototype Tensors.
Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2022

2021
The Challenge of Variable Effort Crowdsourcing and How Visible Gold Can Help.
Proc. ACM Hum. Comput. Interact., 2021

Proceedings of the CSCW 2021 Workshop - Investigating and Mitigating Biases in Crowdsourced Data.
CoRR, 2021

Scalable Uni-directional Pareto Optimality for Multi-Task Learning with Constraints.
CoRR, 2021

The Case for Claim Difficulty Assessment in Automatic Fact Checking.
CoRR, 2021

SCA-Net: A Self-Correcting Two-Layer Autoencoder for Hyper-spectral Unmixing.
CoRR, 2021

A Hybrid 2-stage Neural Optimization for Pareto Front Extraction.
CoRR, 2021

An Information Retrieval Approach to Building Datasets for Hate Speech Detection.
Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks 1, 2021

Aggregating Complex Annotations via Merging and Matching.
Proceedings of the KDD '21: The 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2021

Human-AI Collaboration with Bandit Feedback.
Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, 2021

Pareto Solutions vs Dataset Optima: Concepts and Methods for Optimizing Competing Objectives with Constraints in Retrieval.
Proceedings of the ICTIR '21: The 2021 ACM SIGIR International Conference on the Theory of Information Retrieval, 2021

Investigating and Mitigating Biases in Crowdsourced Data.
Proceedings of the Companion Publication of the 2021 ACM Conference on Computer Supported Cooperative Work and Social Computing, 2021

The Psychological Well-Being of Content Moderators: The Emotional Labor of Commercial Moderation and Avenues for Improving Support.
Proceedings of the CHI '21: CHI Conference on Human Factors in Computing Systems, 2021

Towards Unbiased and Accurate Deferral to Multiple Experts.
Proceedings of the AIES '21: AAAI/ACM Conference on AI, 2021

2020
Annotator Rationales for Labeling Tasks in Crowdsourcing.
J. Artif. Intell. Res., 2020

Understanding and Predicting the Characteristics of Test Collections.
CoRR, 2020

Streaming Singular Value Decomposition for Big Data Applications.
CoRR, 2020

Extracting Optimal Solution Manifolds using Constrained Neural Optimization.
CoRR, 2020

TIME: A Transparent, Interpretable, Model-Adaptive and Explainable Neural Network for Dynamic Physical Processes.
CoRR, 2020

Modeling and Aggregation of Complex Annotations via Annotation Distances.
Proceedings of the WWW '20: The Web Conference 2020, Taipei, Taiwan, April 20-24, 2020, 2020

Efficient Test Collection Construction via Active Learning.
Proceedings of the ICTIR '20: The 2020 ACM SIGIR International Conference on the Theory of Information Retrieval, 2020

Fast, Accurate, and Healthier: Interactive Blurring Helps Moderators Reduce Exposure to Harmful Content.
Proceedings of the Eighth AAAI Conference on Human Computation and Crowdsourcing, 2020

2019
FACTS-IR: fairness, accountability, confidentiality, transparency, and safety in information retrieval.
SIGIR Forum, 2019

A Conceptual Framework for Evaluating Fairness in Search.
CoRR, 2019

CobWeb: A Research Prototype for Exploring User Bias in Political Fact-Checking.
CoRR, 2019

Constructing Test Collections using Multi-armed Bandits and Active Learning.
Proceedings of the World Wide Web Conference, 2019

Mash: software tools for developing interactive and transparent machine learning systems.
Proceedings of the Joint Proceedings of the ACM IUI 2019 Workshops co-located with the 24th ACM Conference on Intelligent User Interfaces (ACM IUI 2019), 2019

Explainable modeling of annotations in crowdsourcing.
Proceedings of the 24th International Conference on Intelligent User Interfaces, 2019

Correlation, Prediction and Ranking of Evaluation Metrics in Information Retrieval.
Proceedings of the Advances in Information Retrieval, 2019

2018
Crowdsourcing and Human Computation: Introduction.
Proceedings of the Encyclopedia of Social Network Analysis and Mining, 2nd Edition, 2018

Neural information retrieval: at the end of the early years.
Inf. Retr. J., 2018

Intelligent topic selection for low-cost information retrieval evaluation: A New perspective on deep vs. shallow judging.
Inf. Process. Manag., 2018

Toward Safer Crowdsourced Content Moderation.
CoRR, 2018

Correlation and Prediction of Evaluation Metrics in Information Retrieval.
CoRR, 2018

Reports of the Workshops Held at the Sixth AAAI Conference on Human Computation and Crowdsourcing.
AI Mag., 2018

Believe it or not: Designing a Human-AI Partnership for Mixed-Initiative Fact-Checking.
Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology, 2018

Crowd vs. Expert: What Can Relevance Judgment Rationales Teach Us About Assessor Disagreement?
Proceedings of the 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, 2018

Your Behavior Signals Your Reliability: Modeling Crowd Behavioral Traces to Ensure Quality Relevance Annotations.
Proceedings of the Sixth AAAI Conference on Human Computation and Crowdsourcing, 2018

Fact Checking and Information Retrieval.
Proceedings of the First Biennial Conference on Design of Experimental Search & Information Retrieval Systems, 2018

Mix and Match: Collaborative Expert-Crowd Judging for Building Test Collections Accurately and Affordably.
Proceedings of the First Biennial Conference on Design of Experimental Search & Information Retrieval Systems, 2018

When Rank Order Isn't Enough: New Statistical-Significance-Aware Correlation Measures.
Proceedings of the 27th ACM International Conference on Information and Knowledge Management, 2018

An Interpretable Joint Graphical Model for Fact-Checking From Crowds.
Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, 2018

2017
Design Activism for Minimum Wage Crowd Work.
CoRR, 2017

Learning to Effectively Select Topics For Information Retrieval Test Collections.
CoRR, 2017

The Many Benefits of Annotator Rationales for Relevance Judgments.
Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, 2017

Bullseye: structured passage retrieval and document highlighting for scholarly search.
Proceedings of the Australasian Computer Science Week Multiconference, 2017

Exploiting Domain Knowledge via Grouped Weight Sharing with Application to Text Categorization.
Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, 2017

Aggregating and Predicting Sequence Labels from Crowd Annotations.
Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, 2017

Active Discriminative Text Representation Learning.
Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, 2017

2016
Neural Information Retrieval: A Literature Review.
CoRR, 2016

Crowdsourcing Information Extraction for Biomedical Systematic Reviews.
CoRR, 2016

MmmTurkey: A Crowdsourcing Framework for Deploying Tasks and Recording Worker Behavior on Amazon Mechanical Turk.
CoRR, 2016

A Correlated Worker Model for Grouped, Imbalanced and Multitask Data.
Proceedings of the Thirty-Second Conference on Uncertainty in Artificial Intelligence, 2016

ArabicWeb16: A New Crawl for Today's Arabic Web.
Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval, 2016

Probabilistic Modeling for Crowdsourcing Partially-Subjective Ratings.
Proceedings of the Fourth AAAI Conference on Human Computation and Crowdsourcing, 2016

Why Is That Relevant? Collecting Annotator Rationales for Relevance Judgments.
Proceedings of the Fourth AAAI Conference on Human Computation and Crowdsourcing, 2016

2015
Combining Crowd and Expert Labels Using Decision Theoretic Active Learning.
Proceedings of the Third AAAI Conference on Human Computation and Crowdsourcing, 2015

Modeling Temporal Crowd Work Quality with Limited Supervision.
Proceedings of the Third AAAI Conference on Human Computation and Crowdsourcing, 2015

A Discriminative Approach to Predicting Assessor Accuracy.
Proceedings of the Advances in Information Retrieval, 2015

2014
Crowdsourcing and Human Computation, Introduction.
Encyclopedia of Social Network Analysis and Mining, 2014

TurKPF: TurKontrol as a Particle Filter.
CoRR, 2014

Workshops Held at the First AAAI Conference on Human Computation and Crowdsourcing: A Report.
AI Mag., 2014

Multidimensional relevance modeling via psychometrics and crowdsourcing.
Proceedings of the 37th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2014

Predicting Next Label Quality: A Time-Series Model of Crowdwork.
Proceedings of the Seconf AAAI Conference on Human Computation and Crowdsourcing, 2014

2013
Crowdsourcing for information retrieval: introduction to the special issue.
Inf. Retr., 2013

Beyond AMT: An Analysis of Crowd Work Platforms.
CoRR, 2013

Crowdsourced Task Routing via Matrix Factorization.
CoRR, 2013

Overview of the TREC 2013 Crowdsourcing Track.
Proceedings of The Twenty-Second Text REtrieval Conference, 2013

SQUARE: Benchmarking Crowd Consensus at MediaEval.
Proceedings of the MediaEval 2013 Multimedia Benchmark Workshop, 2013

Improving bug localization using structured information retrieval.
Proceedings of the 2013 28th IEEE/ACM International Conference on Automated Software Engineering, 2013

SQUARE: A Benchmark for Research on Computing Crowd Consensus.
Proceedings of the First AAAI Conference on Human Computation and Crowdsourcing, 2013

The future of crowd work.
Proceedings of the Computer Supported Cooperative Work, 2013

2012
Evaluating Classifiers Without Expert Labels
CoRR, 2012

Dating Texts without Explicit Temporal Cues
CoRR, 2012

Overview of the TREC 2012 Crowdsourcing Track.
Proceedings of The Twenty-First Text REtrieval Conference, 2012

Parallelizing ListNet training using spark.
Proceedings of the 35th International ACM SIGIR conference on research and development in Information Retrieval, 2012

Crowdsourcing for search evaluation and social-algorithmic search.
Proceedings of the 35th International ACM SIGIR conference on research and development in Information Retrieval, 2012

Inferring missing relevance judgments from crowd workers via probabilistic matrix factorization.
Proceedings of the 35th International ACM SIGIR conference on research and development in Information Retrieval, 2012

Finding and exploring memes in social media.
Proceedings of the 23rd ACM Conference on Hypertext and Social Media, 2012

Improving Quality of Crowdsourced Labels via Probabilistic Matrix Factorization.
Proceedings of the 4th Human Computation Workshop, 2012

Crowdsourcing for usability testing.
Proceedings of the Information, Interaction, Innovation: Celebrating the Past, Constructing the Present and Creating the Future, 2012

2011
Crowdsourcing for information retrieval.
SIGIR Forum, 2011

Crowdsourcing for search and data mining.
SIGIR Forum, 2011

Crowdsourcing 101: putting the WSDM of crowds to work for you.
Proceedings of the Forth International Conference on Web Search and Web Data Mining, 2011

Active learning to maximize accuracy vs. effort in interactive information retrieval.
Proceedings of the Proceeding of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2011

Learning to rank from a noisy crowd.
Proceedings of the Proceeding of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2011

Crowdsourcing for information retrieval: principles, methods, and applications.
Proceedings of the Proceeding of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2011

Mobile options for online public access catalogs.
Proceedings of the iConference 2011, 2011

Supervised language modeling for temporal resolution of texts.
Proceedings of the 20th ACM Conference on Information and Knowledge Management, 2011

Look before you leap: Legal pitfalls of crowdsourcing.
Proceedings of the Bridging the Gulf: Communication and Information in Society, Technology, and Work, 2011

Crowdworker filtering with support vector machine.
Proceedings of the Bridging the Gulf: Communication and Information in Society, Technology, and Work, 2011

Enabling trust in crowd labor relations through identity sharing.
Proceedings of the Bridging the Gulf: Communication and Information in Society, Technology, and Work, 2011

On Quality Control and Machine Learning in Crowdsourcing.
Proceedings of the Human Computation, 2011

Improving Consensus Accuracy via Z-Score and Weighted Voting.
Proceedings of the Human Computation, 2011

2010
Beyond keywords: finding information more accurately and easily using natural language.
PhD thesis, 2010

Crowdsourcing for search evaluation.
SIGIR Forum, 2010

Crowdsourcing Document Relevance Assessment with Mechanical Turk.
Proceedings of the 2010 Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk, 2010

Effective Term Weighting for Sentence Retrieval.
Proceedings of the Research and Advanced Technology for Digital Libraries, 2010

University of Texas mobile library search.
Proceedings of the Navigating Streams in an Information Ecosystem, 2010

Mobile phone search for library catalogs.
Proceedings of the Navigating Streams in an Information Ecosystem, 2010

2009
UMass Amherst and UT Austin @ the TREC 2009 Relevance Feedback Track.
Proceedings of The Eighteenth Text REtrieval Conference, 2009

An improved markov random field model for supporting verbose queries.
Proceedings of the 32nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 2009

Regression Rank: Learning to Meet the Opportunity of Descriptive Queries.
Proceedings of the Advances in Information Retrieval, 2009

2008
Incorporating Relevance and Pseudo-relevance Feedback in the Markov Random Field Model.
Proceedings of The Seventeenth Text REtrieval Conference, 2008

2007
Brown at CL-SR'07: Retrieving Conversational Speech in English and Czech.
Proceedings of the Working Notes for CLEF 2007 Workshop co-located with the 11th European Conference on Digital Libraries (ECDL 2007), 2007

A Dirichlet-Smoothed Bigram Model for Retrieving Spontaneous Speech.
Proceedings of the Advances in Multilingual and Multimodal Information Retrieval, 2007

Natural language processing for information retrieval: the time is ripe (again).
Proceedings of the First Ph.D. Workshop in CIKM, 2007

2006
Recognizing disfluencies in conversational speech.
IEEE Trans. Speech Audio Process., 2006

Early Deletion of Fillers In Processing Conversational Speech.
Proceedings of the Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics, 2006

SParseval: Evaluation Metrics for Parsing Speech.
Proceedings of the Fifth International Conference on Language Resources and Evaluation, 2006

Linguistic Resources for Speech Parsing.
Proceedings of the Fifth International Conference on Language Resources and Evaluation, 2006

Reranking for Sentence Boundary Detection in Conversational Speech.
Proceedings of the 2006 IEEE International Conference on Acoustics Speech and Signal Processing, 2006

PCFGs with Syntactic and Prosodic Indicators of Speech Repairs.
Proceedings of the ACL 2006, 2006

A Look at Parsing and Its Applications.
Proceedings of the Proceedings, 2006

2005
Effective Use of Prosody in Parsing Conversational Speech.
Proceedings of the HLT/EMNLP 2005, 2005

Parsing Biomedical Literature.
Proceedings of the Natural Language Processing, 2005

Parsing and its applications for conversational speech.
Proceedings of the 2005 IEEE International Conference on Acoustics, 2005

2003
SmartElevator: Revitalizing A Legacy Device through Inexpensive Augmentation.
Proceedings of the 23rd International Conference on Distributed Computing Systems Workshops (ICDCS 2003 Workshops), 2003

2002
Making Sensor Networks Practical with Robots.
Proceedings of the Pervasive Computing, 2002

PlantCare: An Investigation in Practical Ubiquitous Systems.
Proceedings of the UbiComp 2002: Ubiquitous Computing, 4th International Conference, Göteborg, Sweden, September 29, 2002

1999
Anatomical information in radiation treatment planning.
Proceedings of the AMIA 1999, 1999


  Loading...