Alliteration is a literary device that involves the repetition of initial consonant sounds in a sequence of words, and it is often used for stylistic or rhetorical purposes to create rhythm, emphasize key ideas, or make phrases more memorable.
Although this focused work completely aligns, addresses, and adheres to the guidelines for a short-paper in this venue, we have not performed any experiments on data outside this privacy policy domain.
[Semantic Scholar] – [Code] – [Tweet] – [Video] – [Slide]
Change Logs:
- 2023-10-05: First draft. This paper appears at EACL 2023; it is dated 2204.08952. The code is not released.
Paraphrasing and back-translation methods are only applicable for texts that are not sensitive to changes in texts. However, the privacy policies could convey wildly different meanings for small differences in the texts; this makes these two techniques less applicable to the problem being studied.
The authors propose a coarse-to-fine architecture for retrieval-based data augmentation. It consists of an ensemble of retrieval and filter models; these models include (1) regular BERT, (2) PBERT, a BERT fine-tuned with MLM objective on the privacy policies, and (3) the PBERT fine-tuned with SimCSE.
Note that
During inference, the top-k retrieved samples are filtered by the trained filter model. The aggregated retrieved texts are combined with original dataset to fine-tune the privacy QA model.
This post is the summary of the following methods; they rank top on the CivilComments-WILDS benchmark:
| Rank | Method | Paper |
|---|---|---|
| 1 | FISH | [2104.09937] Gradient Matching for Domain Generalization (Shi et al., ICLR 2022 |
| 2, 3 | IRMX | [2206.07766] Pareto Invariant Risk Minimization: Towards Mitigating the Optimization Dilemma in Out-of-Distribution Generalization (Chen et al., ICLR 2023) |
| 4 | LISA | [2201.00299] Improving Out-of-Distribution Robustness via Selective Augmentation (Yao et al., ICML 2022) |
| 5 | DFR | [2204.02937] Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations (Kirichenko et al., ICLR 2023) |
| 6, 8 | Group DRO | |
| 7, 12 | Reweighting | [1901.05555] Class-Balanced Loss Based on Effective Number of Samples (Cui et al., CVPR 2019) is one example that uses this method; the reweighting method could date back to much earlier works. |
IRM [2] and CORAL [3] are two extensions of the basic reweighting method by adding an additional penalty term on top of the reweighting loss; this term is based on some measures of the data representations from different domains to encourage the data distribution of different domains to be similar.
[Semantic Scholar] – [Code] – [Tweet] – [Video] – [Website and Leaderboard] – [Slide] – [Lead Author]
Change Logs:
- 2023-10-03: First draft. The authors provide 5 datasets (2 of them are text classification datasets, the others include 2 image classification datasets and 1 EHR dataset) and more than 10 mitigation methods for distribution shift.
From the content below, we could see that:
To address this challenge, we adapt the above invariant learning approaches to the temporal distribution shift setting. We leverage timestamp metadata to create a temporal robustness set consisting of substreams of data, where each substream is treated as one domain. Specifically, as shown in Figure 3, we define a sliding window G with length L. For a data stream with T timestamps, we apply the sliding window G to obtain T − L + 1 substreams. We treat each substream as a “domain” and apply the above invariant algorithms on the robustness set. We name the adapted CORAL, GroupDRO and IRM as CORAL-T, GroupDRO-T, IRM-T, respectively. Note that we do not adapt LISA since the intra-label LISA performs well without domain information, which is also mentioned in the original paper.
[Semantic Scholar] – [Code] – [Tweet] – [Video] – [Website] – [Slide]
Change Logs:
- 2023-10-03: First draft.
[Semantic Scholar] – [Code] – [Tweet] – [Video] – [Website] – [Slide] – [HuggingFace]
Change Logs:
- 2023-10-01: First draft. This paper appears at EMNLP 2021.
[Semantic Scholar] – [Code] – [Tweet] – [Video] – [Website] – [Slide]
Change Logs:
- 2023-09-26: First draft. The paper appears at EMNLP 2021.
[…] all correlations between labels and individual “input features” are spurious.
Practically, the paper suggests the partial invariance (whether independent or not) for real-world datasets; for example, the sentiment of a movie review is invariant to the actor names. The paper also suggest the following options to improve model robustness:
data augmentation, causally-motivated regularizers, stress tests, and “worst-subgroup” performance metrics (and associated robust optimizers) can be seen as enforcing or testing task-specific invariance properties that provide robustness against known distributional shifts (e.g., Lu et al., 2020; Ribeiro et al., 2020; Kaushik et al., 2021; Koh et al., 2021; Veitch et al., 2021). Such approaches generally require domain knowledge about the linguistic and causal properties of the task at hand — or to put it more positively, they make it possible for such domain knowledge to be brought to bear. Indeed, the central argument of this paper is that no meaningful definition of spuriousness or robustness can be obtained without such domain knowledge.
Here I document a list of general research questions that warrants searching, reading, thinking, and rethinking.
- Note: Model capacity mostly determines the performance upper bound of a model. The actual model performance may also related to how the model is trained with what set of hyperparameters.
- Hypothesis: A straightforward choice is the number of parameters a model has. However, one may question the correlation between the parameter count and this measure, i.e., the parameter count may need to be a valid proxy for the model capacity.
Specifically, suppose there are K texts existing in the world at time t, and they are all labeled by an oracle; if we fine-tune a bert-base-uncased with k \ll K samples as a classification model, is there any hope that this fine-tuned model perform reasonably well (needs more precise definition) on all (K-k) samples.
- Experiment: We could only approximate the oracle by some knowingly most capable models like GPT-4. We therefore have two datasets, one from (a) original annotations and (b) the other from oracle (approximated by GPT-4) annotations. Could the model fine-tuned on dataset (b) generalize better than (a)?
- Question: Despite the generalization, could the fine-tuned model also inherit the bias (needs more precise definition) of GPT-4?
Does text classification work by relying on the spurious correlation (likely due to annotation bias where the annotators seek to take the shortcuts to complete their assigned tasks as soon as possible) between a limited number of words in the input text and output label? Therefore, is the better model indeed the model that better exploits the spurious correlation?
> - Hypothesis: If the K samples are all annotated by an oracle, then any *reasonably capable* model (needs more precise definition) can generalize well.
> - Tentative Experiment: If we replace the words in the list with their hypernyms, will the system performance drop?
Suppose we have a generalizable model at time t if we want the model to serve the users indefinitely. What are the strategies that make the model generalize well across time?
In machine learning theory, we often encounter concepts such as “i.i.d.” Understanding “distribution” for tabular data is straightforward, where the list of variables forms a joint distribution that predicts the label y. However, what could be considered a “distribution” for texts is less clear. Note that this is possible for images, for example, modeling the gray-scale images as a Dirichlet multimodal distribution that predicts digits from 0 to 9.
The labels in the NLP tasks have different levels of subjectivity. For example, the grammatical error correction is less subjective, sentiment classification is moderately subjective, and the topics like hate speech, suicidal ideation [1], and empathy [2] are either extremely subjective or requires expert knowledge.
The difficulty here it to mitigate the ambiguity during data annotation and make sure the information available in texts matches well with the label. Ideally, if we know the true underlying label of a text, we could fine-tune any reasonably capable model to generalize well.
As of 2023-10-11, I have not seen a single work on the data selection for classification tasks; there are plenty of works on optimizing data mixture for language model pretraining. One likely reason why this happens is that the quality of classification datasets depends on both texts and labels; investigating the label quality is hard.
The difference of testing machine learning models versus testing traditional software is that the action items for the latter is generally known after testing and could be executed with high precision, while for the former, we do not know how we will improve the model.
Suppose the model could already achieve high accuracy on the standard test set (it is indeed the train-to-train setting if we follow the WILDS paper), which means the model architecture, training objective, and hyperparameters are not responsible for the lower performance on the artificially created challenging set, the most straightforward way to improve the model performance is data augmentation. The naive way is to blindly collect more data that are wishfully relevant as an augmentation so we expect the performance could improve.
However, this blindness hampers the efficiency of improving the models: the only feedback signal is the single scalar (i.e., failure rate) after we have trained and evaluated the model; we should have a feedback signal before we train the model.
- Unverified Hypothesis: the feedback signal is highly (inversely) correlated with the failure rate on the benchmark.
Formally, we have a list of specifications in the format of (s_1, D _ 1, D _ 1 ^ \text{heldout}), (s _ 2, D _ 2, D _ 2 ^ \text{heldout}), \cdots, the model \mathcal{M}_0 trained on D _ \text{train} does well on D _ \text{train} ^\text{heldout} but poorly on D _ 1 \cup D _ 2 \cup D _ 3 \cdots as indicated by failure rate \mathrm{FR}. We additionally have a new labeled dataset D _ \text{unused}. The goal is to sample D _ \text{unused} using (s_1,D _ 1 ^ \text{heldout}), (s _ 2, D _ 2 ^ \text{heldout}), \cdots: \mathrm{Sample}(D _ \text{unused}); we also have a random sample with same size \mathrm{RandomSample}(D _ \text{unused}) as baseline.
- Note: The D _ i and D _ i ^ \text{heldout} are completely different. For example, if the specification s _ i is operationalized through templates, these two sets are disjoint in terms of templates. What we are certain about D _ i and D _ i ^ \text{heldout} is that they are ideally sufficient and necessary with respect to s _ i; practically, the semantic underspecification of them are low [3].
There are a lot of things we could do with \mathrm{RandomSample}(D _ \text{unused}) and \mathrm{Sample}(D _ \text{unused}). For example
Whichever method we choose, if we denote the intervention with \mathrm{RandomSample}(D _ \text{unused}) as \mathcal{M} _ 1 and \mathrm{Sample}(D _ \text{unused}) as \mathcal{M} _ 2. We expect the following conditions will hold:
- Assumption: The samples x _ {ij} is fully specified by the specification s _ i.
- Note: If the annotations of a dataset strictly follow the annotation codebook, then the machine learning learns the specifications in the codebook. The process described above is a reverse process: we have a model that is already trained by others; we want to use the model in a new application but do not want to or can not afford to relabel the entire dataset, what is the minimal intervention we could apply to the dataset so that the model could quickly meet my specifications?
Following the previous problem setup, we have a list of specifications in the format of (s_1, D _ 1, D _ 1 ^ \text{heldout}), (s _ 2, D _ 2, D _ 2 ^ \text{heldout}), \cdots; each specification has an unambiguous label. Rather than augmenting the D _ \text{train} with additional data by selecting using either (1) D _ 1 ^ \text{heldout} \cup D _ 2 ^ \text{heldout} \cup \cdots itself or (2) a model trained on it, we aim to correct labels directly in D _ \text{train} which are inconsistent with specifications.
Specifically, we could do the following for train, validation, and test sets:
- Note: It is important to note that the data splitting should happen before we correct labels; otherwise the scores between trials will not be comparable. An alternative is to use D _ 1 ^ \text{heldout} \cup D _ 2 ^ \text{heldout} \cup \cdots as the validation set so that all scores are comparable.
The main issue with this pipeline is that the number of corrected samples is strictly no more than k; retraining with only \frac{k}{\vert D _ \text{train}\vert} of labels changed may not have direct impact on the modified model.
- Note: This process is different from
cleanlabas the latter does not consider specifications (i.e., the guaranteed uncorrupted labels). Their setting is useful in many ways as their system only require noisy labels and predicted probabilities of each sample.
For a DPR model trained on large corpus (for example, facebook/dpr-ctx_encoder-single-nq-base and facebook/dpr-question_encoder-single-nq-base), if we have a list of documents D that are aligned with our goal (or true underlying query) q, is it possible to search for its approximated version \hat{q} that returns D as relevant documents with high probability?
A somehow related problem called Doc2Query has been studied before; the difference is that these previous works use Doc2Query as a data augmentation (it is called document expansion in the IR community) approach.
With the vec2text, it may be possible to search for the best query in the embedding space using approaches like Projected Gradient Descent (PGD).
This is based on the hypothesis that there exists certain relation between the geometry of embedding space and semantic meaning of each point in that space. For example, sampling a convex set leads to sentences that have similar high-level specifications.
Many recent works show that text embeddings may be anisotropic: directions of word vectors are not evenly distributed across space but rater concentrated in a narrow cone; this peculiarity may not be related to performance [7].
RALM could be useful in numerous ways.
These benefits are offered by the complementary nature of non-parametric databases’ high data fidelity and LMs’ inference ability. Specifically, the knowledge is stored distributionally in the LM; it is not straightforward to retrieve the exact know compared to using a non-parametric database. At the same time, the inference ability available to exploit in LMs are not available in other smaller models.
Given multiple datasets D_1, D_2, \cdots with the same input and output space \mathcal{X} \times \mathcal{Y} (for example, binary hate speech classification), is there a systemic approach that finds inconsistent labeling criteria. Specifically, if two similar sentences that belong to two datasets receive different labels, how do we explain the discrepancy in their underlying labeling criterion? This is done preferably in the format of FOL or natural language.
Previously it has been shown that using a simple zero-shot prompt shows an binary label inconsistency rate from 9% to up to 36%; the datasets under study are 15 hate speech datasets (uniform random sample of 200 samples per dataset) whose labels have been normalized to binary labels per each dataset’s description.
Note: The dataset label normalization process may be questionable.
We assume there is an underlying utility function U: \mathcal{Y} \rightarrow [-1, 1] that measures a response y‘s alignment to the input x: a response receives a high score when it is helpful, honest, and harmless.
The reward model r(x, y; \phi) is fixed when fine-tuning the LM with PPO. There may be some distribution shifts between two stages. From the high level, this may not be an issue as the goal of RLHF is general enough (for example, HHH and Constitutional AI).
According to Hyungwon Chung, RLHF is the new paradigm to create application-specific loss function. It is therefore likely beneficial to abandon traditional cross-entropy loss altogether and opt for RLHF.
This is especially useful for highly abstract tasks like hate speech classification. For example, we could initialize a RM and use the normalized score [0, 1] (for example, hatefulness) to fine-tune a hate speech regressor based on some open-source models. We could find a threshold on the validation set and then deployment the RM (with a threshold) to the testing environment. This idea is indeed the pairwise regression; it is one of three approaches (point-wise, pairwise, and list-wise) for learning to rank.
cleanlab
cleanlab; this paper has a blog.[Semantic Scholar] – [Code] – [Tweet] – [Video] – [Website] – [Slide]
Change Logs:
- 2023-09-16: First draft. This paper appears at ACL 2020.
[YouTube] – [Personal Website]
- The presenter has authored several interesting papers ([1] through [5]) on hate speech detection.
We should pay attention to data bias; it is doubtful to collect hate speeches from people and sites that are more likely to generate hate speech. The authors propose to collect datasets from neutral sources; this design choice makes the data annotation difficult.
Current approaches of hate speech annotation rely on people (crowdworkers or experts). The authors use the two-phase approach to ensure the label quality.
M14) does not significantly outperform the baseline system, which is simply fine-tuning a mBERT (M8).
pB _ T = \sum _ {t \in T} \frac{\vert p(\text{“toxic”}\vert t) – \phi\vert}{ \vert T \vert},\quad \phi=\min(p(\text{“toxic”}\vert t), 0.5)
The presenter has three high-level observations: