Not All Documents Are What You Need for Extracting Instruction Tuning Data

Instruction tuning improves the performance of large language models (LLMs), but it heavily relies on high-quality training data. Recently, LLMs have been used to synthesize instruction data using seed question-answer (QA) pairs. However, these synthesized instructions often lack diversity and tend to be similar to the input seeds, limiting their applicability in real-world scenarios. To address this, we propose extracting instruction tuning data from web corpora that contain rich and diverse knowledge. A naive solution is to retrieve domain-specific documents and extract all QA pairs from them, but this faces two key challenges: (1) extracting all QA pairs using LLMs is prohibitively expensive, and (2) many extracted QA pairs may be irrelevant to the downstream tasks, potentially degrading model performance. To tackle these issues, we introduce EQUAL, an effective and scalable data extraction framework that iteratively alternates between document selection and high-quality QA pair extraction to enhance instruction tuning. EQUAL first clusters the document corpus based on embeddings derived from contrastive learning, then uses a multi-armed bandit strategy to efficiently identify clusters that are likely to contain valuable QA pairs. This iterative approach significantly reduces computational cost while boosting model performance. Experiments on AutoMathText and StackOverflow across four downstream tasks show that EQUAL reduces computational costs by 5-10x and improves accuracy by 2.5 percent on LLaMA-3.1-8B and Mistral-7B

Transmit Beampattern Synthesis for Active RIS-Aided MIMO Radar via Waveform and Beamforming Optimization

In conventional colocated multiple-input multiple-output (MIMO) radars, practical waveform constraints including peak-to-average power ratio, constant or bounded modulus lead to a significant performance reduction of transmit beampattern, especially when the element number is limited. This paper adopts an active reconfigurable intelligent surface (ARIS) to assist the transmit array and discusses the corresponding beampattern synthesis. We aim to minimize the integrated sidelobe-to-mainlobe ratio (ISMR) of beampattern by the codesign of waveform and ARIS reflection coefficients. The resultant problem is nonconvex constrained fractional programming whose objective function and plenty of constraints are variable-coupled. We first convert the fractional objective function into an integral form via Dinkelbach transform, and then alternately optimize the waveform and ARIS reflection coefficients. Three types of waveforms are unifiedly optimized by a consensus alternating direction method of multipliers (CADMM)-based algorithm wherein the global optimal solutions of all subproblems are obtained, while the ARIS reflection coefficients are updated by a concave-convex procedure (CCCP)-based algorithm. The convergence is also analyzed based on the properties of CADMM and CCCP. Numerical results show that ARIS-aided MIMO radars have superior performance than conventional ones due to significant reduction of sidelobe energy.