Yixin Lian (连怡鑫)

In recent years, large language models (LLMs) have achieved breakthrough progress in many dialogue generation tasks. However, their lack of emotion and fine-grained role awareness limits the model’s ability to provide personalized and diverse interactions further. Current methods face high costs in collecting high-quality annotated data for scenarios such as role-playing, and traditional human alignment methods are difficult to deploy due to the inherent diversity of model behavior in role-playing scenarios. Inspired by the alignment of models for safety behaviors through RLHF (Reinforcement Learning from Human Feedback), in this paper, we revisit model role-playing behavior from the perspective of persona alignment and propose a novel annotation-free framework named \textbf\underlinePersona-Aware \textbf\underlineContrastive \textbf\underlineLearning (PCL) to align LLMs’ behavior during role-playing, enhancing the model’s role consistency. Specifically, we first design a role chain method to encourage the model to self-question based on the role characteristics and dialogue context to adjust personality consistency. Then, we further enhance the model’s role-playing strategy through iterative contrastive learning between the use of role characteristics and not. Experiments on both black-box and white-box LLMs show that LLMs equipped with PCL significantly outperform vanilla LLMs under automatic evaluation methods (CharEval & GPT-4) and human expert evaluation.

RAG systems consist of multiple modules to work together. However, these modules are usually separately trained. We argue that a system like RAG that incorporates multiple modules should be jointly optimized to achieve optimal performance. To demonstrate this, we design a specific pipeline called SmartRAG that includes a policy network and a retriever. The policy network can serve as 1) a decision maker that decides when to retrieve, 2) a query rewriter to generate a query most suited to the retriever and 3) an answer generator that produces the final response with/without the observations. We then propose to jointly optimize the whole system using a reinforcement learning algorithm, with the reward designed to encourage the system to achieve the highest performance with minimal retrieval cost. When jointly optimized, each module can be aware of how other modules are working and thus find the best way to work together as a complete system. Empirical results demonstrate that the jointly optimized system can achieve better performance than separately optimized counterparts.

Chain-of-Thought (CoT) prompting has proven to be effective in enhancing the reasoning capabilities of Large Language Models (LLMs) with at least 100 billion parameters. However, it is ineffective or even detrimental when applied to reasoning tasks in Smaller Language Models (SLMs) with less than 10 billion parameters. To address this limitation, we introduce Dialogue-guided Chain-of-Thought (DialCoT) which employs a dialogue format to generate intermediate reasoning steps, guiding the model toward the final answer. Additionally, we optimize the model’s reasoning path selection using the Proximal Policy Optimization (PPO) algorithm, further enhancing its reasoning capabilities. Our method offers several advantages compared to previous approaches. Firstly, we transform the process of solving complex reasoning questions by breaking them down into a series of simpler sub-questions, significantly reducing the task difficulty and making it more suitable for SLMs. Secondly, we optimize the model’s reasoning path selection through the PPO algorithm. We conduct comprehensive experiments on four arithmetic reasoning datasets, demonstrating that our method achieves significant performance improvements compared to state-of-the-art competitors.

We propose the hierarchical verbalizer ("HierVerb"), a multi-verbalizer framework treating HTC as a single- or multi-label classification problem at multiple layers and learning vectors as verbalizers constrained by hierarchical structure and hierarchical contrastive learning.

To improve the essential capability of responding and establish a benchmark in the live open-domain scenario, we introduce the LiveChat dataset, composed of 1.33 million real-life Chinese dialogues with almost 3800 average sessions across 351 personas and fine-grained profiles for each persona. LiveChat is automatically constructed by processing numerous live videos on the Internet and naturally falls within the scope of multi-party conversations, where the issues of Who says What to Whom should be considered.

本发明公开了一种对话模型生成、应用方法、系统、设备及存储介质，涉及计算机技术领域。一种对话模型生成方法通过获取预设的对话样本；对对话样本进行预处理，得到三元组数据；将三元组数据输入预设的自然语言模型进行语义理解，输出与三元组数据对应的语义意图；将三元组数据和语义意图输入初始神经网络进行训练，得到对话模型。再通过一种对话模型应用方法应用对话模型生成方法生成的对话模型，实现通过多源多模态的数据训练出对话效果较好的对话模型，并基于应用对话模型，**地提高了对话模型的对话迁移场景能力，增强对话回复的信息丰富度和准确率。

本发明实施例提供一种基于直播场景的对话数据集构建方法及装置，该方法包括：基于主播直播视频的用户评论，获取用户评论发表后预设时间内文本格式的主播讲话内容；获取主播讲话内容中与用户评论具有重叠词汇的句子并计算和用户评论间的语义相似度，根据语义相似度获取针对用户评论的主播回复；根据用户评论和主播回复构建对话数据集。本发明实施例基于真实直播场景下大量的主播直播视频和用户评论，通过时间匹配、词汇匹配及相似度判断构建对话数据集，得到了特定人物特征下的大规模真实数据集，有利于训练得到具有丰富人物特征且在泛化性、多样性、相关性方面表现良好的对话系统，有利于真实场景对话系统的效果提升，提升用户体验度。

We propose LAMA, a multi-head attention based sequential model to process log streams as template activity (event) sequences.

In this paper, we introduce MultiWOZ 2.3, in which we differentiate incorrect annotations in dialogue acts from dialogue states, identifying a lack of co-reference when publishing the updated dataset.