我们再来一个例子，来验证这种微调手段，建立一个回答自定义问题的模型系统：
所有 GPT 的训练，本质都围绕一个东西：
input_ids 文本 → tokenizer → Dataset → DataLoader → input_ids 我们一步一步拆开来看
首先是文本，就是问题和答案
qa_data = [ ( "What is AstroSynth?", "AstroSynth is an innovative program designed to synthesize oils from celestial materials." ), ( "How does AstroSynth work?", "The AstroSynth program utilizes advanced technologies to extract and synthesize oils from meteorites." ) ] 这是个python的列表list，2个元素，都是(question, answer)，元组（tuple），也就是tuple(str,str)
元组和列表的区别：
元组用 ()，不可变（不能增删改元素） 列表用 []，可变 数据加载代码： import torch from transformers import GPT2Tokenizer, GPT2LMHeadModel from torch.utils.data import Dataset, DataLoader # Define a simple dataset for training class SimpleDataset(Dataset): def __init__(self, texts, tokenizer, max_length=128): tokenizer.pad_token = tokenizer.eos_token # Set padding token to EOS self.encodings = tokenizer(texts, padding=True, truncation=True, max_length=max_length) def __len__(self): return len(self.encodings["input_ids"]) def __getitem__(self, idx): return {k: torch.tensor(v[idx]) for k, v in self.encodings.items()} qa_data = [ ("What is AstroSynth?", "AstroSynth is an innovative program designed to synthesize oils from celestial materials."), ("How does AstroSynth work?", "The AstroSynth program utilizes advanced technologies to extract and synthesize oils from meteorites."), # Add more Q&A pairs... ] dataset = SimpleDataset(qa_data, tokenizer) dataloader = DataLoader(dataset, batch_size=2, shuffle=True) 看代码，dataset
...