So it was replaced by a model based on the 124M GPT-2 model. The 124M model was also trained on the full dataset of 92,931 pairs. To improve on these first two models, the finetuning was run for 1000 steps with the full dataset of 92,931 pairs, and using the medium sized (344M) model as a base. 2000 finetuning steps were used initially, برای دیدن ادامه مطلب اینجا را کلیک کنید and then 4000 steps for a second attempt. The second solution was that the RNN would be trained to distiguish between artificially and manually generated questions. Out of a generated 145 questions, 60 were manually rated as good (41%). These were then passed to the Discriminator, which had been trained for 10 epochs. The advantage of this is that the dataset used to train the generator could be re-used as the “good” questions dataset, and the “bad” questions dataset could be automatically generated by the question generator. In a production environment, the generator should be deployed to a far more powerful device. The latter of these two approaches was chosen, because it removes the task of tagging the results manually, which makes a larger dataset more easily acquired. The initial experimentation with GPT-2 seemed promising, so the data from xinyadu/nqg was used to finetune it and improve results. The disadvantage of this approach is that to manually tag thousands of questions as good or bad would be labour-intensive. Learning relative positioning is particularly important in this instance, because whether the word is before or after the question tag determines whether it is part of the question or not. The neural network architecture chosen for this task was an RNN with two stacked Long Short-Term Memory layers, so that the RNN would be able to learn the significance of relative positioning. To rank the questions, a Recurrent Neural Network (RNN) was trained. Note that as the Discriminator was trained to identify artificial questions, the better questions score lower, not higher. The average score for good questions was 1.467, and the average score for bad questions was 1.670. This suggested that the Discriminator did have some ability to distinguish between good and bad questions. To investigate how the Discriminator accuracy is affected by the number of epochs for which it was trained, the test was re-run with 3 and 30 Epochs. This is plotted below, and shows that as the number of epochs increases, there is improvement, but still significant overlap. If you have any inquiries relating to in which and how to use اطلاعات بیشتر, you can make contact with us at our own page.