Larger and more instructable language models become less reliable

Citations Over TimeTop 1% of 2024 papers

Abstract

The prevailing methods to make large language models more powerful and amenable have been based on continuous scaling up (that is, increasing their size, data volume and computational resources1) and bespoke shaping up (including post-filtering2,3, fine tuning or use of human feedback4,5). However, larger and more instructable large language models may have become less reliable. By studying the relationship between difficulty concordance, task avoidance and prompting stability of several language model families, here we show that easy instances for human participants are also easy for the models, but scaled-up, shaped-up models do not secure areas of low difficulty in which either the model does not err or human supervision can spot the errors. We also find that early models often avoid user questions but scaled-up, shaped-up models tend to give an apparently sensible yet wrong answer much more often, including errors on difficult questions that human supervisors frequently overlook. Moreover, we observe that stability to different natural phrasings of the same question is improved by scaling-up and shaping-up interventions, but pockets of variability persist across difficulty levels. These findings highlight the need for a fundamental shift in the design and development of general-purpose artificial intelligence, particularly in high-stakes areas for which a predictable distribution of errors is paramount.

Related Papers

• → Bespoke Regulation for Bespoke Medicine? A Comparative Analysis of Bioprinting Regulation in Europe, the USA and Australia(2021)17 cited
• Bespoke, Mdre And Challenges In Mdre In Comparison With Bespoke(2012)
• → External Barriers to User-Centred Development of Bespoke Medical Devices in the UK(2013)1 cited
• From Making the Bespoke to Manufacturing the Bespoke(2012)
• → A bespoke model of inclusive education for new arrivals(2020)