Every year, government takes millions of decisions on matters such as individuals' entitlement to social security, their immigration status, and tax liability. There are significant and pervasive concerns about the quality of these decisions. Government case-workers can make poor decisions if they do not collect all the relevant facts or if they misinterpret the relevant legal rules and guidelines. Poor decision-making means more appeals and challenges, increased costs and time, and stress for the individuals involved. Over recent years, the "right first time" agenda has been advanced as a solution to this problem. This article critically examines this agenda and the mechanisms available to government agencies to improve their decision-making. Recognising the difficulties of measuring the quality of decision-making, the article considers how poor decision-making arises and how it might be remedied. I argue that to improve their decision-making, government agencies need to engage in organisational learning. This occurs when individuals within an organisation experience a problematic mismatch between expected and actual results and inquire into it on the organisation's behalf. Organisations learn when they identify appropriate lessons from history which are then encoded into routines that guide future behaviour. The paper examines mechanisms to improve initial decision-making including: re-organising internal decision processes; using feedback from tribunals; making polluters pay; and modifying agency culture.