For different areas of tax e.g. vat, paye etc there are different requirements for the length of time that accounting records should be kept.
However applying the general rule of keeping them for 6 years plus the current year will mean that you have covered all bases.
This means keeping things like bank statements, sales and purchase invoices etc.
So the best thing to do is to get some boxes, label them for each year and put them away in the attic!
However if you are subject to any sort of investigation then HMRC can go back further !