Chronically sick benefit claimants will no longer be required to prove they are still ill every six months, the work and pensions secretary has announced.

Employment support allowance will now continue automatically for those who have lifelong, severe health conditions with no prospect of improvement, Damian Green said.

The testing process, which includes reassessing conditions every six months, has come under intense criticism for failing some of the most seriously ill and disabled.

The move was welcomed by the former work and pensions secretary Iain Duncan Smith, who said he resigned over concerns he could not push the reforms through.

Green said the change was being made because it was pointless to repeatedly test claimants who are not going to get better. He told BBC Radio 4’s Today programme: “I believe in a welfare state where you have got to be hard-headed, but you shouldn’t be hard-hearted.

“We want the welfare state to work for everyone, just as we want the economy to work for everyone, and there are a group of people for whom constant reassessment is pointless and which does increase their stress and anxiety levels.”

Green said it would be a retrograde move to scrap the tests for all ESA claimants, because for the “vast majority of people, work actually helps them”.

But the reforms will help end the anxiety and financial insecurity that claimants may have felt, said Green, who will unveil the plans at the Conservative party conference.

The criteria will be drawn up with health professionals but people with autism and illnesses such as severe Huntington’s or a congenital heart condition are likely to qualify for continuous payments without reassessment.

Green said: “We are building a country that works for everyone – not just the privileged few. A key part of that is making sure that all those who are able to work are given the support and the opportunity to do so. But it also means ensuring that we give full and proper support to those who can’t.

“That includes sweeping away any unnecessary stress and bureaucracy – particularly for the most vulnerable in society.

“If someone has a disease which can only get worse then it doesn’t make sense to ask them to turn up for repeated appointments. If their condition is not going to improve, it is not right to ask them to be tested time after time. So we will stop it.”

The reform will be unveiled at the four-day Conservative party conference, which begins in Birmingham on Sunday.

Debbie Abrahams, shadow work and pensions secretary, said: “As ever with this government though, the devil is in the detail. While the end to repeated assessments will be a relief to those that have been affected, this announcement falls far short of the fundamental shift to a more holistic, person-centred approach we so desperately need.

“Too many sick and disabled people will remain subject to this harmful, ineffective assessment. We will continue to push the Tories for a better deal for disabled people.”

Michelle Mitchell, chief executive of the MS Society, said: “This is a victory for common sense. Frequent reassessments for people with progressive conditions such as MS are too often a waste of time and money. They can leave people with uncertainty and fear of having their support taken away.

“We are therefore delighted that the government have listened to our concerns and have agreed to stop reassessments – albeit for only some ESA claimants.

“This is good news, but there’s still a lot more to do for people with MS, including improving the assessment for ESA and calling for inappropriate reassessments to stop for other vital benefits, like PIP.”

Duncan Smith, whose resignation from the role of work and pensions secretary was seen as an attack on the-then leadership of David Cameron and George Osborne, told the Today programme he completely agreed with the changes.

“We worked to change this process, it was one we inherited and it just functioned badly on this area,” he said.

Smith said he hoped the change formed one part of wider reforms that would see the Department for Work and Pensions and Department of Health collaborate.

“The reason I resigned … was because I felt the disability demand that was in the budget would have absolutely punched a hole straight through what would have been quite a progressive reform, so I welcome this announcement, I think it’s the right thing to do,” he said.

Mark Atkinson, chief executive at disability charity Scope, said: “In the short term this will be a welcome change for some disabled people, and it’s good to see the government recognising that the work capability assessment needs wider reform.

“We’d like to see the government going much further and starting a consultation with disabled people on the wholesale reform of the fitness-for-work test.

“Disabled people are pushing hard to get jobs but still face many barriers to find work and thrive in employment. The test should be the first step in identifying those barriers so the right support can be put in place to help people get back to work.”