More than £42m has been axed from councils' sports and leisure budgets since 2010, according to a BBC survey.
Among the regions which saw the biggest losses were London and north-west England, which saw cuts of more than £12.3m.
Concern was expressed by athletes that cutting facilities was "short-termism" that could impact on communities' health and fitness levels.
Some of the biggest cuts occurred in the North West, where Liverpool City Council closed Woolton Swimming Pool, saving more than £3m.
In the West Midlands, which saw £9.6m of cuts, the region's only 50m pool - in Coventry - was among the facilities to face the axe.
And in London, where budgets were cut by £8.8m, Mornington Crescent Sports Centre in Camden was among the facilities to close.
In other regions, Sheffield lost the Don Valley Stadium, where Olympic heptathlon champion Jessica Ennis-Hill had trained, while Newcastle-upon-Tyne saw the closure of its City Pool in 2013.
David Moorcroft, the Commonwealth Games gold medallist and former chief executive of UK Athletics, said: "In times of cutbacks to public services, rightly or wrongly, sport and leisure is one of the first things to get cut.
"It's really unfortunate because the health and happiness of the nation and communities is based around being able to access facilities that encourage people to take physical activity.
"Ultimately, if we are trying to reduce obesity among young people, you can't really have clubs and volunteers doing all that work. Once a facility is lost, it's gone forever. When you come out of recession, it's very difficult to rebuild it."
Emma Boggis, chief executive of the Sport and Recreation Alliance which represents sports governing bodies in the UK, said she had "some sympathy" with local authorities "and the extreme financial pressures they are under"
"But reducing investment in sport and in leisure facilities is storing up problems for the longer-term," she said.
"Limiting access to leisure facilities will result in greater inactivity and bigger costs to the NHS in terms of tackling conditions like diabetes, cardiovascular disease and depression."
The Department for Culture, Media and Sport said £1bn of public money had been invested into grassroots sport through Sport England.
David Sparks, who chairs the Local Government Association, said councils had had "little choice" but to squeeze budgets.
"The reality is that, within a few years, well over half of the council tax everyone pays will have to be spent on social care," he said.
"With demand on these life and death services continuing to rise and funding from central government continuing to fall, councils will have little choice."
