Taken off the street in 1965 because of construction in the area, the monument returned in time for the anniversary of the battle in 1970. Around this time, the typical September 14 ceremony consisted of a wreath-laying followed by dinner at Antoine's Restaurant. Speakers at these events included US Representative F. Edward Hebert and New Orleans City Councilman James T. Moreau. Usually, these speakers characterized the battle as a fight against tyranny, downplayed racial tension, and left out the White League's white supremacist motivations.
As news of the Liberty Place Commission's ceremonial annual dinners began to fade from the Times Picayune, the monument continued to make the news occasionally as it became a site of protest.
In 1970, the Times-Picayune noted that the monument "carried a defacing smudge of black pitch or paint on it." After numerous protests by black political activists in the early 1970's, Mayor Moon Landrieu's administration attempted to solve the problem by placing an explanatory plaque next to the historical marker which read,
"Although the 'Battle of Liberty Place' and this monument are important parts of New Orleans history, the sentiments in favor of white supremacy expressed thereon are contrary to the philosophy and beliefs of present-day New Orleans."
This gesture satisfied almost no one. In 1976, the NAACP Youth Council requested the monument's removal, while some decried the plaque as "historical revisionism." Furthermore, modern white supremacist groups such as the Ku Klux Klan began to see the memorial as a rallying point for planned marches and demonstrations.
In 1981, the monument nearly left public view at Mayor Ernest "Dutch" Morial's order, sparking a new round of public discussion and protest. Ultimately, the City Council blocked any move or alteration, and the monument remained on Canal Street, although partially hidden behind tall bushes.
The monument left public view in 1989, reportedly for safe keeping, amidst construction on Canal Street. Mayor Sidney Barthelemy pledged to return the marker, though his administration missed the originally stated date for its replacement. The structure stayed in storage until February 1993, when a movement led by David Duke, a former Grand Wizard of the Ku Klux Klan, sued for its return. To the chagrin of many residents, the city obliged but moved the obelisk off of Canal Street to its present site in the curve of Iberville Street, between railroad tracks and the entrance to a parking garage.
In addition to planting the marker in a less obtrusive location, the city removed the plaque championing white supremacy and affixed a new plaque commemorating the Metropolitan Police casualties, several of whom were African-American. Still, heated protests led to a number of arrests at the monument's rededication, including State Representative Avery Alexander, who was 82 years old at the time.
Since no museums offered to house the memorial, it remained in its secluded spot and had fallen out of the public discourse. Vandals removed four small pillars in the midsection of the monument, and graffiti was often found on its plaques. After a 6-1 vote by the City Council in 2015, this statue was one of four Confederate monuments scheduled to be removed. The monument was removed on April 24, 2017 in the middle of the day. The monument is currently housed in a city-owned warehouse.