He was afraid that he would lose the necessary support from the south. He was a president during a tough time when it came to cooperation with the congress. Without the southern support he wouldn't be the president, and he feared that by going on the march he would alienate voters. This is why even when he endorsed them he didn't want to fully support them because he didn't want to make southerners angry since the south was a huge voters base. He didn't want to push civil rights too rapidly.
