It’s a very chicken and egg problem - the US rail network is terrible because no one takes it, and no one takes it because it’s terrible. (And it’s terrible for a host of reasons, including enormous distances, old/slow trains, car culture/car company lobbying, etc., etc.)
The biggest reason behind all of those is that passenger rail rides the same tracks as freight and freight always gets priority.
That's a later development. The downfall started when it was the private railroads who operated the long-distance trains and gave them all the priority they could. (I know a bit of history and context and a long time ago, I rode an Amtrak train myself :))