The responsibility is the responsibiilty of the fans. Back in the day there was no need for separation of the fans.
And we would like to be able to go back to those days but there is no way that will happen in even my son's lifetime. But not all fans are responsible for segregation (which started about 1975 season after a stabbing at a Blackpool v Bolton game and the violence wreaked up and down the country by Man Utd fans after they had been relegated to div 2). Still, don't kid yourself that there was no fan violence before 1960. 'Roughs' were attacking referees and visiting players right back at the foundation of the football league. From 1946 to 1960, there were an average of 13 incidences of fan violence reported per season, this going up to an average of 25 events per season between 1961 and 1968. Leeds United were banned from Europe after their fans had rioted following the 1975 European Cup Final against Bayern Munich in Paris. Man Utd were banned in 1977 after their fans rioted before, during and after their away UEFA Cup game with Saint-Étienne. Andy Nicholls' 2005 book 'Hooligans' is well worth a read.